In this article, I will delve into whether artificial intelligence (AI), particularly the emergence of tools like generative AI (GenAI), will replace classical software outsourcing as we know it.

AI vs software outsourcing: An opportunity or a threat?

We've tested more than 100 anti-malware apps to help you find the top malware protection and removal software for all your devices. When the IBM PC was new, I served as the president of the San ...

The Best Malware Removal and Protection Software for 2023

Study.com analyzed popular schoolwork AI detectors and news reports to explain the pros and cons of AI schoolwork checkers.

A parent and teacher guide to AI detectors like GPTZero

If you’re on a computer and you get a text on your phone, wouldn’t it be great if you could reply without picking it up? Yes, it would, and there are great apps and services that enable just ...

How to send a text message from a computer

Despite economic fears, Gartner predicts spending on software will increase 11% in 2023, up to $879 billion from $790 billion in 2022. Why? The answer is simple. No business can survive in today ...

Four Crucial Steps For Successful Software Implementation

You can have multiple cursors on your computer. However, you will need to use specialized software or hardware configurations. You can use the MouseMax application on Windows to get more than one ...

Can you have 2 cursors on 1 computer?

TRIPS is an important treaty under the WTO. The TRIPS Agreement represented a major expansion in the scope of copyright protection for computer software. TRIPS brought software under the purview of the Berne Convention by defining it as a literary work.

Preserving national integrity: Shunning software piracy

Computer vision at the edge is unlocking actionable ... Dell Validated Designs connects multiple ISVs (independent software vendors) and partners to optimize application-based solutions that ...

Delivering 5 Key Outcomes With Computer Vision At The Edge

Cornell researchers have developed a new optimization tool to estimate motion throughout an input video, which has potential applications in video editing and generative AI video creation.

Novel optimization tool allows for better video motion estimation

Even without a sale, a savvy shopper can find a decent Windows-based laptop computer for less than $1,500 or a basic MacBook starting at around $1,000. But when these portable computers go on sale ...

How to save big on a laptop computer during Amazon's October Prime Day 2023

Optensity

optensity

This is the first part in a multi-part series on building Agents with OpenAI's Assistant API using the Python SDK. What Are Agents? The way I like to look at it, an agent is really just a piece of software leveraging an LLM (Large Language Model) and trying to mimic human behavior. That means it can not only converse and understand language, but it can also perform actions that have an impact on the real world. These actions are typically called tools. In this blog post, we will explore how to build an agent using OpenAI's Assistant API using their Python SDK. Part 1 will be just the skeleton of the assistant. That is, just the conversational part. I chose to build a CLI app on purpose to be framework agnostic. We will purposefully call our implementation an Agent and refer to the OpenAI SDK implementation as an Assistant to easily distinguish between the two. I use the terms and interchangeably when it comes to functions that the Agent is able to call. Part 2 will cover function callin in more detail. tools functions Prerequisites To follow along with this tutorial, you will need the following: Python3 installed on your machine An OpenAI API key Basic knowledge of Python programming OpenAI Assistant Concepts : An Assistant in the Assistants API is an entity configured to respond to user messages. It uses instructions, a chosen model, and tools to interact with functions and provide answers. Assistant : A Thread represents a conversation or dialogue in the Assistants API. It is created for each user interaction and can contain multiple Messages, serving as a container for the ongoing conversation. Thread : A Message is a unit of communication in a Thread. It contains text (and potentially files in the future) and is used to convey user queries or assistant responses within a Thread. Message : A Run is an instance of the Assistant processing a Thread. It involves reading the Thread, deciding whether to call tools, and generating responses based on the model's interpretation of the Thread's Messages. Run Setting Up the Development Environment The first step is to create a virtual environment using and activate it. This will ensure that our dependencies are isolated from the system Python installation: venv python3 -m venv venv
source venv/bin/activate Let's install our only dependency: the package: openai pip install openai Create a file. Let's populate with some basic runtime logic for our CLI app: main.py while True:
    user_input = input("User: ")
    if user_input.lower() == 'exit':
        print("Exiting the assistant...")
        break
    print(f"Assistant: You said {user_input}") Try it out by running : python3 main.py python3 main.py
User: hi
Assistant: You said hi As you can see, the CLI accepts a User message as input, and our genius Assistant doesn't have a brain 🧠 yet so he just repeats the message right back. Not so smart yet. The Agent Now, the fun 😁 (or headaches 🤕) begins. I'll provide all the imports needed for the final class right now, so you don't rack your brain on where things are coming from since I kept imports out of code samples for brevity. Let's start by building an class in a new file : Agent agent.py import time
import openai
from openai.types.beta.threads.run import Run

class Agent:
    def __init__(self, name: str, personality: str):
        self.name = name
        self.personality = personality
        self.client = openai.OpenAI(api_key="sk-*****")
        self.assistant = self.client.beta.assistants.create(
            name=self.name,
            model="gpt-4-turbo-preview"
        ) In the class constructor, we initialize the OpenAI client as a class property by passing our OpenAI API key. Next, we create an class property that maps to our newly created Assistant. We store and as class properties for later use. assistant name personality The argument we are passing to the create method is just for identifying the Assistant in the OpenAI dashboard, and the AI is not actually aware of it at this point. You actually have to pass the name to the which we will see later. name instructions You could already set when creating the Assistant, but it will actually make your Assistant less flexible to dynamic changes. instructions You can update an Assistant by calling , but there is a better place to pass in dynamic values that we will see when we get to Runs. client.beta.assistants.update Note that if you pass here and then again when creating a Run, the Assistant's will be overwritten by the of the run. They do not complement each other, so choose one based on your needs: Assistant level for static instructions or Run level for dynamic instructions. instructions instructions instructions For the model, I chose the model so that we can add function calling in part 2 of this series. You could use if you want to save a few fractions of a penny while giving yourself a migraine of pure frustration down the line when we implement tools. gpt-4-turbo-preview gpt-3.5-turbo GPT 3.5 is terrible at calling tools; the hours I've lost trying to deal with it allow me to say that. 😝 I'll leave it at that, and more on this later. Creating a Thread, Adding Messages, and Retrieving the Last Message After we create an agent, we will need to start a conversation thread. class Agent:
    # ... (rest of code)
    def create_thread(self):
        self.thread = self.client.beta.threads.create() And we will want a way to add messages to that thread: class Agent:
    # ... (rest of code)
    def add_message(self, message):
        self.client.beta.threads.messages.create(
            thread_id=self.thread.id,
            role="user",
            content=message
        ) Note that at the moment, it is only possible to add messages with the role . I believe OpenAI plans on changing this in a future release as this is pretty limiting. user Now, we can get the last message in the thread: class Agent:
    # ... (rest of code)
    def get_last_message(self):
        return self.client.beta.threads.messages.list(
            thread_id=self.thread.id
        ).data[0].content[0].text.value Next, we create an entry point method to test out what we have so far. Currently, the method just returns the last message in the thread. It doesn't actually perform a Run. It's still brainless. run_agent run_agent class Agent:
    # ... (rest of code)
    def run_agent(self):
        message = self.get_last_message()
        return message Back in , we create the agent and our first thread. We add a message to the thread. Then return that same message back to the user, but this time, coming from that live thread. main.py from agent import Agent

agent = Agent(name="Bilbo Baggins",
                      personality="You are the accomplished and renowned adventurer from The Hobbit. You act like you are a bit of a homebody, but you are always up for an adventure. You worry a bit too much about breakfast.")

agent.create_thread()

while True:
    user_input = input("User: ")
    if user_input.lower() == 'exit':
        print("Exiting the agent...")
        break
    agent.add_message(user_input)
    answer = agent.run_agent()
    print(f"Assistant: {answer}") Let's run it: python3 main.py
User: hi
Assistant: hi Still not very smart. Closer to a parrot 🦜 than a hobbit. In the next section, the real fun begins. Creating and Polling a Run When you create a run, you need to periodically retrieve the object to check the status of the run. This is called polling, and it sucks. You need to poll in order to determine what your agent should do next. OpenAI plans to add support for streaming to make this simpler. In the meantime, I will show you how to set up polling in this next section. Run Note the on the following method names which is the standard in Python for indicating that the method is intended for internal use and should not be accessed directly by external code. _ First, let's create a helper method for creating a , and update to call this method: _create_run Run run_agent class Agent:
    # ... (rest of code)
    def get_breakfast_count_from_db(self):
        return 1

    def _create_run(self):
        count = self.get_breakfast_count_from_db()
        return self.client.beta.threads.runs.create(
            thread_id=self.thread.id,
            assistant_id=self.assistant.id,
            instructions=f"""
                Your name is: {self.name}
                Your personality is: {self.personality}

                Metadata related to this conversation:
                {{
                    "breakfast_count": {count}
                }}
            """,
        )

    def run_agent(self):
        run = self._create_run() # add this line
        message = self.get_last_message()
        return message Notice how we pass the and to create a run. thread.id assistant.id Remember how I said at the beginning that there was a better place to pass in dynamic instructions and data? That would be the parameter when creating the Run. In our case, we could have the breakfast be fetched from a database. This will allow you to easily pass in different relevant dynamic data every time you want to trigger an answer. instructions count Now, your agent is aware of the world changing around it and can act accordingly. I like to have a metadata JSON object in my instructions that keeps relevant dynamic context. This allows me to pass in data while being less verbose and in a format that the LLM understands really well. Don't run this yet; it won't work because we aren't waiting for the run to complete when we are getting the last message, so it will still be the last user message. Let's solve this by building out our polling mechanism. First, we will need a way to repeatedly and easily retrieve a run, so let's add a method: _retrieve_run class Agent:
    # ... (rest of code)
    def _retrieve_run(self, run: Run):
        return self.client.beta.threads.runs.retrieve(
            run_id=run.id, thread_id=self.thread.id) Notice how we need to pass both the and to find a specific run. run.id thread.id Add a method to our Agent class: _poll_run class Agent:
    # ... (rest of code)
    def _cancel_run(self, run: Run):
        self.client.beta.threads.runs.cancel(
            run_id=run.id, thread_id=self.thread.id)

    def _poll_run(self, run: Run):
        status = run.status
        start_time = time.time()
        while status != "completed":
            if status == 'failed':
                raise Exception(f"Run failed with error: {run.last_error}")
            if status == 'expired':
                raise Exception("Run expired.")

            time.sleep(1)
            run = self._retrieve_run(run)
            status = run.status

            elapsed_time = time.time() - start_time
            if elapsed_time > 120:  # 2 minutes
                self._cancel_run(run)
                raise Exception("Run took longer than 2 minutes.") 🥵 Phew, that's a lot... Let's unpack it. receives a object as an argument and extracts the current Run . All the available statuses can be found in the OpenAI . We'll just use a few that suit our current purpose. _poll_run Run status docs We now run a while loop to check for a completed status while handling a few error scenarios. The actual billing of the Assistant API is a bit murky, so to be on the safe side, I opted to cancel my runs after 2 minutes. Even though there is an status for when OpenAI cancels runs after 10 minutes. If a run takes more than 2 minutes, you probably have a problem anyway. expired Since I also don't want to poll every few milliseconds, I throttle my request by only polling every 1 second until I hit the 2-minute mark and cancel my run. You can adjust this to whatever you see fit. Each iteration after the delay, we fetch the Run status again. Now, let's plug all that into our method. You will notice we first create the run with then we poll with until we get an answer or an error is thrown, and finally when the polling is finished, we retrieve the last message from the thread which will now be from the agent. run_agent _create_run _poll_run We then return the message to our runtime loop, so it can be sent back to the user. class Agent:
    # ... (rest of code)
    def run_agent(self):
        run = self._create_run()
        self._poll_run(run) # add this line
        message = self.get_last_message()
        return message Voilà, now when you run your agent again, you will get a reply from our friendly Agent: python3 main.py
User: hi
Assistant: Hello there! What adventure can we embark on today? Or perhaps, before we set out, we should think about breakfast. Have you had yours yet? I've had mine, of course – can't start the day without a proper breakfast, you know.
User: how many breakfasts have you had?
Assistant: Ah, well, I've had just 1 breakfast today. But the day is still young, and there's always room for a second, isn't there? What about you? How can I assist you on this fine day? In part 2, we will add the ability for our Agent to call tools. You can find the full code on my . GitHub Thank you for your reading. Happy to hear any thoughts and feedback in the comments. Follow me on Linkedin for more content like this: https://www.linkedin.com/in/jean-marie-dalmasso-1b5473141/

How to Build an Agent With an OpenAI Assistant in Python - Part 1: Conversational

পাইথনে ওপেনএআই সহকারী দিয়ে কীভাবে এজেন্ট তৈরি করবেন - অংশ 1: কথোপকথন

So erstellen Sie einen Agenten mit einem OpenAI-Assistenten in Python – Teil 1: Konversational

Cómo crear un agente con un asistente OpenAI en Python - Parte 1: Conversacional

Comment créer un agent avec un assistant OpenAI en Python - Partie 1 : Conversationnel

पायथन में ओपनएआई असिस्टेंट के साथ एक एजेंट कैसे बनाएं - भाग 1: संवादात्मक

Python で OpenAI アシスタントを使用してエージェントを構築する方法 - パート 1: 会話型

Python에서 OpenAI 도우미를 사용하여 에이전트를 구축하는 방법 - 1부: 대화형

Como construir um agente com um assistente OpenAI em Python - Parte 1: Conversacional

Как создать агент с помощью OpenAI Assistant на Python. Часть 1: Диалог

Python'da OpenAI Asistanıyla Aracı Nasıl Oluşturulur - Bölüm 1: Konuşmaya Dayalı

Cách xây dựng một tác nhân với trợ lý OpenAI trong Python - Phần 1: Đàm thoại

如何使用 Python 中的 OpenAI 助手构建代理 - 第 1 部分：对话式

Primer on Large Language Model (LLM) Inference Optimizations: 2. Introduction to Artificial Intelligence (AI) Accelerators

Primer on Large Language Model (LLM) Inference Optimizations: 2. Úvod do akcelerátorů umělé inteligence (AI)

Introducción a las optimizaciones de inferencia de modelos de lenguaje grandes (LLM): 2. Introducción a los aceleradores de inteligencia artificial (IA)

Manual sobre as optimizacións de inferencia do modelo de linguaxe grande (LLM): 2. Introdución aos aceleradores de intelixencia artificial (IA)

वृहद भाषा मॉडल (एलएलएम) अनुमान अनुकूलन पर प्राइमर: 2. कृत्रिम बुद्धिमत्ता (एआई) त्वरक का परिचय

Introduzione alle ottimizzazioni dell'inferenza del Large Language Model (LLM): 2. Introduzione agli acceleratori dell'intelligenza artificiale (IA)

大規模言語モデル (LLM) 推論最適化入門: 2. 人工知能 (AI) アクセラレータの概要

Primer on Large Language Model (LLM) დასკვნის ოპტიმიზაცია: 2. შესავალი ხელოვნური ინტელექტის (AI) ამაჩქარებლებში

Үлкен тілдік үлгідегі (LLM) қорытындыны оңтайландырудағы праймер: 2. Жасанды интеллект (AI) үдеткіштеріне кіріспе

Том хэлний загвар (LLM) дүгнэлтийг оновчтой болгох праймер: 2. Хиймэл оюун ухаан (AI) хурдасгуурын танилцуулга

Horudhac ku saabsan Qaabka Luqadda Weyn (LLM) Kordhinta Farshaxanka: 2. Horudhaca Sirdoonka Artificial (AI) Dardariyeyaasha

Primer on Large Language Model (LLM) Inference Optimizations: 2. Panimula sa Artificial Intelligence (AI) Accelerators

Sách nhập môn về Tối ưu hóa suy luận mô hình ngôn ngữ lớn (LLM): 2. Giới thiệu về bộ tăng tốc trí tuệ nhân tạo (AI)

THE SCIENCE OF LIGHT-PRODUCTION

He did not return to camp as Norman of Torn had ordered

THE CORONATION DAY

Tanks: The Deciding Battle of the War of 1932 

Tanques: la batalla decisiva de la guerra de 1932

Chars : la bataille décisive de la guerre de 1932

टैंक: 1932 के युद्ध की निर्णायक लड़ाई

戦車: 1932 年の戦争の決戦

Tanques: A Batalha Decisiva da Guerra de 1932

Xe tăng: Trận chiến quyết định năm 1932

坦克：1932 年战争的决定性战役

We all use JPG photos on our websites and apps. In this post, I’ll show how you can reduce image sizes by an additional 20–50% with a single line of code. This is accomplished by carefully analyzing the way JPG works and changing its logic. Consider a bear Can you tell the difference between the two bear photos below? The first is a photo compressed with JPG at quality=90. The other is still a JPG photo, that for the life of me looks identical, and takes up half the space compared to the other. You can download them to see for yourself. Show them in full screen, and check their file size. No trickery involved — the output is still a JPG image, that looks the same, and takes up much less space. If you want to see I’m not playing tricks with some cherry-picked example, you can play around andupload your own photos right here. How come JPG sucks I remember almost 15 years ago when I was an undergrad in Electrical Engineering, I sat in a classroom and was taught how JPEG works. I was in awe at the ingenuity of the engineers that handcrafted a method that takes an image and describes its content with much fewer numbers than you’d get by say, enumerating the values of each pixel independently. I wasn’t wrong in being impressed. Despite the provocative title for this post — I’m actually still impressed with the engineers who in the 1980s figured out a pretty good way to lower an image file size while making visually minor changes. But they left a lot of optimization opportunities untouched. And to see that, we need to understand how it works. How does JPG work You can skip this portion if you’re not interested in a trip down memory lane, but in order to improve JPG, you really need to understand how it works. So I’m going to briefly skim over what happens in regular old JPG compression. All the numeric examples are ripped straight off of Wikipedia, which has an . excellent write-up on this Step 1: divide the image into 8x8 blocks of pixels JPG applies the same logic for every block of 8x8 pixel values and applies the same logic for each color channel independently. So you can understand JPG just by looking at what it does to a single 8x8 block of pixel values. For example, let’s look at one block, and pretend this is the pixel values: Step 2: apply DCT transform to the block, and round down low values for some components DCT transform is a very close cousin to Fourier transform. It transforms pixel intensity values to spatial coefficients. You can read about it , but for our purposes, we can focus on the upshot. here After applying DCT transform, you end up with another 8x8 value, but they tend to have higher values in the upper left part of the block. For the example above, you’d get the following values 8x8 values: Not only do the values on the bottom right tend to be smaller, but they also tend to describe information about the block that is often less visually noticeable when omitted! This is because these values describe changes in the pixel intensity that go back and forth very quickly between neighboring pixels, and our visual system tends to “average them out”. So engineers in the 1980s sat and came up with something called a “quantization table”. It’s just a bunch of constants, that express our belief that keeping track of the values in the upper left corner of the DCT coefficient block is more important than keeping information about stuff that goes on in the bottom right. We’re just going to divide each value in the block by its corresponding quantization table value, and round the result. We use smaller values on the top left, and higher values on the bottom right. This process is , and . It just captures the intuition of engineers about how some coefficients are going to be less noticeable than others and doesn’t take into account all that are the actual contents in this specific image. done regardless of the content of the image regardless of how visually noticeable this rounding process is in context Quantization table. Take all the values in the above block, and divide by these numbers Here’s the result when you do this division: Get it? You get a bunch of 0s, which you can get away with not saving. Also, the few values that aren’t 0 tend to repeat or be very small, which takes fewer bits to save. There’s only a little bit more detail like running Huffman encoding and run-length encoding — but that’s details that matter exceeds the scope of this post. You now know pretty much how JPG works. What are we missing here I think just describing how the process works, it’s a bit obvious what JPG misses out on. . So it has no understanding of whether a certain loss of information would introduce an edge or not. It also doesn’t care if we’re losing information at the background or smack dab in the center of a person’s eyes. Can’t blame them — this protocol predates the time when this understanding was technically something we could accomplish. JPG doesn’t look at the picture . A machine learning model can solve that as well — you can directly optimize for a quantization table that empirically reduced visual distortion. JPG uses the same quantization table regardless of the image content . What if we can lower one coefficient from 1 to 0, and trade away another coefficient going up from 5 to 6, looking almost exactly the same? We’d gain space because it’s easier to encode. JPG doesn’t look at the entire block to optimize it further Deep learning to the rescue I might write a follow-up blog post on the details of the model we’ve trained. But the upshot is, in each weakness of the original JPG I’ve described above, you get an opportunity to improve results. Modern convolutional neural nets can do a great job at approximating the way we perceive images visually (with some strong disclaimers around adversarial examples which fool them and not humans). Once we learn how to visually determine if our change to the image content was visually identifiable, you can think of the very broad goal of “improve JPG” as well defined optimization process of “tweak every pixel in the image, and see if the result is visually unnoticeable and results in smaller DCT coefficients in a given block”. Let’s focus on how the first part is achievable — the latter part can be thought of as any numeric optimization procedure. What can we do to look at two patches of an image, and ask “do they look the same to a visual system that humans have?”. A taste of the solution: learning to see So, how do we use deep learning models to tell us how visually similar two image patches are? It turns out that taking a convolutional neural net that was trained on e.g. image classification, and lopping chopping it off midway through, you get internal representations of the image that are useful for visual comparison. So, if you think of a small patch of an image, let’s say 128x128 pixels. You can tweak their values as you see fit to make JPG take up less space. So let’s say you have some guess about a change to the image that would make it take up less space like zero out a bunch of coefficients in a certain block. You end up with: The original image patch, which we can run through the model. The output might be 256 values that capture what the entire patch looks like to the model. Another 256 values represent how the model understands the modified image. Now, you can take some distance measures like their euclidean distance, and call that your rate of distortion. That already works pretty well, but something even better you can do is have humans annotate “on a scale of 1 to 5, how different are these two patches?”, and train a secondary model that learns to predict given these two sets of 256 numbers, a single number fo similarity score. More to come This post is getting a bit long, so I’ll write up further details on how we optimize the image in the future. If you don’t care about the details and just want to fiddle with a live demo, you can try out the result for yourself right . here Also published . here

JPEG Sucks but Deep Learning Can Help It Improve

Astounding Stories of Super-Science August 1931: VOL. VII, NO. 2 - The Midget From the Island

Stories, News & Mentions about OPTENSITY

Optensity’s Stories on HackerNoon

Optensity's News & Mentions