It was Friday evening and I needed to make myself lunch after work. I found myself staring into my fridge wondering what to cook. I had leftovers from last night, however, I needed to make something fresh and exciting. I thought to myself, why not build a personal AI kitchen assistant to analyze the ingredients I have and suggests delicious dishes tailored precisely to my pantry? This is where Large Language Models (LLMs) stepped into the kitchen with me. LLMs are a type of artificial intelligence that have been trained on a massive dataset of text (and code), including a vast trove of recipes; meaning that they can do so much more than just finding a suitable recipe. They can become your culinary genius on demand. literally So, how does it all come together? In this article, I'll detail how I: Developed an LLM app utilizing GPT-3.5 turbo LLM to assess available ingredients and recommend recipes Deploy the LLM app on for production, enabling integration into a user interface for seamless communication via my mobile device Agenta Developing The LLM App I’ll be using OpenAI gpt-3.5-turbo in this application, however, I’d also be including various directives of the turbo model and also gpt-4. Use whichever model you think is appropriate for the task. Additionally, I will make use of to deploy my LLM application, as it is my preferred open-source platform for creating robust AI applications with LLMs. Agenta provides the necessary tools for managing and evaluating prompts. Agenta Here are the capabilities of Agenta: Quickly experiment with and compare prompts in any LLM workflow (chain-of-prompts, Retrieval Augmented Generation (RAG), LLM agents...) Swiftly generate test sets and golden datasets for evaluation Evaluate your application using pre-existing or custom evaluators Annotate and A/B test your applications with human feedback Collaborate with product teams on prompt engineering and evaluation Deploy your application with a single click through the UI, CLI, or github workflows By using Agenta, I am able to concentrate on constructing my LLM app and streamline the entire development cycle of the application, thereby enhancing the speed of experimentation. I’d be using Python to build the kitchen assistant. Creating LLM app directory and dependency file Create a directory named and a file named and add the following packages inside the file: and kitchen_assistant requirements.txt agenta openai If you have something like the image attached below, you’re good to go. Environment variable & LLM app development Now that you have the assistant folder and have included the packages inside the file, proceed to create a new file named and include your OPENAI secret key to make the LLM app work. requirements.txt .env Ensure that the file is inside the folder. You should have something like this: .env kitchen_assistant OPENAI_API_KEY=sk-vxxxxxxxxxxxxx # replace with your secret key Next would be to create a file named , and copy the following code into it. I’ll explain what the code is doing in a minute. app.py # Stdlib Imports from typing import Dict, Any # Third Party Imports import agenta as ag # type: ignore from openai import AsyncOpenAI SYSTEM_PROMPT = "You are my AI kitchen assistant. Help me find creative recipes based on my ingredients, provide step-by-step cooking instructions, and advise me on food safety practices." CHAT_LLM_GPT = [ "gpt-3.5-turbo-16k", "gpt-3.5-turbo-0301", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613", "gpt-4", ] # Initialize agenta sdk and update the configuration parameters ag.init() ag.config.default( temperature=ag.FloatParam(0.2), max_tokens=ag.IntParam(-1, -1, 4000), prompt_system=ag.TextParam(SYSTEM_PROMPT), model=ag.MultipleChoiceParam("gpt-3.5-turbo", CHAT_LLM_GPT), ) # Initialize async openai sdk client = AsyncOpenAI() @ag.entrypoint async def suggest(inputs: ag.MessagesInput = ag.MessagesInput()) -> Dict[str, Any]: """ Suggest delicious dish tailored precisely based on the ingredients (inputs). Args: inputs (ag.MessagesInput): messages input for chat completion Returns: dict: The llm response (along with additional information). """ messages = [{"role": "system", "content": ag.config.prompt_system}] + inputs max_tokens = ag.config.max_tokens if ag.config.max_tokens != -1 else None chat_completion = await client.chat.completions.create( model=ag.config.model, messages=messages, temperature=ag.config.temperature, max_tokens=max_tokens, ) token_usage = chat_completion.usage.dict() # type: ignore return { "message": chat_completion.choices[0].message.content, **{"usage": token_usage}, "cost": ag.calculate_token_usage(ag.config.model, token_usage), } The imports are essentially importing the required modules that are needed to ensure the code works correctly. System prompt sets the context for every conversation that happens between the user (you and I) and the LLM application. You’d be pissed at your friend who in the middle of a finance conversation with you starts talking about something unrelated, right? Imagine a Large Language Model (LLM) as a super-powered reader. To really grasp what it's reading, it needs context – the words and sentences surrounding whatever it's focused on. This 'context window' is like a spotlight: the bigger the window, the more information the LLM can use to understand and generate meaningful responses. Next to the system prompt is a list of defined models that can be used when testing the user prompt for the LLM app in Agenta. You get to choose whichever model to see how it performs. The command is used to initialize the Agenta SDK. Subsequently, is where you define crucial parameters that dictate the behavior of the kitchen AI assistant. These parameters include: ag.init() ag.config.default(...) Regulates the creativity of responses (lower temperature = more predictable, higher = more unexpected). temperature: Determines the maximum length of the AI's response (measured in tokens, which roughly correspond to words). max_tokens: Sets the initial instructions or context provided to the LLM, specifying it as your kitchen assistant. prompt_system: Specifies the LLM to be used (with a range of GPT model versions as options). model: The line initializes the OpenAI Async SDK, enabling your code to establish a connection and interact with OpenAI's language models. client = AsyncOpenAI() To register your LLM app with Agenta and use it in their playground, you need to define an entry point. I used the decorator on my function. This tells Agenta that this function is where the action starts. Agenta then automatically converts this function into an API endpoint (using FastAPI), making it accessible through their web interface. @ag.entrypoint suggest Publishing The LLM App Now that your AI kitchen assistant is ready, we should publish it to Agenta for experimentation. To do this, you can install the Agenta CLI in a virtual environment using your preferred Python package installer or globally on your system. However, I advise against the latter to avoid potential conflicts with your system packages. Once the installation is complete, make sure to activate the virtual environment. I will proceed with publishing the LLM app to Agenta Cloud, enabling me to deploy the LLM application to a production environment and later, create a UI application for mobile use. If you wish to publish to the cloud version of Agenta as well, I suggest creating an first and then returning to follow the steps below. account If you prefer not to deploy the app to the cloud version of agenta, you can the open-source version on your local machine by following these steps. It will be a slightly different process, but not overly complex. run Create a new project for the LLM app To publish the application in Agenta, we must first create an empty project. Execute the following command in the directory where your application code is located: (.venv) [kitchen_assistant]$ agenta init This will prompt you to enter the project name, Agenta host, API key (if using the cloud or enterprise version), organization, and how you want to initialize the app. Once the app initialization is successfully done, a blank project will be generated in Agenta along with a config.toml file in the same directory, containing all project information. The config.toml file and the empty project in Agenta Cloud will look like the following. Publishing/Serving the first app variant With the project created in Agenta cloud, we need to add the first app variant to it. This can be done by running the following command: agenta variant serve app.py This will create a new app variant in Agenta under the name . Here, is the name of the codebase containing the LLM app logic, while is a default configuration created for that codebase. Each new app variant created from the web interface or from the CLI will always have the name format . app.default app default <codebase_name>.<configuration_name> An is a distinct version of an LLM app, featuring different parameters or code but keeping the same inputs and outputs. This feature enables you to experiment with alternative methods within a single app. For instance, you could have two variants of the kitchen assistant app, each using a different system prompt. app variant Executing the serve command above will create a container for the application featuring a REST API endpoint. This endpoint will be used in Agenta when interacting with the application. The CLI will also display the URL of the endpoint, which can be used to navigate to the application API documentation or playground in Agenta. Copy the playground URI and paste it into your browser. After doing so, you will see the following user interface. It may take 10-20 seconds to load as it sets up the playground for you. You have successfully published your LLM app to Agenta and would need to experiment with it to ensure that it functions as intended. In the chat section right after “User”, write the ingredients and click on the “Run” button. You’ll see that the AI kitchen assistant is considering what you can make with the ingredients: When it’s done processing the ingredients that you inputted, you’ll get suggestion on what you can make. And so, you own a Kitchen AI assistant that suggests recipes based on your ingredients. Did you observe something after each use? You view the total tokens used, the run's cost, and the time taken! Don't hold back now, keep experimenting with the LLM app. Customize the system prompt to match a particular culture that you want, adjust the temperature, and switch the model if needed. Deploying the LLM App To An Environment After using the playground to find a good configuration for your AI Kitchen assistant, it’s time to deploy the application. By deploying the application, you can integrate it to a UI (web and/or mobile) application or anywhere since the deployed app will be an endpoint. You can also later change the configuration from the playground without having to update the code. Clicking on the “Publish” button will bring up a modal with environments you want to deploy the LLM application to. Agenta provides a way to deploy an application to multiple environments: development, staging, and production. Each environment having its own unique configuration. Since the LLM app works and it’s the weekend, we can deploy to production :-). Click on the "production" checkbox and then the "Publish" button. You will receive a success notification informing you that the LLM app variant has been deployed. Look at the side navbar, you’re going to see different sections. I’ll keep what each of them do: The playground is a user-friendly environment in Agenta where you can create and test new app variants. Here, you can adjust parameters, input data, and observe the outputs from your app. Playground: The test set is another user-friendly environment where you can create high-quality datasets. A golden test set, also known as a ground truth test set, consists of various inputs and their expected correct answers. Test Sets: Another environment that enables you to generate automated assessments using various evaluators: , , , , , , and . The purpose of conducting evaluations is to methodically evaluate the outcomes of the LLM application and compare various versions to determine the most optimal one. Another objective is to set a benchmark for the application and evaluate any potential risks. Evaluations: exact match similarity match regex test JSON field match AI critique code evaluation webhook test At times, you might require assessing your models' performance through human judgement. This is where the Human Evaluation environment becomes valuable. It enables you to carry out and to gauge your models' performance using human judgement. Annotations: A/B tests single model tests Endpoints are where you can access the deployed application code. You can view the code only after configuring your application successfully and deploying it to a specific environment. Currently, Agenta supports just three languages to integrate the deployed LLM app: Python, cURL, and TypeScript. If you’d like to add more languages or a specific language, feel free to contribute by raising an and tackling it. ;-) issue To ensure the deployed LLM app works in production, choose cURL in the languages drop-down, click "Copy," update the content value with the ingredients you have, and run it in your terminal. You should receive a result similar to the one below. Conclusion How does it feel to have a Personal Kitchen AI assistant? My partner feels like she has a sous chef that handles recipe making for her. Quite amusing, isn't it? XD In this article, you learnt what LLMs are, how to create a kitchen assistant app using the GPT-3.5-turbo model to analyze your available ingredients and offer recipe suggestions. You also learned how to deploy the LLM app on Agenta for practical use. As an extra tip, I will be a user interface using Svelte and deploy it on Github Pages. I will design it to allow you to input the URI of your LLM app, enabling personal interaction. In the upcoming article, I will demonstrate how to generate test sets for conducting human evaluations to ensure that the LLM app's output aligns with your expectations. building
