These 13 Hidden Open-Source Libraries Will Help You Become an AI Wizard 🧙‍♂️🪄

I have been building AI applications for the past 4 years and contributing to major AI tooling platforms for a while now. Over this period, I have used many tools and frameworks for building; AI agents that actually work in the real world. Tools for AI agents. End-to-end RAG applications. I have curated a coveted list of open-source tools and frameworks that will help you craft robust and reliable AI applications. 🔥 Feel free to explore their GitHub repositories, contribute to your favorites, and support them by starring the repositories. 1. Composio 👑 - Build Reliable Agents 10x Faster I have tried building many agents, and honestly, while it is easy to create them, it is an entirely different ball game to get them right. Building efficient AI agents that actually work requires efficient toolsets. This is where Composio comes into the picture. Composio lets you augment your AI agents with robust tools and integrations to accomplish AI workflows. They provide native support for Python and Javascript. Python Get started with the following pip command. pip install composio-core Add a GitHub integration. composio add github Composio handles user authentication and authorization on your behalf. Here is how you can use the GitHub integration to start a repository. from openai import OpenAI from composio_openai import ComposioToolSet, App openai_client = OpenAI(api_key="******OPENAIKEY******") # Initialise the Composio Tool Set composio_toolset = ComposioToolSet(api_key="**\\*\\***COMPOSIO_API_KEY**\\*\\***") ## Step 4 # Get GitHub tools that are pre-configured actions = composio_toolset.get_actions(actions=[Action.GITHUB_ACTIVITY_STAR_REPO_FOR_AUTHENTICATED_USER]) ## Step 5 my_task = "Star a repo ComposioHQ/composio on GitHub" # Create a chat completion request to decide on the action response = openai_client.chat.completions.create( model="gpt-4-turbo", tools=actions, # Passing actions we fetched earlier. messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": my_task} ] ) Run this Python script to execute the given instruction using the agent. Javascript You can Install it using npm, yarn, or pnpm. npm install composio-core Define a method to let the user connect their GitHub account. import { OpenAI } from "openai"; import { OpenAIToolSet } from "composio-core"; const toolset = new OpenAIToolSet({ apiKey: process.env.COMPOSIO_API_KEY, }); async function setupUserConnectionIfNotExists(entityId) { const entity = await toolset.client.getEntity(entityId); const connection = await entity.getConnection('github'); if (!connection) { // If this entity/user hasn't already connected, the account const connection = await entity.initiateConnection(appName); console.log("Log in via: ", connection.redirectUrl); return connection.waitUntilActive(60); } return connection; } Add the required tools to the OpenAI SDK, and pass the entity name on to the executeAgent function. async function executeAgent(entityName) { const entity = await toolset.client.getEntity(entityName) await setupUserConnectionIfNotExists(entity.id); const tools = await toolset.get_actions({ actions: ["github_activity_star_repo_for_authenticated_user"] }, entity.id); const instruction = "Star a repo ComposioHQ/composio on GitHub" const client = new OpenAI({ apiKey: process.env.OPEN_AI_API_KEY }) const response = await client.chat.completions.create({ model: "gpt-4-turbo", messages: [{ role: "user", content: instruction, }], tools: tools, tool_choice: "auto", }) console.log(response.choices[0].message.tool_calls); await toolset.handle_tool_call(response, entity.id); } executeGithubAgent("joey") Execute the code, and let the agent do the work for you. Composio works with famous frameworks like LangChain, LlamaIndex, CrewAi, etc. For more information, visit the official docs, and also, for even complex examples, visit the example sections of the repository. Star the Composio.dev repository ⭐ 2. Julep - Framework for Building Stateful Agents Developing AI applications, especially those requiring long-term memory, presents significant challenges. Julep is solving for this problem. It is an open-source framework for building production-ready stateful AI agents. They provide a built-in state management system that helps in efficient context storage and retrieval. Context storage helps maintain conversation continuity, ensuring that interactions with the AI remain coherent and contextually relevant over time. Get started with the following pip command. pip install julep Here is how it works. from julep import Client from pprint import pprint import textwrap import os base_url = os.environ.get("JULEP_API_URL") api_key = os.environ.get("JULEP_API_KEY") client = Client(api_key=api_key, base_url=base_url) #create agent agent = client.agents.create( name="Jessica" model="gpt-4", tools=[] # Tools defined here ) #create a user user = client.users.create( name="Anon", about="Average nerdy tech bro/girl spending 8 hours a day on a laptop, ) #create a session situation_prompt = """You are Jessica. You're a stuck-up Cali teenager. You basically complain about everything. You live in Bel-Air, Los Angeles and drag yourself to Curtis High School when necessary. """ session = client.sessions.create( user_id=user.id, agent_id=agent.id, situation=situation_prompt ) #start a conversation user_msg = "hey. what do u think of Starbucks?" response = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": user_msg, "name": "Anon", } ], recall=True, remember=True, ) print("\n".join(textwrap.wrap(response.response[0][0].content, width=100))) They also support Javascript. Check out their documentation for more. Star the Julep repository ⭐ 3. E2B - Code Interpreting for AI Apps If I am building an AI app with code execution capabilities, such as an AI tutor or AI data analyst, E2B's Code Interpreter will be my go-to tool. E2B Sandbox is a secure cloud environment for AI agents and apps. It allows AI to run safely for long periods, using the same tools as humans, such as GitHub repositories and cloud browsers. They offer native Code Interpreter SDKs for Python and Javascript/Typescript. The Code Interpreter SDK allows you to run AI-generated code in a secure small VM - E2B sandbox - for AI code execution. Inside the sandbox is a Jupyter server you can control from their SDK. Get started with E2B with the following command. npm i @e2b/code-interpreter Execute a program. import { CodeInterpreter } from '@e2b/code-interpreter' const sandbox = await CodeInterpreter.create() await sandbox.notebook.execCell('x = 1') const execution = await sandbox.notebook.execCell('x+=1; x') console.log(execution.text) // outputs 2 await sandbox.close() For more on how to work with E2B, visit their official documentation. Star the E2B repository ⭐ 4. Camel-ai - Build Communicative AI Systems Solving for scalable multi-agent collaborative systems can unlock many potential in building AI applications. Camel is well-positioned for this. It is an open-source framework offering a scalable approach to studying multi-agent systems' cooperative behaviors and capabilities. If you intend to build a multi-agent system, Camel can be one of the best choices available in the open-source scene. Get started by installing it with pip. pip install camel-ai Here is how to use Camel. from camel.messages import BaseMessage as bm from camel.agents import ChatAgent sys_msg = bm.make_assistant_message( role_name='stone', content='you are a curious stone wondering about the universe.') #define agent agent = ChatAgent( system_message=sys_msg, message_window_size=10, # [Optional] the length of chat memory ) # Define a user message usr_msg = bm.make_user_message( role_name='prof. Claude Shannon', content='what is information in your mind?') # Sending the message to the agent response = agent.step(usr_msg) # Check the response (just for illustrative purposes) print(response.msgs[0].content) Voila, you have your first AI agent. For more information, refer to their official documentation. Star the camel-ai repository ⭐ 5. CopilotKit - Build AI Copilots for React Apps Look no further if you want to include AI capabilities in your existing React application. The CopilotKit lets you use GPT models to automate interaction with your application's front and back end. It is a ready-made Copilot that you can integrate with your application or any code you can access (OSS). It offers React components like text areas, popups, sidebars, and chatbots to augment any application with AI capabilities. Get started with CopilotKit using the following command. npm i @copilotkit/react-core @copilotkit/react-ui A CopilotKit must wrap all components interacting with CopilotKit. You should also start with CopilotSidebar (swap to a different UI provider later). "use client"; import { CopilotKit } from "@copilotkit/react-core"; import { CopilotSidebar } from "@copilotkit/react-ui"; import "@copilotkit/react-ui/styles.css"; export default function RootLayout({children}) { return ( {children} ); } You can check their documentation for more information. Star the CopilotKit repository ⭐ 6. Aider - The AI Pair-programmer Imagine having a pair-programmer who’s always helpful and never annoying. Well, now you do! Aider is an AI-powered pair programmer that can start a project, edit files, or work with an existing Git repository and more from the terminal. It works with leading LLMs like GPT4o, Sonnet 3.5, DeepSeek Coder, Llama 70b, etc. You can get started quickly like this: pip install aider-chat # Change directory into a git repo cd /to/your/git/repo # Work with Claude 3.5 Sonnet on your repo export ANTHROPIC_API_KEY=your-key-goes-here aider # Work with GPT-4o on your repo export OPENAI_API_KEY=your-key-goes-here aider For more details, see the installation instructions and other documentation. Star the Aider repository ⭐ 7. Haystack - Build Composable RAG Pipelines There are plenty of frameworks for building AI pipelines, but if I want to integrate production-ready end-to-end search pipelines into my application, Haystack is my go-to. Whether it's RAG, Q&A, or semantic searches, Haystack's highly composable pipelines make development, maintenance, and deployment a breeze. Their clean and modular approach is what sets them apart. Haystack lets you effortlessly integrate rankers, vector stores, and parsers into new or existing pipelines, making it easy to turn your prototypes into production-ready solutions. Haystack is a Python-only framework; you can install it using. pip. pip install haystack-ai Now, build your first RAG Pipeline with Haystack components. import os from haystack import Pipeline, PredefinedPipeline import urllib.request os.environ["OPENAI_API_KEY"] = "Your OpenAI API Key" urllib.request.urlretrieve("https://www.gutenberg.org/cache/epub/7785/pg7785.txt", "davinci.txt") indexing_pipeline = Pipeline.from_template(PredefinedPipeline.INDEXING) indexing_pipeline.run(data={"sources": ["davinci.txt"]}) rag_pipeline = Pipeline.from_template(PredefinedPipeline.RAG) query = "How old was he when he died?" result = rag_pipeline.run(data={"prompt_builder": {"query":query}, "text_embedder": {"text": query}}) print(result["llm"]["replies"][0]) For more tutorials and concepts, check out their documentation. Star the Haystack repository ⭐ 8. Pgvectorscale - Fastest Vector Database Modern RAG applications are incomplete without vector databases. These store documents (texts, images) as embeddings, enabling users to search for semantically similar documents. Pgvectorscale is an extension of PgVector, a vector database from PostgreSQL. It can seamlessly integrate with existing Postgres databases. If you are building an application with vector stores, this is a no-brainer. Pgvectorscale has outperformed Pinecone's storage-optimized index (s1). And it costs 75% less. You can install it from the source, use a package manager like Yum, Homebrew, apt, etc., or use a Docker container. To get started with it, compile and install. # install prerequisites ## rust curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ## pgrx cargo install --locked cargo-pgrx cargo pgrx init --pg16 pg_config #download, build and install pgvectorscale cd /tmp git clone --branch https://github.com/timescale/pgvectorscale cd pgvectorscale/pgvectorscale cargo pgrx install --release Connect to your database: psql -d "postgres:// : @ : / " Create the pgvectorscale extension: CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE; The CASCADE automatically installs pgvector. Create a table with an embedding column. For example: CREATE TABLE IF NOT EXISTS document_embedding ( id BIGINT PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, metadata JSONB, contents TEXT, embedding VECTOR(1536) ) For more information on how to use this, check out the repository. Star the Pgvectorscale repository ⭐ 9. GPTCache - Semantic Caching for AI Apps LLMs are expensive. If you are building an app that requires more extended conversations with chat models and do not want to max out credit cards, you need caching. However, traditional caching is of no use here. This is where GPTCache comes into the picture. It is a semantic caching tool from Zilliz, the parent organization of the Milvus vector store. It lets you store conversations in your preferred vector stores. Before sending a query to the LLM, it searches the vector store; if there is a hit, it fetches it. Otherwise, it routes the request to the model. For more information, visit the official documentation page. Star the GPTCache repository ⭐ 10. Mem0 (EmbedChain) - Build Personalized LLM Apps Mem0 provides a smart, self-improving memory layer for Large Language Models. It lets you add persistent memory for users, agents, and sessions. Consider Mem0 if you are building a chatbot or Q&A system on custom data. Get started with Mem0 using. pip. pip install mem0ai Here is how to use Mem0 to add a memory layer to Large Language Models. from mem0 import Memory # Initialize Mem0 m = Memory() # Store a memory from any unstructured text result = m.add("I am working on improving my tennis skills. Suggest some online courses.", user_id="Alice", metadata={"category": "hobbies"}) print(result) # Created memory: Improving her tennis skills. Looking for online suggestions. # Retrieve memories all_memories = m.get_all() print(all_memories) # Search memories related_memories = m.search(query="What are Alice's hobbies?", user_id="alice") print(related_memories) # Update a memory result = m.update(memory_id="m1", data="Likes to play tennis on weekends") print(result) # Get memory history history = m.history(memory_id="m1") print(history) Refer to the official documentation for more. Star the Mem0 ( Embedchain) repository ⭐ 11. FastEmbed - Embed Documents Faster Speed of execution is paramount in software development, and it is even more important when building an AI application. Usually, embedding generation can take a long time, slowing down the entire pipeline. However, this should not be the case. FastEmbed from Qdrant is a fast, lightweight Python library built for embedding generation. It uses ONNX runtime instead of Pytorch, making it faster. It also supports most of the state-of-the-art open-source embedding models. To get started with FastEmbed, install it using pip. pip install fastembed # or with GPU support pip install fastembed-gpu Here is how you can create the embeddings of the documents. from fastembed import TextEmbedding from typing import List # Example list of documents documents: List[str] = [ "This is built to be faster and lighter than other embedding libraries, e.g. Transformers, Sentence-Transformers, etc.", "FastEmbed is supported by and maintained by Quadrant." ] # This will trigger the model download and initialization embedding_model = TextEmbedding() print("The model BAAI/bge-small-en-v1.5 is ready to use.") embeddings_generator = embedding_model.embed(documents) # reminder this is a generator embeddings_list = list(embedding_model.embed(documents)) # You can also convert the generator to a list, and that to a Numpy array len(embeddings_list[0]) # Vector of 384 dimensions Check out their repository for more information. Star the FastEmbed repository ⭐ 12. Instructor - Structured Data Extraction From LLMs If you have played with LLM outputs, you know it can be challenging to validate structured responses. Instructor is an open-source tool that streamlines the validation, retry, and streaming of LLM outputs. It uses Pydantic for Python and Zod for JS/TS for data validation and supports various model providers beyond openAI. Get started with the Instructor using the following command. npm i @instructor-ai/instructor zod openai Now, here is how you can extract structured data from LLM responses. import Instructor from "@instructor-ai/instructor"; import OpenAI from "openai" import { z } from "zod" const oai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY ?? undefined, organization: process.env.OPENAI_ORG_ID ?? undefined }) const client = Instructor({ client: oai, mode: "TOOLS" }) const UserSchema = z.object({ // Description will be used in the prompt age: z.number().describe("The age of the user"), name: z.string() }) // User will be of type z.infer const user = await client.chat.completions.create({ messages: [{ role: "user", content: "Jason Liu is 30 years old" }], model: "gpt-3.5-turbo", response_model: { schema: UserSchema, name: "User" } }) console.log(user) // { age: 30, name: "Jason Liu" } For more information, visit the official documentation page. Star the Instructor ⭐ 13. LiteLLM - Drop-in Replacement for LLMs in OpenAI Format Let's be honest; we all have screamed at some point because a new model provider does not follow the OpenAI SDK format for text, image, or embedding generation. However, with LiteLLM, using the same implementation format, you can use any model provider (Claude, Gemini, Groq, Mistral, Azure AI, Bedrock, etc.) as a drop-in replacement for OpenAI models. They also support load-balancing, fallbacks, and spend tracking across 100+ LLMs. Install LiteLLM using pip. pip install litellm Here is how you can use the Claude-2 model as a drop-in replacement for GPT models. from litellm import completion import os # LiteLLM with OpenAI Models os.environ["OPENAI_API_KEY"] = "your-API-key" response = completion( model="gpt-3.5-turbo", messages=[{ "content": "Hello, how are you?","role": "user"}] ) # LiteLLM with Claude Models os.environ["ANTHROPIC_API_KEY"] = "your-API-key" response = completion( model="claude-2", messages=[{ "content": "Hello, how are you?","role": "user"}] ) For more, refer to their official documentation. Star the LiteLLM ⭐ Do you use or have built some other cool tool or framework? Let me know about them in the comments :) I have been building AI applications for the past 4 years and contributing to major AI tooling platforms for a while now. Over this period, I have used many tools and frameworks for building; AI agents that actually work in the real world. Tools for AI agents. End-to-end RAG applications. AI agents that actually work in the real world. Tools for AI agents. End-to-end RAG applications. I have curated a coveted list of open-source tools and frameworks that will help you craft robust and reliable AI applications. 🔥 Feel free to explore their GitHub repositories, contribute to your favorites, and support them by starring the repositories. 1. Composio 👑 - Build Reliable Agents 10x Faster Composio I have tried building many agents, and honestly, while it is easy to create them, it is an entirely different ball game to get them right. Building efficient AI agents that actually work requires efficient toolsets. This is where Composio comes into the picture. Composio lets you augment your AI agents with robust tools and integrations to accomplish AI workflows. They provide native support for Python and Javascript. Python Python Get started with the following pip command. pip pip install composio-core pip install composio-core Add a GitHub integration. composio add github composio add github Composio handles user authentication and authorization on your behalf. Here is how you can use the GitHub integration to start a repository. from openai import OpenAI from composio_openai import ComposioToolSet, App openai_client = OpenAI(api_key="******OPENAIKEY******") # Initialise the Composio Tool Set composio_toolset = ComposioToolSet(api_key="**\\*\\***COMPOSIO_API_KEY**\\*\\***") ## Step 4 # Get GitHub tools that are pre-configured actions = composio_toolset.get_actions(actions=[Action.GITHUB_ACTIVITY_STAR_REPO_FOR_AUTHENTICATED_USER]) ## Step 5 my_task = "Star a repo ComposioHQ/composio on GitHub" # Create a chat completion request to decide on the action response = openai_client.chat.completions.create( model="gpt-4-turbo", tools=actions, # Passing actions we fetched earlier. messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": my_task} ] ) from openai import OpenAI from composio_openai import ComposioToolSet, App openai_client = OpenAI(api_key="******OPENAIKEY******") # Initialise the Composio Tool Set composio_toolset = ComposioToolSet(api_key="**\\*\\***COMPOSIO_API_KEY**\\*\\***") ## Step 4 # Get GitHub tools that are pre-configured actions = composio_toolset.get_actions(actions=[Action.GITHUB_ACTIVITY_STAR_REPO_FOR_AUTHENTICATED_USER]) ## Step 5 my_task = "Star a repo ComposioHQ/composio on GitHub" # Create a chat completion request to decide on the action response = openai_client.chat.completions.create( model="gpt-4-turbo", tools=actions, # Passing actions we fetched earlier. messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": my_task} ] ) Run this Python script to execute the given instruction using the agent. Javascript You can Install it using npm , yarn , or pnpm . npm yarn pnpm npm install composio-core npm install composio-core Define a method to let the user connect their GitHub account. import { OpenAI } from "openai"; import { OpenAIToolSet } from "composio-core"; const toolset = new OpenAIToolSet({ apiKey: process.env.COMPOSIO_API_KEY, }); async function setupUserConnectionIfNotExists(entityId) { const entity = await toolset.client.getEntity(entityId); const connection = await entity.getConnection('github'); if (!connection) { // If this entity/user hasn't already connected, the account const connection = await entity.initiateConnection(appName); console.log("Log in via: ", connection.redirectUrl); return connection.waitUntilActive(60); } return connection; } import { OpenAI } from "openai"; import { OpenAIToolSet } from "composio-core"; const toolset = new OpenAIToolSet({ apiKey: process.env.COMPOSIO_API_KEY, }); async function setupUserConnectionIfNotExists(entityId) { const entity = await toolset.client.getEntity(entityId); const connection = await entity.getConnection('github'); if (!connection) { // If this entity/user hasn't already connected, the account const connection = await entity.initiateConnection(appName); console.log("Log in via: ", connection.redirectUrl); return connection.waitUntilActive(60); } return connection; } Add the required tools to the OpenAI SDK, and pass the entity name on to the executeAgent function. executeAgent async function executeAgent(entityName) { const entity = await toolset.client.getEntity(entityName) await setupUserConnectionIfNotExists(entity.id); const tools = await toolset.get_actions({ actions: ["github_activity_star_repo_for_authenticated_user"] }, entity.id); const instruction = "Star a repo ComposioHQ/composio on GitHub" const client = new OpenAI({ apiKey: process.env.OPEN_AI_API_KEY }) const response = await client.chat.completions.create({ model: "gpt-4-turbo", messages: [{ role: "user", content: instruction, }], tools: tools, tool_choice: "auto", }) console.log(response.choices[0].message.tool_calls); await toolset.handle_tool_call(response, entity.id); } executeGithubAgent("joey") async function executeAgent(entityName) { const entity = await toolset.client.getEntity(entityName) await setupUserConnectionIfNotExists(entity.id); const tools = await toolset.get_actions({ actions: ["github_activity_star_repo_for_authenticated_user"] }, entity.id); const instruction = "Star a repo ComposioHQ/composio on GitHub" const client = new OpenAI({ apiKey: process.env.OPEN_AI_API_KEY }) const response = await client.chat.completions.create({ model: "gpt-4-turbo", messages: [{ role: "user", content: instruction, }], tools: tools, tool_choice: "auto", }) console.log(response.choices[0].message.tool_calls); await toolset.handle_tool_call(response, entity.id); } executeGithubAgent("joey") Execute the code, and let the agent do the work for you. Composio works with famous frameworks like LangChain, LlamaIndex, CrewAi, etc. For more information, visit the official docs , and also, for even complex examples, visit the example sections of the repository. docs example Star the Composio.dev repository ⭐ Star the Composio.dev repository ⭐ 2. Julep - Framework for Building Stateful Agents Developing AI applications, especially those requiring long-term memory, presents significant challenges. Julep is solving for this problem. It is an open-source framework for building production-ready stateful AI agents. They provide a built-in state management system that helps in efficient context storage and retrieval. Context storage helps maintain conversation continuity, ensuring that interactions with the AI remain coherent and contextually relevant over time. Get started with the following pip command. pip pip install julep pip install julep Here is how it works. from julep import Client from pprint import pprint import textwrap import os base_url = os.environ.get("JULEP_API_URL") api_key = os.environ.get("JULEP_API_KEY") client = Client(api_key=api_key, base_url=base_url) #create agent agent = client.agents.create( name="Jessica" model="gpt-4", tools=[] # Tools defined here ) #create a user user = client.users.create( name="Anon", about="Average nerdy tech bro/girl spending 8 hours a day on a laptop, ) #create a session situation_prompt = """You are Jessica. You're a stuck-up Cali teenager. You basically complain about everything. You live in Bel-Air, Los Angeles and drag yourself to Curtis High School when necessary. """ session = client.sessions.create( user_id=user.id, agent_id=agent.id, situation=situation_prompt ) #start a conversation user_msg = "hey. what do u think of Starbucks?" response = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": user_msg, "name": "Anon", } ], recall=True, remember=True, ) print("\n".join(textwrap.wrap(response.response[0][0].content, width=100))) from julep import Client from pprint import pprint import textwrap import os base_url = os.environ.get("JULEP_API_URL") api_key = os.environ.get("JULEP_API_KEY") client = Client(api_key=api_key, base_url=base_url) #create agent agent = client.agents.create( name="Jessica" model="gpt-4", tools=[] # Tools defined here ) #create a user user = client.users.create( name="Anon", about="Average nerdy tech bro/girl spending 8 hours a day on a laptop, ) #create a session situation_prompt = """You are Jessica. You're a stuck-up Cali teenager. You basically complain about everything. You live in Bel-Air, Los Angeles and drag yourself to Curtis High School when necessary. """ session = client.sessions.create( user_id=user.id, agent_id=agent.id, situation=situation_prompt ) #start a conversation user_msg = "hey. what do u think of Starbucks?" response = client.sessions.chat( session_id=session.id, messages=[ { "role": "user", "content": user_msg, "name": "Anon", } ], recall=True, remember=True, ) print("\n".join(textwrap.wrap(response.response[0][0].content, width=100))) They also support Javascript. Check out their documentation for more. documentation Star the Julep repository ⭐ Star the Julep repository ⭐ 3. E2B - Code Interpreting for AI Apps If I am building an AI app with code execution capabilities, such as an AI tutor or AI data analyst, E2B's Code Interpreter will be my go-to tool. E2B Sandbox is a secure cloud environment for AI agents and apps. It allows AI to run safely for long periods, using the same tools as humans, such as GitHub repositories and cloud browsers. They offer native Code Interpreter SDKs for Python and Javascript/Typescript. The Code Interpreter SDK allows you to run AI-generated code in a secure small VM - E2B sandbox - for AI code execution. Inside the sandbox is a Jupyter server you can control from their SDK. E2B sandbox Get started with E2B with the following command. npm i @e2b/code-interpreter npm i @e2b/code-interpreter Execute a program. import { CodeInterpreter } from '@e2b/code-interpreter' const sandbox = await CodeInterpreter.create() await sandbox.notebook.execCell('x = 1') const execution = await sandbox.notebook.execCell('x+=1; x') console.log(execution.text) // outputs 2 await sandbox.close() import { CodeInterpreter } from '@e2b/code-interpreter' const sandbox = await CodeInterpreter.create() await sandbox.notebook.execCell('x = 1') const execution = await sandbox.notebook.execCell('x+=1; x') console.log(execution.text) // outputs 2 await sandbox.close() For more on how to work with E2B, visit their official documentation . documentation Star the E2B repository ⭐ Star the E2B repository ⭐ 4. Camel-ai - Build Communicative AI Systems Solving for scalable multi-agent collaborative systems can unlock many potential in building AI applications. Camel is well-positioned for this. It is an open-source framework offering a scalable approach to studying multi-agent systems' cooperative behaviors and capabilities. If you intend to build a multi-agent system, Camel can be one of the best choices available in the open-source scene. Get started by installing it with pip . pip pip install camel-ai pip install camel-ai Here is how to use Camel. from camel.messages import BaseMessage as bm from camel.agents import ChatAgent sys_msg = bm.make_assistant_message( role_name='stone', content='you are a curious stone wondering about the universe.') #define agent agent = ChatAgent( system_message=sys_msg, message_window_size=10, # [Optional] the length of chat memory ) # Define a user message usr_msg = bm.make_user_message( role_name='prof. Claude Shannon', content='what is information in your mind?') # Sending the message to the agent response = agent.step(usr_msg) # Check the response (just for illustrative purposes) print(response.msgs[0].content) from camel.messages import BaseMessage as bm from camel.agents import ChatAgent sys_msg = bm.make_assistant_message( role_name='stone', content='you are a curious stone wondering about the universe.') #define agent agent = ChatAgent( system_message=sys_msg, message_window_size=10, # [Optional] the length of chat memory ) # Define a user message usr_msg = bm.make_user_message( role_name='prof. Claude Shannon', content='what is information in your mind?') # Sending the message to the agent response = agent.step(usr_msg) # Check the response (just for illustrative purposes) print(response.msgs[0].content) Voila, you have your first AI agent. For more information, refer to their official documentation . documentation Star the camel-ai repository ⭐ Star the camel-ai repository ⭐ 5. CopilotKit - Build AI Copilots for React Apps Look no further if you want to include AI capabilities in your existing React application. The CopilotKit lets you use GPT models to automate interaction with your application's front and back end. It is a ready-made Copilot that you can integrate with your application or any code you can access (OSS). It offers React components like text areas, popups, sidebars, and chatbots to augment any application with AI capabilities. Get started with CopilotKit using the following command. npm i @copilotkit/react-core @copilotkit/react-ui npm i @copilotkit/react-core @copilotkit/react-ui A CopilotKit must wrap all components interacting with CopilotKit. You should also start with CopilotSidebar (swap to a different UI provider later). CopilotKit CopilotSidebar "use client"; import { CopilotKit } from "@copilotkit/react-core"; import { CopilotSidebar } from "@copilotkit/react-ui"; import "@copilotkit/react-ui/styles.css"; export default function RootLayout({children}) { return ( {children} ); } "use client"; import { CopilotKit } from "@copilotkit/react-core"; import { CopilotSidebar } from "@copilotkit/react-ui"; import "@copilotkit/react-ui/styles.css"; export default function RootLayout({children}) { return ( {children} ); } You can check their documentation for more information. documentation Star the CopilotKit repository ⭐ Star the CopilotKit repository ⭐ 6. Aider - The AI Pair-programmer Imagine having a pair-programmer who’s always helpful and never annoying. Well, now you do! Aider is an AI-powered pair programmer that can start a project, edit files, or work with an existing Git repository and more from the terminal. It works with leading LLMs like GPT4o, Sonnet 3.5, DeepSeek Coder, Llama 70b, etc. You can get started quickly like this: pip install aider-chat # Change directory into a git repo cd /to/your/git/repo # Work with Claude 3.5 Sonnet on your repo export ANTHROPIC_API_KEY=your-key-goes-here aider # Work with GPT-4o on your repo export OPENAI_API_KEY=your-key-goes-here aider pip install aider-chat # Change directory into a git repo cd /to/your/git/repo # Work with Claude 3.5 Sonnet on your repo export ANTHROPIC_API_KEY=your-key-goes-here aider # Work with GPT-4o on your repo export OPENAI_API_KEY=your-key-goes-here aider For more details, see the installation instructions and other documentation . installation instructions documentation Star the Aider repository ⭐ Star the Aider repository ⭐ 7. Haystack - Build Composable RAG Pipelines There are plenty of frameworks for building AI pipelines, but if I want to integrate production-ready end-to-end search pipelines into my application, Haystack is my go-to. Whether it's RAG, Q&A, or semantic searches, Haystack's highly composable pipelines make development, maintenance, and deployment a breeze. Their clean and modular approach is what sets them apart. Haystack lets you effortlessly integrate rankers, vector stores, and parsers into new or existing pipelines, making it easy to turn your prototypes into production-ready solutions. Haystack is a Python-only framework; you can install it using. pip . pip pip install haystack-ai pip install haystack-ai Now, build your first RAG Pipeline with Haystack components. import os from haystack import Pipeline, PredefinedPipeline import urllib.request os.environ["OPENAI_API_KEY"] = "Your OpenAI API Key" urllib.request.urlretrieve("https://www.gutenberg.org/cache/epub/7785/pg7785.txt", "davinci.txt") indexing_pipeline = Pipeline.from_template(PredefinedPipeline.INDEXING) indexing_pipeline.run(data={"sources": ["davinci.txt"]}) rag_pipeline = Pipeline.from_template(PredefinedPipeline.RAG) query = "How old was he when he died?" result = rag_pipeline.run(data={"prompt_builder": {"query":query}, "text_embedder": {"text": query}}) print(result["llm"]["replies"][0]) import os from haystack import Pipeline, PredefinedPipeline import urllib.request os.environ["OPENAI_API_KEY"] = "Your OpenAI API Key" urllib.request.urlretrieve("https://www.gutenberg.org/cache/epub/7785/pg7785.txt", "davinci.txt") indexing_pipeline = Pipeline.from_template(PredefinedPipeline.INDEXING) indexing_pipeline.run(data={"sources": ["davinci.txt"]}) rag_pipeline = Pipeline.from_template(PredefinedPipeline.RAG) query = "How old was he when he died?" result = rag_pipeline.run(data={"prompt_builder": {"query":query}, "text_embedder": {"text": query}}) print(result["llm"]["replies"][0]) For more tutorials and concepts, check out their documentation . documentation Star the Haystack repository ⭐ Star the Haystack repository ⭐ 8. Pgvectorscale - Fastest Vector Database Modern RAG applications are incomplete without vector databases. These store documents (texts, images) as embeddings, enabling users to search for semantically similar documents. Pgvectorscale is an extension of PgVector, a vector database from PostgreSQL. It can seamlessly integrate with existing Postgres databases. If you are building an application with vector stores, this is a no-brainer. Pgvectorscale has outperformed Pinecone's storage-optimized index (s1). And it costs 75% less. You can install it from the source, use a package manager like Yum, Homebrew, apt, etc., or use a Docker container. To get started with it, compile and install. # install prerequisites ## rust curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ## pgrx cargo install --locked cargo-pgrx cargo pgrx init --pg16 pg_config #download, build and install pgvectorscale cd /tmp git clone --branch https://github.com/timescale/pgvectorscale cd pgvectorscale/pgvectorscale cargo pgrx install --release # install prerequisites ## rust curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ## pgrx cargo install --locked cargo-pgrx cargo pgrx init --pg16 pg_config #download, build and install pgvectorscale cd /tmp git clone --branch https://github.com/timescale/pgvectorscale cd pgvectorscale/pgvectorscale cargo pgrx install --release Connect to your database: psql -d "postgres:// : @ : / " psql -d "postgres:// : @ : / " Create the pgvectorscale extension: CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE; CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE; The CASCADE automatically installs pgvector . CASCADE pgvector Create a table with an embedding column. For example: CREATE TABLE IF NOT EXISTS document_embedding ( id BIGINT PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, metadata JSONB, contents TEXT, embedding VECTOR(1536) ) CREATE TABLE IF NOT EXISTS document_embedding ( id BIGINT PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, metadata JSONB, contents TEXT, embedding VECTOR(1536) ) For more information on how to use this, check out the repository . repository Star the Pgvectorscale repository ⭐ Star the Pgvectorscale repository ⭐ 9. GPTCache - Semantic Caching for AI Apps LLMs are expensive. If you are building an app that requires more extended conversations with chat models and do not want to max out credit cards, you need caching. However, traditional caching is of no use here. This is where GPTCache comes into the picture. It is a semantic caching tool from Zilliz, the parent organization of the Milvus vector store. It lets you store conversations in your preferred vector stores. Before sending a query to the LLM, it searches the vector store; if there is a hit, it fetches it. Otherwise, it routes the request to the model. For more information, visit the official documentation page. documentation Star the GPTCache repository ⭐ Star the GPTCache repository ⭐ 10. Mem0 (EmbedChain) - Build Personalized LLM Apps Mem0 provides a smart, self-improving memory layer for Large Language Models. It lets you add persistent memory for users, agents, and sessions. Consider Mem0 if you are building a chatbot or Q&A system on custom data. Get started with Mem0 using. pip . pip pip install mem0ai pip install mem0ai Here is how to use Mem0 to add a memory layer to Large Language Models. from mem0 import Memory # Initialize Mem0 m = Memory() # Store a memory from any unstructured text result = m.add("I am working on improving my tennis skills. Suggest some online courses.", user_id="Alice", metadata={"category": "hobbies"}) print(result) # Created memory: Improving her tennis skills. Looking for online suggestions. # Retrieve memories all_memories = m.get_all() print(all_memories) # Search memories related_memories = m.search(query="What are Alice's hobbies?", user_id="alice") print(related_memories) # Update a memory result = m.update(memory_id="m1", data="Likes to play tennis on weekends") print(result) # Get memory history history = m.history(memory_id="m1") print(history) from mem0 import Memory # Initialize Mem0 m = Memory() # Store a memory from any unstructured text result = m.add("I am working on improving my tennis skills. Suggest some online courses.", user_id="Alice", metadata={"category": "hobbies"}) print(result) # Created memory: Improving her tennis skills. Looking for online suggestions. # Retrieve memories all_memories = m.get_all() print(all_memories) # Search memories related_memories = m.search(query="What are Alice's hobbies?", user_id="alice") print(related_memories) # Update a memory result = m.update(memory_id="m1", data="Likes to play tennis on weekends") print(result) # Get memory history history = m.history(memory_id="m1") print(history) Refer to the official documentation for more. documentation Star the Mem0 ( Embedchain) repository ⭐ Star the Mem0 ( Embedchain) repository ⭐ 11. FastEmbed - Embed Documents Faster Speed of execution is paramount in software development, and it is even more important when building an AI application. Usually, embedding generation can take a long time, slowing down the entire pipeline. However, this should not be the case. FastEmbed from Qdrant is a fast, lightweight Python library built for embedding generation. It uses ONNX runtime instead of Pytorch, making it faster. It also supports most of the state-of-the-art open-source embedding models. To get started with FastEmbed, install it using pip . pip pip install fastembed # or with GPU support pip install fastembed-gpu pip install fastembed # or with GPU support pip install fastembed-gpu Here is how you can create the embeddings of the documents. from fastembed import TextEmbedding from typing import List # Example list of documents documents: List[str] = [ "This is built to be faster and lighter than other embedding libraries, e.g. Transformers, Sentence-Transformers, etc.", "FastEmbed is supported by and maintained by Quadrant." ] # This will trigger the model download and initialization embedding_model = TextEmbedding() print("The model BAAI/bge-small-en-v1.5 is ready to use.") embeddings_generator = embedding_model.embed(documents) # reminder this is a generator embeddings_list = list(embedding_model.embed(documents)) # You can also convert the generator to a list, and that to a Numpy array len(embeddings_list[0]) # Vector of 384 dimensions from fastembed import TextEmbedding from typing import List # Example list of documents documents: List[str] = [ "This is built to be faster and lighter than other embedding libraries, e.g. Transformers, Sentence-Transformers, etc.", "FastEmbed is supported by and maintained by Quadrant." ] # This will trigger the model download and initialization embedding_model = TextEmbedding() print("The model BAAI/bge-small-en-v1.5 is ready to use.") embeddings_generator = embedding_model.embed(documents) # reminder this is a generator embeddings_list = list(embedding_model.embed(documents)) # You can also convert the generator to a list, and that to a Numpy array len(embeddings_list[0]) # Vector of 384 dimensions Check out their repository for more information. repository Star the FastEmbed repository ⭐ Star the FastEmbed repository ⭐ 12. Instructor - Structured Data Extraction From LLMs If you have played with LLM outputs, you know it can be challenging to validate structured responses. Instructor is an open-source tool that streamlines the validation, retry, and streaming of LLM outputs. It uses Pydantic for Python and Zod for JS/TS for data validation and supports various model providers beyond openAI. Get started with the Instructor using the following command. npm i @instructor-ai/instructor zod openai npm i @instructor-ai/instructor zod openai Now, here is how you can extract structured data from LLM responses. import Instructor from "@instructor-ai/instructor"; import OpenAI from "openai" import { z } from "zod" const oai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY ?? undefined, organization: process.env.OPENAI_ORG_ID ?? undefined }) const client = Instructor({ client: oai, mode: "TOOLS" }) const UserSchema = z.object({ // Description will be used in the prompt age: z.number().describe("The age of the user"), name: z.string() }) // User will be of type z.infer const user = await client.chat.completions.create({ messages: [{ role: "user", content: "Jason Liu is 30 years old" }], model: "gpt-3.5-turbo", response_model: { schema: UserSchema, name: "User" } }) console.log(user) // { age: 30, name: "Jason Liu" } import Instructor from "@instructor-ai/instructor"; import OpenAI from "openai" import { z } from "zod" const oai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY ?? undefined, organization: process.env.OPENAI_ORG_ID ?? undefined }) const client = Instructor({ client: oai, mode: "TOOLS" }) const UserSchema = z.object({ // Description will be used in the prompt age: z.number().describe("The age of the user"), name: z.string() }) // User will be of type z.infer const user = await client.chat.completions.create({ messages: [{ role: "user", content: "Jason Liu is 30 years old" }], model: "gpt-3.5-turbo", response_model: { schema: UserSchema, name: "User" } }) console.log(user) // { age: 30, name: "Jason Liu" } For more information, visit the official documentation page. documentation Star the Instructor ⭐ Star the Instructor ⭐ 13. LiteLLM - Drop-in Replacement for LLMs in OpenAI Format Let's be honest; we all have screamed at some point because a new model provider does not follow the OpenAI SDK format for text, image, or embedding generation. However, with LiteLLM, using the same implementation format, you can use any model provider (Claude, Gemini, Groq, Mistral, Azure AI, Bedrock, etc.) as a drop-in replacement for OpenAI models. They also support load-balancing, fallbacks, and spend tracking across 100+ LLMs. Install LiteLLM using pip . pip pip install litellm pip install litellm Here is how you can use the Claude-2 model as a drop-in replacement for GPT models. from litellm import completion import os # LiteLLM with OpenAI Models os.environ["OPENAI_API_KEY"] = "your-API-key" response = completion( model="gpt-3.5-turbo", messages=[{ "content": "Hello, how are you?","role": "user"}] ) # LiteLLM with Claude Models os.environ["ANTHROPIC_API_KEY"] = "your-API-key" response = completion( model="claude-2", messages=[{ "content": "Hello, how are you?","role": "user"}] ) from litellm import completion import os # LiteLLM with OpenAI Models os.environ["OPENAI_API_KEY"] = "your-API-key" response = completion( model="gpt-3.5-turbo", messages=[{ "content": "Hello, how are you?","role": "user"}] ) # LiteLLM with Claude Models os.environ["ANTHROPIC_API_KEY"] = "your-API-key" response = completion( model="claude-2", messages=[{ "content": "Hello, how are you?","role": "user"}] ) For more, refer to their official documentation . documentation Star the LiteLLM ⭐ Star the LiteLLM ⭐ Do you use or have built some other cool tool or framework? Let me know about them in the comments :)