How to Build an AI-Search-Powered Personal Assistant App

In this article we would like to show you how to combine two technologies ( and ) and create a LangChain Anthropic search-powered personal assistant. More and more AI products are being created by participants in The next opportunity for developers is our joint hackathon with , where each participant has the chance to create their own using the latest technology from . Here we’ll talk about one application of AI that is growing more and more important: . our hackathons. Google Cloud Vertex AI AI application Google Cloud search engines What is it? A search-powered personal assistant is a digital assistant that uses to help users with tasks such as These assistants use search algorithms to gather and analyse data from various sources, presenting it to users in a useful and concise manner. search engine technology finding information, making reservations, setting reminders and sending messages. Prominent examples of search-powered personal assistants are , , and . These assistants effectively use their search capabilities to provide accurate and relevant information, complete ltasks, and improve their responses as they interact with the user. Google Assistant Siri Alexa Cortana Introducing Anthropic's Claude Anthropic is a research organization focused on developing advanced AI systems. Their latest creation, Claude, is a next-generation AI assistant designed to be helpful, honest, and harmless. This cutting-edge model ensures a high degree of reliability and predictability in various tasks. Key features of Claude include: Versatile conversational and text processing capabilities Maintaining user safety and privacy as its top priority Claude's primary use cases are: Summarization Search Creative and collaborative writing Q&A Coding assistance These features make Claude an ideal AI tool for a breadth of applications, empowering users across different domains. Introduction to LangChain LangChain is a versatile tool for building end-to-end language model applications. It provides a robust framework that simplifies the process of creating, managing, and deploying Language Learning Models (LLMs). LLMs are advanced AI models designed to understand, generate, and manipulate human-like text in various languages and tasks. Key features of LangChain include: Efficient management of prompts for LLMs The ability to create chains of tasks for complex workflows Adding state to the AI, allowing it to remember information from previous interactions These capabilities make LangChain a powerful and user-friendly platform for harnessing the potential of language models in diverse applications. Prerequisites Basic knowledge of Python Basic knowledge of Typescipt and/or React Access to Anthropic's Claude API Access to SerpApi's Web Search API Outline Initializing the Project Building the Front-End for an AI Assistant App with Claude and LangCHain Writing the Project Files Testing the AI Assistant App Discussion Initializing the Project app.py (Flask app entrypoint) 🐍 : To start, make sure you have Flask installed in your environment. You can do this using : Install Flask pip pip install Flask : Create a new directory for your project and navigate to it: Create a new directory mkdir claude-langchain cd claude-langchain : It's a good practice to use a virtual environment when working with Python projects. You can create one using or any other tool of your choice: Set up a virtual environment (optional) venv python -m venv venv source venv/bin/activate (Linux/Mac) venv\Scripts\activate (Windows) file: Create a file to write your Flask application code: Create a main.py main.py touch app.py # Linux/Mac echo.>app.py # Windows : Open the file in your favorite code editor and add the following code: Write your Flask application code main.py from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!' if __name__ == '__main__': app.run() : Save the file and run the following command in your terminal/command prompt: Run the Flask application main.py python main.py : Open your preferred web browser and navigate to . You should see "Hello, World!" displayed on the webpage. Open your browser http://127.0.0.1:5000/ And that's it! You've successfully initialized a Flask project and created a simple application. .env 🌏 : To easily manage environment variables with a file, we'll use the package. At the same time, let's also install . Install both packages using : Install python-dotenv and langchain .env python-dotenv langchain pip pip install python-dotenv langchain file: Create a file in the root directory of your project: Create a .env .env touch .env # Linux/Mac echo.>.env # Windows : Open the file in your favorite code editor and add your environment variables. Each variable should be on a new line, in the format KEY=VALUE: Add environment variables .env ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxxxxxxx SERPAPI_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Please note that you need to have both API key for model and 's web search service. Anthropic's Claude SerpAPI : Modify your file to load the environment variables from the file using . Update your as follows: Load environment variables main.py .env python-dotenv main.py import os from flask import Flask from dotenv import load_dotenv load_dotenv() app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!' if __name__ == '__main__': app.run() file in version control: It's important not to share sensitive information like secret keys in version control. If you're using Git, add the following line to your file (create one if you don't have it): Ignore the .env .gitignore .env Now, your Flask project is set up to use environment variables from the file. You can add more variables as needed and access them using . Remember to keep the file private and never commit it to version control. .env os.environ.get('KEY') .env Building the Front-End for an AI Assistant App with Claude and LangChain This tutorial is designed for intermediate users who have a basic understanding of Node.js, npm, React, and Typescript. We'll be using a stack that includes these technologies, along with Tailwind CSS for styling. This stack was chosen for its robustness, versatility, and the strong community support for these technologies. In addition, we'll be integrating Anthropic's Claude model and LangChain, two powerful AI technologies that will allow our app to generate accurate, human-like text responses. Installing Node.js and NPM Download the Node.js installer for your OS from the . official site Follow the installation prompts to install Node.js and npm. The LTS (Long Term Support) version is recommended for most users. Once installed, verify the installation by checking the versions of Node.js and npm from your terminal: node -v npm -v Setting Up the Project Environment Installing Create React App is a command-line utility that assists us in creating a new React.js application. We'll install it globally via npm: Create React App (CRA) npm install -g create-react-app Creating a New React Project with Typescript We'll use CRA with the Typescript template to create a new project named . ai-assistant-claude npx create-react-app ai-assistant-claude --template typescript This command creates a new directory called in our current directory, housing a new React application with Typescript support. ai-assistant-claude Integrating TailwindCSS Installing TailwindCSS The steps followed in this tutorial are based on the . Refer to these docs for more up-to-date instructions. official Tailwind CSS documentation We'll start by installing TailwindCSS and initializing the library in our project: npm install -D tailwindcss npx tailwindcss init Configuring Template Paths Next, we configure our template paths by adding them to the file. The signifies the lines you'll be adding: tailwind.config.js ++ /** @type {import('tailwindcss').Config} */ module.exports = { -- content: [], ++ content: [ ++ "./src/**/*.{js,jsx,ts,tsx}", ++ ], theme: { extend: {}, }, plugins: [], } Adding Tailwind Directives Lastly, we'll add the directives for each of Tailwind's layers to our file: @tailwind ./src/index.css @tailwind base; @tailwind components; @tailwind utilities; And voila! Tailwind CSS is now integrated into our project. Installing Required Libraries Before we proceed to the coding section, let's finalize our preparations by installing the necessary libraries such as , , , and . fontawesome react-markdown axios react-hook-form Installing FontAwesome for Icons npm i --save @fortawesome/fontawesome-svg-core npm install --save @fortawesome/free-solid-svg-icons npm install --save @fortawesome/react-fontawesome Installing React Markdown for Rendering Markdown Content npm install --save react-markdown With these steps completed, your project is set up with all the necessary tools and libraries. Next, we'll start building the AI assistant app that uses Anthropic's Claude API and LangChain for generating accurate and human-like text responses. Troubleshooting If you run into any issues during installation or setup, here are a few common solutions: For issues related to Node.js or npm, try reinstalling them or check the official and docs. Node.js npm If you have problems with Create React App, consult the for assistance. CRA documentation For Tailwind CSS issues, refer to the . Tailwind CSS documentation For any other issues, consult the documentation of the respective libraries or post your issues on StackOverflow or relevant GitHub repositories. Writing the Project Files In this section, we'll go back to the Flask app we initialized earlier and add new endpoints, such as and . These will serve as the endpoints for our simple chat and advanced chat features (the latter being powered by Google search results). /ask /search Let's start with importing our necessary modules: from flask import Flask, jsonify, request from dotenv import load_dotenv from langchain.chat_models import ChatAnthropic from langchain.chains import ConversationChain from langchain.agents import Tool from langchain.agents import AgentType from langchain.utilities import SerpAPIWrapper from langchain.agents import initialize_agent from langchain.memory import ConversationBufferMemory from langchain.prompts.chat import ( ChatPromptTemplate, MessagesPlaceholder, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate, ) load_dotenv() app = Flask(__name__) The above section imports all the necessary packages and initializes our Flask application. Developing the Backend Creating the Basic Endpoints We will start by creating a basic endpoint ( ) to test if our server is running correctly: / @app.route('/') def hello_world(): return 'Hello, World!' By visiting the root URL, we should receive the response "Hello, World!", indicating that our server is running as expected. Creating the /ask Endpoint This endpoint processes messages for the basic chat feature of our application. It reads JSON data from the request, processes the messages, and generates a response using the Anthropic's Claude model and LangChain. @app.route('/ask', methods=['POST']) def ask_assistant(): # The code for /ask endpoint goes here Extracting Messages from the Request First, we need to check whether any data is provided and extract the messages from it. data = request.get_json() if not data: return jsonify({"error": "No data provided"}), 400 messages = data.get("message") Generating the Response The following code segment generates the chat response using LangChain's model, and the to structure our conversation. The conversation history is stored using the . ChatAnthropic() ChatPromptTemplate ConversationBufferMemory llm = ChatAnthropic() input = "" message_list = [] for message in messages: if message['role'] == 'user': message_list.append( HumanMessagePromptTemplate.from_template(message['content']) ) input = message['content'] elif message['role'] == 'assistant': message_list.append( AIMessagePromptTemplate.from_template(message['content']) ) # Adding SystemMessagePromptTemplate at the beginning of the message_list message_list.insert(0, SystemMessagePromptTemplate.from_template( "The following is a friendly conversation between a human and an AI. The AI is talkative and " "provides lots of specific details from its context. The AI will respond with plain string, replace new lines with \\n which can be easily parsed and stored into JSON, and will try to keep the responses condensed, in as few lines as possible." )) message_list.insert(1, MessagesPlaceholder(variable_name="history")) message_list.insert(-1, HumanMessagePromptTemplate.from_template("{input}")) prompt = ChatPromptTemplate.from_messages(message_list) memory = ConversationBufferMemory(return_messages=True) conversation = ConversationChain(memory=memory, prompt=prompt, llm=llm) result = conversation.predict(input=input) Sending the Response After generating the response, we replace newlines in the result and return it as a JSON object. print(result) return jsonify({"status": "success", "message": result}) Creating the /search Endpoint The endpoint is similar to but includes search functionality to provide more detailed responses. We use the to add this feature. /search /ask SerpAPIWrapper @app.route('/search', methods=['POST']) def search_with_assistant(): data = request.get_json() if not data: return jsonify({"error": "No data provided"}), 400 messages = data.get("message") llm = ChatAnthropic() # Get the last message with 'user' role user_messages = [msg for msg in messages if msg['role'] == 'user'] last_user_message = user_messages[-1] if user_messages else None # If there is no user message, return an error response if not last_user_message: return jsonify({"error": "No user message found"}), 400 input = last_user_message['content'] search = SerpAPIWrapper() tools = [ Tool( name = "Current Search", func=search.run, description="useful for when you need to answer questions about current events or the current state of the world" ), ] chat_history = MessagesPlaceholder(variable_name="chat_history") memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True) agent_chain = initialize_agent( tools, llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True, memory=memory, agent_kwargs = { "memory_prompts": [chat_history], "input_variables": ["input", "agent_scratchpad", "chat_history"] } ) result = agent_chain.run(input=input) print(result) return jsonify({"status": "success", "message": result}) Running the Flask App Finally, we add the standard boilerplate to run our Flask app. if __name__ == '__main__': app.run() Testing the Backend Should everything goes well, here's our final backend code. from flask import Flask, jsonify, request from dotenv import load_dotenv from langchain.chat_models import ChatAnthropic from langchain.chains import ConversationChain from langchain.agents import Tool from langchain.agents import AgentType from langchain.utilities import SerpAPIWrapper from langchain.agents import initialize_agent from langchain.memory import ConversationBufferMemory from langchain.prompts.chat import ( ChatPromptTemplate, MessagesPlaceholder, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate, ) load_dotenv() app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!' @app.route('/ask', methods=['POST']) def ask_assistant(): data = request.get_json() if not data: return jsonify({"error": "No data provided"}), 400 messages = data.get("message") llm = ChatAnthropic() input = "" message_list = [] for message in messages: if message['role'] == 'user': message_list.append( HumanMessagePromptTemplate.from_template(message['content']) ) input = message['content'] elif message['role'] == 'assistant': message_list.append( AIMessagePromptTemplate.from_template(message['content']) ) # Adding SystemMessagePromptTemplate at the beginning of the message_list message_list.insert(0, SystemMessagePromptTemplate.from_template( "The following is a friendly conversation between a human and an AI. The AI is talkative and " "provides lots of specific details from its context. The AI will respond with plain string, replace new lines with \\n which can be easily parsed and stored into JSON, and will try to keep the responses condensed, in as few lines as possible." )) message_list.insert(1, MessagesPlaceholder(variable_name="history")) message_list.insert(-1, HumanMessagePromptTemplate.from_template("{input}")) prompt = ChatPromptTemplate.from_messages(message_list) memory = ConversationBufferMemory(return_messages=True) conversation = ConversationChain(memory=memory, prompt=prompt, llm=llm) result = conversation.predict(input=input) print(result) return jsonify({"status": "success", "message": result}) @app.route('/search', methods=['POST']) def search_with_assistant(): data = request.get_json() if not data: return jsonify({"error": "No data provided"}), 400 messages = data.get("message") llm = ChatAnthropic() # Get the last message with 'user' role user_messages = [msg for msg in messages if msg['role'] == 'user'] last_user_message = user_messages[-1] if user_messages else None # If there is no user message, return an error response if not last_user_message: return jsonify({"error": "No user message found"}), 400 input = last_user_message['content'] search = SerpAPIWrapper() tools = [ Tool( name = "Current Search", func=search.run, description="useful for when you need to answer questions about current events or the current state of the world" ), ] chat_history = MessagesPlaceholder(variable_name="chat_history") memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True) agent_chain = initialize_agent( tools, llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True, memory=memory, agent_kwargs = { "memory_prompts": [chat_history], "input_variables": ["input", "agent_scratchpad", "chat_history"] } ) result = agent_chain.run(input=input) print(result) return jsonify({"status": "success", "message": result}) if __name__ == '__main__': app.run() Now, let's test our app. Run the backend app with this command, also please make sure that our virtual environment is activated. flask run We'll know everything is going to be fine if our terminal returns this output. Without further ado, let's test our two endpoints, and . To distinguish between the two, let's send each of them similar payload. Open your REST API testing and documentation software, or just use cURL. But in this tutorial, I'll use . /ask /search Insomnia Let's first call the endpoint with the following payload. Let's say I've asked about the sequel of the video game " ". Which it will answer incorrectly. This is to be expected as the model has training cutoff as of the end of 2021, in which there were no announcements about the sequel yet. /ask The Legend of Zelda: Breath of the Wild How about the endpoint? If you notice our code from earlier, this endpoint is handled with a more sophisticated chain, which utilise Agent. /search By using Agent, we can give the AI more power in decision making, by providing it with more tools than just its own model, which as we already demonstrated, has its own flaws. To demonstrate how the endpoint work, let's call it with the same payload as before. /search This time, it worked nicely! How does it work under the hood? let's look at the output in our terminal. Interestingly enough, this time the AI model didn't answer right away, but seemed to 'ponder' about what to do to answer. It decided to answer after it made the observation from the web search result that "Based on the news search, it looks like a sequel called The Legend of Zelda: Tears of the Kingdom was announced". So, by that logic, shouldn't we just use the endpoint then? as it is more accurate, and therefore, smarter? Not quite. Normally, in a chatbot-based app, the bot is expected to retain the context of previous conversations, so that it can provide responses as if it 'remembers' the whole context of the conversation. Let's see if endpoint can do just that. /search /search Let's test if the endpoint can recall what we just asked it. /search This happened because while the LangChain library has memory chain feature which can be used to retain past conversations, web server, and the REST API service built on it is inherently stateless. Which means, the web server will treat each request as a new request. But, haven't we included the previous conversations as payload? That's a good question. Turns out, the Agent chain used in LangChain currently doesn't support handling composed prompts, which consist of both requests and responses of both user and the AI. We mainly use this to provide example conversations for the model, further tuning the model to our desired response. The Agent, on the other hand, work by receiving a single instruction and develop its chains of thought around it. Which is why, no matter how long our conversations are, the agent will only respond to the latest request. Let's test our endpoint to notice the difference. /ask This time, it answered using our past conversations as added context! By now we realized that we need both of the endpoints to build our AI Assistant app. But how do we incorporate both the outdated but always remember endpoint with the forgetful but thorough and up-to-date endpoint? By building the front-end, of course! /ask /search Developing the Frontend App.tsx Let's navigate back to the project directory. In this project, we'll start modifying our React components, beginning with the entry point file, . ai-assistant-claude App.tsx import React from 'react'; import logo from './logo.svg'; import './App.css'; import { ChatClient } from './ChatClient'; function App() { return ( ); } export default App; In the above code snippet, we import the component, which will be created in a subsequent step. We then include the component within a element. This sets up the structure for our AI assistant app using React components. ChatClient ChatClient.tsx import React, { useState } from 'react'; import { ChatInput } from './ChatInput'; import { ChatHistory } from './ChatHistory'; export interface Message { content: string; role: string; } export const ChatClient: React.FC = () => { const [messages, setMessages] = useState >([]); const [isLoading, setIsLoading] = useState(false) const handleSimpleChat = (message: string) => { // Send the message and past chat history to the backend // Update messages state with the new message let newMessages = [...messages, { content: message, role: 'user' }] setMessages(newMessages); let postData = { message: newMessages } setIsLoading(true) fetch('/ask', { method: 'POST', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify(postData), }) .then((response) => response.json()) .then((data) => { if (data.status === "success") { setMessages([...newMessages, { content: data.message, role: 'assistant' }]) } setIsLoading(false) console.log('Success:', data); }) .catch((error) => { console.error('Error:', error); setIsLoading(false) }); }; const handleAdvancedChat = (message: string) => { // Trigger AI agent with Google Search functionality // Update messages state with the new message and AI response let newMessages = [...messages, { content: message, role: 'user' }] setMessages(newMessages); let postData = { message: newMessages } setIsLoading(true) fetch('/search', { method: 'POST', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify(postData), }) .then((response) => response.json()) .then((data) => { if (data.status === "success") { setMessages([...newMessages, { content: data.message, role: 'assistant' }]) } console.log('Success:', data); setIsLoading(false) }) .catch((error) => { console.error('Error:', error); setIsLoading(false) }); }; return ( AI Assistant with Claude and LangChain ); }; This component serves as the primary user interface for our AI assistant. As shown, it incorporates a component to display the conversation history and a component for inputting text. ChatHistory ChatInput The component processes input from the component, sends requests to the backend using this input, and then displays a loading status. If the request is successfully processed, the component will also show the response received from the backend. ChatInput ChatHistory.tsx import React from 'react'; import { ReactMarkdown } from 'react-markdown/lib/react-markdown'; import { Message } from './ChatClient'; interface ChatHistoryProps { messages: Array ; isLoading: boolean } export const ChatHistory: React.FC = ({ messages, isLoading }) => { return ( {messages.map((message, index) => ( {message.content} ))} {isLoading && ( {/* Put your desired loading content here */} Thinking... )} ); }; Fortunately, TailwindCSS offers built-in utility classes for simple animations, such as . This class elegantly helps indicate that a request is awaiting a response. In this component, we also differentiate the positioning of messages from the "user" and the "assistant." animate-pulse ChatInput.tsx import React, { useState } from 'react'; interface ChatInputProps { onSimpleChat: (message: string) => void; onAdvancedChat: (message: string) => void; } export const ChatInput: React.FC = ({ onSimpleChat, onAdvancedChat }) => { const [input, setInput] = useState(''); const handleInputChange = (event: React.ChangeEvent ) => { setInput(event.target.value); }; const handleSubmit = (handler: (message: string) => void) => { handler(input); setInput(''); }; return ( handleSubmit(onSimpleChat)} className="ml-2 px-4 py-2 font-semibold text-gray-600 bg-white dark:text-gray-400 dark:bg-gray-800 border border-gray-300 rounded-md hover:bg-gray-200 dark:hover:bg-gray-700 focus:outline-none" > Ask handleSubmit(onAdvancedChat)} className="ml-2 px-4 py-2 font-semibold text-white bg-blue-500 border border-blue-600 rounded-md hover:bg-blue-400 focus:outline-none" > Ask and Search ); }; Lastly, we added two buttons to our text input. The first button is used to send the input to the endpoint, which processes the input using the AI model without any additional enhancements. /ask This endpoint is context-aware. The second button, aptly named "Ask and Search," sends the input to the endpoint. In addition to processing the input via the AI model, this button also triggers an AI-driven web search if the situation requires it. /search index.html  -- React App ++ Claude AI Assistant You need to enable JavaScript to run this app. tag. To begin the development, run `npm start` or `yarn start`. To create a production bundle, use `npm run build` or `yarn build`. --> As a finishing touch, we update the title of our app in the page by changing it from " " to " ." index.html React App Claude AI Assistant package.json { "name": "ai-assistant-claude", "version": "0.1.0", "private": true, ++ "proxy": "http://localhost:5000", "dependencies": { Lastly, we add a proxy configuration to the file and set it to . This helps us bypass CORS limitations arising from using different ports. package.json http://localhost:5000 Testing the AI Assistant App To begin testing, first ensure that the backend app (claude-langchain) is already running. Next, change the directory to the front-end app (ai-assistant-claude) and start the app using the following command: npm start The app may take a moment to build. Once it's ready, it will automatically open in your browser at . localhost:3000 Let's test both the context awareness and search feature! First, let's inquire about another video game whose release hasn't been announced yet in 2021 – the sequel to the latest Yakuza game by . Additionally, we'll ask if this game features the beloved character Kazuma Kiryu from previous games. To do this, click the " " button. SEGA Ask Give it a few seconds to think... Surprisingly, the AI answered incorrectly. Yakuza: Like a Dragon is indeed the latest Yakuza game but stars a different protagonist, Ichiban Kasuga. Let's rephrase the question and use the " " button this time. Ask and Search Now, the AI took longer to decide whether it needed to search the web. After determining that a web search was necessary and finding a satisfactory answer, it returned the information to the client/front-end. This time, it returns the title "Like a Dragon Gaiden: The Man Who Erased His Name," which indeed features Kazuma Kiryu as the protagonist. Fascinating, isn't it? The agent, powered by a Large Language Model, predicts what actions are necessary given the available resources (search) and provides a satisfactory answer. LangChain makes it easy to connect and "chain" the model, prompts (instructions to the model), and other features such as agents and tools. Conclusion We hope you enjoyed building this app and learning about these technologies. For those interested in diving deeper, you can access the completed project for the and . front-end backend And if you want to build with the latest technologies realized by our partners, join us on next AI challenge! *please see lablab.ai for all terms and conditions