Creating a Retrieval-Augmented Generation (RAG) application allows you to leverage the capabilities of language models while grounding their responses in specific, reliable information you provide to the model. This guide will walk you through building a RAG application using for the web framework, the for the language model, and as your knowledge base. NextJS OpenAI API Dewy By the end of this tutorial, you'll understand how to integrate these technologies to reduce hallucinations in language model responses and ensure the information provided is relevant and accurate. What is Dewy? ​ Dewy is a knowledge base designed to simplify RAG applications by managing the extraction of knowledge from your documents and implementing semantic search over the extracted content. Providing recent, relevant domain knowledge in the form of documents ensures the models have the right information to answer your questions, without needing to hallucinate. Using large documents (ie PDF's) for Retrieval-Augmented Generation (RAG) poses several challenges: imposed by language models restrict the amount of text that can be analyzed in a single query, requiring documents to be summarized or "chunked" into smaller pieces Token limits Automatically from documents designed for humans requires complex parsing and post-processing extracting information over time, making it difficult to maintain the accuracy and relevancy of the RAG system's outputs. Documents may change Finding the portions of a document that are most relevant to a given conversation requires sophisticated techniques. semantic search Passing the entire content of all the documents through the LLM for every question isn’t practical. Dewy (and RAG in general) addresses these by doing some work upfront – extracting and indexing the content – so that it can do less at query time. Dewy addresses these challenges by automating the extraction, indexing, and retrieval of information from your documents. Getting Started ​ This guide will walk you through how to create a simple RAG-powered chatbot. The final code is available if you'd rather skip to the end and start hacking 😉. as an example Prerequisites ​ Basic knowledge of JavaScript and React. A NextJS environment set up on your local machine. A copy of Dewy running on your local machine (see Dewy's if you need help here). installation instructions Access to the OpenAI API platforms. Step 1: Set Up Your NextJS Project ​ : Create a new NextJS app by running the following command in your terminal: Initialize a new NextJS project npx create-next-app@latest my-rag-app Navigate into your new project directory: cd my-rag-app Install client librarires for the OpanAI API and Dewy. Install required packages: npm install openai dewy-ts ai : Set up environment variables for the OpenAI API key and your Dewy instance. Create a file in the root of your NextJS project and add the following lines: Prepare environment variables .env.local OPENAI_API_KEY=<your_openai_api_key_here>
DEWY_ENDPOINT=localhost:8000
DEWY_COLLECTION=main Step 2: Create an API endpoint to add documents ​ // app/api/documents/route.ts

import { Dewy } from 'dewy-ts';

export const runtime = 'edge'

// Create a Dewy client
const dewy = new Dewy({
    BASE: process.env.DEWY_ENDPOINT
})

export async function POST(req: Request) {
    // Pull the document's URL out of the request
    const formData = await req.formData();
    const url = formData.get('url');

    const document = await dewy.default.addDocument({
        collection: process.env.DEWY_COLLECTION,
        url,
    });

    return NextResponse.json({document_id: result.id})
} This API handler receives a form containing a document and indexes it in the knowledgebase. Dewy takes care of downloading the document, extracting information from it and making that information available as searchable chunks. Step 3: Create an API endpoint for chat generation ​ : This function will take the user's query and the retrieved documents from Dewy, and send a request to the OpenAI API to generate a response. The key is to format the prompt to include relevant information from the retrieved documents. Create a generation function //app/api/chat/utils.tsx

import OpenAI from 'openai';
import { Dewy } from 'dewy-ts'; 

// Create Dewy and OpenAI clients
const openai = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY
})
const dewy = new Dewy({
    BASE: process.env.DEWY_ENDPOINT
})

export async function generate({query}) {
    // Search Dewy for chunks relevant to the given query.
    const context = await dewy.default.retrieveChunks({
        collection: process.env.DEWY_COLLECTION,
        query: query, 
        n: 10,
    });

    // Build an augmented prompt providing the retrieved chunks as context for the LLM.
    const prompt = [{
        role: 'system',
        content: `You are a helpful assistant.
            You will take into account any CONTEXT BLOCK 
            that is provided in a conversation.
            START CONTEXT BLOCK
            ${context.results.map((c: any) => c.chunk.text).join("\n")}
            END OF CONTEXT BLOCK`,
    } ]

    // Call the OpenAI chat completion API to generate a response
    const messages =  [...prompt, [{role: 'user': content: 'Tell me about RAG'}]]
    const res = await openai.chat.completions.create({
        messages,
        model: 'gpt-3.5-turbo',
        temperature: 0.7,
    })

    return res
} This function handles chat messages by calling the generation function we just created and streaming back the generated response in real-time. Create the route handler: // app/api/chat/route.tsx

import { OpenAIStream, StreamingTextResponse } from 'ai';

import { generate } from "./utils";

export async function POST(req: Request) {
    const json = await req.json()
    const { messages } = json

    // Generate a response to the updated conversation
    const response = await generate(messages);

    // Convert the response into a friendly text-stream
    const stream = OpenAIStream(response);

    // Respond with the stream
    return new StreamingTextResponse(stream);
    } Step 4: Build the Frontend ​ This component creates a simple form with a text box and submit button, for sending URL's to the document creating route we created earlier. Basic form for loading Documents: // app/components/AddFromUrl.tsx

import React, { useState, FormEvent } from 'react';

export default function AddFromUrl(props) {
    async function onSubmit(event: FormEvent<HTMLFormElement>) {
        event.preventDefault()
    
        const formData = new FormData(event.currentTarget)
        await fetch('/api/documents', {
            method: 'POST',
            body: formData,
        })
    }
    
    return (
        <form onSubmit={onSubmit} {...props}>
            <input type="text" name="url" placeholder="URL to load..."/>
            <button type="submit">Load</button>
        </form>
    )
} : Use NextJS pages to build a user interface where users can input their queries. This will involve creating a form in the file. Create a simple chat UI pages/index.js // app/page.tsx

 'use client';
 
 import { useChat } from 'ai/react';
 import AddFromUrl from './components/AddFromUrl';
 
 export default function Chat() {
     const { messages, input, handleInputChange, handleSubmit } = useChat();
     return (
         <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
         {messages.map(m => (
             <div key={m.id} className="whitespace-pre-wrap">
             {m.role === 'user' ? 'User: ' : 'AI: '}
             {m.content}
             </div>
         ))}
     
         <form onSubmit={handleSubmit}>
             <input
             className="fixed bottom-10 w-full max-w-md p-2 mb-8 border border-gray-300 rounded "
             value={input}
             placeholder="Say something..."
             onChange={handleInputChange}
             />
         </form>
         <AddFromUrl className="fixed bottom-0 w-full max-w-md p-2 mb-8 border border-gray-300 rounded shadow-xl"/>
         </div>
     );
 } Try it out! Build your application and run it locally: npm run dev You should see a simple chat UI like the following: Managing Documents for RAG Using Dewy's Admin Console ​ In addition to API endpoints for managing documents programmatically, Dewy provides a GUI admin console. You can see the admin console in a browser at port 8000 (ie if you're running Dewy locally). https://localhost:8000 Dewy's admin console is designed to streamline the management of documents used for Retrieval-Augmented Generation (RAG) applications. By offering an intuitive interface and comprehensive features, it helps you fine-tune your knowledge bases, ensuring the AI generates responses that are both accurate and relevant. Here's how you can use Dewy's admin console to manage your documents effectively: Adding a Document ​ Upload or input new documents into your Dewy knowledge base through the admin console. Once a document is added, you can immediately observe how it influences the AI-generated results. This is useful for assessing the utility of new information and ensuring it aligns with the desired output quality and relevance. Exploring Extracted Information ​ Dewy's console allows you to get into the specifics of how information is extracted from each document. You can view structured data extracted from the text, making it easier to understand how the document might influence generation. This exploration aids in fine-tuning the extraction process, ensuring that the most relevant pieces of information are highlighted and utilized in the RAG process. Sample Queries ​ The admin console lets you test sample queries against your knowledge base. This helps when evaluating how well the RAG system retrieves relevant document chunks based on different inputs. By observing what is returned for each sample query, youd can quickly gauge the effectiveness of your current document set and retrieval algorithms, making it easier to identify areas for improvement. Conclusion and Key Takeaways ​ By building this RAG application, you've learned how to reduce hallucinations by providing specific, relevant information to your Gen AI application. This approach mitigates common issues such as hallucinations by ensuring the AI's responses are grounded in accurate information, and it addresses the challenge of managing large documents by intelligently retrieving only the most relevant information for each query. Also appears . here

This writer has a vested interest be it monetary, business, or otherwise, with 1 or more of the products or companies mentioned within.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

This story contains AI-generated text. The author has used AI either for research, to generate outlines, or write the text itself. 

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Creating a RAG chatbot with NextJS, OpenAI & Dewy

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Creating a Question-Answering CLI with Dewy and LangChain.js

10 ways how designing a chat bot is different from designing an app

10 Tips to Get the Most out of ChatGPT

10 Best AI Chatbot Builder for Your Business in 2022

11 Of The Best Artificial Intelligence Enterprise Chatbots in 2022

Creating a Question-Answering CLI with Dewy and LangChain.js

10 ways how designing a chat bot is different from designing an app

10 Tips to Get the Most out of ChatGPT

10 Best AI Chatbot Builder for Your Business in 2022

11 Of The Best Artificial Intelligence Enterprise Chatbots in 2022

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps