RAG Aplicacionkunapi Precisión kichay: Neo4j chaymanta LangChain kaqwan Yachaypa Gráficos kaqninta aprovechay

Gráfico kutichiy-yapachisqa miray ( GraphRAG ) kallpata tarichkan chaymanta ñawpaq vector maskay kutichiy ñankunaman kallpasapa yapayman tukuchkan. Kay ruwayqa aprovechan estructurado naturalezata grafico bases de datos kaqmanta, mayqinkunachus willayta nodos hina chaymanta relaciones hina organizanku, ukhunta chaymanta contextualidadta hurqusqa willayta aswan allinchaypaq.

Gráficos hatun kanku heterogéneo chaymanta huknin hukninwan tinkisqa willayta huk ruwasqa ruwaypi rikuchiypi chaymanta waqaychaypi, mana kallpachakuywan sasachakuy tinkiykuna chaymanta atributokuna imaymana willay layakuna kaqpi hap'iypi. Chaywanpas, vectorpa willay tantanakunaqa sapa kutim chayna estructurasqa willakuykunawan maqanakunku, kallpankuqa mana estructurasqa willakuykunata hatun dimensional vectores nisqawan hapiypim kachkan. RAG ruwanaykipi, estructurasqa grafico willayta vector maskanawan mana estructurasqa qillqawan tinkiyta atikunki iskaynin pachamanta aswan allin kaqta aypanaykipaq. Chaytan kay blog qelqapi rikuchisunchis.

Yachaypa Gráficos nisqakunaqa ancha allinmi, Ichaqa ¿Imaynatataq Hukta Ruwawaq?

Yachaypa siq’in ruwayqa aswan sasachakuymi. Chayqa willakuykunata huñuy hinaspa estructurayta ruwan, chaymi necesitakun ukhunchasqa hamutay iskaynin dominiomanta hinallataq grafico modeladomanta.

Kay ruwayta pisiyachinapaqqa LLM nisqawanmi experimentarqaniku. Simimanta, contextomanta ukhunchasqa hamut’ayninkuwan, LLMkuna yachay grafico ruway ruwaypi hatun partekunata automatizayta atinku. Qillqasqa willakuykunata qawarispaqa, kay modelokunam entidadkunata riqsiyta atinku, imayna tupasqankuta hamutayta atinku, chaynallataqmi yuyaychanmanku imaynatam aswan allinta qawarichikunmanku huk estructura de gráfico nisqapi.

Kay experimentokuna ruwasqaykurayku, LangChain kaqman yaparqayku ñawpaq kaq versión gráfico ruway módulo kaqmanta, chayta kay blog qillqasqapi rikuchisaqku.

Chay codigoqa GitHub nisqapim kachkan.

Neo4j Pachamamapa Kamachiynin

Huk Neo4j instanciata churanayki tiyan. Kay blog qillqasqapi ejemplokunata qatipay. Aswan facil ñanqa huk mana qullqiyuq instanciata qallariymi Neo4j Aura kaqpi , mayqinchus Neo4j willaypa tiyapuyninpa puyu instanciankunata qun. Hukninpi, Neo4j willaypa tiyapuyninpa huk kiti instanciantapas churayta atikunki Neo4j Escritorio ruwana uraykachispa chaymanta huk kiti willaypa instanciata ruwaspa.

 os.environ["OPENAI_API_KEY"] = "sk-" os.environ["NEO4J_URI"] = "bolt://localhost:7687" os.environ["NEO4J_USERNAME"] = "neo4j" os.environ["NEO4J_PASSWORD"] = "password" graph = Neo4jGraph()

Chaymanta, huk OpenAI llaveta qunayki tiyan , imaynachus paykunap modelonkuta kay blog qillqapi llamk'achisaqku.

Datos Ingestion nisqa

Kay rikuchiypaqqa Elizabeth Ipa Wikipidiya p'anqanta llamk'achisunchik. LangChain kargaqkunata llamk'achiyta atiykuman, Wikipidiyamanta qillqakunata mana pantasqa apamunapaq, rakinapaqpas.

 # Read the wikipedia article raw_documents = WikipediaLoader(query="Elizabeth I").load() # Define chunking strategy text_splitter = TokenTextSplitter(chunk_size=512, chunk_overlap=24) documents = text_splitter.split_documents(raw_documents[:3])

Tiempom chayamun huk grafico ruwanapaq chay hurqusqa qillqakunamanta. Kaypaq, huk LLMGraphTransformermodule ruwasqayku mayqinchus anchata pisiyachin huk yachay grafico ruwayta chaymanta waqaychayta huk grafico willaypa tiyapuyninpi.

 llm=ChatOpenAI(temperature=0, model_name="gpt-4-0125-preview") llm_transformer = LLMGraphTransformer(llm=llm) # Extract graph data graph_documents = llm_transformer.convert_to_graph_documents(documents) # Store to neo4j graph.add_graph_documents( graph_documents, baseEntityLabel=True, include_source=True )

Mayqin LLM yachay grafico mirachiy cadena llamk'achiyta munanki chayta riqsichiyta atikunki. Kunan, OpenAI kaqmanta chaymanta Mistral kaqmanta ruwanakuna waqyay modelokunallata yanapayku. Ichaqa, hamuq pachakunapi LLM akllayta mastariyta yuyaykuyku. Kay ejemplopiqa, aswan qhipa GPT-4 nisqawanmi llamk’achkayku. Reparay, ruwasqa graficopa calidadninqa anchata dependen ima modelotachus llamk'achkanki chaymanta. Teoría nisqapiqa, sapa kutim aswan atiyniyuq kaqninta servichikuyta munanki. LLM grafico tikraqkuna grafiku qillqakunata kutichinku, chaymanta Neo4j kaqman add_graph_documents ruwaywan apamuyta atinku. baseEntityLabel tupuqa huk yapasqata qun Entidad sapa nodoman etiqueta, indexación chaymanta tapuy ruwayta kallpachaspa. include_source parámetro nodukunata paqarimuq qillqankuman t'inkikun, willay qatiy atiy chaymanta contexto hamut'ayta yanapan.

Neo4j Navegador kaqpi ruwasqa grafico qhawayta atikunki.

Reparay kay siq'iqa paqarichisqa siq'imanta huk rakillatam rikuchin.

RAG nisqapaq Híbrido Recuperación

Gráfico mirachiymanta qhipaman, huk híbrido kutichiy ruwayta llamk'achisunchik, mayqinchus vector chaymanta palabra clave índices kaqwan grafico kutichiywan RAG ruwanakunapaq huñun.

Siqiqa huk kutichiy ruwayta rikuchin, huk llamk'aq tapuyta ruwasqanwan qallarispa, chaymanta huk RAG kutichiyman pusasqa. Kay kutichiyqa simikuna chaymanta vector maskanakunata llamk'achin mana ruwasqa qillqa willayta maskanapaq chaymanta yachay graficomanta huñusqan willakuywan tinkin. Neo4j iskaynin llalliq simikuna chaymanta vector indiskunayuq kasqanrayku, kimsantin kutichiy akllanakuna huklla willaypa tiyapuynin sistemawan ruwayta atikunki. Kay pukyuta huñusqa willakuykunatam huk LLM nisqaman mikuchinku, chaynapi tukupay kutichiyta paqarichinankupaq hinaspa chayachinankupaq.

Mana Estructurasqa Willayta Retriever

Neo4jVector.from_existing_graph ruwayta llamk'achiy atikunki iskaynin llalliq simita chaymanta vector kutichiyta qillqakunaman yapanaykipaq. Kay ñanqa llalliq simi chaymanta vector maskana indiskunata huk hibrido maskana ruwaypaq ruwan, Documento sutichasqa nodukunata qhawaspa. Chaymantapas, qillqa churay chanikunata yupan sichus mana kanchu.

 vector_index = Neo4jVector.from_existing_graph( OpenAIEmbeddings(), search_type="hybrid", node_label="Document", text_node_properties=["text"], embedding_node_property="embedding" )

Chaymantaqa vector indice nisqataqa similarity_search nisqa ruwaywanmi waqyayta atikunman.

Gráfico nisqamanta hurquq

Hukninpiqa, huk gráfico kutichiyta wakichiyqa aswan involucrado ichaqa aswan libre kayta qun. Kay rikch'anaqa huk hunt'asqa qillqa indiswanmi llamk'achinqa, tupaq nodukunata riqsinanpaq chaymanta chiqan barrionkuta kutichinanpaq.

Gráfico kutichiyqa qallarikun yaykusqapi tupaq entidadkunata riqsichispa. Sasalla kananpaq, LLM kaqman kamachiyku runakuna, organizacionkuna, maypi kasqankuta riqsichinankupaq. Chayta aypanapaq, LCEL nisqawan musuq yapasqa with_structured_output nisqa ruwaywan kayta aypanapaq llamk'achisunchik.

 # Extract entities from text class Entities(BaseModel): """Identifying information about entities.""" names: List[str] = Field( ..., description="All the person, organization, or business entities that " "appear in the text", ) prompt = ChatPromptTemplate.from_messages( [ ( "system", "You are extracting organization and person entities from the text.", ), ( "human", "Use the given format to extract information from the following " "input: {question}", ), ] ) entity_chain = prompt | llm.with_structured_output(Entities)

Chayta pruebasun:

 entity_chain.invoke({"question": "Where was Amelia Earhart born?"}).names # ['Amelia Earhart']

Hatun, kunan tapuypi entidadkunata tariyta atispanchik, hunt'asqa qillqa indiswan yachay graficoman mapa ruwanapaq. Ñawpaqta, huk hunt'asqa qillqa indisota chaymanta huk ruwayta riqsichinanchik tiyan mayqinchus hunt'asqa qillqa tapuykunata paqarichinqa mayqinkunachus huk chhika pantasqa qillqayta saqin, mayqinmantachus mana anchatachu kaypi yaykusunchik.

 graph.query( "CREATE FULLTEXT INDEX entity IF NOT EXISTS FOR (e:__Entity__) ON EACH [e.id]") def generate_full_text_query(input: str) -> str: """ Generate a full-text search query for a given input string. This function constructs a query string suitable for a full-text search. It processes the input string by splitting it into words and appending a similarity threshold (~2 changed characters) to each word, then combines them using the AND operator. Useful for mapping entities from user questions to database values, and allows for some misspelings. """ full_text_query = "" words = [el for el in remove_lucene_chars(input).split() if el] for word in words[:-1]: full_text_query += f" {word}~2 AND" full_text_query += f" {words[-1]}~2" return full_text_query.strip()

Kunanqa llapanta huñusun.

 # Fulltext index query def structured_retriever(question: str) -> str: """ Collects the neighborhood of entities mentioned in the question """ result = "" entities = entity_chain.invoke({"question": question}) for entity in entities.names: response = graph.query( """CALL db.index.fulltext.queryNodes('entity', $query, {limit:2}) YIELD node,score CALL { MATCH (node)-[r:!MENTIONS]->(neighbor) RETURN node.id + ' - ' + type(r) + ' -> ' + neighbor.id AS output UNION MATCH (node)<-[r:!MENTIONS]-(neighbor) RETURN neighbor.id + ' - ' + type(r) + ' -> ' + node.id AS output } RETURN output LIMIT 50 """, {"query": generate_full_text_query(entity)}, ) result += "\n".join([el['output'] for el in response]) return result

structured_retriever llamk'ayqa qallarikun entidadkunata tarispa user tapuypi. Chaymanta, tarisqa entidades patapi itera ruwan chaymanta huk plantilla Cypher kaqwan llamk'achin barrio nodos relevantes kaqmanta kutichinanpaq. ¡Pruebasun!

 print(structured_retriever("Who is Elizabeth I?")) # Elizabeth I - BORN_ON -> 7 September 1533 # Elizabeth I - DIED_ON -> 24 March 1603 # Elizabeth I - TITLE_HELD_FROM -> Queen Of England And Ireland # Elizabeth I - TITLE_HELD_UNTIL -> 17 November 1558 # Elizabeth I - MEMBER_OF -> House Of Tudor # Elizabeth I - CHILD_OF -> Henry Viii # and more...

Qhipa kaq Retriever

Qallariypi nisqa hina, mana estructurasqa chaymanta grafico kutichiyta huñusaqku huk LLM kaqman pasasqa qhipa contexto ruwanapaq.

 def retriever(question: str): print(f"Search query: {question}") structured_data = structured_retriever(question) unstructured_data = [el.page_content for el in vector_index.similarity_search(question)] final_data = f"""Structured data: {structured_data} Unstructured data: {"#Document ". join(unstructured_data)} """ return final_data

Pythonwan ruwachkasqanchikmanhina, f-string nisqawan lluqsiykunata tinkiyta atisunman.

Cadena RAG nisqamanta sut’inchaspa

RAGpa componente de recuperación nisqatam allinta ruwarqayku. Chaymanta, huk tapuyta riqsichiyku, chaymi aprovechan contexto qusqa chay integrado híbrido retriever kaqmanta kutichiyta ruwanapaq, RAG cadena implementacionta tukuchispa.

 template = """Answer the question based only on the following context: {context} Question: {question} """ prompt = ChatPromptTemplate.from_template(template) chain = ( RunnableParallel( { "context": _search_query | retriever, "question": RunnablePassthrough(), } ) | prompt | llm | StrOutputParser() )

Tukuchanapaq, ñawpaqman rispa pruebayta atisunman RAG híbrido implementacionninchikta.

 chain.invoke({"question": "Which house did Elizabeth I belong to?"}) # Search query: Which house did Elizabeth I belong to? # 'Elizabeth I belonged to the House of Tudor.'

Hinallataq huk tapuy wakmanta qillqana ruwayta churarqani, RAG kadena rimanakuypaq churanakunaman tupachiyta atichispa, chaymanta qatipay tapuykunata saqin. Vector chaymanta palabra clave maskana ñankunata llamk'achisqaykurayku, qatiq tapuykunata wakmanta qillqanayku tiyan maskana ruwayniyku allinchaypaq.

 chain.invoke( { "question": "When was she born?", "chat_history": [("Which house did Elizabeth I belong to?", "House Of Tudor")], } ) # Search query: When was Elizabeth I born? # 'Elizabeth I was born on 7 September 1533.'

Qawawaqmi: ¿Hayk’aqmi nacerqan? ñawpaqta musuqmanta qillqasqa karqan ¿Hayk'aqmi Elizabeth I nacerqan? . Chaymantaqa musuqmanta qillqasqa tapukuywanmi tupaqnin contextota hurqurqaku, tapukuyta kutichirqaku.

Yachaymanta Gráficos Made Facil

LLMGraphTransformer riqsichisqawan, yachay graficokuna ruway ruwayqa kunan aswan sumaq chaymanta aswan yaykuy atiy kanan tiyan, pillapas RAG ruwayninku allinchayta maskaqpaq aswan facil kanqa ukhunwan chaymanta contextowan yachay grafikukuna qusqankuwan. Kayqa qallariyllaraqmi achka allinchakuykunata yuyaykusqaykurayku.

Sichus LLMkunawan graficokuna paqarichisqaykumanta hamut'aykunata, yuyaychaykunata utaq tapukuyniykikunata kanki, ama iskayrayaychu chayayta.

Chay código nisqa kaypi tarikun GitHub nisqapi .