Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Background


Method


Experiments
4.1 Multi-hop Reasoning Performance
4.2 Reasoning with Distractors
4.3 Generalization to Real-World knowledge
4.4 Run-time Analysis
4.5 Memorizing Knowledge


Related Work


Conclusion, Acknowledgements, and References Background Background Background Method Method Method Experiments
4.1 Multi-hop Reasoning Performance
4.2 Reasoning with Distractors
4.3 Generalization to Real-World knowledge
4.4 Run-time Analysis
4.5 Memorizing Knowledge Experiments Experiments 4.1 Multi-hop Reasoning Performance 4.1 Multi-hop Reasoning Performance 4.2 Reasoning with Distractors 4.2 Reasoning with Distractors 4.3 Generalization to Real-World knowledge 4.3 Generalization to Real-World knowledge 4.4 Run-time Analysis 4.4 Run-time Analysis 4.5 Memorizing Knowledge 4.5 Memorizing Knowledge Related Work Related Work Related Work Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References A. Dataset Dataset B. In-context Reasoning with Distractors B. In-context Reasoning with Distractors C. Implementation Details C. Implementation Details D. Adaptive Learning Rate D. Adaptive Learning Rate E. Experiments with Large Language Models E. Experiments with Large Language Models 3 Method Addressing these challenges, we propose RECKONING (REasoning through dynamiC KnOwledge eNcodING), which solves reasoning problems by memorizing the provided contextual knowledge, and then using this encoded knowledge when prompted with downstream questions. Specifically, RECKONING uses bi-level optimization to learn a set of meta-parameters primed to encode relevant knowledge in a limited number of gradient steps. The model can then use its updated weights to solve reasoning problems over this knowledge, without further presentation of the knowledge itself RE C O N ING without further presentation of the knowledge itself Authors:
(1) Zeming Chen, EPFL (zeming.chen@epfl.ch);
(2) Gail Weiss, EPFL (antoine.bosselut@epfl.ch);
(3) Eric Mitchell, Stanford University (eric.mitchell@cs.stanford.edu)';
(4) Asli Celikyilmaz, Meta AI Research (aslic@meta.com);
(5) Antoine Bosselut, EPFL (antoine.bosselut@epfl.ch). Authors: Authors: (1) Zeming Chen, EPFL (zeming.chen@epfl.ch); (2) Gail Weiss, EPFL (antoine.bosselut@epfl.ch); (3) Eric Mitchell, Stanford University (eric.mitchell@cs.stanford.edu)'; (4) Asli Celikyilmaz, Meta AI Research (aslic@meta.com); (5) Antoine Bosselut, EPFL (antoine.bosselut@epfl.ch). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

RECKONING Method: Bi-Level Optimization for Dynamic Knowledge Encoding and Robust Reasoning

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

AI for All: Operationalizing Diversity and Inclusion Requirements for AI Systems

Exploiting Memorization: Understanding the CLM Objective for Knowledge Encoding in LLMs

A Beginners Guide to the Gradient Descent Algorithm

Demystifying Different Variants of Gradient Descent Optimization Algorithm

DL03: Gradient Descent

AI for All: Operationalizing Diversity and Inclusion Requirements for AI Systems

Exploiting Memorization: Understanding the CLM Objective for Knowledge Encoding in LLMs

A Beginners Guide to the Gradient Descent Algorithm

Demystifying Different Variants of Gradient Descent Optimization Algorithm

DL03: Gradient Descent

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps