New Story
Think-and-Execute: The Limitations That We Face
by Transcompiler: Learn How to Translate CodeMarch 21st, 2025
Too Long; Didn't Read
A possible limitation of our approach is that we focus on algorithmic reasoning, as we believe it is the best setting to assess LLMs’ capabilities
Table of Links
B Details of Think-and-Execute
C Prompts Used in Our Experiments
D Human-written Pseudocode Prompts
F Generated Pseudocode Prompts
7 Limitations and Discussion
A possible limitation of our approach is that we focus on algorithmic reasoning, as we believe it is the best setting to assess LLMs’ capabilities in understanding a complex logic and carrying out a sequence of reasoning step, following the logic. However, we believe that THINK-AND-EXECUTE can be applied to other domains of reasoning that require following a long sequence of reasoning steps, such as multi-hop reasoning (Ji et al., 2020) and symbolic reasoning (Madaan & Yazdanbakhsh, 2022).
This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.
Authors:
(1) Hyungjoo Chae, Yonsei University;
(2) Yeonghyeon Kim, Yonsei University;
(3) Seungone Kim, KAIST AI;
(4) Kai Tzu-iunn Ong, Yonsei University;
(5) Beong-woo Kwak, Yonsei University;
(6) Moohyeon Kim, Yonsei University;
(7) Seonghwan Kim, Yonsei University;
(8) Taeyoon Kwon, Yonsei University;
(9) Jiwan Chung, Yonsei University;
(10) Youngjae Yu, Yonsei University;
(11) Jinyoung Yeo, Yonsei University.
L O A D I N G
. . . comments & more!
. . . comments & more!
