Optimizing Scoring Models: Effective Prompting Formats
by Writings, Papers and Blogs on Text ModelsSeptember 25th, 2024
Too Long; Didn't Read
This section illustrates the use of prompting formats for scorer large language models (LLMs), focusing on examples of Q_begin, Q_end, and A_begin formats when the "QA" pattern is present. It emphasizes that the "QA" pattern is eliminated in instruction-tuned scorer models like text-bison when employing the Q_begin and Q_end formats.
Authors:
(1) Chengrun Yang, Google DeepMind and Equal contribution;
(2) Xuezhi Wang, Google DeepMind;
(3) Yifeng Lu, Google DeepMind;
(4) Hanxiao Liu, Google DeepMind;
(5) Quoc V. Le, Google DeepMind;
(6) Denny Zhou, Google DeepMind;
(7) Xinyun Chen, Google DeepMind and Equal contribution.
Table of Links
2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms
3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression
3.2 Traveling Salesman Problem (TSP)
4 Application: Prompt Optimization and 4.1 Problem Setup
5 Prompt Optimization Experiments and 5.1 Evaluation Setup
5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt
7 Conclusion, Acknowledgments and References
B Prompting Formats for Scorer Llm
C Meta-Prompts and C.1 Meta-Prompt for Math Optimization
C.2 Meta-Prompt for Prompt Optimization
D Prompt Optimization Curves on the Remaining Bbh Tasks
E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions
B PROMPTING FORMATS FOR SCORER LLM
Figure 14, 15, and 16 show examples of the Q_begin, Q_end, and A_begin prompting formats when the “QA” pattern is present. The “QA” pattern is eliminated when prompting instruction-tuned scorer models like text-bison with the Q_begin and Q_end formats (Figure 17 and 18).
This paper is available on arxiv under CC0 1.0 DEED license.
L O A D I N G
. . . comments & more!
. . . comments & more!
