This story draft by @escholar has not been reviewed by an editor, YET.

Critique Ability of Large Language Models: CriticBench: Statistics and Examples

EScholar: Electronic Academic Papers for Scholars HackerNoon profile picture
0-item

Table of Links

Abstract and 1. Introduction

  1. Definition of Critique Ability

  2. Construction of CriticBench

    3.1 Data Generation

    3.2 Data Selection

  3. Properties of Critique Ability

    4.1 Scaling Law

    4.2 Self-Critique Ability

    4.3 Correlation to Certainty

  4. New Capacity with Critique: Self-Consistency with Self-Check

  5. Conclusion, References, and Acknowledgments


A. Notations

B. CriticBench: Sources of Queries

C. CriticBench: Data Generation Details

D. CriticBench: Data Selection Details

E. CriticBench: Statistics and Examples

F. Evaluation Settings

E CRITICBENCH: STATISTICS AND EXAMPLES

E.1 STATISTICS

Table 2 presents the detailed statistics of CRITICBENCH and each subset.


Table 2: The statistics of CRITICBENCH and each subset.

E.2 EXAMPLES

Figure 8, 9 and 10 provide examples in CRITICBENCH.


Authors:

(1) Liangchen Luo, Google Research ([email protected]);

(2) Zi Lin, UC San Diego;

(3) Yinxiao Liu, Google Research;

(4) Yun Zhu, Google Research;

(5) Jingbo Shang, UC San Diego;

(6) Lei Meng, Google Research ([email protected]).

This paper is available on arxiv under CC BY 4.0 DEED license.


Catchpoint
L O A D I N G
. . . comments & more!

About Author

EScholar: Electronic Academic Papers for Scholars HackerNoon profile picture
EScholar: Electronic Academic Papers for Scholars@escholar
We publish the best academic work (that's too often lost to peer reviews & the TA's desk) to the global tech community
Read my storiesAbout @escholar

Topics

#large-language-models-(llms)#critical-thinking-in-ai#model-evaluation-framework#criticbench-benchmark#self-critique-and-enhancement#natural-language-processing#benchmarking-ai-performance#machine-learning-evaluation

Around The Web...

Read on Terminal Reader Terminal
Read this story w/o Javascript Lite

Join HackerNoonloading
Latest technology trends. Customized Experience. Curated Stories. Publish Your Ideas

Trending Topics

blockchaincryptocurrencyhackernoon-top-storyprogrammingsoftware-developmenttechnologystartuphackernoon-booksBitcoinbooks