Authors:
(1) Wanru Zhao, University of Cambridge, Shanghai AI Laboratory with Equal contribution;
(2) Yaxin Du, Shanghai Jiao Tong University with Equal contribution;
(3) Nicholas D. Lane, University of Cambridge and Flower Labs;
(4) Siheng Chen, Shanghai AI Laboratory and Shanghai Jiao Tong University;
(5) Yanfeng Wang, Shanghai AI Laboratory and Shanghai Jiao Tong University. Table of Links Abstract and Introduction
Motivation and Setup: How low-quality data affects the performance of Collaborative Training
Proposed Workflow for Data Quality Control
Experiments
Conclusion and Future Work, and References
A. Related Work
B. Heterogeneity Settings
C. Experimental Details
D. Ablation study of Unified Scoring with Anchor Data
E. Examples for low-and high- quality Data E EXAMPLES FOR LOW- AND HIGH- QUALITY DATA To model real-world scenario, we designed three types of low-quality data generation rules. Cut The cut type data simulates scenarios where an answer exceeds the word limit, resulting in the truncation of the last few words. Practically, we establish a threshold of 100 tokens, retaining only the initial tokens up to this limit, as demonstrated in Table 5. Deletion Deletion type of data refers to missing important terminologies. In practice, we delete part of the tokens in the answer to simulate the missing terminology or some nouns, shown in Table 4. Exchange Exchange type low-quality data is characterized by the complete irrelevance between questions and their corresponding answers. In this approach, we swap answers among question-answer pairs to illustrate this form of low-quality data. Table 3 demonstrates examples of this exchange, where the first two samples involve swapping answers within the Medalpaca-flashcards dataset (Han et al., 2023). The remaining two samples, drawn from PMC-Llama (Wu et al., 2023), undergo a process where their rationales for answer choices are exchanged, and a different, incorrect choice is selected to ensure a clear distinction in the option outputs. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Wanru Zhao, University of Cambridge, Shanghai AI Laboratory with Equal contribution; (2) Yaxin Du, Shanghai Jiao Tong University with Equal contribution; (3) Nicholas D. Lane, University of Cambridge and Flower Labs; (4) Siheng Chen, Shanghai AI Laboratory and Shanghai Jiao Tong University; (5) Yanfeng Wang, Shanghai AI Laboratory and Shanghai Jiao Tong University. Authors: Authors: (1) Wanru Zhao, University of Cambridge, Shanghai AI Laboratory with Equal contribution; (2) Yaxin Du, Shanghai Jiao Tong University with Equal contribution; (3) Nicholas D. Lane, University of Cambridge and Flower Labs; (4) Siheng Chen, Shanghai AI Laboratory and Shanghai Jiao Tong University; (5) Yanfeng Wang, Shanghai AI Laboratory and Shanghai Jiao Tong University. Table of Links Abstract and Introduction Motivation and Setup: How low-quality data affects the performance of Collaborative Training Proposed Workflow for Data Quality Control Experiments Conclusion and Future Work, and References A. Related Work B. Heterogeneity Settings C. Experimental Details D. Ablation study of Unified Scoring with Anchor Data E. Examples for low-and high- quality Data Abstract and Introduction Abstract and Introduction Motivation and Setup: How low-quality data affects the performance of Collaborative Training Motivation and Setup: How low-quality data affects the performance of Collaborative Training Proposed Workflow for Data Quality Control Proposed Workflow for Data Quality Control Experiments Experiments Conclusion and Future Work, and References Conclusion and Future Work, and References A. Related Work A. Related Work B. Heterogeneity Settings B. Heterogeneity Settings C. Experimental Details C. Experimental Details D. Ablation study of Unified Scoring with Anchor Data D. Ablation study of Unified Scoring with Anchor Data E. Examples for low-and high- quality Data E. Examples for low-and high- quality Data E EXAMPLES FOR LOW- AND HIGH- QUALITY DATA To model real-world scenario, we designed three types of low-quality data generation rules. Cut The cut type data simulates scenarios where an answer exceeds the word limit, resulting in the truncation of the last few words. Practically, we establish a threshold of 100 tokens, retaining only the initial tokens up to this limit, as demonstrated in Table 5. Cut Deletion Deletion type of data refers to missing important terminologies. In practice, we delete part of the tokens in the answer to simulate the missing terminology or some nouns, shown in Table 4. Deletion Exchange Exchange type low-quality data is characterized by the complete irrelevance between questions and their corresponding answers. In this approach, we swap answers among question-answer pairs to illustrate this form of low-quality data. Table 3 demonstrates examples of this exchange, where the first two samples involve swapping answers within the Medalpaca-flashcards dataset (Han et al., 2023). The remaining two samples, drawn from PMC-Llama (Wu et al., 2023), undergo a process where their rationales for answer choices are exchanged, and a different, incorrect choice is selected to ensure a clear distinction in the option outputs. Exchange This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Enhancing Data Quality: Examples for Low-and High-quality Data

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Spatial-Photonic Ising Machines and the Next Era of Computing

AI as the "Bad Student" in Class

AI Will Not Kill Quantum Computing

AI's Unstoppable Energy Appetite: A Looming Crisis

Beyond the Algorithm: How Training Data Can Make or Break a Generative AI Model

Spatial-Photonic Ising Machines and the Next Era of Computing

AI as the "Bad Student" in Class

AI Will Not Kill Quantum Computing

AI's Unstoppable Energy Appetite: A Looming Crisis

Beyond the Algorithm: How Training Data Can Make or Break a Generative AI Model

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps