Enhancing Data Quality: Examples for Low-and High-quality Data

Table of Links

• Abstract and Introduction
• Motivation and Setup: How low-quality data affects the performance of Collaborative Training
• Proposed Workflow for Data Quality Control
• Experiments
• Conclusion and Future Work, and References
• A. Related Work
• B. Heterogeneity Settings
• C. Experimental Details
• D. Ablation study of Unified Scoring with Anchor Data
• E. Examples for low-and high- quality Data

E EXAMPLES FOR LOW- AND HIGH- QUALITY DATA

To model real-world scenario, we designed three types of low-quality data generation rules.

Cut The cut type data simulates scenarios where an answer exceeds the word limit, resulting in the truncation of the last few words. Practically, we establish a threshold of 100 tokens, retaining only the initial tokens up to this limit, as demonstrated in Table 5.

Deletion Deletion type of data refers to missing important terminologies. In practice, we delete part of the tokens in the answer to simulate the missing terminology or some nouns, shown in Table 4.

Exchange Exchange type low-quality data is characterized by the complete irrelevance between questions and their corresponding answers. In this approach, we swap answers among question-answer pairs to illustrate this form of low-quality data. Table 3 demonstrates examples of this exchange, where the first two samples involve swapping answers within the Medalpaca-flashcards dataset (Han et al., 2023). The remaining two samples, drawn from PMC-Llama (Wu et al., 2023), undergo a process where their rationales for answer choices are exchanged, and a different, incorrect choice is selected to ensure a clear distinction in the option outputs.