Authors:
(1) Goran Muric, InferLink Corporation, Los Angeles, (California gmuric@inferlink.com);
(2) Ben Delay, InferLink Corporation, Los Angeles, California (bdelay@inferlink.com);
(3) Steven Minton, InferLink Corporation, Los Angeles, California (sminton@inferlink.com). Authors: Authors: (1) Goran Muric, InferLink Corporation, Los Angeles, (California gmuric@inferlink.com); (2) Ben Delay, InferLink Corporation, Los Angeles, California (bdelay@inferlink.com); (3) Steven Minton, InferLink Corporation, Los Angeles, California (sminton@inferlink.com). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 1.1 Motivation 1.1 Motivation 2 Related Work and 2.1 Prompting techniques 2 Related Work and 2.1 Prompting techniques 2.2 In-context learning 2.2 In-context learning 2.3 Model interpretability 2.3 Model interpretability 3 Method 3 Method 3.1 Generating questions 3.1 Generating questions 3.2 Prompting LLM 3.2 Prompting LLM 3.3 Verbalizing the answers and 3.4 Training a classifier 3.3 Verbalizing the answers and 3.4 Training a classifier 4 Data and 4.1 Clinical trials 4 Data and 4.1 Clinical trials 4.2 Catalonia Independence Corpus and 4.3 Climate Detection Corpus 4.2 Catalonia Independence Corpus and 4.3 Climate Detection Corpus 4.4 Medical health advice data and 4.5 The European Court of Human Rights (ECtHR) Data 4.4 Medical health advice data and 4.5 The European Court of Human Rights (ECtHR) Data 4.6 UNFAIR-ToS Dataset 4.6 UNFAIR-ToS Dataset 5 Experiments 5 Experiments 6 Results 6 Results 7 Discussion 7 Discussion 7.1 Implications for Model Interpretability 7.1 Implications for Model Interpretability 7.2 Limitations and Future Work 7.2 Limitations and Future Work Reproducibility Reproducibility Acknowledgment and References Acknowledgment and References A Questions used in ICE-T method A Questions used in ICE-T method 4.4 Medical health advice data This dataset comprises a collection of sentences related to the medical domain, each accompanied by a label indicating whether the sentence offers medical advice. The labels can be one of three values: “strong advice”, “weak advice”, or “no advice”. (Yu et al., 2019) For the purpose of binary classification task we combined “strong advice” and “weak advice” into a single class: “advice”. The dataset includes approximately 8,000 samples, which have been divided into training and test datasets following the 80/20 rule. 4.5 The European Court of Human Rights (ECtHR) Data The European Court of Human Rights (ECtHR) hears allegations that a state has breached human rights provisions of the European Convention of Human Rights (ECHR) (Chalkidis et al., 2019). The dataset for each case includes a series of facts in form of paragraphs extracted from the case description. Additionally, each case is associated with specific articles of the European Convention on Human Rights (ECHR) that may have been violated. In many cases, multiple articles are violated at the same time. To make this a binary categorization problem, we adopted a binary labeling system. Cases are marked with a “1” if any ECHR articles are violated, and a “0” if no violations are detected. 4.6 UNFAIR-ToS Dataset The UNFAIR-ToS dataset contains 50 relevant online consumer contracts, i.e. Terms of Service (ToS) from on-line platforms (e.g., YouTube, Ebay, Facebook, etc.). Each agreement has been annotated at the sentence level to identify various types of potentially unfair clauses, which could infringe upon user rights under European consumer law. This dataset categorizes unfair terms into eight distinct groups: Arbitration, Unilateral Change, Content Removal, Jurisdiction, Choice of Law, Limitation of Liability, Unilateral Termination, and Contract by Using (Lippi et al., 2019). To transform the analysis into a binary classification problem, we re-labelled each sentence as either “unfair” if it contains any type of the identified unfair terms, or “not unfair’ if it does not fall into these categories. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Facebook

Improving Binary Classification with LLM-Generated Questions

Diverse NLP Datasets for Real-World Text Classification

Read My Stories

Too Long; Didn't Read

Get Free Unlimited IP Geo Data With IPinfo Lite

Medical and Legal Text Datasets for Binary Classification Tasks

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Diverse NLP Datasets for Real-World Text Classification

Getting the Most out of a Large Language Model

Deductive Verification of Chain-of-Thought Reasoning in LLMs

Solving the AI Hallucination Problem with Self-Verifying Natural Programs

Breaking Down Deductive Reasoning Errors in LLMs

Deductively Verifiable Chain-of-Thought Reasoning

Diverse NLP Datasets for Real-World Text Classification

Getting the Most out of a Large Language Model

Deductive Verification of Chain-of-Thought Reasoning in LLMs

Solving the AI Hallucination Problem with Self-Verifying Natural Programs

Breaking Down Deductive Reasoning Errors in LLMs

Deductively Verifiable Chain-of-Thought Reasoning

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps