Authors:
(1) Goran Muric, InferLink Corporation, Los Angeles, (California gmuric@inferlink.com);
(2) Ben Delay, InferLink Corporation, Los Angeles, California (bdelay@inferlink.com);
(3) Steven Minton, InferLink Corporation, Los Angeles, California (sminton@inferlink.com). Authors: Authors: (1) Goran Muric, InferLink Corporation, Los Angeles, (California gmuric@inferlink.com); (2) Ben Delay, InferLink Corporation, Los Angeles, California (bdelay@inferlink.com); (3) Steven Minton, InferLink Corporation, Los Angeles, California (sminton@inferlink.com). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 1.1 Motivation 1.1 Motivation 2 Related Work and 2.1 Prompting techniques 2 Related Work and 2.1 Prompting techniques 2.2 In-context learning 2.2 In-context learning 2.3 Model interpretability 2.3 Model interpretability 3 Method 3 Method 3.1 Generating questions 3.1 Generating questions 3.2 Prompting LLM 3.2 Prompting LLM 3.3 Verbalizing the answers and 3.4 Training a classifier 3.3 Verbalizing the answers and 3.4 Training a classifier 4 Data and 4.1 Clinical trials 4 Data and 4.1 Clinical trials 4.2 Catalonia Independence Corpus and 4.3 Climate Detection Corpus 4.2 Catalonia Independence Corpus and 4.3 Climate Detection Corpus 4.4 Medical health advice data and 4.5 The European Court of Human Rights (ECtHR) Data 4.4 Medical health advice data and 4.5 The European Court of Human Rights (ECtHR) Data 4.6 UNFAIR-ToS Dataset 4.6 UNFAIR-ToS Dataset 5 Experiments 5 Experiments 6 Results 6 Results 7 Discussion 7 Discussion 7.1 Implications for Model Interpretability 7.1 Implications for Model Interpretability 7.2 Limitations and Future Work 7.2 Limitations and Future Work Reproducibility Reproducibility Acknowledgment and References Acknowledgment and References A Questions used in ICE-T method A Questions used in ICE-T method A Questions used in ICE-T method This is the list of questions used for each task in ICE-T technique. For each task, there are five questions in total, where the question with id = 0 is used in a prompt in a zero-shot approach. Other questions are secondary questions obtained as explained in Section 3.1: DRUG-ABUSE DRUG-ABUSE 0 Is there any indication in the patient’s records of current or past drug abuse? 1 Has the patient been prescribed medication with a high potential for abuse? 2 Are there any notes in the patient’s records indicating substance abuse or dependency issues? 3 Has the patient previously sought treatment for substance abuse or addiction? 4 Are there any irregularities in the patient’s prescription history that suggest misuse, such as early refill requests? ALCOHOL-ABUSE ALCOHOL-ABUSE 0 Has the patient consumed alcohol beyond the weekly recommended limits recently? 1 Did the patient consume more than 14 units of alcohol in the past week? 2 Has the patient engaged in binge drinking sessions in the last month? 3 Does the patient frequently consume alcohol on more than 5 days in a week? 4 Has the patient expressed concerns about controlling their alcohol intake recently? ENGLISH ENGLISH 0 Does the patient speak English, according to their medical records? 1 Is there a language preference indicated in the patient’s medical records? 2 Does the medical record include an interpreter request for non-English languages? 3 Is English listed as the patient’s primary language in the medical records? 4 Have previous medical consultations been conducted in English, as noted in the patient’s records? MAKES-DECISIONS MAKES-DECISIONS 0 Is there evidence that the patient makes their own medical decisions? 1 Does the patient have a documented history of discussing treatment options with their healthcare provider? 2 Has the patient previously filled out an advanced directive or a living will? 3 Does the patient regularly attend medical appointments alone? 4 Have there been instances where the patient explicitly expressed their treatment preferences or declined certain medical interventions? ABDOMINAL ABDOMINAL 0 Is there a record of the patient undergoing intraabdominal surgery, small or large intestine resection, or experiencing a small bowel obstruction? 1 Has the patient ever undergone any form of intraabdominal surgery? 2 Has the patient had a resection of either the small or large intestine? 3 Is there a record of the patient experiencing a small bowel obstruction? 4 Has the patient had any surgical intervention related to the digestive system that is not explicitly mentioned above? MAJOR-DIABETES MAJOR-DIABETES 0 Does the patient have any major diabetes-related complications such as amputation, kidney damage, skin conditions, retinopathy, nephropathy, or neuropathy? 1 Has the patient experienced any significant changes in their vision or been diagnosed with retinopathy? 2 Is there any history of skin conditions, wounds that heal poorly, or any amputations? 3 Does the patient have a history of kidney damage or been diagnosed with nephropathy? 4 Has the patient reported any persistent numbness, pain, or tingling in their extremities indicating neuropathy? ADVANCED-CAD ADVANCED-CAD 0 Is the patient currently taking two or more medications to treat CAD? 1 Is the patient currently being treated for coronary artery disease (CAD)? 2 Is the patient taking any medication to manage CAD symptoms? 3 Is the patient prescribed more than one medication specifically for CAD? 4 Are the medications the patient is taking to treat CAD being taken concurrently? MI-6MOS MI-6MOS 0 Has the patient had a myocardial infarction within the past 6 months from the most recent record date? 1 Has the patient reported any chest pain or symptoms consistent with a myocardial infarction in the past 6 months? 2 Has the patient undergone any cardiac diagnostic tests, such as an ECG or troponin levels, in the past 6 months? 3 Did the patient receive any treatment specifically for myocardial infarction, such as medication, stenting, or bypass surgery, in the past 6 months? 4 Is there any notation in the patient’s medical records of a confirmed diagnosis of myocardial infarction within the past 6 months? DIETSUPP-2MOS DIETSUPP-2MOS 0 Has the patient taken any dietary supplements, excluding Vitamin D, in the past 2 months? 1 Have you taken any vitamins or dietary supplements in the past 2 months, besides Vitamin D? 2 Are there any non-prescription supplements you have consumed regularly in the last 60 days? 3 Did you start or continue taking any herbal or nutritional supplements recently, except Vitamin D? 4 Excluding Vitamin D, have you used any health or wellness supplements since two months ago? ASP-FOR-MI ASP-FOR-MI 0 Is the patient using aspirin to prevent myocardial infarction based on their medical records? 1 Is there a history of cardiovascular disease in the patient’s medical records? 2 Has the patient been prescribed aspirin for long-term use? 3 Do the medical records indicate a doctor’s recommendation for aspirin to prevent myocardial infarction? 4 Is there any mention of HBA1C HBA1C 0 Have any HbA1c test results been listed in the records with a value between 6.5 and 9.5 1 Are there any HbA1c test results listed in the records? 2 Do any HbA1c test results fall within the 6.5 to 9.5 3 Is the HbA1c value specifically mentioned for each test result? 4 Have all the HbA1c test results been recorded and updated in the records? CREATININE CREATININE 0 Is there any indication in the patient’s records of serum creatinine levels above the upper limit of normal? 1 Has the patient undergone any recent serum creatinine tests? 2 Are the results of the patient’s serum creatinine tests documented in their medical records? 3 Do the documented serum creatinine levels exceed the established normal range? 4 Has there been any physician commentary or notes indicating concern over the patient’s serum creatinine levels? UNFAIR-ToS UNFAIR-ToS 0 Does this sentence from company’s Terms of Service violate the European Council Directive on unfair terms in consumer contracts? 1 Does the sentence create a significant imbalance between the parties’ rights and obligations to the detriment of the consumer? 2 Is the term clearly understandable and transparent to a typical consumer? 3 Does the term determine the main subject matter of the contract, or the appropriateness of the price or remuneration, in a way that is unfair to the consumer? 4 Has the consumer been given an opportunity to negotiate the terms of the contract? ECtHR ECtHR 0 Considering all the facts, did a state breach human rights provisions of the European Convention of Human Rights? 1 Did the state fail to provide adequate protection to an individual or group, thereby violating Article 2 or 3 concerning the right to life and prohibition of torture? 2 Was there any discrimination in the state’s actions or laws that contravenes Article 14 or Protocol 12 regarding the prohibition of discrimination? 3 Did the state unlawfully interfere with privacy, family life, freedom of expression, or freedom of assembly and association, as protected under Articles 8 to 11? 4 Was there a failure by the state to ensure a fair trial or access to a court, in violation of Article 6 or 13, concerning the right to a fair trial and the right to an effective remedy? Climate detection Climate detection 0 Is a given paragraph climate-related? 1 Does the paragraph discuss weather, temperature trends, or climate patterns? 2 Does the paragraph mention the impact of human activity on the environment? 3 Does the paragraph reference scientific studies or data on climate change? 4 Does the paragraph address policy or actions taken to mitigate climate impacts? Health advice Health advice 0 Does the given sentence contain a medical advice? 1 Does the sentence recommend or suggest a specific medication, treatment, or remedy? 2 Does the sentence advise on health behaviors, such as diet, exercise, or sleep patterns? 3 Does the sentence include any wording that implies or directly states a medical diagnosis or prognosis? 4 Does the sentence suggest consulting a healthcare professional or seeking medical attention? Catalonia indep. Catalonia indep. 0 Does the given sentence speak in favor of Catalonia independence? 1 Does the sentence express a positive opinion about Catalonia’s political autonomy? 2 Does the sentence criticize the Spanish government’s policies towards Catalonia? 3 Does the sentence highlight benefits or advantages of Catalonia being independent? 4 Does the sentence encourage actions or steps towards achieving independence for Catalonia? This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

ICE-T’s Challenges and Paths for Future Development

ICE-T Outperforms Zero-Shot in NLP Tasks Across Multiple Domains

Read My Stories

Too Long; Didn't Read

Get Free Unlimited IP Geo Data With IPinfo Lite

Key Questions in the ICE-T Method for Patient Assessment

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Diverse NLP Datasets for Real-World Text Classification

Getting the Most out of a Large Language Model

Deductive Verification of Chain-of-Thought Reasoning in LLMs

Solving the AI Hallucination Problem with Self-Verifying Natural Programs

Breaking Down Deductive Reasoning Errors in LLMs

Deductively Verifiable Chain-of-Thought Reasoning

Diverse NLP Datasets for Real-World Text Classification

Getting the Most out of a Large Language Model

Deductive Verification of Chain-of-Thought Reasoning in LLMs

Solving the AI Hallucination Problem with Self-Verifying Natural Programs

Breaking Down Deductive Reasoning Errors in LLMs

Deductively Verifiable Chain-of-Thought Reasoning

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps