Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Data 2.1 Data Sources 2.1 Data Sources 2.2 SS and SI Categories 2.2 SS and SI Categories 3 Methods 3.1 Lexicon Creation and Expansion 3.1 Lexicon Creation and Expansion 3.2 Annotations 3.2 Annotations 3.3 System Description 3.3 System Description 4 Results 4.1 Demographics and 4.2 System Performance 4.1 Demographics and 4.2 System Performance 5 Discussion 5 Discussion 5.1 Limitations 5.1 Limitations 6 Conclusion, Reproducibility, Funding, Acknowledgments, Author Contributions, and References 6 Conclusion, Reproducibility, Funding, Acknowledgments, Author Contributions, and References SUPPLEMENTARY SUPPLEMENTARY Guidelines for Annotating Social Support and Social Isolation in Clinical Notes Guidelines for Annotating Social Support and Social Isolation in Clinical Notes Other Supervised Models Other Supervised Models 4 RESULTS 4.1 Demographics The demographic characteristics of patients within the annotated cohort are detailed in Table 2. Notably, the patient composition at MSHS was younger and more diverse as compared to patients at WCM. 4.2 System Performance Analysis was conducted using the gold-standard, manually annotated data. The macro-averaged precision, recall, and f-scores for classifying fine- and coarse-grained SS and SI categories at note level are provided in Table 3. At MSHS, the RBS outperformed the LLM-based system for both fine- and coarse-grained classification. For the fine-grained categories, the RBS achieved macro-averaged f-score of 0.90 compared to 0.62 for the LLM. For coarse-grained classification, the RBS had macro-averaged f-score of 0.89 versus 0.65 for the LLM. At WCM, the RBS outperformed the LLM for fine-grained classification with macro-averaged f-scores of 0.82 versus 0.81, respectively. The coarse-grained categories were similar, with a macro-averaged f-score of 0.85 for the RBS compared to 0.82 for FLAN. The performance of the zero-shot FLAN-T5-XL model is provided in Supplementary Table S8. Comparison to ICD codes: There were zero visits associated with the annotated clinical notes where SI was captured by the structured ICD codes (ICD-10: ‘Z60.2’, ‘V60.3’, ‘Z60.4’, ‘Z60.9’; ICD-9: ‘V60.3’, ‘V62.4’ [see Supplementary Table S9]). Without the NLP pipelines, the presence of SI would otherwise be missed both sites. Comparison to ICD codes: This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv Authors:
(1) Braja Gopal Patra, Weill Cornell Medicine, New York, NY, USA and co-first authors;
(2) Lauren A. Lepow, Icahn School of Medicine at Mount Sinai, New York, NY, USA and co-first authors;
(3) Praneet Kasi Reddy Jagadeesh Kumar. Weill Cornell Medicine, New York, NY, USA;
(4) Veer Vekaria, Weill Cornell Medicine, New York, NY, USA;
(5) Mohit Manoj Sharma, Weill Cornell Medicine, New York, NY, USA;
(6) Prakash Adekkanattu, Weill Cornell Medicine, New York, NY, USA;
(7) Brian Fennessy, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(8) Gavin Hynes, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(9) Isotta Landi, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(10) Jorge A. Sanchez-Ruiz, Mayo Clinic, Rochester, MN, USA;
(11) Euijung Ryu, Mayo Clinic, Rochester, MN, USA;
(12) Joanna M. Biernacka, Mayo Clinic, Rochester, MN, USA;
(13) Girish N. Nadkarni, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(14) Ardesheer Talati, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA;
(15) Myrna Weissman, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA;
(16) Mark Olfson, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA, New York State Psychiatric Institute, New York, NY, USA, and Columbia University Irving Medical Center, New York, NY, USA;
(17) J. John Mann, Columbia University Irving Medical Center, New York, NY, USA;
(18) Alexander W. Charney, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(19) Jyotishman Pathak, Weill Cornell Medicine, New York, NY, USA. Authors: Authors: (1) Braja Gopal Patra, Weill Cornell Medicine, New York, NY, USA and co-first authors; (2) Lauren A. Lepow, Icahn School of Medicine at Mount Sinai, New York, NY, USA and co-first authors; (3) Praneet Kasi Reddy Jagadeesh Kumar. Weill Cornell Medicine, New York, NY, USA; (4) Veer Vekaria, Weill Cornell Medicine, New York, NY, USA; (5) Mohit Manoj Sharma, Weill Cornell Medicine, New York, NY, USA; (6) Prakash Adekkanattu, Weill Cornell Medicine, New York, NY, USA; (7) Brian Fennessy, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (8) Gavin Hynes, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (9) Isotta Landi, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (10) Jorge A. Sanchez-Ruiz, Mayo Clinic, Rochester, MN, USA; (11) Euijung Ryu, Mayo Clinic, Rochester, MN, USA; (12) Joanna M. Biernacka, Mayo Clinic, Rochester, MN, USA; (13) Girish N. Nadkarni, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (14) Ardesheer Talati, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA; (15) Myrna Weissman, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA; (16) Mark Olfson, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA, New York State Psychiatric Institute, New York, NY, USA, and Columbia University Irving Medical Center, New York, NY, USA; (17) J. John Mann, Columbia University Irving Medical Center, New York, NY, USA; (18) Alexander W. Charney, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (19) Jyotishman Pathak, Weill Cornell Medicine, New York, NY, USA.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Natural Language Processing for Risk Assessment: Identifying SI/SS in Psychiatric Notes

Developing Rule and LLM-Based Systems to Identify Mentions of Fine-Grained Categories

Natural Language Processing

Too Long; Didn't Read

Make resilience your competitive advantage

Extracting Social Support and Isolation Info From Clinical Notes: Demo and System Performance

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Supervised Models for Clinical Text: Evaluating SVM and BERT Performance

Extracting Social Support and Social Isolation Information From Clinical Psychiatry Notes

How We Used EHR Data to Develop the SS/SI NLP Pipeline

Extracting Social Support and Isolation Info From Clinical Psychiatry Notes: SS and SI Categories

How We Collected Lexicons for Fine-Grained Categories of SS and SI Using an Iterative Method

Supervised Models for Clinical Text: Evaluating SVM and BERT Performance

Extracting Social Support and Social Isolation Information From Clinical Psychiatry Notes

How We Used EHR Data to Develop the SS/SI NLP Pipeline

Extracting Social Support and Isolation Info From Clinical Psychiatry Notes: SS and SI Categories

How We Collected Lexicons for Fine-Grained Categories of SS and SI Using an Iterative Method

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps