Enhancing Cross-Domain Generalizability in Social Determinants of Health Extraction with Prompt-Tuning Large Language Models.
Peng Chenga, Zehao Yua, Kaleb E Smithb, Wei‐Hsuan Lo‐Ciganicc, Jiang Biana, Yonghui Wua d
- aDepartment of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
- bNVIDIA, Santa Clara, California, USA
- cDepartment of Pharmaceutical Outcomes & Policy, College of Pharmacy, University of Florida, Gainesville, Florida, USA
- dCancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, Florida, USA
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
The progress in natural language processing (NLP) using large language models (LLMs) has greatly improved patient information extraction from clinical narratives. However, most methods based on the fine-tuning strategy have limited transfer learning ability for cross-domain applications. This study proposed a novel approach that employs a soft prompt-based learning architecture, which introduces trainable prompts to guide LLMs toward desired outputs. We examined two types of LLM architectures, including encoder-only GatorTron and decoder-only GatorTronGPT, and evaluated their performance for the extraction of social determinants of health (SDoH) using a cross-institution dataset from the 2022 n2c2 challenge and a cross-disease dataset from the University of Florida (UF) Health. The results show that decoder-only LLMs with prompt tuning achieved better performance in cross-domain applications. GatorTronGPT achieved the best F1 scores for both datasets, outperforming traditional fine-tuned GatorTron by 8.9% and 21.8% in a cross-institution setting, and 5.5% and 14.5% in a cross-disease setting.