Search papers, labs, and topics across Lattice.
The paper introduces PaReGTA, a framework that encodes longitudinal EHR data into visit-level text, learns domain-adapted visit embeddings via contrastive fine-tuning of a sentence-embedding model, and aggregates these embeddings using hybrid temporal pooling. This approach addresses the limitations of sparse representations and the data demands of sequence models by leveraging pre-trained LLMs for encoding temporal information in EHR data. Experiments on a migraine patient cohort demonstrate that PaReGTA outperforms sparse baselines in migraine type classification, while deep sequential models struggled.
Even with limited data, LLMs can encode temporal EHR data into fixed-dimensional patient representations that outperform traditional sparse methods and unstable deep sequential models.
Temporal information in structured electronic health records (EHRs) is often lost in sparse one-hot or count-based representations, while sequence models can be costly and data-hungry. We propose PaReGTA, an LLM-based encoding framework that (i) converts longitudinal EHR events into visit-level templated text with explicit temporal cues, (ii) learns domain-adapted visit embeddings via lightweight contrastive fine-tuning of a sentence-embedding model, and (iii) aggregates visit embeddings into a fixed-dimensional patient representation using hybrid temporal pooling that captures both recency and globally informative visits. Because PaReGTA does not require training from scratch but instead utilizes a pre-trained LLM, it can perform well even in data-limited cohorts. Furthermore, PaReGTA is model-agnostic and can benefit from future EHR-specialized sentence-embedding models. For interpretability, we introduce PaReGTA-RSS (Representation Shift Score), which quantifies clinically defined factor importance by recomputing representations after targeted factor removal and projecting representation shifts through a machine learning model. On 39,088 migraine patients from the All of Us Research Program, PaReGTA outperforms sparse baselines for migraine type classification while deep sequential models were unstable in our cohort.
