LiFT: Does Instruction Fine-Tuning Improve In-Context Learning for Longitudinal Modelling by Large Language Models?

2026-03-25

arXiv.org

AI Summary

This work addresses the challenge that large language models struggle to effectively capture temporal dynamics and rare events in longitudinal natural language processing tasks—such as mental health monitoring and stance evolution—particularly under data-scarce conditions where zero- or few-shot performance is limited. To overcome this, the authors propose LiFT, a novel framework that uniquely integrates instruction tuning with longitudinal modeling through a model-agnostic, unified multitask design. LiFT incorporates temporal dependencies, curriculum learning, and time-conditioning mechanisms to enhance the model’s in-context learning capabilities. Experiments demonstrate that LiFT significantly improves performance across five longitudinal datasets in zero-, one-, and three-shot settings, with notable gains in sensitivity to minority classes and recent historical context, while also achieving strong cross-task generalization.

Abstract

Longitudinal NLP tasks require reasoning over temporally ordered text to detect persistence and change in human behavior and opinions. However, in-context learning with large language models struggles on tasks where models must integrate historical context, track evolving interactions, and handle rare change events. We introduce LiFT, a longitudinal instruction fine-tuning framework that unifies diverse longitudinal modeling tasks under a shared instruction schema. LiFT uses a curriculum that progressively increases temporal difficulty while incorporating few-shot structure and temporal conditioning to encourage effective use of past context. We evaluate LiFT across five datasets. Models trained on longitudinal tasks with different levels of temporal granularity are tested for generalisability on two separate datasets. Across models with different parameter sizes (OLMo (1B/7B), LLaMA-8B, and Qwen-14B), LiFT consistently outperforms base-model ICL, with strong gains on out-of-distribution data and minority change events.

Problem

• longitudinal modeling
• in-context learning
• data scarcity
• temporal text
• rare event detection

Innovation

• in-context learning
• longitudinal modeling
• instruction fine-tuning
• temporal conditioning
• low-resource NLP