Improving Transfer Learning for Sequence Labeling Tasks by Adapting Pre-trained Neural Language Models
Pretrained language models exhibit limited cross-domain transferability and insufficient contextual adaptability in sequence labeling tasks. To address these challenges, this work proposes three key innovations: (1) a multi-task learning framework that incorporates external knowledge signals by jointly modeling auxiliary tasks such as event trigger detection; (2) a modified autoregressive large language model architecture enabling bidirectional inter-layer information flow to enhance local sequence awareness; and (3) a generative in-context learning paradigm for sequence labeling, supporting few-shot adaptation without parameter updates. Evaluated on cross-domain event detection, the approach achieves significant performance gains over strong baselines. Experimental results demonstrate that this targeted transfer learning paradigm effectively unlocks the potential of pretrained models for structured prediction tasks, improving both generalization across domains and contextual sensitivity in label assignment.