Closing the Gap at CRAC 2026: Two-Stage Adaptation for LLM-Based Multilingual Coreference Resolution

2026-05-16

AI Summary

This study addresses the limited cross-lingual generalization of large language models in multilingual coreference resolution by proposing a two-stage fine-tuning approach based on Gemma-3-27B. The method first trains a multilingual base adapter and then a task-specific adapter using dataset-targeted data. It innovatively incorporates head-based XML-style structured annotations together with a local re-indexing mechanism, enabling iterative document annotation. This framework substantially enhances model robustness across diverse languages, document lengths, and heterogeneous annotation schemes. Evaluated on the CRAC 2026 official test set, the approach achieves an average CoNLL F1 score of 74.32, ranking first among large language model submissions and third overall.

Abstract

We present our submission to the LLM track of the 2026 Computational Models of Reference, Anaphora and Coreference (CRAC 2026) shared task. With an average CoNLL F1 score of 74.32 on the official test set, our system ranked first in the LLM track, and third overall. Our system is based on the Gemma-3-27b model, fine-tuned using a two-stage strategy with a multilingual base adapter followed by dataset-specific adapters. We represent mention spans by their headword using an XML-inspired format with local reindexing and annotate documents iteratively. These design choices proved effective across languages, document lengths, and annotation guidelines.

Problem

• multilingual coreference resolution
• large language models
• cross-lingual generalization
• anaphora
• reference resolution

Innovation

• two-stage adaptation
• multilingual coreference resolution
• adapter-based fine-tuning
• headword-based mention representation
• iterative annotation