Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding

2026-08-09

AI Summary

This work addresses the challenge that large language models (LLMs) struggle to accurately retrieve and reason over the correct temporal version of evolving official documents—such as legal or tax regulations—corresponding to a given query date. To this end, the authors introduce TIDE, a novel benchmark comprising 644 Bangladeshi customs documents spanning 1969–2025 and 3,050 expert-validated question-answer pairs, which explicitly frames version resolution as a core task in time-aware question answering. Evaluation of nine prominent LLMs across three settings—parametric memory, golden context, and retrieval-augmented generation—employs a rigorous three-judge LLM scoring mechanism with hard date gating and multi-calendar date parsing. Results reveal that even the best-performing model achieves only a 68.5% macro-averaged accuracy, with implicit version resolution at 59.7% and an alarmingly low 26.7% accuracy in recognizing when a provided document version is inapplicable, underscoring models’ overreliance on parametric knowledge rather than authoritative textual sources.

Abstract

Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates. In contrast to encyclopedic knowledge, where an old fact is simply overwritten, an amendment is itself an official text that states what it replaces and when it takes effect, and the earlier version stays correct for its validity period. The central challenge is therefore version resolution, that is, identifying the version in force on the queried date. Existing temporal QA datasets treat time only as an annotation, so version resolution stays untested. We present TIDE, an expert-verified benchmark of 3,050 QA pairs over 644 official customs instruments issued between 1969 and 2025 by the Government of Bangladesh, covering eight task types over deeply code-mixed documents that are heterogeneous in layout and dated in two calendars. In addition, we evaluate nine recent LLMs under a single protocol across parametric, gold-context, and retrieval access, scored by a three-judge LLM council with a hard date gate separating correct meaning from correct time. The best macro-averaged accuracy is only 68.5%. Resolving a version from an implicit date reaches 59.7%, and detecting that the supplied version does not govern the query reaches only 26.7%. Models are more likely to find correct versions than to reject incorrect ones, and they tend to follow a confident parametric answer over the supplied authoritative text. All code and data are available at https://github.com/icsetepa44/TIDE

Problem

• temporal document understanding
• version resolution
• time-aware QA
• evolving documents
• temporal reasoning

Innovation

• version resolution
• temporal document understanding
• TIDE benchmark
• time-aware QA
• code-mixed legal documents