Context Window Failures in Relational Foundation Models

๐Ÿ“… 2026-08-31
๐Ÿ“ˆ Citations: 0
โœจ Influential: 0
๐Ÿ“„ PDF
๐Ÿค– AI Summary
็ ”็ฉถ้’ˆๅฏนๅ…ณ็ณปๅž‹ๅŸบ็ก€ๆจกๅž‹ๅค„็†้ซ˜ๅŸบๆ•ฐๆ•ฐๆฎๆ—ถ็š„็ช—ๅฃ้™ๅˆถ้—ฎ้ข˜๏ผŒ้€š่ฟ‡ๅผ•ๅ…ฅๅˆๆˆ้‡‘่žๆ•ฐๆฎ้›†Animusๅนถ้‡‡็”จๆ—ถ้—ด้ข„่šๅˆๆ–นๆณ•๏ผŒๆ˜พ่‘—ๆ้ซ˜ไบ†้ข„ๆต‹ๆ€ง่ƒฝใ€‚
๐Ÿ“ Abstract
Recent Relational Deep Learning architectures have been proposed as foundation models for multi-table relational data, yet they impose constrained neighborhood budgets that force row truncation when an entity has many related records. We introduce Animus, a synthetic financial dataset in which predicting customer income requires aggregating up to tens of thousands of transactions. On the raw representation, three recently proposed models (RT, Griffin, RelGT) achieve $R^2 \le 0.18$; a single, routine, temporal pre-aggregation step recovers $R^2$ up to $0.65$. This questions whether current relational foundation models are ready for high-cardinality real-world data.
Problem

Research questions and friction points this paper is trying to address.

Context Window Failures
Relational Foundation Models
High-cardinality Data
Innovation

Methods, ideas, or system contributions that make the work stand out.

Context Window Failures
Relational Foundation Models
Temporal Pre-aggregation
High-cardinality Data
๐Ÿ”Ž Similar Papers
D
Denis Oliveira Correa
Kunumi Institute, Brazil
F
Francisco Galuppo Azevedo
Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil