When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Tasks

2026-04-29

AI Summary

This study investigates the representational burden imposed on large language models when explicit two-dimensional structured tasks—such as matrix transposition, Conway’s Game of Life, and LU decomposition—are serialized into one-dimensional inputs. To this end, the authors introduce the concept of “serialization friction” and develop a dual-path contrastive framework built upon the same language backbone: one path processes purely textual sequential inputs, while the other incorporates visual enhancements to preserve two-dimensional spatial layout information. Experimental results demonstrate that the visual-augmented path consistently and significantly outperforms the text-only path across all tasks, with the performance gap widening as problem scale increases. Moreover, errors in the text-only condition exhibit pronounced spatial structure, underscoring the critical importance of retaining two-dimensional structural cues for effective model reasoning.

Abstract

Large language models (LLMs) conventionally process structured inputs as 1D token sequences. While natural for prose, such linearization may introduce additional representational burden for tasks whose computation depends directly on explicit 2D structure, because row--column alignment and local neighborhoods are no longer directly expressed in the input. We study this setting, which we refer to as serialization friction, on a small diagnostic testbed of synthetic tasks with explicit 2D structure: matrix transpose, Conway's Game of Life, and LU decomposition. To examine this question, we compare a text-only language pathway over serialized inputs with a vision-augmented pathway, built on the same language backbone, that receives the same underlying content rendered in task-faithful 2D layout, yielding a system-level comparison between two end-to-end input pathways. Across the tasks and settings we study, the visual pathway consistently outperforms the textual pathway; the gap often widens at larger dimensions, and error patterns under serialization become increasingly spatially structured. These findings indicate that the relationship between input representation and model performance on such tasks warrants further investigation, and suggest that preserving task-relevant 2D layout is a promising direction for structured 2D tasks.

Problem

• serialization friction
• structured tasks
• 2D structure
• large language models
• input representation

Innovation

• serialization friction
• structured 2D tasks
• vision-augmented LLMs
• input representation
• matrix reasoning