PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark

2026-07-03

AI Summary

This work addresses the challenge that existing text-to-image models struggle to generate academic posters adhering to scientific communication standards—such as legible labels, prescribed aspect ratios, and avoidance of fabricated charts. To tackle this, the authors propose a “placeholder-first” protocol that reframes poster generation as an auditable instruction-following task, decoupling layout design from chart synthesis and explicitly prohibiting the model from generating charts. They introduce PosterHarness, an auditable framework that, for the first time, quantitatively evaluates placeholder accuracy, blankness, aspect-ratio compliance, and information traceability. The system integrates a vision-language model, a deterministic synthesizer, and auditing scripts into an end-to-end verifiable pipeline. Experiments across 12 papers demonstrate that the method reduces synthetic charts from 34 to zero, significantly improves resolution, minimizes blank space, and enhances visual preference over Paper2Poster. All code and components are publicly released.

Abstract

Text-rich image models can now design poster-scale layouts, but we lack ways to measure whether they honor scientific communication contracts: legible labels, prescribed aspect ratios, and -- above all -- abstaining from fabricated scientific figures. We present POSTERHARNESS, an auditable harness reframing poster generation as measurable instruction-following tasks, with a pilot benchmark and failure taxonomy. POSTERHARNESS uses a placeholder-first contract to separate two jobs models otherwise conflate. The model performs visual-summary design: typography, reading path, color, and background -- but never draws data-bearing figures. Every figure region must be an empty labeled placeholder; a deterministic compositor inserts real source-paper figures at detected coordinates. This makes properties measurable: placeholder count and ID accuracy, blankness, aspect-ratio compliance, abstention from synthesized graphics, public-text hygiene, and source-figure provenance -- with failures logged as explicit rejections, not hidden in plausible-looking output. We instantiate the harness on 12 papers (6 HEP, 6 AI/ML-adjacent) and report three findings. (i) A counterfactual probe shows the placeholder contract drives VLM-counted synthesized figures from 34 to 0 across three papers. (ii) A failure taxonomy identifies blocking contracts: placeholder geometry, placeholder QA, template critic, and public text. (iii) Comparison with Paper2Poster shows a trade-off: PosterHarness yields higher-resolution artifacts, lower white-canvas fraction, and stronger VLM visual preference; the deterministic baseline retains slightly more PosterQuiz-style information and runs faster. We report this as regime characterization, not a superiority claim. All artifacts, prompts, manifests, and audit scripts are released as a reusable evaluation component.

Problem

• scientific poster generation
• instruction-following benchmark
• fabricated figures
• auditable evaluation
• visual communication contract

Innovation

• instruction-following benchmark
• placeholder-first contract
• auditable evaluation
• scientific poster generation
• deterministic compositor