Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging

2026-07-12

AI Summary

This work addresses the challenge of evaluating large language models (LLMs) on complex, domain-intensive computational imaging tasks, which existing general-purpose programming benchmarks inadequately capture. The authors introduce Imaging-101, the first standardized evaluation benchmark tailored to computational imaging, comprising 57 expert-validated tasks spanning six scientific disciplines. All tasks are unified into a canonical four-stage pipeline—preprocessing, forward modeling, inverse problem solving, and visualization—and assessed through a multi-granular framework encompassing planning, function-level unit testing, and end-to-end reconstruction. Experiments across seven state-of-the-art LLMs reveal substantial performance gaps in algorithm selection, adherence to physical conventions, and pipeline integration—deficiencies far more pronounced than those exposed by generic benchmarks—thereby highlighting critical directions for developing domain-specialized AI agents.

Abstract

Computational imaging, which recovers hidden signals from indirect, noisy measurements, underpins quantitative discovery across scientific disciplines, yet building a correct reconstruction pipeline demands deep domain expertise and remains laborious even for domain scientists. We introduce Imaging-101, a benchmark of 57 expert-verified computational imaging tasks spanning six scientific domains, each grounded in a peer-reviewed paper and canonicalized into a standardized four-stage pipeline (preprocessing, forward physics modeling, inverse solver, and visualization) Three evaluation tracks (planning, function-level unit tests, and end-to-end reconstruction) probe distinct agent capabilities across the full pipeline. Evaluating seven frontier LLMs uncovers systematic challenges in applying coding agents to computational imaging that go beyond those exposed by general coding benchmarks, spanning algorithm selection, physical convention handling, and pipeline integration. These findings highlight concrete capability gaps and point toward skill-augmented, domain-specialized agents as a practical path to reliable computational imaging assistance.

Problem

• computational imaging
• LLM coding agents
• reconstruction pipeline
• scientific domains
• benchmarking

Innovation

• computational imaging
• LLM coding agents
• benchmark
• domain-specialized agents
• pipeline integration