STEMGym: Benchmarking Sequential Decision-Making under Dose Budgets in Autonomous Electron Microscopy

2026-06-28

AI Summary

In dose-constrained autonomous scanning transmission electron microscopy (STEM), efficiently acquiring atomic-resolution images while minimizing sample damage remains a critical challenge. This work introduces STEMGym, an open-source Gymnasium benchmark platform encompassing five materials, three difficulty levels, and four task categories, along with a unified evaluation metric—dose efficiency curve area under the curve (DEC-AUC). Systematic experiments reveal, for the first time, that perception models exert a far greater influence on overall efficiency than navigation strategies: a simple CNN paired with raster scanning improves DEC-AUC by 5.5× (0.287 vs. 0.052), whereas advanced navigation yields marginal gains. Moreover, general-purpose vision-language models underperform specialized CNNs by approximately 13× in defect analysis. These findings redirect the focus of machine learning development in autonomous electron microscopy toward perception rather than navigation.

Abstract

A central premise of autonomous scientific imaging is that smarter navigation, whether Bayesian, RL-based, or otherwise adaptive, is the principal lever for sample-efficient acquisition. We present evidence to the contrary in scanning transmission electron microscopy (STEM), an atomic-resolution imaging modality whose every measurement deposits damaging electron dose. We introduce STEMGym, an open-source Gymnasium benchmark of 15 physics-simulated STEM worlds spanning five materials, three difficulty levels, and four characterisation tasks, scored by the Dose-Efficiency Curve area (DEC-AUC), a single scalar capturing the information-vs-dose Pareto frontier. Across 33 agent configurations under realistic dose budgets, the dominant determinant of dose efficiency is the analyst (perception) pipeline, not the navigator: pairing a trained CNN analyst with naïve raster scanning raises DEC-AUC by 5.5x over a CNN-free raster baseline (0.287 vs.\ 0.052), while substituting Bayesian or adaptive finite-state-machine navigation for raster yields no statistically significant further gain. Production-tier vision-language models further underperform task-specific CNNs by {\sim}13x on crystallographic defect analysis. By decoupling perception, navigation, and planning under a unified dose budget, STEMGym reframes where ML effort should be invested in autonomous electron microscopy and provides the measurement infrastructure to test it.

Problem

• autonomous electron microscopy
• dose budget
• sequential decision-making
• sample-efficient acquisition
• STEM

Innovation

• autonomous electron microscopy
• dose-efficient imaging
• sequential decision-making
• perception-navigation decoupling
• STEMGym benchmark