Sim-to-Real Betting on the E-Process: Bringing "simulators" to anytime-valid confidence sequences

2026-06-22

AI Summary

This work addresses the challenge of efficiently and reliably inferring the mean performance of robotic systems in real-world environments from simulation data. It introduces, for the first time, a novel integration of sim-to-real performance estimation with betting-based E-processes to construct anytime-valid confidence sequences. By combining a scaled simulator with an anytime-valid inference framework, the method dynamically generates statistically rigorous confidence bounds on true performance. Experimental results on synthetic data demonstrate that the proposed approach efficiently produces accurate confidence certificates for mean performance estimates, thereby offering a principled statistical guarantee for sim-to-real performance evaluation.

Abstract

This note describes an integration of the sim-to-real performance estimate with betting (from Chen et al.) and the safe anytime-valid inference (from Ramdas et al.). Using the scaled simulators. The method produces efficient, reliable certificates for the mean estimate, an approach that is especially valuable in robot performance testing. This note gives a primary, self-contained account of the construction; preliminaries of the respective methods are kept at a minimum, and one shall refer to the original works for full detail. Some synthetic examples demonstrating the proposed algorithm can be found at https://github.com/ISUSAIL/Bet4Sim2Real-EProcess.

Problem

• sim-to-real
• confidence sequences
• robot performance testing
• anytime-valid inference
• mean estimation

Innovation

• sim-to-real
• betting
• anytime-valid inference
• confidence sequences
• performance certification