Sailing Towards Zero-Shot State Estimation using Foundation Models Combined with a UKF

2025-09-04

AI Summary

Traditional state estimation relies heavily on labor-intensive system identification and extensive prior data. To address this, we propose FM-UKF—a novel framework that integrates a Transformer-based foundation model with analytical sensor models directly into the Unscented Kalman Filter (UKF) architecture, enabling zero-shot state estimation across unseen dynamical systems and sensor configurations. Unlike conventional end-to-end approaches, FM-UKF requires no training data and generalizes to previously unencountered physical systems. Evaluated on a newly constructed, high-fidelity benchmark of container ship dynamics, it achieves accuracy and robustness comparable to both classical UKF and state-of-the-art end-to-end models, while significantly reducing deployment overhead. Its core innovation lies in decoupling sensor-specific dependencies inherent in end-to-end methods, thereby establishing an interpretable, transferable paradigm that fuses foundation models with classical filtering. We publicly release the benchmark dataset to advance zero-shot state estimation research.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

State estimation in control and systems engineering traditionally requires extensive manual system identification or data-collection effort. However, transformer-based foundation models in other domains have reduced data requirements by leveraging pre-trained generalist models. Ultimately, developing zero-shot foundation models of system dynamics could drastically reduce manual deployment effort. While recent work shows that transformer-based end-to-end approaches can achieve zero-shot performance on unseen systems, they are limited to sensor models seen during training. We introduce the foundation model unscented Kalman filter (FM-UKF), which combines a transformer-based model of system dynamics with analytically known sensor models via an UKF, enabling generalization across varying dynamics without retraining for new sensor configurations. We evaluate FM-UKF on a new benchmark of container ship models with complex dynamics, demonstrating a competitive accuracy, effort, and robustness trade-off compared to classical methods with approximate system knowledge and to an end-to-end approach. The benchmark and dataset are open sourced to further support future research in zero-shot state estimation via foundation models.

Problem

• Zero-shot state estimation for unseen systems
• Reducing manual deployment effort via foundation models
• Generalizing across dynamics without retraining sensors

Innovation

• Transformer-based dynamics model with UKF
• Combines learned dynamics with known sensor models
• Enables zero-shot generalization across sensor configurations