NeRF-based Visualization of 3D Cues Supporting Data-Driven Spacecraft Pose Estimation

2025-09-18

AI Summary

This study addresses the limited interpretability of data-driven spacecraft pose estimation models. We propose a NeRF-based, gradient-guided 3D visual cue attribution method that jointly optimizes a pre-trained pose estimator and a NeRF image generator. By leveraging backpropagated gradients to dynamically steer NeRF rendering, our approach explicitly reconstructs the critical 3D structural cues—such as edges, symmetry axes, and component topology—upon which the model’s pose predictions depend. Crucially, it operates without ground-truth 3D annotations or additional supervision, enabling localization of implicit geometric features influencing pose estimation and establishing an interpretable mapping between supervision signals and internal network representations. Experiments demonstrate its effectiveness in recovering essential 3D visual cues and significantly enhancing the transparency and trustworthiness of black-box pose estimators. The method provides a novel paradigm for on-orbit autonomous spacecraft diagnostics and human–machine collaborative decision-making.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

On-orbit operations require the estimation of the relative 6D pose, i.e., position and orientation, between a chaser spacecraft and its target. While data-driven spacecraft pose estimation methods have been developed, their adoption in real missions is hampered by the lack of understanding of their decision process. This paper presents a method to visualize the 3D visual cues on which a given pose estimator relies. For this purpose, we train a NeRF-based image generator using the gradients back-propagated through the pose estimation network. This enforces the generator to render the main 3D features exploited by the spacecraft pose estimation network. Experiments demonstrate that our method recovers the relevant 3D cues. Furthermore, they offer additional insights on the relationship between the pose estimation network supervision and its implicit representation of the target spacecraft.

Problem

• Visualizing 3D cues for spacecraft pose estimation
• Understanding decision process of neural pose estimators
• Recovering 3D features used by estimation networks

Innovation

• NeRF-based image generator visualization
• Gradient back-propagation through pose network
• Recovers 3D cues for spacecraft pose