From Decoupled to Coupled: Robustness Verification for Learning-based Keypoint Detection with Joint Specifications

2026-03-05

AI Summary

Existing keypoint detection models are sensitive to input perturbations and lack verifiable robustness guarantees regarding the interdependencies among keypoints. This work proposes the first coupled robustness verification framework, which jointly models the deviations of all keypoints via mixed-integer linear programming (MILP) to formally verify the collective robustness of heatmap-based detectors. By introducing reachable heatmap sets and polyhedral joint constraints, the method overcomes the conservativeness inherent in traditional decoupled, independent verification approaches and enables, for the first time, provably robust analysis of entire keypoint configurations. Experimental results demonstrate that under stringent error thresholds, the proposed framework significantly improves verification success rates and can either generate counterexamples or provide formal robustness certificates.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Keypoint detection underpins many vision tasks, including pose estimation, viewpoint recovery, and 3D reconstruction, yet modern neural models remain vulnerable to small input perturbations. Despite its importance, formal robustness verification for keypoint detectors is largely unexplored due to high-dimensional inputs and continuous coordinate outputs. We propose the first coupled robustness verification framework for heatmap-based keypoint detectors that bounds the joint deviation across all keypoints, capturing their interdependencies and downstream task requirements. Unlike prior decoupled, classification-style approaches that verify each keypoint independently and yield conservative guarantees, our method verifies collective behavior. We formulate verification as a falsification problem using a mixed-integer linear program (MILP) that combines reachable heatmap sets with a polytope encoding joint deviation constraints. Infeasibility certifies robustness, while feasibility provides counterexamples, and we prove the method is sound: if it certifies the model as robust, then the keypoint detection model is guaranteed to be robust. Experiments show that our coupled approach achieves high verified rates and remains effective under strict error thresholds where decoupled methods fail.

Problem

• keypoint detection
• robustness verification
• joint specifications
• input perturbations
• heatmap-based models

Innovation

• coupled robustness verification
• keypoint detection
• mixed-integer linear programming
• formal verification
• heatmap-based models