Certification of Machine Learning Models via Directional Sharpness

2026-06-23

AI Summary

Existing model certification methods struggle to reliably assess generalization under training perturbations, and conventional proxy metrics—such as test accuracy or sharpness—are either prone to failure or computationally expensive. This work proposes directional sharpness, a novel proxy metric that evaluates the sensitivity of the loss landscape along specific directions, thereby introducing directionality into sharpness measurement for the first time. The method enables efficient and robust prediction of generalization performance and supports trustworthy verification through model auditing combined with zero-knowledge proofs, without exposing training data. Empirical results demonstrate that directional sharpness exhibits stronger correlation with generalization error, more accurately identifies poorly generalizing models, and incurs low computational overhead, making it well-suited for practical certification scenarios.

Abstract

In machine learning, model certification has been identified as an important method for gaining assurance about a model's trustworthiness and quality. A model's quality is largely determined by its ability to generalize, i.e., to perform well on data beyond what it was trained on. It is not possible to certify generalization directly, however, as it depends on unknown data and is not directly measurable. Proxies such as test accuracy can be misleading when the training process is perturbed (intentionally or accidentally), and metrics such as sharpness -- which has an empirically supported link to generalization -- are computationally expensive and can also serve as unreliable signals when training deviates from a prescribed procedure. In this work, we propose directional sharpness, a metric designed to efficiently and reliably indicate generalization despite potential training deviations. We provide empirical and analytical evidence that directional sharpness (1) correlates more strongly with generalization than existing metrics and (2) identifies models with poor generalization more reliably than existing metrics. Furthermore, directional sharpness is efficiently computable in model auditing settings, where the verifier has access to training data, and via zero-knowledge proofs that certify quality without revealing training data.

Problem

• model certification
• generalization
• sharpness
• training deviation
• trustworthiness

Innovation

• directional sharpness
• generalization
• model certification
• zero-knowledge proofs
• sharpness