Data Reconstruction: Identifiability and Optimization with Sample Splitting

2026-02-09

AI Summary

This work addresses the lack of theoretical guarantees in existing training data reconstruction methods regarding the uniqueness of solutions to the Karush–Kuhn–Tucker (KKT) conditions, as well as their limited optimization efficacy in identifiable settings. Focusing on two-layer neural networks with polynomial activations, the paper establishes, for the first time, sufficient conditions under which the original data can be uniquely identified via KKT-based analysis. To enhance non-convex optimization and escape poor stationary points, the authors propose a general, curvature-aware sample splitting strategy. By integrating identifiability theory with this novel splitting technique, the method consistently achieves significant improvements in reconstruction performance across diverse settings, demonstrating both its effectiveness and broad applicability.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Feature Construction/Reformulation
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance

Abstract

Training data reconstruction from KKT conditions has shown striking empirical success, yet it remains unclear when the resulting KKT equations have unique solutions and, even in identifiable regimes, how to reliably recover solutions by optimization. This work hereby focuses on these two complementary questions: identifiability and optimization. On the identifiability side, we discuss the sufficient conditions for KKT system of two-layer networks with polynomial activations to uniquely determine the training data, providing a theoretical explanation of when and why reconstruction is possible. On the optimization side, we introduce sample splitting, a curvature-aware refinement step applicable to general reconstruction objectives (not limited to KKT-based formulations): it creates additional descent directions to escape poor stationary points and refine solutions. Experiments demonstrate that augmenting several existing reconstruction methods with sample splitting consistently improves reconstruction performance.

Problem

• data reconstruction
• identifiability
• optimization
• KKT conditions
• sample splitting

Innovation

• data reconstruction
• identifiability
• KKT conditions
• sample splitting
• optimization