🤖 AI Summary
To address the dual challenges of label switching and Byzantine failures in finite mixture model learning under distributed settings, this paper proposes Distance-Filtered Mixture Reduction (DFMR), a hyperparameter-free, Byzantine-robust divide-and-conquer estimator. DFMR introduces the first Byzantine-resilient aggregation mechanism for mixture models in non-Euclidean spaces, leveraging a pairwise distance-based filtering strategy to simultaneously align component parameters and prune outlying local estimates. Theoretically, DFMR achieves statistically optimal convergence rates under mild regularity conditions. Empirically, it significantly outperforms existing methods—including Mixture Reduction (MR)—on both synthetic benchmarks and the MNIST handwritten digit recognition task, demonstrating superior robustness against Byzantine adversaries while maintaining high statistical efficiency.
📝 Abstract
This paper proposes two split-and-conquer (SC) learning estimators for finite mixture models that are tolerant to Byzantine failures. In SC learning, individual machines obtain local estimates, which are then transmitted to a central server for aggregation. During this communication, the server may receive malicious or incorrect information from some local machines, a scenario known as Byzantine failures. While SC learning approaches have been devised to mitigate Byzantine failures in statistical models with Euclidean parameters, developing Byzantine-tolerant methods for finite mixture models with non-Euclidean parameters requires a distinct strategy. Our proposed distance-based methods are hyperparameter tuning free, unlike existing methods, and are resilient to Byzantine failures while achieving high statistical efficiency. We validate the effectiveness of our methods both theoretically and empirically via experiments on simulated and real data from machine learning applications for digit recognition. The code for the experiment can be found at https://github.com/SarahQiong/RobustSCGMM.