COWs and their Hybrids: A Statistical View of Custom Orthogonal Weights

2025-10-17

AI Summary

In high-energy physics, accurately extracting signal components from mixed signal-background distributions remains a longstanding challenge. To address the limited flexibility of conventional sPlot methods, this paper proposes the Control-Orthogonal Weights (COWs) framework: it constructs signal weights based on discriminative variables and enforces orthogonality via control variables; formally defines statistical assumptions for the weights and establishes theoretical guarantees of unbiasedness, consistency, and asymptotic normality; and jointly optimizes weight estimation and uncertainty quantification using maximum likelihood estimation within a mixture model. Experiments demonstrate that COWs significantly improve signal extraction accuracy and robustness under complex conditions—including nonstationary backgrounds and high-dimensional correlated structures—thereby providing a scalable, statistically principled paradigm for high-dimensional, nonlinear background modeling.

Technology Category

• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

A recurring challenge in high energy physics is inference of the signal component from a distribution for which observations are assumed to be a mixture of signal and background events. A standard assumption is that there exists information encoded in a discriminant variable that is effective at separating signal and background. This can be used to assign a signal weight to each event, with these weights used in subsequent analyses of one or more control variables of interest. The custom orthogonal weights (COWs) approach of Dembinski, et al.(2022), a generalization of the sPlot approach of Barlow (1987) and Pivk and Le Diberder (2005), is tailored to address this objective. The problem, and this method, present interesting and novel statistical issues. Here we formalize the assumptions needed and the statistical properties, while also considering extensions and alternative approaches.

Problem

• Separating signal from background events in high energy physics data analysis
• Developing custom orthogonal weights for signal extraction from mixed distributions
• Formalizing statistical assumptions and properties for improved inference methods

Innovation

• Custom orthogonal weights generalize sPlot approach
• Assigns signal weights using discriminant variable
• Tailored for signal-background separation in physics