redundancy measurement

This paper addresses the challenge of accurately quantifying complex (e.g., nonlinear, multimodal, non-functional) dependencies between continuous variables. We propose Predictive Dependence (PD), a fully nonparametric and interpretable dependence measure defined as the expected relative loss in predictive accuracy for $Y$ when ignoring $X$, bounded in $[0,1]$ and identically zero under independence. PD is grounded in kernel density estimation and conditional distribution reconstruction, augmented by Monte Carlo approximation and adaptive bandwidth selection—ensuring rigorous statistical guarantees and exact independence detection. Extensive evaluation across over 90,000 real-world and synthetic datasets demonstrates that PD consistently outperforms state-of-the-art methods—including HSIC and distance correlation—with particularly pronounced gains in weakly nonlinear and multimodal dependency settings.

Existing probabilistic forecasting evaluation metrics primarily emphasize predictive accuracy while neglecting their practical utility in downstream decision-making tasks, leading to a misalignment between evaluation and application. To address this, we propose a data-driven evaluation alignment framework that formulates the learning of a surrogate evaluation function as an end-to-end optimization problem. Leveraging proper scoring rule theory, our approach employs a neural network-parameterized weighted scoring rule to automatically learn an evaluation function aligned with downstream objectives—without assuming any prior cost structure. This work is the first to formalize evaluation alignment as a learnable problem, combining theoretical rigor with engineering scalability. Experiments on synthetic and real-world regression tasks demonstrate its effectiveness: it significantly reduces the gap between evaluation scores and downstream decision utility, enabling rapid, task-adaptive model selection and hyperparameter tuning.

This study addresses how to effectively visualize both quantifiable statistical uncertainty and expert-derived qualitative confidence in predictive contexts to support decision-making by non-expert users. Through three preregistered human-subject experiments (N=923), it systematically compares the impact of juxtaposing versus integrating qualitative confidence indicators (e.g., textual labels, icons) with statistical confidence intervals (encoded via color, transparency, or blurred outlines) in time-series line charts. This work presents the first systematic evaluation in time-series forecasting of multiple non-textual visual encodings for jointly representing dual forms of uncertainty. Findings demonstrate that designs employing color and blurred outlines successfully convey qualitative confidence and significantly alter users’ judgment patterns, offering empirical validation and actionable design guidelines for visualizing multidimensional uncertainty.

Quantifying and comparing statistical dependence between random variables X and Y across distinct sample sets is challenging under non-stationary distributions, as conventional dependence measures—such as mutual information—lack comparability under distributional shift. Method: We propose Cross Mutual Information (CMI), the first formally defined, cross-sample comparable dependence measure. CMI jointly estimates the joint and marginal distributions from two independent samples to assess consistency in the X–Y dependence structure, thereby overcoming the fundamental incomparability of standard mutual information under distribution shift. Contribution/Results: We establish a tight theoretical connection between CMI and the coefficient of determination (R²) in linear regression. In extensive simulations across diverse nonlinear dependence structures, CMI consistently outperforms baseline methods and exhibits robustness to distribution drift. This framework provides an interpretable, generalizable tool for cross-subject or cross-task dependence modeling in high-dimensional non-stationary domains such as neuroimaging.

Downstream probing only assesses task-relevant information in representations, failing to characterize critical properties—such as equivariance, invariance, and disentanglement—that govern interpretability and generalization; moreover, existing evaluation frameworks lack standardization, modularity, and cross-modal applicability. Method: We propose the first representation quality assessment framework that transcends downstream tasks, employing controlled factorial probe design to systematically quantify informativeness, equivariance, invariance, and disentanglement. The framework is modular, interpretable, and supports cross-modal analysis (e.g., image and speech). Contribution/Results: It establishes the first standardized, multi-dimensional semantic attribute disentanglement protocol. Experiments reveal substantial divergence in intrinsic representation properties—even among models with comparable downstream performance—enabling fine-grained representation understanding, diagnosis, and optimization. This work introduces a novel paradigm and practical toolkit for representation evaluation beyond task-specific metrics.

Existing segmentation evaluation metrics often lack transparency and modularity, making them ill-suited for diverse tasks such as transparent object, specular surface, or lesion segmentation. This work proposes a unified evaluation framework that decomposes metrics into five modular components: prediction representation, target extraction, target matching, score computation, and metric reporting. For the first time, it systematically analyzes the implicit assumptions and design limitations of mainstream binary segmentation metrics through this modular lens. The framework enables task-aware customization of evaluation protocols, reveals evolutionary trajectories among existing metrics, and is accompanied by an open-source toolkit. By offering a principled and interpretable foundation, this approach paves the way for developing more rational and adaptable segmentation evaluation methodologies.

Traditional confidence interval plots in multi-model climate prediction visualization often obscure individual model characteristics, leading users to misinterpret the underlying distribution—frequently assuming normality where none exists. To address this, this work proposes a Weighted Multi-Forecast Visualization (MFV) approach that leverages visual variables such as line width and opacity, combined with a downsampling strategy, to preserve accurate perception of the true predictive distribution while effectively conveying additional forecast attributes. Through a preregistered experimental design and large-scale user study, results demonstrate that MFV significantly improves users’ accuracy in identifying predictive distributions and reduces erroneous assumptions of normality. Moreover, the weighted MFV variant successfully overcomes the expressive limitations of conventional summary-based visualizations without compromising distributional fidelity.

This work addresses the performance bottleneck in joint probabilistic forecasting of multivariate time series arising from the entangled modeling of marginal distributions and cross-series dependencies. To resolve this, the authors propose WIRED, a novel approach that decouples adaptive marginal prediction via expert aggregation from dependency structure reconstruction based on copulas. Specifically, marginal forecasts are generated using a CRPS-weighted adaptive ensemble, while dependencies are captured through Gaussian or Student-t copulas. The framework is rigorously evaluated via rolling-origin validation and ablation studies on both synthetic data and the EuStockMarkets benchmark. Results demonstrate the efficacy of the proposed architecture and reveal limitations in current CRPS-weighting strategies, showing that simple equal-weighting or bootstrap aggregation remains competitive for marginal modeling—offering new insights for designing regularized ensembles in probabilistic forecasting.

This work addresses the misalignment between conventional uncertainty quantification metrics—such as negative log-likelihood and expected calibration error—and the utility of downstream decision-making, which often renders them poor proxies for real-world decision value. To bridge this gap, the paper introduces a “decision-aligned” evaluation principle, systematically exposing the mismatch between widely used scoring rules and common decision tasks. Building on decision theory and proper scoring rules, the authors propose a class of prior-weighted utility-based metrics that directly reflect the impact of predictive uncertainty on decision outcomes. Empirical evaluations across multiple benchmarks and real-world scenarios demonstrate that the proposed metric consistently correlates strongly with actual decision utility, significantly outperforming traditional approaches and offering a principled foundation for decision-relevant uncertainty assessment.

This work addresses the challenge that existing calibration tests for conditional quantile predictors struggle to handle distributional shifts and discrepancies in information sets, lacking feature-aware, continuous monitoring capabilities. The authors propose a distribution-free, game-theoretic sequential auditing framework that formally defines conditional quantile calibration under varying feature information sets—a notion not previously established—and provides finite-time detection guarantees without requiring independent and identically distributed data. By integrating contextual linear betting strategies with nonparametric e-processes, the method enables interpretable, feature-level calibration audits. Empirical evaluations demonstrate that the framework effectively detects significant miscalibration in state-of-the-art time series models, such as Chronos-2, across critical features.