Approximate full-conformal multi-task regression with reproducing kernels

2026-07-01

AI Summary

This work addresses the challenge of constructing fully conformal prediction regions in multi-task regression, which are typically intractable due to their reliance on an infinite ensemble of predictors. The authors propose a computationally feasible approximation within a vector-valued reproducing kernel Hilbert space, providing theoretical guarantees of achieving the desired coverage level under both known and estimated covariance matrices. In the case of known covariance, they derive an upper bound on the volume of the prediction region and establish its tightness. Empirical evaluations on synthetic data demonstrate that the proposed method substantially outperforms split conformal prediction, yielding significantly smaller prediction regions while maintaining accurate coverage.

Abstract

Multi-task regression aims at jointly solving multiple regression problems, called tasks. Compared to solving each task separately, better performances can be achieved as long as the tasks are sufficiently related. Full-conformal prediction is a framework that formulates a data-dependent prediction-region containing the unknown output-vector at any prescribed confidence level. However, explicit computation of this prediction-region is intractable in general since it requires training infinitely many predictors. The present work focuses on multi-task regression in a Reproducing Kernel Hilbert Space (RKHS) of vector-valued functions. This computational issue is addressed by designing an approximating predictionregion containing the full-conformal one. This construction is carried out in two scenarios: piq when the inter-task covariance-matrix is known, and piiq when this matrix is estimated. In terms of volume, the tightness of this approximation is assessed theoretically by means of an upper-bound in the first scenario. It is also empirically proved to improve upon the split-conformal prediction on synthetic data in both scenarios.

Problem

• multi-task regression
• full-conformal prediction
• prediction region
• computational intractability
• Reproducing Kernel Hilbert Space

Innovation

• approximate full-conformal prediction
• multi-task regression
• reproducing kernel Hilbert space
• vector-valued RKHS
• prediction region