MELCOT: A Hybrid Learning Architecture with Marginal Preservation for Matrix-Valued Regression

2025-09-27

AI Summary

To address the loss of spatial structure and excessive memory overhead in high-dimensional matrix-valued regression, this paper proposes MELCOT—a hybrid architecture integrating a Marginal Estimation (ME) module and a Learnable Cost Optimal Transport (LCOT) module. The ME module preserves local spatial structure via marginal modeling, while the LCOT module employs deep neural networks to adaptively learn transport costs, enabling effective global feature representation. By synergistically combining classical statistical modeling with deep learning, MELCOT achieves both low memory footprint and high computational efficiency without compromising expressive power. Extensive experiments across diverse matrix regression tasks—including brain imaging, remote sensing, and recommender systems—demonstrate that MELCOT consistently outperforms state-of-the-art baselines, validating its structural fidelity, generalizability, and practical utility.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Regression is essential across many domains but remains challenging in high-dimensional settings, where existing methods often lose spatial structure or demand heavy storage. In this work, we address the problem of matrix-valued regression, where each sample is naturally represented as a matrix. We propose MELCOT, a hybrid model that integrates a classical machine learning-based Marginal Estimation (ME) block with a deep learning-based Learnable-Cost Optimal Transport (LCOT) block. The ME block estimates data marginals to preserve spatial information, while the LCOT block learns complex global features. This design enables MELCOT to inherit the strengths of both classical and deep learning methods. Extensive experiments across diverse datasets and domains demonstrate that MELCOT consistently outperforms all baselines while remaining highly efficient.

Problem

• Matrix-valued regression with preserved spatial structure
• Hybrid architecture combining classical and deep learning
• Efficient high-dimensional regression with global feature learning

Innovation

• Hybrid architecture combines classical and deep learning
• Marginal estimation preserves spatial structure in data
• Learnable-cost optimal transport captures global features