Inference on the Significance of Modalities in Multimodal Generalized Linear Models

2026-01-22

AI Summary

This study addresses the lack of effective statistical inference methods for assessing the significance of individual modalities in high-dimensional multimodal generalized linear models. The authors propose a novel measure, termed “expected relative entropy,” to quantify the information gain contributed by a specific modality after adjusting for others, and develop a bias-corrected test statistic based on this quantity. For the first time in the high-dimensional multimodal setting, the method provides computable confidence intervals and p-values. Theoretical analysis establishes that the proposed estimator is consistent and asymptotically follows a non-central chi-squared distribution. Extensive simulations demonstrate favorable finite-sample performance, and application to real multimodal neuroimaging data successfully identifies significant modalities, confirming the method’s practical utility.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Despite the popular of multimodal statistical models, there lacks rigorous statistical inference tools for inferring the significance of a single modality within a multimodal model, especially in high-dimensional models. For high-dimensional multimodal generalized linear models, we propose a novel entropy-based metric, called the expected relative entropy, to quantify the information gain of one modality in addition to all other modalities in the model. We propose a deviance-based statistic to estimate the expected relative entropy, prove that it is consistent and its asymptotic distribution can be approximated by a non-central chi-squared distribution. That enables the calculation of confidence intervals and p-values to assess the significance of the expected relative entropy for a given modality. We numerically evaluate the empirical performance of our proposed inference tool by simulations and apply it to a multimodal neuroimaging dataset to demonstrate its good performance on various high-dimensional multimodal generalized linear models.

Problem

• multimodal
• statistical inference
• significance
• high-dimensional
• generalized linear models

Innovation

• expected relative entropy
• multimodal generalized linear models
• statistical inference
• deviance-based statistic
• non-central chi-squared distribution