Spectral Analysis of Molecular Kernels: When Richer Features Do Not Guarantee Better Generalization

2025-10-15

AI Summary

This study challenges the conventional assumption that spectral richness of molecular kernels inherently improves generalization, investigating whether increased spectral diversity actually enhances predictive performance. Method: Leveraging the QM9 dataset, we conduct the first comprehensive spectral analysis across multiple molecular kernels, integrating kernel ridge regression, spectral truncation, and four spectral complexity measures to systematically evaluate the relationship between spectral characteristics and prediction accuracy across seven molecular property prediction tasks. Contribution/Results: We find a significant negative correlation between spectral richness and prediction accuracy; retaining only the top 2% of dominant eigenvalues recovers nearly all predictive performance, demonstrating that essential information is highly concentrated in a low-dimensional spectral subspace. These results reveal that spectral redundancy may impede generalization, providing a new design principle for molecular kernels and underscoring the theoretical value of spectral analysis in elucidating generalization mechanisms of graph kernels.

Technology Category

• Machine Learning: Kernel Methods
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Understanding the spectral properties of kernels offers a principled perspective on generalization and representation quality. While deep models achieve state-of-the-art accuracy in molecular property prediction, kernel methods remain widely used for their robustness in low-data regimes and transparent theoretical grounding. Despite extensive studies of kernel spectra in machine learning, systematic spectral analyses of molecular kernels are scarce. In this work, we provide the first comprehensive spectral analysis of kernel ridge regression on the QM9 dataset, molecular fingerprint, pretrained transformer-based, global and local 3D representations across seven molecular properties. Surprisingly, richer spectral features, measured by four different spectral metrics, do not consistently improve accuracy. Pearson correlation tests further reveal that for transformer-based and local 3D representations, spectral richness can even have a negative correlation with performance. We also implement truncated kernels to probe the relationship between spectrum and predictive performance: in many kernels, retaining only the top 2% of eigenvalues recovers nearly all performance, indicating that the leading eigenvalues capture the most informative features. Our results challenge the common heuristic that "richer spectra yield better generalization" and highlight nuanced relationships between representation, kernel features, and predictive performance. Beyond molecular property prediction, these findings inform how kernel and self-supervised learning methods are evaluated in data-limited scientific and real-world tasks.

Problem

• Analyzing spectral properties of molecular kernels for generalization insights
• Investigating why richer kernel features don't guarantee better accuracy
• Challenging the heuristic that spectral richness improves generalization performance

Innovation

• Spectral analysis of molecular kernels on QM9 dataset
• Truncated kernels retain top 2% eigenvalues for performance
• Richer spectral features do not guarantee better accuracy