Attributing Preprocessing Invariance in Spectral Foundation Models

2026-08-14

AI Summary

This study addresses the issue of erroneous invariance attribution in preprocessing for spectral foundation models by proposing a decoupled evaluation paradigm that distinguishes the contributions of normalization from representation learning. Through Raman spectroscopy analysis, controlled experiments, and numerical validation, we demonstrate that the invariance observed in multimodal models primarily stems from parameter-free preprocessing normalization rather than learned representations. These findings reveal that current models fail to surpass simple normalization baselines, effectively correcting prevailing cognitive biases regarding spectral representation learning capabilities within the field. Consequently, this work establishes a new benchmark for evaluation methodologies, emphasizing the necessity of rigorously isolating preprocessing effects when assessing the true efficacy of spectral foundation models.

Abstract

Preprocessing invariance is an appealing goal for spectral foundation models: a frozen model should remain useful when laboratories preprocess spectra differently. It is usually measured by training a classifier under one preprocessing pipeline and testing it under another, with preserved accuracy read as evidence of learning. We revisit that reading, using a Raman foundation model as a case study. Such models normalize their inputs before any learned parameter is applied. If that normalization maps two differently preprocessed spectra to the same vector, the encoder receives identical inputs, so the invariance cannot be attributed to learning. For a normalization that uses each spectrum's own statistics, this happens exactly when one spectrum is a positive multiple of the other plus a constant. Several standard preprocessing operations take that form. The encoder should therefore be measured against the normalization alone, which has no learned parameters. On six Raman evaluation datasets, the model does not measurably outperform its own normalization. It improves on raw spectra, but so does the normalization alone. Training does improve the encoder over random initialization, and a controlled experiment shows that it learns to ignore a transformation only when that transformation reaches it. A numerical test settles which transformations a given normalization removes. Across released systems in five modalities, most normalizations already remove transformations of that form, and several of those systems claim that invariance as learned. Replicating the comparison on two of them shows no gain either.

Problem

• Spectral Foundation Models
• Preprocessing Invariance
• Normalization
• Attribution
• Raman Spectroscopy

Innovation

• Preprocessing Invariance
• Spectral Foundation Models
• Normalization Artifact
• Evaluation Methodology
• Raman Spectroscopy