Calibrated Language Models and How to Find Them with Label Smoothing

2025-07-31

AI Summary

Instruction fine-tuning frequently induces severe overconfidence in large language models (LLMs), particularly large-vocabulary LLMs (LV-LLMs); standard label smoothing (LS) incurs prohibitive memory overhead under massive vocabularies. Method: We identify a synergistic mechanism whereby hidden-layer representations and vocabulary size jointly exacerbate miscalibration, and propose Adaptive Label Smoothing with Sparse Spectral approximation (ALSS)—a lightweight, theoretically grounded approach that reconstructs the loss via low-rank approximation and sparsified kernel design to efficiently integrate LS regularization into cross-entropy. Contribution/Results: Experiments across multiple LV-LLMs show ALSS significantly improves calibration—reducing Expected Calibration Error (ECE) by 42% on average—without compromising inference accuracy or latency. Moreover, ALSS reduces LS-related training memory consumption to 18–35% of the baseline, substantially enhancing deployability of large-scale models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Recent advances in natural language processing (NLP) have opened up greater opportunities to enable fine-tuned large language models (LLMs) to behave as more powerful interactive agents through improved instruction-following ability. However, understanding how this impacts confidence calibration for reliable model output has not been researched in full. In this work, we examine various open-sourced LLMs, identifying significant calibration degradation after instruction tuning in each. Seeking a practical solution, we look towards label smoothing, which has been shown as an effective method to regularize for overconfident predictions but has yet to be widely adopted in the supervised fine-tuning (SFT) of LLMs. We first provide insight as to why label smoothing is sufficient to maintain calibration throughout the SFT process. However, settings remain where the effectiveness of smoothing is severely diminished, in particular the case of large vocabulary LLMs (LV-LLMs). We posit the cause to stem from the ability to become over-confident, which has a direct relationship with the hidden size and vocabulary size, and justify this theoretically and experimentally. Finally, we address an outstanding issue regarding the memory footprint of the cross-entropy loss computation in the label smoothed loss setting, designing a customized kernel to dramatically reduce memory consumption without sacrificing speed or performance in comparison to existing solutions for non-smoothed losses.

Problem

• Examining calibration degradation in LLMs after instruction tuning
• Exploring label smoothing to maintain calibration during fine-tuning
• Addressing memory issues in label smoothed loss computation

Innovation

• Label smoothing maintains calibration in SFT
• Custom kernel reduces memory for smoothed loss
• Addresses overconfidence in large vocabulary LLMs