A Survey on Training-free Alignment of Large Language Models

2025-08-12

AI Summary

This study addresses the challenge of value alignment for large language models (LLMs) in resource-constrained settings, where conventional fine-tuning is infeasible—particularly when preserving factual knowledge, avoiding parameter updates, and ensuring compatibility with both closed- and open-source models are required. Method: We systematically survey training-free alignment techniques, proposing the first unified taxonomy spanning three stages: pre-decoding (e.g., prompt engineering), in-decoding (e.g., decoding strategy modulation), and post-decoding (e.g., output correction). Our analysis integrates perspectives from both LLMs and multimodal models to characterize underlying mechanisms and inherent limitations. Contribution: We introduce the first structured, principled taxonomy of training-free alignment methods; provide a reproducible practical guideline; identify concrete open challenges and future research directions; and thereby significantly enhance the safety, universality, and deployment efficiency of alignment techniques across diverse model architectures and operational constraints.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The alignment of large language models (LLMs) aims to ensure their outputs adhere to human values, ethical standards, and legal norms. Traditional alignment methods often rely on resource-intensive fine-tuning (FT), which may suffer from knowledge degradation and face challenges in scenarios where the model accessibility or computational resources are constrained. In contrast, training-free (TF) alignment techniques--leveraging in-context learning, decoding-time adjustments, and post-generation corrections--offer a promising alternative by enabling alignment without heavily retraining LLMs, making them adaptable to both open-source and closed-source environments. This paper presents the first systematic review of TF alignment methods, categorizing them by stages of pre-decoding, in-decoding, and post-decoding. For each stage, we provide a detailed examination from the viewpoint of LLMs and multimodal LLMs (MLLMs), highlighting their mechanisms and limitations. Furthermore, we identify key challenges and future directions, paving the way for more inclusive and effective TF alignment techniques. By synthesizing and organizing the rapidly growing body of research, this survey offers a guidance for practitioners and advances the development of safer and more reliable LLMs.

Problem

• Ensuring LLM outputs align with human values ethically and legally
• Overcoming resource-intensive fine-tuning limitations in model alignment
• Exploring training-free alignment methods for diverse LLM environments

Innovation

• Training-free alignment without heavy retraining
• Leveraging in-context learning and decoding adjustments
• Post-generation corrections for adaptable alignment