Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation

2025-10-16

AI Summary

To address inconsistent responses from large language models (LLMs) to semantically equivalent queries in Retrieval-Augmented Generation (RAG) systems, this paper proposes a layer-wise consistency-aware model fusion method. Without requiring extensive human annotation, the approach constructs synthetic triplets and introduces a triplet loss function that explicitly enforces consistency among intermediate-layer activations. It further designs a dynamic weight fusion mechanism guided by inter-layer activation similarity to synergistically integrate knowledge from multiple specialized models. The key innovation lies in explicitly modeling hidden-layer activation consistency as the primary fusion criterion, thereby jointly enhancing semantic robustness and generation stability. Experimental results demonstrate that the fused model improves response similarity—measured via embedding-based cosine similarity—by 47.5% over strong baselines, significantly boosting output consistency and reliability in industrial-scale RAG deployments.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Retrieval-Augmented Generation (RAG) systems leverage Large Language Models (LLMs) to generate accurate and reliable responses that are grounded in retrieved context. However, LLMs often generate inconsistent outputs for semantically equivalent inputs, a problem compounded by the scarcity of consistency-focused training data and the limitations of current fine-tuning techniques in enhancing output consistency. We propose a new approach combining systematic synthetic data generation, triplet loss for better embeddings, and a novel layer-wise model merging approach. Using consistency-aware weights derived from intermediate layer activations, our method effectively integrates knowledge from specialized models. Experimental results how that our merged model significantly enhances output consistency, achieving a ~47.5% improvement in response similarity over the baseline, thus offering a practical solution for increasing the reliability of an industrial RAG system.

Problem

• Addresses inconsistent LLM outputs for equivalent semantic inputs
• Solves scarcity of consistency-focused training data limitations
• Improves reliability of industrial RAG system generation

Innovation

• Layer-wise model merging with consistency-aware weights
• Systematic synthetic data generation for training
• Triplet loss optimization for improved embeddings