MIH-TCCT: Mitigating Inconsistent Hallucinations in LLMs via Event-Driven Text-Code Cyclic Training

2025-02-13

AI Summary

Large language models (LLMs) frequently exhibit cross-context inconsistent hallucinations in natural language processing (NLP) tasks, undermining reliability and trustworthiness. Method: This paper proposes an event-driven text-to-code cyclic training framework that establishes a bidirectional closed loop: text-to-code generation followed by code-to-text reverse distillation, enabling logical consistency transfer. It introduces an innovative event-alignment mechanism and a task-agnostic, multi-stage cyclic fine-tuning paradigm—overcoming the task dependency inherent in existing synthetic-data-based approaches. Contribution/Results: Evaluated on three mainstream LLMs, the method significantly reduces inconsistency hallucination rates across two representative NLP tasks while preserving original task performance without degradation. It offers a generalizable, lightweight, and architecture-agnostic solution for hallucination mitigation, advancing the state of the art in reliable LLM deployment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent methodologies utilizing synthetic datasets have aimed to address inconsistent hallucinations in large language models (LLMs); however,these approaches are primarily tailored to specific tasks, limiting their generalizability. Inspired by the strong performance of code-trained models in logic-intensive domains, we propose a novel framework that leverages event-based text to generate corresponding code and employs cyclic training to transfer the logical consistency of code to natural language effectively. Our method significantly reduces inconsistent hallucinations across three leading LLMs and two categories of natural language tasks while maintaining overall performance. This framework effectively alleviates hallucinations without necessitating adaptation to downstream tasks, demonstrating generality and providing new perspectives to tackle the challenge of inconsistent hallucinations.

Problem

• Reduces inconsistent hallucinations in LLMs
• Uses event-driven text-code cyclic training
• Enhances logical consistency in natural language

Innovation

• event-driven text-code cyclic training
• leverages event-based text generation
• transfers logical consistency effectively