🤖 AI Summary
Existing hallucination detection methods focus narrowly on factual consistency, overlooking potential creative value and struggling to balance accuracy with creativity across diverse scientific tasks. Method: We propose HIC-Bench—a novel benchmark framework that systematically distinguishes *Intelligent Hallucination* (IH) from *Defective Hallucination* (DH). It evaluates both dimensions across ten open-ended scientific innovation tasks using a dual-axis metric: creativity (integrating Torrance Tests of Creative Thinking with hallucination-specific dimensions) and factual deviation. Innovations include an IH/DH binary classification paradigm, Dynamic Hallucination Prompting (DHP), a multidimensional metrics matrix, cross-disciplinary task design, ensemble evaluation by multiple LLMs, and human validation. Results: IH and DH exhibit a nonlinear relationship; creativity and factual accuracy can be jointly optimized; and hallucinations—when appropriately structured—can serve as catalysts for scientific innovation.
📝 Abstract
Hallucinations in large language models (LLMs) are commonly regarded as errors to be minimized. However, recent perspectives suggest that some hallucinations may encode creative or epistemically valuable content, a dimension that remains underquantified in current literature. Existing hallucination detection methods primarily focus on factual consistency, struggling to handle heterogeneous scientific tasks and balance creativity with accuracy. To address these challenges, we propose HIC-Bench, a novel evaluation framework that categorizes hallucinations into Intelligent Hallucinations (IH) and Defective Hallucinations (DH), enabling systematic investigation of their interplay in LLM creativity. HIC-Bench features three core characteristics: (1) Structured IH/DH Assessment. using a multi-dimensional metric matrix integrating Torrance Tests of Creative Thinking (TTCT) metrics (Originality, Feasibility, Value) with hallucination-specific dimensions (scientific plausibility, factual deviation); (2) Cross-Domain Applicability. spanning ten scientific domains with open-ended innovation tasks; and (3) Dynamic Prompt Optimization. leveraging the Dynamic Hallucination Prompt (DHP) to guide models toward creative and reliable outputs. The evaluation process employs multiple LLM judges, averaging scores to mitigate bias, with human annotators verifying IH/DH classifications. Experimental results reveal a nonlinear relationship between IH and DH, demonstrating that creativity and correctness can be jointly optimized. These insights position IH as a catalyst for creativity and reveal the ability of LLM hallucinations to drive scientific innovation.Additionally, the HIC-Bench offers a valuable platform for advancing research into the creative intelligence of LLM hallucinations.