Large Language Models as Innovators: A Framework to Leverage Latent Space Exploration for Novelty Discovery

2025-07-18

AI Summary

Large language models (LLMs) face a fundamental trade-off in generative creativity: they tend to reproduce memorized training data, struggling to simultaneously ensure novelty and semantic relevance. Existing prompt-engineering approaches rely on domain-specific heuristics, exhibiting poor generalizability and limited scalability. To address this, we propose a model-agnostic, rule-free creative generation framework operating in the latent embedding space of LLMs. Our method constructs a continuous, differentiable semantic manifold over LLM embeddings, enabling controllable conceptual divergence and semantics-guided exploration. The framework natively supports cross-domain and multimodal inputs, automatically adapting to task-specific requirements while substantially reducing reliance on manual prompt design. Empirical evaluation across diverse creative tasks—including brainstorming, analogical reasoning, and conceptual blending—demonstrates significant improvements in both novelty and coherence. Results underscore its viability as a general-purpose, human-AI collaborative innovation partner.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Innovative idea generation remains a core challenge in AI, as large language models (LLMs) often struggle to produce outputs that are both novel and relevant. Despite their fluency, LLMs tend to replicate patterns seen during training, limiting their ability to diverge creatively without extensive prompt engineering. Prior work has addressed this through domain-specific heuristics and structured prompting pipelines, but such solutions are brittle and difficult to generalize. In this paper, we propose a model-agnostic latent-space ideation framework that enables controlled, scalable creativity by navigating the continuous embedding space of ideas. Unlike prior methods, our framework requires no handcrafted rules and adapts easily to different domains, input formats, and creative tasks. This paper introduces an early-stage prototype of our method, outlining the conceptual framework and preliminary results highlighting its potential as a general-purpose co-ideator for human-AI collaboration.

Problem

• LLMs struggle to generate novel and relevant outputs
• Existing solutions are brittle and hard to generalize
• Need for controlled, scalable creativity in AI ideation

Innovation

• Leverages latent-space exploration for novelty
• Model-agnostic framework for scalable creativity
• No handcrafted rules, adapts to domains