LUCIFER: Language Understanding and Context-Infused Framework for Exploration and Behavior Refinement

2025-06-09

AI Summary

To address model mismatch in dynamic environments caused by rapid obsolescence of prior knowledge, this paper proposes a human–AI collaborative decision-making framework that transforms domain experts’ real-time, context-aware observations into structured signals for autonomous behavior optimization. Methodologically, it introduces: (1) a novel dual-role large language model (LLM) mechanism—simultaneously enabling contextual representation learning and zero-shot exploration guidance; (2) an attention-space alignment mechanism to dynamically couple human contextual knowledge with reinforcement learning (PPO/SAC) processes; and (3) a hierarchical decision architecture integrating LLaMA-3, Qwen2, GPT-4, and context-structured encoding. Evaluated across multiple dynamic simulation tasks, the approach improves exploration efficiency by 37% and achieves significantly higher decision quality than single-layer goal-conditioned policies, empirically validating the critical role of contextual injection in enhancing system adaptability.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

In dynamic environments, the rapid obsolescence of pre-existing environmental knowledge creates a gap between an agent's internal model and the evolving reality of its operational context. This disparity between prior and updated environmental valuations fundamentally limits the effectiveness of autonomous decision-making. To bridge this gap, the contextual bias of human domain stakeholders, who naturally accumulate insights through direct, real-time observation, becomes indispensable. However, translating their nuanced, and context-rich input into actionable intelligence for autonomous systems remains an open challenge. To address this, we propose LUCIFER (Language Understanding and Context-Infused Framework for Exploration and Behavior Refinement), a domain-agnostic framework that integrates a hierarchical decision-making architecture with reinforcement learning (RL) and large language models (LLMs) into a unified system. This architecture mirrors how humans decompose complex tasks, enabling a high-level planner to coordinate specialised sub-agents, each focused on distinct objectives and temporally interdependent actions. Unlike traditional applications where LLMs are limited to single role, LUCIFER integrates them in two synergistic roles: as context extractors, structuring verbal stakeholder input into domain-aware representations that influence decision-making through an attention space mechanism aligning LLM-derived insights with the agent's learning process, and as zero-shot exploration facilitators guiding the agent's action selection process during exploration. We benchmark various LLMs in both roles and demonstrate that LUCIFER improves exploration efficiency and decision quality, outperforming flat, goal-conditioned policies. Our findings show the potential of context-driven decision-making, where autonomous systems leverage human contextual knowledge for operational success.

Problem

• Bridging gap between outdated agent models and dynamic environments
• Translating human contextual input into autonomous system actions
• Enhancing exploration efficiency and decision quality via LLM integration

Innovation

• Hierarchical decision-making with RL and LLMs
• LLMs as context extractors and exploration facilitators
• Attention space aligns human insights with learning