value prediction

This paper addresses two-stage stochastic optimization problems with contextual information. Method: We propose a novel “single-scenario optimal solving” paradigm: under fixed recourse matrices and linear second-stage costs, we theoretically establish for the first time that such problems reduce to point-estimate optimization over a single scenario. We develop a joint learning-and-optimization framework featuring a decision-optimal structured loss function, which trains a parametric forecasting model to produce point predictions explicitly tailored to optimal decisions. Contribution/Results: On synthetic inventory control and real-world bike-sharing dispatch tasks, our approach reduces decision cost by 12–23% compared to conventional “predict-then-optimize” pipelines and distributional forecasting baselines, while cutting computational overhead by an order of magnitude—significantly enhancing end-to-end decision-making efficacy.

This work addresses a critical limitation in existing latent-variable world models, which rely on average prediction error over training data for training and selection—a metric that fails to reflect actual controller performance due to a mismatch between the evaluation distribution and the distribution queried by the planner. The authors propose instead to center model assessment on the discrepancy between predicted and true costs over states reachable by the planner. They establish, for the first time, a rigorous theoretical link between this discrepancy and control suboptimality, proving it provides a valid upper bound on performance loss, whereas conventional prediction errors neither bound nor track performance. Leveraging control theory, spectral analysis, and non-normal operator theory, they decompose the discrepancy into an intrinsic manifold residual and an off-manifold divergence term, and introduce a fidelity score to quantify alignment of the planner’s reachable distribution. Experiments on synthetic systems and model predictive control confirm that the proposed metric reliably tracks control performance, while single-step prediction error shows virtually no correlation.

Addressing the dual challenges of inherent stochasticity and non-differentiable evaluation metrics in physical spatiotemporal forecasting, this paper proposes a novel model-based reinforcement learning paradigm that reformulates prediction as sequential planning. Methodologically, we construct a generative world model to simulate high-fidelity, diverse future states and employ domain-specific non-differentiable metrics—such as extreme-event hit rate—as sparse reward signals. We design a beam-search–guided, reward-driven imagination mechanism and introduce an iterative pseudo-labeling self-training strategy. Crucially, our framework enables end-to-end optimization of non-differentiable objectives without gradient approximation. Experiments demonstrate substantial reductions in overall prediction error alongside marked improvements in long-tail event detection. This work establishes a new pathway toward interpretable and robust forecasting for complex physical systems.

This work investigates how to achieve optimal policies in Markov decision processes (MDPs) that incorporate future information—such as reference trajectories or predictions—by leveraging model predictive control (MPC). The authors formulate MPC as a class of parameterized policies and train them end-to-end via reinforcement learning. Their key contribution lies in establishing, for the first time, the precise structural conditions under which MPC can exactly represent the optimal value function and policy, thereby providing a theoretical foundation for MPC as a structured function approximator with formal guarantees. Empirical validation on a point-mass racing task with future reference trajectories demonstrates that the proposed approach learns policies approaching optimality, confirming its effectiveness.

This work addresses the challenge that large language model agents struggle with effective forward-looking planning in long-horizon tasks due to the absence of an internalized world model. To overcome this limitation, the authors propose a capability-first, three-stage training framework comprising World-Model Agent Mid-training (WM-AMT), Format-Guided Supervised Fine-Tuning (FE-SFT), and Foresight-Conditioned Reinforcement Learning (FC-RL). This framework enables a single autoregressive model to generate coherent future state rollouts and calibrate the likelihood of plan success. Evaluated on search and mathematical reasoning tasks, the approach significantly outperforms existing baselines, achieving— for the first time—a well-calibrated internal world model with genuine predictive capacity, thereby effectively bridging the gap between output format alignment and planning competence.

This work addresses the absence of a unified closed-loop learning environment that enables agents to continuously learn from real-world events and forecast future outcomes. To bridge this gap, we propose FutureWorld—the first framework that formulates real-time future prediction as a reinforcement learning environment. By integrating a closed-loop mechanism of prediction, outcome realization, and parameter update, FutureWorld effectively prevents answer leakage and supports continual learning. Built upon open-source large language models and grounded in real-world event feedback, the framework establishes a daily-updated benchmark for training and evaluation. Experimental results over consecutive days demonstrate the efficacy of our approach, setting a new state-of-the-art baseline and significantly advancing agents’ predictive capabilities.

This work addresses the high computational cost and limited generalization of traditional state-value-based approaches in planning tasks by proposing a novel supervised learning framework that directly trains a regularized Q-function. The method models the Q-function using a graph neural network and introduces an action-discriminative regularization term to enhance the distinction between actions selected by the teacher policy and those not selected. To our knowledge, this is the first systematic effort in planning-by-learning to replace state-value functions with Q-functions. Evaluated across ten planning domains, the approach significantly outperforms existing state-value-based policies, achieving performance on par with the state-of-the-art planner LAMA-first while substantially reducing per-step inference cost, thereby improving both inference efficiency and policy robustness.

Current evaluation standards for world models suffer from inconsistency and a lack of task-specific focus, particularly in embodied decision-making scenarios, often leading to claims unsupported by empirical evidence. This work proposes a decision-centric evaluation framework featuring a novel L0–L7 assessment hierarchy that clearly distinguishes diagnostic metrics from those measuring decision utility. Emphasizing core capabilities such as counterfactual reasoning, closed-loop rollouts, and policy optimization, the framework introduces a new benchmark protocol centered on action fidelity, policy ranking consistency, and uncertainty calibration. This protocol is operationalized through counterfactual intervention analysis, closed-loop policy rollouts, value prediction tasks, and model exploitability tests, systematically addressing critical gaps in existing evaluation practices and establishing clear, reproducible validation criteria for world model research.

Traditional scenario tree construction methods prioritize probabilistic fidelity but often fail to guarantee strong downstream control performance. This work proposes a novel control-performance-driven paradigm, formulating scenario allocation as an attention-based policy optimization problem. By leveraging reinforcement learning, the approach directly maximizes closed-loop control returns and generates compact branches that emphasize high-impact events under a fixed tree structure. The method incorporates an asymmetric critic to stabilize training and is evaluated against baselines such as Wasserstein reduction. In a risk-averse battery arbitrage task, it consistently achieves the highest returns across varying forecast ensemble sizes, significantly outperforming classical scenario reduction and certainty-equivalent control while demonstrating superior tail-risk management.

This work addresses the limited generalization of reinforcement learning agents to new tasks, which often necessitates training from scratch. To overcome this, the authors propose Outcome-Predictive State Representations (OPSRs) and the OPSR Skill framework, which construct compact, task-agnostic state abstractions and define reusable abstract actions—referred to as skills—on top of these representations. This approach is the first to jointly abstract both states and actions, enabling cross-task skill transfer without requiring task-specific preprocessing, while preserving policy optimality. Empirical results demonstrate that OPSR-based skills significantly accelerate learning across multiple unseen tasks, confirming their strong generalization capability and effectiveness.