The World Leaks the Future: Harness Evolution for Future Prediction Agents

2026-04-17

AI Summary

This work addresses the challenge of making accurate predictions in scenarios requiring critical decisions before outcomes are known, where existing methods struggle to effectively leverage dynamic information generated during reasoning. The authors propose Milkyway, a system that introduces an “internal feedback” mechanism and a “backtesting” strategy to continuously refine predictions on unresolved questions without updating the underlying large language model. Built upon a self-evolving agent architecture, Milkyway integrates a persistent prediction-augmentation module with temporal contrastive learning to dynamically track predictive factors, gather supporting evidence, and quantify uncertainty. Evaluated on the FutureX and FutureWorld benchmarks, Milkyway achieves substantial performance gains, improving scores from 44.07 to 60.90 and from 62.22 to 77.96, respectively, significantly outperforming current state-of-the-art approaches.

Technology Category

• Multiagent Systems: Multiagent Learning
• Reasoning under Uncertainty: Sequential Decision Making
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Many consequential decisions must be made before the relevant outcome is known. Such problems are commonly framed as \emph{future prediction}, where an LLM agent must form a prediction for an unresolved question using only the public information available at the prediction time. The setting is difficult because public evidence evolves while useful supervision arrives only after the question is resolved, so most existing approaches still improve mainly from final outcomes. Yet final outcomes are too coarse to guide earlier factor tracking, evidence gathering and interpretation, or uncertainty handling. When the same unresolved question is revisited over time, temporal contrasts between earlier and later predictions can expose omissions in the earlier prediction process; we call this signal \emph{internal feedback}. We introduce \emph{Milkyway}, a self-evolving agent system that keeps the base model fixed and instead updates a persistent \emph{future prediction harness} for factor tracking, evidence gathering and interpretation, and uncertainty handling. Across repeated predictions on the same unresolved question, \emph{Milkyway} extracts internal feedback and writes reusable guidance back into the harness, so later predictions on that question can improve before the outcome is known. After the question is resolved, the final outcome provides a \emph{retrospective check} before the updated harness is carried forward to subsequent questions. On FutureX and FutureWorld, Milkyway achieves the best overall score among the compared methods, improving FutureX from 44.07 to 60.90 and FutureWorld from 62.22 to 77.96.

Problem

• future prediction
• internal feedback
• uncertainty handling
• evidence gathering
• factor tracking

Innovation

• future prediction
• internal feedback
• self-evolving agent
• prediction harness
• temporal contrast