xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems

2026-09-07

AI Summary

为解决LLM在处理实际生活问题时面临的开放性、非正式及上下文依赖请求的问题，通过xDailyBench这一包含248个任务的新基准进行评估。

Abstract

Large language models (LLMs) are increasingly used for everyday assistance, yet existing benchmarks only partially reflect the requests users naturally make in practice. Real-world requests are often open-ended, casually specified, and context-dependent, requiring models not only to follow explicit instructions but also to infer unstated needs from user background and situational context. We introduce xDailyBench, a benchmark of 248 carefully curated tasks spanning 51 scenarios across personal life, white-collar work, learning and research, and cross-domain activities. The tasks are grounded in requests that users have actually completed or genuinely intended to accomplish with AI, and are evaluated with fine-grained binary rubrics covering both explicit and implicit requirements. We evaluate 11 frontier models under standardized agentic settings. The best models achieve a task-level score of 75.6\%, while all models perform substantially worse on implicit than explicit requirements, with gaps no less than 9 percentage points. These results reveal implicit requirement inference as a persistent bottleneck for reliably satisfying real-world everyday user needs.

Problem

• large language models
• real-world requests
• implicit requirements

Innovation

• xDailyBench
• implicit requirement inference
• real-world everyday user needs