CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design

2026-09-14

AI Summary

为解决现有基准对专业工程工作流程覆盖不足的问题，通过引入CADWorld基准来评估计算机使用代理在FreeCAD中长交互周期内的表现。

Abstract

Computer-use agents are increasingly evaluated in realistic desktop environments, but existing benchmarks provide limited coverage of professional engineering workflows whose outputs are persistent, structured artifacts. Mechanical computer-aided design (CAD) is a particularly demanding setting: an agent must manipulate geometry and constraints over long interaction horizons while producing a native project whose dimensions, construction structure, and downstream engineering state remain valid. We introduce \textbf{CADWorld}, a benchmark for long-horizon computer use in FreeCAD. CADWorld contains 200 tasks spanning 11 mechanical-CAD workflow categories, including sketching, part modeling, assembly, CAM, FEM, measurement, mesh processing, and technical drawing. Agents operate through screenshots and GUI actions, while success is determined by task-specific executable checks over saved FreeCAD artifacts and auxiliary outputs, covering geometric properties, parametric structure, constraints, manufacturing state, and simulation results. Across seven current agents on the full benchmark, the strongest agent achieves 17.5\% success, compared with an 87.0\% expert reference pass. We find that weaker agents often fail before producing a valid artifact, whereas stronger agents increasingly fail on structural, geometric, and construction-process requirements. CADWorld therefore exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows. Project accessible at https://cad-world.github.io.

Problem

• Computer-Use Agents
• Mechanical CAD
• Long-Horizon Interactions
• Engineering Workflows
• Persistent Artifacts

Innovation

• CADWorld
• long-horizon computer use
• FreeCAD
• executable checks
• engineering workflows