Can Agent Conquer Web? Exploring the Frontiers of ChatGPT Atlas Agent in Web Games

2025-10-30

AI Summary

This study systematically evaluates the capabilities—and limitations—of the ChatGPT Atlas agent in concurrent real-time interaction and logical reasoning tasks, using dynamic web-based games as a novel benchmark. Method: We construct a browser-native evaluation framework comprising four representative environments—T-Rex Runner, Sudoku, Flappy Bird, and Stein.world—leveraging Atlas’s intrinsic web perception, intent parsing, and end-to-end cursor/keyboard control capabilities to enable fully autonomous, tool-free gameplay; performance is quantified uniformly via in-game scores. Results: Atlas significantly outperforms human baselines on logic-intensive tasks (e.g., Sudoku), yet fails catastrophically on millisecond-critical real-time control tasks (e.g., Flappy Bird), often failing to clear the first level. Contribution: We introduce the first browser-game evaluation paradigm specifically designed for LLM-based agents, empirically revealing a fundamental capability imbalance: strong symbolic reasoning coupled with weak temporal control—a critical insight for advancing agent architecture and training paradigms.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

OpenAI's ChatGPT Atlas introduces new capabilities for web interaction, enabling the model to analyze webpages, process user intents, and execute cursor and keyboard inputs directly within the browser. While its capacity for information retrieval tasks has been demonstrated, its performance in dynamic, interactive environments remains less explored. In this study, we conduct an early evaluation of Atlas's web interaction capabilities using browser-based games as test scenarios, including Google's T-Rex Runner, Sudoku, Flappy Bird, and Stein.world. We employ in-game performance scores as quantitative metrics to assess performance across different task types. Our results show that Atlas performs strongly in logical reasoning tasks like Sudoku, completing puzzles significantly faster than human baselines, but struggles substantially in real-time games requiring precise timing and motor control, often failing to progress beyond initial obstacles. These findings suggest that while Atlas demonstrates capable analytical processing, there remain notable limitations in dynamic web environments requiring real-time interaction. The website of our project can be found at https://atlas-game-eval.github.io.

Problem

• Evaluating Atlas agent's web interaction in dynamic gaming environments
• Assessing performance differences between logical and real-time tasks
• Identifying limitations in real-time motor control for web agents

Innovation

• Browser-based cursor and keyboard input execution
• Webpage analysis and user intent processing
• Game performance scores as quantitative metrics