Software Engineering Agents for Embodied Controller Generation : A Study in Minigrid Environments

2025-10-24

AI Summary

This work introduces Software Engineering Agents (SWE-Agents) to embodied control policy generation for the first time, focusing on 20 diverse controller synthesis tasks in MiniGrid. Recognizing the strong dependence of solution discovery on design information, we systematically evaluate how varying information access—such as full environment source-code visibility versus interactive exploration—affects agent performance. We propose Mini-SWE-Agent, a novel architecture integrating static code analysis, dynamic environment interaction, and multi-turn feedback loops, establishing the first SWE-Agent evaluation framework tailored to embodied control. Experiments demonstrate that combining open-source access with interactive exploration significantly improves task success rate (+32.5%), confirming the critical role of information accessibility in reasoning quality. This study establishes a new paradigm and benchmark for program synthesis and controllable policy generation in embodied AI.

Technology Category

• Intelligent Robots: Embodied AI
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Software Engineering Agents (SWE-Agents) have proven effective for traditional software engineering tasks with accessible codebases, but their performance for embodied tasks requiring well-designed information discovery remains unexplored. We present the first extended evaluation of SWE-Agents on controller generation for embodied tasks, adapting Mini-SWE-Agent (MSWEA) to solve 20 diverse embodied tasks from the Minigrid environment. Our experiments compare agent performance across different information access conditions: with and without environment source code access, and with varying capabilities for interactive exploration. We quantify how different information access levels affect SWE-Agent performance for embodied tasks and analyze the relative importance of static code analysis versus dynamic exploration for task solving. This work establishes controller generation for embodied tasks as a crucial evaluation domain for SWE-Agents and provides baseline results for future research in efficient reasoning systems.

Problem

• Evaluating SWE-Agents for embodied controller generation tasks
• Comparing agent performance with different information access levels
• Analyzing static code analysis versus dynamic exploration importance

Innovation

• Adapted Mini-SWE-Agent for embodied controller generation
• Compared agent performance across information access conditions
• Analyzed static code analysis versus dynamic exploration importance