Skilled AI Agents for Embedded and IoT Systems Development

2026-03-19

AI Summary

This work addresses the challenge that existing AI methods struggle to jointly model the tight coupling between software logic and physical hardware behavior in hardware-in-the-loop (HIL) development of embedded and IoT systems, often leading to deployment failures. To tackle this, the authors propose a skill-oriented agent architecture tailored for HIL scenarios and introduce IoT-SkillsBench, a novel real-hardware evaluation benchmark. The framework systematically assesses AI agents across multiple platforms, peripherals, and task complexities through three agent configurations enhanced by skill augmentation, structured expert knowledge injection, and real-hardware validation. Experimental results demonstrate that, over 378 real-world deployments, agents equipped with human-expert-derived skills achieve near-perfect cross-platform task success rates, substantially outperforming baseline approaches.

Technology Category

• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Large language models (LLMs) and agentic systems have shown promise for automated software development, but applying them to hardware-in-the-loop (HIL) embedded and Internet-of-Things (IoT) systems remains challenging due to the tight coupling between software logic and physical hardware behavior. Code that compiles successfully may still fail when deployed on real devices because of timing constraints, peripheral initialization requirements, or hardware-specific behaviors. To address this challenge, we introduce a skills-based agentic framework for HIL embedded development together with IoT-SkillsBench, a benchmark designed to systematically evaluate AI agents in real embedded programming environments. IoT-SkillsBench spans three representative embedded platforms, 23 peripherals, and 42 tasks across three difficulty levels, where each task is evaluated under three agent configurations (no-skills, LLM-generated skills, and human-expert skills) and validated through real hardware execution. Across 378 hardware validated experiments, we show that concise human-expert skills with structured expert knowledge enable near-perfect success rates across platforms.

Problem

• hardware-in-the-loop
• embedded systems
• IoT
• AI agents
• LLMs

Innovation

• skills-based agentic framework
• hardware-in-the-loop (HIL)
• IoT-SkillsBench
• embedded systems
• expert knowledge integration