SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets

2026-07-09

AI Summary

This study addresses critical challenges in decentralized energy markets, where autonomous agents, despite enhancing utility, often compromise system safety and generate spurious liquidity by neglecting physical constraints, compounded by the absence of evaluation frameworks that jointly consider performance and trustworthiness. To bridge this gap, we propose SolarChain-Eval—the first multidimensional trustworthy evaluation framework integrating physical constraints and transparent intervention logs. We model market governance as a Gymnasium-compatible Markov decision process and introduce an LLM-driven planner and auditor to enable risk-aware interventions and full traceability. Experimental results demonstrate that purely reinforcement learning–based agents, while efficient, exhibit unsafe behaviors; omitting physics-based penalties leads to ineffective actions; and while the LLM audit layer enhances auditability and mitigates certain risks, it cannot substitute for well-designed reward functions.

Abstract

As agentic AI systems are increasingly applied to cyber-physical environments, their evaluation requires assessment of both task performance and trustworthiness. In decentralized energy markets, autonomous agents may improve market utility, but may also exploit invalid physical data, create artificial liquidity, and produce unstable governance decisions. Therefore, we propose SolarChain-Eval, a physics-constrained benchmark for evaluating trustworthy economic agents. It formulates market governance as a Gymnasium-compatible Markov Decision Process, where agents make hourly decisions. SolarChain-Eval evaluates each policy across multiple dimensions, including market utility, physical safety, slippage, action smoothness, spatial fairness, and auditability. To support agentic evaluation, SolarChain-Eval incorporates an LLM-based Planner/Auditor layer. The Planner defines episode-level action bounds and audit rules, while the Auditor reviews and revises high-risk actions. All interventions are recorded through structured logs, including trigger signals, proposed actions, revised actions, and audit rationales. Experiments with static, random, myopic, RL, and RL+LLM policies reveal a clear utility-safety trade-off. RL agents improve market utility but can still produce unsafe behavior. When the physics penalty is removed, reward-maximizing agents exploit invalid generation and increase artificial liquidity. The LLM Planner/Auditor improves auditability and mitigates selected risks, but it cannot fully compensate for a misspecified reward function. These results indicate that trustworthy agentic AI evaluation requires both physical constraints and transparent intervention traces. We release data and code as open access on GitHub for replicability.

Problem

• trustworthy AI
• decentralized energy markets
• physics-constrained evaluation
• economic agents
• agentic safety

Innovation

• physics-constrained benchmark
• trustworthy agentic AI
• decentralized energy markets
• LLM-based Planner/Auditor
• Markov Decision Process