🤖 AI Summary
This work addresses the challenge of enabling agents to implicitly convey internal state information through their actions in communication-constrained environments, thereby facilitating accurate external observation. The authors propose a method that directly embeds state observability into the reinforcement learning reward function, guiding the policy to actively expose informative state signals while preserving primary task performance. By integrating reinforcement learning with observability-aware optimization, the approach successfully trains control policies with high observability in an aerial tracking task. Experimental results demonstrate that the resulting policies significantly enhance the accuracy of state reconstruction by external observers, with negligible degradation to the main task performance.
📝 Abstract
Physical or operational constraints often impose communications limitations on autonomous agents. Such limitations complicate monitoring or multiagent coordination. Even when strong communications are absent, some information may still be available. The remainder of the relevant agent state may be reconstructed via estimation. The actions taken by an agent are a potential source of information -- as the agent interacts with the environment, these actions may be observed even in the absence of explicit communication. We investigate using actions to estimate the state of an agent, using reinforcement learning to develop policies which make the estimation problem more tractable. Policy observability is encouraged through the training reward and is analyzed using simulation of the trained agent. In an aircraft tracking problem a policy with enhanced observability is found that has minimal impact on nominal task performance.