Testing and Evaluation of Agentic AI Systems In Military Command and Control
研究针对军事指挥控制中代理AI系统的测试与评估问题，通过分析240个实践案例，提出保障声明并探讨现有方法的有效性。
0 citationsRead paper
Selected work
Representative Papers
Monitoring Agentic Systems Before They're Reliable
This study addresses the challenge of effectively monitoring early-stage agent systems, where structural flaws often obscure task-level errors. The authors propose a three-dimensional (quality, suitability, efficiency) and three-granularity (intra-run, inter-run, structural) monitoring and triaging framework tailored for low-maturity agent systems. They introduce a novel system maturity staging model based on the coefficient of variation and monitoring granularity, integrated with a severity classification adapted from FMEA to guide human review. The resulting transferable monitoring architecture supports document-driven, multi-stage workflows, enhanced by a synthetic testbed with controlled error injection. Experimental results demonstrate that structural defects significantly mask task-level signals; 97% of issues can be automatically traced, with only 2% requiring human intervention, and each granularity level precisely identifies its corresponding defect type (coefficients of variation: 0.02, 1.25, and 0.00, respectively).
0 citationsRead paper
Recent publications
Latest Papers
Testing and Evaluation of Agentic AI Systems In Military Command and Control
研究针对军事指挥控制中代理AI系统的测试与评估问题，通过分析240个实践案例，提出保障声明并探讨现有方法的有效性。
0 citationsRead paper
Monitoring Agentic Systems Before They're Reliable
This study addresses the challenge of effectively monitoring early-stage agent systems, where structural flaws often obscure task-level errors. The authors propose a three-dimensional (quality, suitability, efficiency) and three-granularity (intra-run, inter-run, structural) monitoring and triaging framework tailored for low-maturity agent systems. They introduce a novel system maturity staging model based on the coefficient of variation and monitoring granularity, integrated with a severity classification adapted from FMEA to guide human review. The resulting transferable monitoring architecture supports document-driven, multi-stage workflows, enhanced by a synthetic testbed with controlled error injection. Experimental results demonstrate that structural defects significantly mask task-level signals; 97% of issues can be automatically traced, with only 2% requiring human intervention, and each granularity level precisely identifies its corresponding defect type (coefficients of variation: 0.02, 1.25, and 0.00, respectively).
0 citationsRead paper