Evaluating Agentic Harness Systems for Autonomous Computational Pathology

2026-07-01

AI Summary

This work proposes ACP-Bench, the first end-to-end agent evaluation benchmark for autonomous computational pathology, designed to assess the capability of intelligent agents to translate high-level pathological analysis objectives into executable, traceable, and clinically compliant workflows. The framework integrates nine foundation models and three types of code-generating agents, producing 369 execution trajectories across 41 pathological tasks, and incorporates expert process auditing, diagnostic performance evaluation, and safety review mechanisms. Experimental results indicate that current agents perform relatively well in workflow initiation and report generation but exhibit significant weaknesses in critical stages such as tool invocation, result binding, and reflective correction, leading to low end-to-end task completion rates. These findings underscore the necessity of enhancing system reliability and consistency prior to clinical deployment.

Abstract

Autonomous computational pathology (ACP) converts high-level pathology analysis goals into executable, traceable and clinically bounded workflows. Realizing this capability requires adapting general agentic harness systems to pathology-specific tasks, tools, evidence standards and clinical claim boundaries. We contribute ACP-Bench, a framework that adapts existing harness systems from computational pathology support toward ACP workflow capability. ACP-Bench evaluates 41 pathology workflow tasks, including 24 biomarker, 7 morphology and 10 prognosis tasks spanning 6 body-system groups and 9 endpoint families. The benchmark evaluates 9 models and 3 harness groups (Claude Code, Codex and Open Code), yielding 369 complete trajectories. ACP-Bench evaluates each trajectory across workflow execution, diagnostic performance and clinical-boundary alignment, combining expert-adjudicated process audits, diagnostic assessment and pathologist-validated safety review. Across evaluated systems, workflow initiation, task interpretation and diagnostic reporting were more mature than tool-bound execution, result binding and reflective workflow revision, and formal end-to-end completion remained rare. ACP-Bench provides a reusable standard for auditing whether agentic systems can operationalize pathology workflows before claims of reliable clinical autonomy.

Problem

• Autonomous Computational Pathology
• Agentic Systems
• Clinical Workflow
• Pathology Benchmarking
• Clinical Autonomy

Innovation

• Autonomous Computational Pathology
• Agentic Harness Systems
• ACP-Bench
• Clinical Workflow Evaluation
• Diagnostic Autonomy