Using profiles of cognitive capability to assess AI suitability for workplace tasks

2026-08-26

AI Summary

为解决AI在工作场所任务分配问题，本文提出一种基于共同认知能力集的评估方法，通过分析AI系统和工作任务的认知需求来评估AI适用性。

Abstract

Organisations deploying AI face a scoping problem: which tasks can be automated, which should remain with humans, and which are best shared between the two. Aggregate benchmark scores provide little insight into where systems will succeed or fail in practice, while human judgements of model capabilities quickly become outdated. We introduce a pipeline that profiles agents and tasks using a shared set of core cognitive capabilities. Cognitive capability profiling infers an agent's capabilities from performance on a benchmark battery annotated for the cognitive demands of each item. Task requirements weighting elicits from domain experts the relative importance of these same capabilities for their work. As both use a common set of cognitive dimensions, they can be updated independently as models and roles change, and combined to estimate AI suitability at the level of a domain, organisation, role, or individual duty. We validate capability recovery on synthetic agents, profile six AI systems, and elicit task requirements from 410 employees across six occupational domains. AI systems differed more across cognitive dimensions than across model families, while workplace activities converged on a shared cognitive core. The resulting scores provide a comparative scoping tool for identifying promising candidates for piloting and areas where current systems are unlikely to be well suited. We discuss extending the framework to profile human workers alongside AI systems, moving from AI suitability towards human-machine task allocation.

Problem

• AI suitability
• cognitive capabilities
• workplace tasks
• automation
• human-machine task allocation

Innovation

• Cognitive Capability Profiling
• Task Requirements Weighting
• AI Suitability Estimation