Towards the Terminator Economy: Assessing Job Exposure to AI through LLMs

2024-07-27

arXiv.org

AI Summary

This study systematically assesses occupational exposure to and substitution risk from artificial intelligence—particularly large language models (LLMs)—at the task level. Method: We propose a novel, task-level dual-index framework—Task Exposure to AI (TEAI) and Task Replacement by AI (TRAI)—grounded in open-source LLMs (e.g., Llama, Phi), the O*NET task database, expert human annotations, and statistical modeling, yielding a reproducible, quantifiable assessment framework. Contribution/Results: We find TEAI positively correlates with cognitive ability but negatively with social skill requirements; approximately one-third of U.S. occupations exhibit high exposure, concentrated among high-education roles. Counterintuitively, TEAI is significantly positively associated with employment and wage growth from 2003–2023, suggesting a net long-term labor-market–enhancing effect of AI. TRAI reveals substantial intra-occupational heterogeneity in substitutability, underscoring the prevalence of human–AI complementarity rather than wholesale replacement.

Technology Category

• Philosophy and Ethics of AI: AI & Jobs/Labor
• Humans and AI: AI for Accessibility
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

AI and related technologies are reshaping jobs and tasks, either by automating or augmenting human skills in the workplace. Many researchers have been working on estimating if and to what extent jobs and tasks are exposed to the risk of being automatized by AI-related technologies. Our work tackles this issue through a data-driven approach by: (i) developing a reproducible framework that uses cutting-edge open-source large language models to assess the current capabilities of AI and robotics in performing job-related tasks; (ii) formalizing and computing a measure of AI exposure by occupation, the Task Exposure to AI (TEAI) index, and a measure of Task Replacement by AI (TRAI), both validated through a human user evaluation and compared with the state of the art. Our results show that the TEAI index is positively correlated with cognitive, problem-solving and management skills, while it is negatively correlated with social skills. Applying the index to the US, we obtain that about one-third of US employment is highly exposed to AI, primarily in high-skill jobs requiring a graduate or postgraduate level of education. We also find that AI exposure is positively associated with both employment and wage growth in 2003-2023, suggesting that AI has an overall positive effect on productivity. Considering specifically the TRAI index, we find that even in high-skill occupations, AI exhibits high variability in task substitution, suggesting that AI and humans complement each other within the same occupation, while the allocation of tasks within occupations is likely to change. All results, models, and code are freely available online to allow the community to reproduce our results, compare outcomes, and use our work as a benchmark to monitor AI's progress over time.

Problem

• Assessing job exposure to AI using large language models
• Measuring AI's impact on employment and wage growth
• Evaluating task replacement variability across occupations

Innovation

• Uses open-source LLMs for AI capability assessment
• Develops TEAI and TRAI indexes for job exposure
• Validates indexes via human evaluation and comparison