Crashing Waves vs. Rising Tides: Preliminary Findings on AI Automation from Thousands of Worker Evaluations of Labor Market Tasks

2026-04-01

AI Summary

This study investigates whether AI automation impacts labor market tasks through abrupt “tsunami-like” disruptions or gradual “tidal” progression. Leveraging the U.S. Department of Labor’s O*NET taxonomy, the authors construct an analytical framework for large language model (LLM) capability evolution by integrating over 3,000 textual task descriptions with more than 17,000 worker assessments, incorporating human judgments on both task completion time and quality to model cross-task and cross-temporal performance trends. The work proposes and empirically validates an “AI automation continuum” theory, challenging prevailing narratives of sudden AI capability leaps and demonstrating that incremental, tide-like advancement is the dominant pattern. The findings project that by Q3 2025, LLMs will complete tasks requiring 3–4 hours of human effort with 50%–65% success rates, rising to average success rates of 80%–95% by 2029.

Abstract

We propose that AI automation is a continuum between: (i) crashing waves where AI capabilities surge abruptly over small sets of tasks, and (ii) rising tides where the increase in AI capabilities is more continuous and broad-based. We test for these effects in preliminary evidence from an ongoing evaluation of AI capabilities across over 3,000 broad-based tasks derived from the U.S. Department of Labor O*NET categorization that are text-based and thus LLM-addressable. Based on more than 17,000 evaluations by workers from these jobs, we find little evidence of crashing waves (in contrast to recent work by METR), but substantial evidence that rising tides are the primary form of AI automation. AI performance is high and improving rapidly across a wide range of tasks. We estimate that, in 2024-Q2, AI models successfully complete tasks that take humans approximately 3-4 hours with about a 50% success rate, increasing to about 65% by 2025-Q3. If recent trends in AI capability growth persist, this pace of AI improvement implies that LLMs will be able to complete most text-related tasks with success rates of, on average, 80%-95% by 2029 at a minimally sufficient quality level. Achieving near-perfect success rates at this quality level or comparable success rates at superior quality would require several additional years. These AI capability improvements would impact the economy and labor market as organizations adopt AI, which could have a substantially longer timeline.

Problem

• AI automation
• labor market tasks
• crashing waves
• rising tides
• LLM capabilities

Innovation

• AI automation
• large language models
• labor market tasks
• rising tides
• human evaluation