Working with AI: Measuring the Occupational Implications of Generative AI

2025-07-10

AI Summary

This study quantifies the real-world impact of generative AI across occupations. Method: Leveraging 200,000 anonymized Microsoft Bing Copilot workplace dialogues and an occupational task database, we propose the first AI-applicability scoring framework grounded in authentic human–AI interaction data. Using NLP techniques, we perform fine-grained classification of user activities and integrate task success rates with impact breadth to compute occupation-level AI suitability scores. Contribution/Results: Information retrieval and text generation are the most frequently assisted activities; knowledge-intensive occupations—including computer science, administrative support, and sales—exhibit the highest AI applicability; and higher-education, higher-wage occupations are systematically more amenable to AI augmentation. Critically, this work pioneers the systematic integration of large-scale, real-world interaction data into occupational impact analysis, delivering a reproducible and scalable empirical framework for assessing the economic effects of AI.

Technology Category

• Humans and AI: Applications
• Application Domains: Humanities & Computational Social Science
• Philosophy and Ethics of AI: AI & Jobs/Labor

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Given the rapid adoption of generative AI and its potential to impact a wide range of tasks, understanding the effects of AI on the economy is one of society's most important questions. In this work, we take a step toward that goal by analyzing the work activities people do with AI, how successfully and broadly those activities are done, and combine that with data on what occupations do those activities. We analyze a dataset of 200k anonymized and privacy-scrubbed conversations between users and Microsoft Bing Copilot, a publicly available generative AI system. We find the most common work activities people seek AI assistance for involve gathering information and writing, while the most common activities that AI itself is performing are providing information and assistance, writing, teaching, and advising. Combining these activity classifications with measurements of task success and scope of impact, we compute an AI applicability score for each occupation. We find the highest AI applicability scores for knowledge work occupation groups such as computer and mathematical, and office and administrative support, as well as occupations such as sales whose work activities involve providing and communicating information. Additionally, we characterize the types of work activities performed most successfully, how wage and education correlate with AI applicability, and how real-world usage compares to predictions of occupational AI impact.

Problem

• Measure AI's impact on work activities and occupations
• Analyze AI assistance success rates and scope
• Assess wage and education correlation with AI applicability

Innovation

• Analyzing user-AI interaction data
• Classifying AI-assisted work activities
• Computing AI applicability scores