Early Impacts of M365 Copilot

2025-04-15

AI Summary

This study investigates the impact of the generative AI assistant Microsoft 365 Copilot on knowledge workers’ office productivity. We conducted a six-month, large-scale randomized controlled trial (RCT) across 56 enterprises, involving over 6,000 employees—the first long-term causal evaluation of generative AI in authentic corporate settings. Leveraging behavioral log analysis, task completion time measurements, and usage frequency statistics, we found that users reduced weekly email reading time by 0.5 hours and accelerated document completion by 12%; nearly 40% of authorized employees sustained high-frequency usage over time. Our key contributions are threefold: (1) establishing quantifiable productivity gains from generative AI in real-world office workflows; (2) identifying a positive association between early adoption behavior and sustained performance improvement; and (3) providing a methodological framework for empirically evaluating AI-augmented knowledge work.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Generation
• Intelligent Robots: Embodied AI

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Advances in generative AI have rapidly expanded the potential of computers to perform or assist in a wide array of tasks traditionally performed by humans. We analyze a large, real-world randomized experiment of over 6,000 workers at 56 firms to present some of the earliest evidence on how these technologies are changing the way knowledge workers do their jobs. We find substantial time savings on common core tasks across a wide range of industries and occupations: workers who make use of this technology spent half an hour less reading email each week and completed documents 12% faster. Despite the newness of the technology, nearly 40% of workers who were given access to the tool used it regularly in their work throughout the 6-month study.

Problem

• Analyze M365 Copilot's impact on knowledge workers' efficiency
• Measure time savings in email and document tasks
• Assess adoption rates of AI tools in workplaces

Innovation

• Generative AI assists diverse human tasks
• Large-scale randomized experiment with 6000 workers
• Substantial time savings in email and documents