What Copilot’s Earliest Users Reveal About Generative AI in the Workplace...

What Copilot’s Earliest Users Reveal About Generative AI in the Workplace:

An initial examination of its impact on productivity, creativity, and time management.

Last year, Microsoft launched Copilot for Microsoft 365 with the aim of reducing digital debt and enhancing productivity, allowing people to concentrate on uniquely human tasks. The pressing question now is: will generative AI truly revolutionise work, and in what ways?

Recent data indicates that the productivity benefits are substantial. Through a combination of surveys and experiments, they sought to understand how Copilot is transforming work by learning alongside  customers as they scale. The findings are clear: Copilot boosts productivity and creativity, and saves time. In fact, 77% of early users are reluctant to return to their previous methods of working without it.

And this is only the beginning. As the adoption of generative AI in the workplace expands, the potential extends beyond individual productivity to elevating the capabilities of entire organisations. Copilot establishes a new standard, empowering every employee with skills to write, design, code, analyse data, and more. It enhances expertise, elevating work from good to exceptional.

Key insights from Microsoft’s research include:

• 70% of Copilot users reported increased productivity, and 68% noted improved work quality.
• Users completed tasks (searching, writing, summarising) 29% faster.
• Catching up on missed meetings was nearly four times faster.
• 64% of users spent less time processing email.
• 85% found they could draft initial versions more quickly.
• 75% said Copilot helped them find what they needed in their files.
• 77% did not want to give up using Copilot once they had started.
• Perceived Productivity Gains with Copilot
  
  

  In a survey of 297 participants from our Copilot for Microsoft 365 Early Access Program, users reported significant productivity gains and time savings, echoing the benefits developers experienced with GitHub Copilot.

Productivity and Efficiency:

• 70% felt more productive.
• 73% completed tasks faster.
• 64% spent less time processing email.
• 85% could draft initial versions more quickly.
• 87% found it easier to start a first draft.
• 86% found it easier to catch up on missed content.
• 84% took action more readily after meetings.
• 75% saved time finding necessary files.
• 71% saved time on routine tasks.

Quality and Creativity:

• 68% saw an improvement in work quality.
• 57% felt more creative.
• 68% found it easier to initiate the creative process.
• 72% generated more ideas while writing

Time Management Shifts:

• 67% saved time for more important tasks.
• Users reported daily time savings of 14 minutes on average, equating to 1.2 hours weekly.
• 22% saved over 30 minutes a day.
• Of those saving over 30 minutes daily, 53% used it for focused work, with only 7% on administrative tasks like data entry.

Interestingly, 16% used the extra time for additional meetings, which could be counterproductive given already high meeting loads. These time savings are remarkable, but it is crucial to reinvest this time wisely.

Employee Value of Copilot

To gauge the value of Copilot, Microsoft asked early users about their experiences. An overwhelming 77% would not want to stop using it. Most users preferred access to Copilot over perks like a free lunch at work, whether on a monthly (88%), bi-weekly (79%), or weekly (77%) basis. Moreover, 30% said access to Copilot would influence their choice of employer.

Impact on Meetings, Email, Information Search, and Writing

Microsoft’s research quantified the productivity gains of Copilot through observational studies across four key areas of knowledge work: meetings, email, information search, and writing. The digital burden people experience is tangible. Heavy Microsoft 365 users receive over 250 emails a day, exchange 150 chats, and participate in three times as many Teams meetings globally since 2022

Study Insights: Copilot’s Impact on Daily Tasks

Study #1: A Day in the Life

In our first study, we simulated a typical workday for a knowledge worker by assigning a series of tasks. We divided 147 participants into two groups: one using Copilot and the other not. Participants were asked to complete three tasks: searching for information across multiple sources, summarising a meeting recording, and writing a blog post.

Findings:

Efficiency: Copilot users completed tasks 29% faster, taking 29 minutes and 42 seconds on average, compared to 42 minutes and 6 seconds for those without Copilot.

Writing Task: A first draft with Copilot took 8 minutes and 12 seconds versus 13 minutes and 48 seconds without it.

Accuracy: There was no statistically significant difference in accuracy between the two groups.

Quality: A panel of Large Language Model (LLM) reviewers scored the blogs written with and without Copilot and found no significant difference in quality.

Effort Reduction: 85% of Copilot users reported a reduced effort in completing tasks.

Study #2: The Missed Meeting

This focused on Copilot’s usage in Microsoft Teams, where its application has been most extensive. Sixty Microsoft employees were divided into two groups to summarise a 35-minute recorded meeting that included crosstalk, debates, and off-topic discussions to simulate a real meeting environment.

Findings:

Speed: Copilot users summarized the meeting nearly 4 times faster (3.8 times), taking 11 minutes and 13 seconds compared to 42 minutes and 34 seconds without Copilot.

Productivity: Copilot users felt twice as productive.

Effort: Users found the task 58% less draining.

Comprehensiveness: There was a slight decrease in comprehensiveness; Copilot users included 11 out of 15 key details, compared to 12 by the control group.

Study #3: Email Effectiveness

A blind panel of 62 participants evaluated the clarity and conciseness of various email messages, some composed with Copilot and some without. Participants were unaware of which emails were written with Copilot.

Findings:

Clarity: Emails written with Copilot were rated 18% clearer.

Conciseness: Emails written with Copilot were rated 19% more concise.

Quality: Copilot-assisted emails scored 25% higher on the “Couldn’t have said it better myself” metric.

Study #4: The Strain of Searching

In this study, 163 participants (80 with Copilot and 83 without) were tasked with gathering information from various sources, mimicking a scenario where they were joining the procurement department of a fictional company. They had to find information about the company’s procurement policy (from files), the manager’s needs (from emails), and the timing of upcoming meetings (from the calendar).

Findings:

Speed: Copilot users completed the task 27% faster, taking 17 minutes and 54 seconds compared to 24 minutes and 18 seconds without Copilot.

Accuracy: There was no statistically significant difference in accuracy between the groups.

Role-Specific Challenges and Opportunities with Copilot

Copilot offers significant potential to drive organizational impact across various functions. By adopting a learn-it-all approach, Microsoft started with our own salespeople and customer service agents to explore these opportunities.

Sales

They surveyed 133 Microsoft salespeople who use Copilot for Sales at least weekly. Here’s what we found:

Time Savings: Users saved an average of 90 minutes per week.

Productivity: 83% reported increased productivity.

Administrative Work: 79% experienced a reduction in administrative tasks.

Workflow: 78% said it helps them stay in the flow of work.

Task Completion: 73% found tasks easier to complete.

CRM Management: 68% maintained their CRM systems with less effort.

Customer Interaction: 67% spent more time with customers.

Personalisation: 64% were able to better personalise customer engagements.

Information Retrieval: 58% found CRM information more quickly.

Customer Service

Microsoft also assessed Copilot’s impact on their Customer Service and Support (CSS) team, one of the largest customer service organisations in the world, between April 2023 and July 2023. The study included 6,500 agents using Copilot and a control group of 5,000 agents who did not.

Case Resolution: Agents using Copilot had a 12% reduction in time spent resolving cases.

Independent Resolution: 10% of cases requiring peer collaboration were resolved independently with Copilot, reducing customer hold times.

Security

With 3 million unfilled cybersecurity positions worldwide and increasing cyber threats, there’s an urgent need for Copilot in the cybersecurity workforce. Microsoft studied the impact of Copilot for Security on 149 new-in-career security analysts, dividing them into two groups (one with Copilot and one without) to identify attacker scripts, create incident reports, and identify remediation steps.

Accuracy: Copilot users were 44% more accurate.

Speed: Copilot users were 26% faster.

Quality Improvement: 86% said Copilot improved their work quality.

Effort Reduction: 83% reported reduced effort.

Productivity: 86% felt more productive.

Future Use: 90% wanted Copilot for similar future tasks.

Transforming Business Processes with AI

To understand the broader potential of generative AI in transforming business processes, we surveyed 18,100 people across 12 countries and six key functions. Respondents highlighted that they spend more time searching for information (27% of their day) than creating (24%), communicating (24%), or consuming information (25%), and only half of the information consumed is necessary for their job.

Respondents identified areas where generative AI could improve job performance:

• Sales: 75% cited identifying sales opportunities, and 74% cited unifying marketing and sales data.
• Customer Service: 70% highlighted intelligently routing issues to appropriate agents, and 68% noted detecting trends in agent-customer interactions.
• Finance: 73% pointed to simplifying financial reporting, and 72% to validating data quality.

The Path Forward

The data underscores the substantial personal productivity gains from Copilot. However, transitioning from individual experimentation to an AI-powered organisation requires a new approach to work. Treating Copilot as a capable, patient, and always-available assistant will yield the greatest benefits.

Build Daily Habits: Like exercising or learning a new language, realising Copilot’s productivity gains requires daily practice. Those who start early will gain an advantage.

Think Like a Manager: Maximising Copilot’s potential involves delegating tasks, exercising good judgment, and applying subject-matter expertise. Employees need to manage their AI assistant effectively and know when to rely on their own intelligence.

Utilise Reclaimed Time Wisely: Leaders should guide employees on how to use the time saved with Copilot intentionally. Managers can focus more on coaching their teams, salespeople on building customer relationships and closing deals, and marketers on innovating new products. This shift allows everyone to invest more energy in impactful, value-adding work