Microsoft 365 Copilot rollouts slowed by data security, ROI concerns
With the promise of greater office worker productivity, Microsoft 365 Copilot generated a lot of business interest even before it launched last November. (It was also for a while renamed Copilot for Microsoft 365; that changed earlier this month.)
That initial enthusiasm prompted many Microsoft 365 customers to put the generative AI (genAI) assistant into the hands of employees. But for many organizations, those rollouts have been largely limited to small trials at this stage. The reason — data security concerns and questions over the value the tool provides, said Max Goss, director analyst at Gartner.
“Microsoft has done a very good job of generating interest; most organizations that we speak to now are trying it in some form or another,” Goss said. “What we are not seeing, though, is that those pilots translate into broader deployments.”
A Gartner survey of 132 IT leaders at companies of a variety sizes in June — around half with 10,000 or more employees — showed that 60% of respondents have started pilot projects to deploy Microsoft 365 Copilot. But just 6% had finished their pilots at that point and were actively planning large-scale deployments. And only 1% had completed a Copilot deployment to all eligible office workers in their organization.
Almost half — 48% of respondents — plan to move from their existing pilot projects to small-scale deployments later this year or in 2025.
“People want to use it — they’re excited about it — they’re just not quite sure how to use it, and that is hurting Microsoft from a deployment perspective,” said Goss, one of the report authors. (Full details of the survey methodology and findings can be found here.)
Others offer a similar perspective on early business uptake. “I would characterize it as ‘cautious optimism,’” said Dave Schubmehl, research vice president for IDC’s Conversational AI and Intelligent Knowledge Discovery research. “I think a lot of organizations are experimenting with Copilot.”
Based on his own conversations with CIOs and IT leaders at organizations actively deploying M365 Copilot, Schubmehl gave a rough estimate of progress so far: around 20% are rolling out the AI assistant widely across their organization, another 20% are deploying it to select departments, and the remaining 60% are at a pilot or testing stage.
Where things stand on M365 Copilot deployments.
Gartner
Microsoft itself has talked up Copilot adoption without providing specific figures.
During Microsoft’s Q4 FY24 earnings call, CEO Satya Nadella said that M365 Copilot adoption is growing fast, with a 60% increase in customers quarter over quarter. The number of customers more with than 10,000 seats more than doubled during this time, Nadella said. Microsoft has also highlighted some large customer deployments, noting Vodafone and Cognizant have purchased licenses for tens of thousands of employees.
Microsoft didn’t respond to a request for comment on the progress of M365 Copilot deployments and the number of business customers and individual users.
Data security and governance
The ability for M365 Copilot to access organization’s data is one barrier to wider adoption.
Around 64% of respondents in the Gartner survey reported that information governance and security risks required significant time and resources to deal with, with concerns about data “oversharing” causing 40% to delay rollouts by three months or more. And 57% of respondents opted to manage risk levels by limiting their rollout to “low-risk or trusted” users.
M365 Copilot’s large language models (LLMs) process responses based on an organization’s M365 data — all the files, emails, calendars and chat conversation data in applications and storage tools such as Outlook, Teams, Word, SharePoint and OneDrive. In theory, this shouldn’t be a problem: M356 Copilot follows customer’s existing user permissions and data security controls around documents stored in M365.
But if sensitive documents aren’t labelled correctly, they can be accessed by M365 Copilot when prompted to do so, with contents then surfaced in responses to users.
This could allow payroll or company financial data to be exposed, to give some drastic examples.
“This is, of course, not Copilot’s fault,” said Goss, “this is just the fact that you’re putting Copilot into an environment where permissions haven’t really been taken care of for years.”
The ability for M365 Copilot to surface sensitive data took has taken many businesses by surprise, said Brian Vecci, Field CTO at data protection software vendor Varonis.
The removal of a 300-seat minimum purchase requirement for M365 Copilot deployments in January encouraged a wider range of organizations to start pilots, but many were unprepared to deploy it more widely once the risks became apparent. “They’d give it to a few users in IT — 10 here, a dozen here, maybe a couple dozen users — and very quickly they realized there were significant privacy and security issues,” he said.
Managing which employees have access to certain files isn’t a new challenge for business, but the introduction of genAI assistants can substantially increase the risk of that sensitive data being exposed, either accidentally by an employee or by a malicious actor.
“The better the information retrieval tool, the better your information governance has to be. And Copilot is, among many things, a very good information retrieval tool,” said Goss.
Business value remains hard to pin down
From an end user perspective, there’s plenty of interest in the tool. Almost all (98%) respondents in the Gartner survey said employees are excited to use it. Once granted a license, they want to keep it: 90% of respondents said employees would “fight to retain access.”
But embedding the genAI assistant into employees’ workflow is a different story: 72% of respondents said employees struggle to integrate it into their daily routines, and 57% report that user engagement declines quickly after it’s implemented.
At $30 per user each month for the enterprise version, price is a key factor in determining value, but it’s not the only one. Businesses also find that deploying M365 Copilot requires significant resource investment outside of licensing costs, such as change management efforts. Many respondents (73%) said that M365 Copilot deployments required higher change management efforts than expected, and 87% said end users require frequent engagement and education when the AI assistant is introduced.
“One of our survey respondents put it perfectly: ‘The learning curve is longer than expected, particularly as the expectation is there’s no learning curve,’” said Goss. “We’re not talking about learning a new software or learning a UI, we’re talking about a new way of working. As a result of that, we are seeing change management being perhaps one of the biggest blockers.”
Combined, the governance and security challenges, user experience, and change management needs can make it difficult for some businesses to make the case for organization-wide M365 Copilot. Although the tool helped users save time, only 3% of respondents in the Gartner survey said M365 Copilot provides “significant” value at this stage. The majority said it is “somewhat valuable” with signs it will live up to expectations in future.
It can be difficult to track the return on investment of genAI tools in the workplace, and M365 Copilot is no different. While there are examples where the value is clearer, said Schubmehl — an engineering services firm that finds it can reduce time engineers spend with clients leading to lower project costs, for example — ROI is often more elusive for business.
Then there’s the question of how employees should use the time they saved. “Should I use it to attend another meeting? Should I use it to go and take personal time and take my dog for a walk? It’s much more nuanced and that’s the challenge,” said Goss.
Of course, it’s still early days for the use of genAI tools in the workplace and for M365 Copilot as a product. Microsoft recently announced M365 Copilot “Wave 2,” with additional features in apps such as Excel, Outlook, and Teams, and, Pages, a new collaborative document that aims to bring the assistant into the workflow of teams and serve as a productivity tool. AI agents also offer the potential to expand what M365 Copilot can do in terms of automating business processes. And it’s likely there will be more updates to core features and underlying performance going forward.
“We’re not talking about a product that’s more than a year old from a GA perspective, so I think it’s fair to say that you should expect some level of immaturity, and that will improve,” he said.
However, Goss expects businesses will continue to have doubts about moving to wider M365 Copilot deployments, at least until the software matures to the point where value is no longer in question, and/or the cost model changes so organizations feel more comfortable tackling the challenges involved in deployments.
“If those two things don’t change, and Copilot remains a similar product at a similar cost, then I think it could be a similar conversation in a year’s time,” he said.
