We're sunsetting PodQuest on 2025-07-28. Thank you for your support!
Export Podcast Subscriptions
cover of episode Managing the Future of Work: Microsoft’s AI Perspective

Managing the Future of Work: Microsoft’s AI Perspective

2025/1/21
logo of podcast Cold Call

Cold Call

AI Deep Dive AI Chapters Transcript
People
J
Jared Spataro
Topics
Jared Spataro: 我认为人工智能技术发展的一个关键因素是人的因素。人们需要学习如何使用这项技术,克服对它的恐惧,并认识到它对个人和管理者的好处。我们应该积极采用人工智能技术,但不要试图彻底改造整个公司,这可能会导致不必要的混乱。我个人使用人工智能工具来了解人工智能领域的最新发展,因为创新的速度太快,人类无法独自跟上。目前,人工智能主要作为个人和团队的助手,但随着人工智能助手数量的激增,未来可能会出现整合,并出现一个能够帮助用户选择合适助手的更高级别系统。信息工作者的大部分时间都花在了沟通和协调上,而人工智能助手可以及时地解决这个问题。Copilot 等人工智能工具可以显著提高生产力,节省大量时间。我们的研究表明,Copilot 可以帮助用户节省超过 10 个小时的时间,并且在实验中,用户完成任务的速度提高了 29%。人工智能工具可以缩小经验差距,让年轻员工和经验丰富的员工都能从中受益。微软将人工智能视为一种通用的推理引擎,它可以作为一种强大的自然语言用户界面,并且检索增强生成 (RAG) 技术是近年来商业领域最重要的进步之一。它允许人工智能系统对数据进行推理并给出答案。人工智能技术不仅可以节省时间,还可以提高工作效率本身。下一代人工智能技术将实现流程自动化,从而在功能层面提高生产力。人工智能可以提高协作效率,减少不必要的沟通。人工智能可以提高会议效率,扮演熟练会议主持人的角色,并改进会议后的体验,使人们更容易获取会议信息。人工智能工具会使会议的目的更加清晰,并促进人们对会议目标达成共识。人工智能可以帮助弥合线上线下会议之间的差距。企业面临的技术型问题和适应型问题不同,人工智能在解决适应型问题方面发挥着重要作用。企业应该循序渐进地采用人工智能技术,不要试图一步到位。人工智能技术的应用既需要适应员工的现有技能水平,也需要员工提升技能以适应新的技术。微软优先考虑那些对用户来说既简单又实用的 AI 应用。企业可以根据自身需求和员工偏好调整人工智能工具的设置。人工智能工具可以改变人们使用邮箱的方式,提高效率并减少邮件数量。微软通过内部推广 Copilot 和开展试点项目来进行内部重组和员工技能再培训。企业应该逐步采用人工智能技术,让员工在实践中学习和适应。微软与 OpenAI 的合作关系促进了人工智能技术的创新和发展。微软致力于负责任地开发和应用人工智能技术,并与社会各界合作。微软与工会合作,以应对人工智能技术带来的劳动力市场变化。微型证书等技能认证将变得越来越重要,以帮助员工适应人工智能技术带来的变化。未来 20 年人工智能技术带来的变革将比过去 20 年更为显著,它将重塑企业和经济。 Bill Kerr: 在与 Jared Spataro 的对话中,我主要负责引导话题,并就人工智能技术对未来工作的影响提出问题,例如人工智能对生产力的影响、人工智能在协作中的作用、以及人工智能技术带来的风险和伦理考量等。 我与 Jared Spataro 探讨了微软内部如何采用人工智能技术,以及微软如何与 OpenAI 合作,共同推动人工智能技术的发展。我们还讨论了企业应该如何逐步采用人工智能技术,以及如何帮助员工适应人工智能技术带来的变化。

Deep Dive

Chapters
This chapter explores Microsoft's strategies for integrating AI into its products and services, focusing on productivity gains and the leveling of the playing field for younger workers. It highlights the significant time savings and productivity improvements observed in early user trials and experiments.
  • 77% of early Copilot users didn't want to go back to working without it
  • Users estimated saving over 10 hours per month with Copilot
  • Experiments showed a 29% time savings with Copilot, without quality loss
  • AI levels the playing field for younger and less experienced workers

Shownotes Transcript

Translations:
中文

Before we begin, we have a couple of questions. What do you love about Cold Call? What do you want less of? What would make Cold Call even better? Tell us. Head over to hbr.org slash podcast survey to share your thoughts. We want to make the show even better, but we need your help to do that. So head to hbr.org slash podcast survey. Thank you.

If you're a regular listener, you know that we've tackled the topic of AI multiple times over the past year, and for good reason. It's a topic that's changing how we live and work in ways large and small. So it won't surprise you to know that we aren't the only ones at Harvard Business School who are thinking about this.

With that in mind, we'd like to reshare an episode of Harvard Business School's Managing the Future of Work podcast, a show hosted by professors Bill Kerr and Joe Fuller that is all about the forces reshaping the nature of work. In this episode from February 2024, Bill Kerr talks to Jared Spataro of Microsoft

about how the tech giant is experimenting its way from AI assistance to autonomous agents, as well as responsible AI and upskilling. We thought this would be a great discussion to get you thinking about what AI has in store for all of us in 2025. And if you have some thoughts on that, we'd love to hear from you. Email us at coldcall at hbs.edu.

might sound counterintuitive, but one aspect of the propagation of AI innovation is absolutely the human element. People have to learn it. They have to learn not to be afraid of it. They have to learn that it can be beneficial for them as individuals and as, you know, in the case of being a leader or manager in those circumstances.

And so that's what's emerged for me is I would just say, hey, get after it, you know, get after it with alacrity. But but don't try and, you know, jump over and say we're going to transform our entire firm, you know, from the ground up by reinventing everything that I feel like is unnecessary tumult. It's early 2024 and generative AI is fast becoming pervasive. Its impact will be both earth shattering and mundane.

While many organizations are experimenting with it, fewer yet making wholesale changes to take advantage of it. What's clear, though, is that AI will reshape the global economy and the world of work. The International Monetary Fund estimates that 40% of jobs globally and 60% in advanced economies are exposed to elimination or change as a result of AI.

The IMF also warns that the technology has the potential to increase inequality. And while AI is sparking innovation across the spectrum, dominant technology players have a distinct advantage in terms of their resources and market share. What can we expect from these leading suppliers of productivity tools as AI works its way through the enterprise and into our smart devices?

Welcome to the Managing the Future of Work podcast from Harvard Business School. I'm your host, Bill Kerr. My guest today is Jared Spataro, Corporate Vice President of Modern Work and Business Applications at Microsoft. We'll talk about how Microsoft internally and through its investments in OpenAI is AI enabling its ubiquitous products from Office apps to Teams collaboration software.

We'll consider use cases and crucially, how Microsoft sees organizations adapting business processes, jobs, and tasks, and managing the risks of the powerful technology. We'll talk about how the company is gearing up and skilling itself internally to carry out its AI strategy. And we'll look at the bigger picture of ethical and regulatory considerations.

Jared, welcome to the podcast. Great to be here. Thanks for having me, Bill. Jared, why don't we start with a bit of your personal background and how you came to lead modern work and business applications at Microsoft? Well, sure. I've been here for a long time, 18 years at this point. And throughout my history at Microsoft, I've always worked on what we've called essentially our productivity businesses. So started with applications like Office and

And then over time has expanded into these areas like CRM and ERP, kind of large areas where businesses are investing in technology. So it's given me a really nice rounded view of how businesses use technology to run their operations. And with the pace of change being so fast right now, what's your personal strategy for keeping up to date with what AI is doing and how it might influence Microsoft's products?

You know, interestingly enough, I use AI to keep up to date. And that, I think, is going to become one of the themes that you'll hear from me in our conversation today today.

And the pace of innovation, the pace of change has picked up so much over the last few years that there's no way that humans can keep up with it on their own. The old pattern simply of, you know, scanning the news is far from adequate. So these days, you know, I use AI based agents to help me keep up with AI based agents. I think it's the only way to go.

Sound advice. And so maybe we'll ask you to continue and tell us what are those agents saying are the biggest trends right now? And also what might be around the horizon for us? I tend to think of us being in an era that is characterized by AI as an assistant, an assistant to individuals, an assistant to groups.

You see it as a standalone assistant in products like ChatGPT and like the Microsoft Copilot. You see it as an embedded assistant into many different applications. We see just about every software vendor on the planet building these assistants into their interfaces.

But increasingly, I think you'll see with the explosion of these assistants, kind of people scratching their head and saying, man, I don't know which assistant to go to for what at this point. It's too much. And so I also anticipate that in the coming months and years, we'll see some consolidation. There's almost like a meta layer that's going to help the...

crossing the chasm from the earliest users to the general mainstream and beyond. That's right. I think the context is important. Almost 60%, based on our telemetry, of the average information worker's time is actually spent in communication and coordination just to do the rest of their job. So for instance, it's in meetings or in chats or it's used in email. And that percentage is growing month by month and it doesn't look like it's flattening out.

So what people actually tell us in the qualitative work that we've done is that they say, man, I hardly have time to do the job I was hired for. And that backdrop, that context, I think is important because the assistant era of AI is arriving just in time. Yeah, neither of us are going to be able to see the listeners, but I'm sure a number of heads were nodding when you talked about the overload that some of these communication tasks have had for us. Yeah.

Is there any early data on the impact for individual productivity that comes with AI, generative AI tools? I think that there are two things that, two data points that really stick out to me there. We've asked users, for instance, you know, after you have used in particular a product from Microsoft called Copilot, you know, do you want to go back to working without it? And 77% of the people who had early access to Copilot as a product said, I never want to go back to working without it.

The other one that has been particularly interesting in terms of sentiment is that users, our best users of Copilot are estimating that they save north of more than 10 hours per month. That said, anyone who's in business for a while knows that people lie and they in particular really lie when they answer these types of surveys. So we've done some work to just go in and actually run experiments where we have groups that use the tool Copilot and then control groups that don't. We give them a series of tasks

working across a corpus of information. On average, users saved 29% of their time to get the work done. So they were 29% faster without any real degradation in quality.

So that gives me hope. You know, if you say I can give you 30 percent of your time back once you figure out how to use the tool across this broad swath of information worker tasks, that's that's pretty attractive, you know, certainly attractive for individuals if they can harness the excess, the surplus that's generated. Very attractive for organizations as well. Yeah. One of the.

Things that a couple of academic kind of experiments have also found is not only that productivity gain, but that it really accrues to the youngest workers or the newest to the job, that it kind of really can flatten out parts of the experience curve.

Is that something you've also found? Absolutely. We're very excited about that because it does level the playing field. It helps people essentially kind of close the gap that experience ends up creating. But we would like to get that same productivity boost even for experienced workers. So that's some of what we're digging into right now.

That's great. So tell us a little bit about how Microsoft is viewing AI. Like, what's the strategy or the approach that it's taking towards it? And how are you kind of staging it's, you know, building AI into the many products and services of Microsoft?

Well, let's take a step back for a moment. For a large part of 2023, there was this sense that, okay, I get it. You basically have a technology that can do things like summarize and answer basic questions. But what we have found during that last year was not only can it do some of those key skills, but we really believe that what we have here is a general purpose reasoning engine.

And I always pause for effect, you know, when I'm talking with, with my customers about that, because we've just never had that before. We have, we have computers that essentially reason over math based formulations of, of problems because they're, they are driven based on math. Essentially. This is the first time that we have something that is a language driven reasoning engine. And that's really important. Uh,

As we look at the basic implications of that, it means that it can become an incredible natural language user interface. That makes a lot of sense. You can speak to computing resources going forward. But even more important than that, what we have found is that there's a pattern that has emerged that many of our listeners will have heard of that's called RAG, retrieval augmented generation. And it essentially says this. If you ask me a question and you give me essentially a scoop, a bucket full of data to

to reference as I answer that question, can take the reference data and reason across that reference data to get you the right answer. As long as the right answer is in that bucket full of data, we call that a context window. And that pattern is retrieval augmented generation. It's become, from my perspective, the single biggest advance for business that we've seen over the last 12 months. It's very, very significant.

And it means that you can take these large language models as they continue to get better and better at reasoning and really have them reason over your data, your finance data, your HR data, any type of data to help you make decisions. And that's a very exciting area of innovation right now in business with generative AI. Wow, that seems very transformative. You talked about saving time, getting those 10 hours back for yourself, which I'm sure everyone wants.

But the capacity to have that superpower come towards your actual meaningful work, the things that you're doing to learn from the data seems significant. Do you also anticipate that level of improvement in the productivity of the work itself? We do. And I noted earlier that we're in this kind of assistant generation where we're all about helping people. I would call it almost incrementally improve their performance, their productivity, which

But the next generation moves us out of kind of just individuals and even small groups into process oriented. Maybe you could even call it function oriented productivity. Many of the processes that are so common and so core to the way a firm operates today can be automated. So closing the books at quarter's end totally can be automated. Right.

Reviewing sales pipeline, that can be automated. You know, answering customer questions, even finding the next best thing to do with customers, even unearthing the right people to target for new customers. All that can be automated and much of that work because it requires a logical progression, a reasoning over data is done by people today.

So that second generation, we believe, really moves us into process automation, where we do start to get some pretty important gains, not at the individual level, but at kind of the functional level in terms of how work is getting done. So again, we start to see this technology used in an increasingly sophisticated way to drive returns. Another area that I know you've been thinking a lot about is collaboration. So tell us about how you're seeing AI's next generation towards collaborative work and

Recognizing the process functions that you were just describing have some element of collaboration, but there's many things that would be cross-functional that would also be collaborative. Interestingly enough, what we've seen the tools do, these digital tools do over the course of, let's call it the last two decades, is

is that at the individual level, they have driven quite a bit of productivity. A spreadsheet is a great example. It helps me do things that I otherwise would have taken days to do in sometimes minutes, sometimes seconds.

But the downside to them, speaking as a purveyor of these tools, is that they've made it incredibly easy, frictionless even has been our claim, and it's really true, to communicate with other people. And although that sounds like nirvana, what has actually happened is we are over-communicating. We are communicating and in some ways coordinating too much.

If a person has a question, instead of going to yet another person to ask the question, they could go to a bank of answers or an AI agent that would understand what's happening across the firm. It actually would significantly improve what the efficiency of the overall processes would be. So one aspect of what we're trying to do is actually get people out of the business of just having to service other people in very mundane ways.

Think of how much time we spend servicing our inbox. One of the things we can do for collaboration, interestingly enough, is reduce how much we have to go to each other for some of the mundane aspects of work where we could be much more efficient. Well, let's just continue on the other use cases for meetings. What are some of the pieces you're envisioning for the future workplace and office?

There are two aspects to meetings that have caught our attention over the last couple of years. The first is that the in-meeting experience can be much more efficient than it is. What we find with experiments and research is that when meetings are facilitated by skilled

humans who really know how to run a meeting, they are much, much more effective. They're much better at reaching, for instance, decisions or pursuing outcomes. But the problem is not everybody's a skilled meeting facilitator, even though that's a core part of what many information workers do every day.

So one of the most exciting use cases that I've been involved in is building essentially a co-pilot for a meeting that actually helps the meeting to be more effective than it otherwise could be, playing the role, again, of that skilled human facilitator. And the fact is, we just don't have enough of those folks. We don't hire them. It's not a job that we do, but it is a job that AI can do very well.

Then over the course of the pandemic, we saw meetings explode. So the time in meetings actually doubled for the average Teams user, the user of Microsoft Teams. And we saw large meetings start to balloon. More and more people are joining meetings, sometimes as a fly on the wall, sometimes to hear that five minutes of an hour long meeting that is pertinent to their job.

So the after meeting experience, making it easier for people to not attend a meeting, but instead get the value that they would need out of the interactions. That's incredibly important. You ask Copilot what decisions were made? What did David say? You know, what did David and Sally talk about when it comes to this topic? It can extract all of that out and summarize it. So those two things, I think meeting goers should rejoice as they think about what AI is going to be able to do for them. But they'll have to learn how to use the tech.

Yeah, Jerry, I agree that there's far too many meetings at my employer and I'm sure at many other employers. There's also a function or a role of meetings that's not just about coordinating on a solution. It's about...

all coming to terms that this is the way that we're going to go forward and soliciting buy-in. There's these extra things. As you think about a future that's going to have more of this AI collaborative role in organizing stuff,

Do you anticipate that there is going to be resistance from people that will look at the outcome and say, wait a minute, that's not what I would have agreed to or bought in? Or will we also kind of learn to lean in more on these types of prompts and directions for success?

seeing the path forward. I think what these tools will do is that they will shine more of a spotlight onto the why of a meeting. Why are we having this meeting? Why have we invited the people that we've invited? And it will take some time. That has been one of my biggest observations is as the technology makes us in some ways much more proficient, maybe, um,

much more able to dig into the details of our work than it does push us to not gloss over those details any longer, but really ensure that we're surfacing them. So to be very specific, I think there are meetings where it's about building consensus and support around a particular direction.

But we often don't say that's the reason we're meeting. And I think we're going to start to want to state that, you know, hey, this is a this is a build energy meeting or this is a build buy in meeting. We want you to come. And that's the reason we want you to come. I haven't seen too many meeting invites that start that way.

Jared, there's another layer of this, which is in-person meetings versus virtual teams-based meetings. Do you see, I mean, clearly the AI could play a role in both of those. Is that just kind of an extra dimension to this or would the technology almost prefer us to be in one format or another? You know, what's very interesting to me is the technology can help us bridge that gap. So as an example,

Some of the things we've been experimenting with is watching when it looks as though people who are online and not in person want to actually participate in the conversation but are having a hard time. A simple prompt that says, hey, John's had his hand up now for five minutes and nobody's paying attention to him, you know, can help get the attention of people who are in a room who are quite engaged understandably with each other because there's all sorts of very rich cues and clues going on there in person physically as

I think we'll start to see these intelligent meeting moderators pop up as a real best practice. One of the areas that you've talked about is the difference between technical and adaptive business problems. Can you unpack that distinction for us and tell us about the role of AI and managers in addressing them?

you bet, adaptive leadership. In times of change, it identifies that there are basically two types of business problems that employees face. One is what has been labeled technical business problems. And these are problems that have been solved before, where essentially there is a state of the art. A lot of operational problems tend to be technical, even in things like healthcare, things that become routine surgeries over time become technical. There's a whole host of

technical problems out there. Then you turn over into what has been labeled through the research adaptive problems. This is in many ways where employees are stepping into a new domain or a domain where there's a lot of innovation or change.

It's not as well charted. You can't just find an expert. And here there's a different set of skills that are required. It's a bit more test and learn. It might seem a little bit tentative, but the tentativeness actually has changed into a faster cycle time because you kind of take a step forward and evaluate, take another step forward and evaluate. It's a good way of describing what we're experiencing right now with AI.

Some customers are approaching this era of AI as if it were a technical problem. But the truth is, I think that approach is going to fail. It's too incremental. I like to think of it as essentially the industrial revolution for information work. I think it'll be that impactful. That's the definition of what we just talked about as an adaptive challenger problem. Yeah, it's a great segue into thinking about this span of organizations that will be

The pre-AI firm, then there's going to be AI-assisted, then there's going to be semi-autonomous agents, and then you can go to some of the more integrated functional AI that you described a little bit earlier. And obviously, Microsoft as a company has long served companies all across the technology spectrum, from very limited technology to some of the most advanced technology.

What's kind of the vision you're having or the landscape that you're anticipating in terms of organizations across the spectrum? If you're with one of the companies that's new to this game, early stages, what do you typically suggest is the next experiment that the company should run, how they should test reconfiguring jobs? How do they get started moving further down the path? Right now, I'm seeing three generations.

Generation one is about these assistants for individuals and groups. It requires a reworking of what you might call muscle memory in terms of how core tasks are done. It drives, you know, pretty substantial incremental productivity. Gen two for me ends up being process automation. And in particular, as we talked about process automation focused on functions. So in the finance function, in the marketing function, in the sales function, we're starting to see innovation there. These things really matter in the efficiency of the

firm. And then Gen 3, we hadn't quite gotten there yet, but this is, as you indicated, this is the place where you start to get semi-autonomous agents that are running at what I call the firm level. I think of it as enterprise transformation. You have semi-autonomous agents that are working across functions that are now starting to interact directly with your customers on your behalf that come back to humans when they reach essentially a place that they've been told, hey, it's material enough. I want you to get human input.

And this is an interesting place to be. It's a future that's not too far off, not nearly as far off as people might think it would be. And it really is, from my perspective, a reconfiguring of the firm.

If I were to then hit your second question, gosh, if people are just getting started, what should they do? I would say be careful not to try and jump the generations. It is important to gain experience, to gain skill, to gain a sense of how these things are impacting your industry, your business. And that might sound a little, you know, a little strange coming from a technology vendor who seems to want to sell everything to everyone all at once, but

But this stepwise evolution of understanding how AI can help your business, I'm finding as I observe different businesses and different industries operate, is actually really important. I suppose this wouldn't be too different from other technologies, but I'm struck by with generative AI, both how simple and user-friendly it can be.

in the sense of someone that, you know, barely knows anything about computers and technology, can understand the prompt and, you know, ask questions and particularly have some kind of input into their workplace over to some of the most complicated frontier type conversations that one can also hear related to the technology. Do you have a sense of

How much is going to come and try to meet individual workers where they are versus the workers are going to have to significantly revise their skill set in order to participate in what lies ahead? I think it's an and. I think we'll see both. But if I talk about our company strategy for a moment, we have really picked some of the mundane cases that are

that are remarkably or surprisingly interesting for users. So having an assistant within your Outlook email client, as an example, that helps you go through your email, that's very mundane, but it's incredibly delightful. You know, here's a summary of this long thread and what I'd recommend you do next.

The first time I read that, I thought, oh my goodness, gosh, I'm not going to do email without this. The same can be said as you move into something like Excel or Word, where you are analyzing numbers or trying to start with a blank sheet and write things. An assistant there that can help you get started or help you dive deep into the numbers, incredibly useful.

We're also trying to innovate in experiences that allow you to pull with that pattern of RAG, retrieval augmented generation, across a whole set of data so you can have really sophisticated, very deep conversations about a domain. But what's interesting here is I have to teach people as I'm working with them, I have to teach them, boy, you have to think of this almost as if it were a human on the other side.

and talk about the domain as if it were a human. Recognize it's not a one-shot pattern like we often see in search. You put in a keyword and hope it works.

But instead you ask a question, something comes back, you redirect it a little bit. The multi-turn, multi-step nature of the process is part of what's required to get sophisticated answers and to really get value. So we see it all, but we're not at all shy about starting with the mundane because we think there will be real benefits by starting there. And Jerry, just continuing on that, and we'll use the email kind of curation as a great example.

You can imagine that there's individuals within the workplace have preferences and they can adjust various things about their outlook structure and how they approach the email. There might also be corporate level requests, especially if it starts to get into the collaboration elements or the things that go beyond individual productivity. Is there...

a way that you calibrate that to environments and to the work that a firm is doing? Or what's kind of the approach to handling those different perspectives on how we might best curate the email in an employee's inbox? Well, after people start to use the tool with email, if I just

get really specific for a moment. They typically will have two reactions and this is coming up over and over. Number one, they start to really get curious about how powerful it is. So they'll start to say, well, wait a second, you know, like I kind of sequentially go through email every day and people shove stuff in my inbox and

what if I change my pattern and every hour I just ask AI, what are the three most important things that have landed in email? And, you know, what do you think I should do about them? That's a totally different mindset. It typically changes people's way of interacting with, you know, their inbox. Copilot is very good at it. It knows who your

manager is it can flag those things it understands who a customer is so hey a customer just wrote to you it's it's good at actually giving you its perspective on what's important that has landed in your inbox then it gets even more exciting when it can give its perspective on how you should answer you know hey your boss just wrote you he asked if you're on you know project xyz and i see that you've had several meetings you want me to summarize what's going on real quick and give you a draft of how you could respond to them that starts to get very exciting for people

So that is one pattern that I see happening. Then the second thing that I'm seeing is that you actually change the use of your tool. A lot of email traffic that we see, particularly within a firm, is just questions. Hey, Susie, can you send that over to me? Hey, Brian, I remember that you mentioned something two days ago in a meeting. Can you remind me where you found that?

And today we're finding that Copilot can help you kind of cut through that. You don't have to go to the person any longer. So it actually can have the effect of reducing email. So those are the types of things that we're seeing. People are just starting to use it to adapt to their own workflow, but they're also changing that flow of work as they learn what the technology is capable of.

With such significant change, can you tell us how Microsoft is internally approaching the restructuring and the reskilling that it needs to do in order to have an effective AI strategy and to compete in, you know,

for customers in a very AI-influenced economy? What's happening inside the organization? There basically are two prongs to it. Number one, we have aggressively rolled out our own AI assistant called Copilot to every function, every employee. And we're just asking them, hey, get proficient. We're providing a lot of coaching in that content, but we're also creating small groups because we find that the

the viral nature of learning how to use the tool is really important. So we start with this very horizontal use case. You might be in finance, you might be in sales. Hey, here's a tool. Here are some immediate ways you can use it in your job. Please get started and tell us what you're learning.

That's step one. Step two is that then we have worked within the functions to just identify specific business problems where we can say, gosh, it feels like AI should be able to help us out here. Why don't we run a pilot and actually see, again, with control groups that aren't using the technology and groups that learn to use it in different ways and see if there are places that we can gain efficiencies. We've seen some of our best results, particularly in customer service here, because that's so

essentially information intensive and answering and solving people's questions. And we've started to see some good results in sales as well as again, there's kind of this matching the customer and their needs to what we have to sell them. And we're doing more and more each function is kind of learning how to become AI practitioners is the way we think about it. But that one, two bunch of horizontal and then functionally based pilots focused on specific problems and

That's how we're doing it. And what's interesting is it's very natural skilling rather than like timeout, the coach blows the whistle. We're all going to take a day and take a YouTube based AI training course. People are learning based on their real jobs, the content of their jobs, and they're being rewarded for being innovative and demonstrating that AI is making a difference in those jobs.

It sounds like your prescription to organizations a little bit earlier that they not leapfrog or attempt to leapfrog various waves of this, but instead go through the waves because that's going to be the most natural way for employees to learn how to handle the new technology.

That's right. It might sound counterintuitive, but one aspect of the propagation of AI innovation is absolutely the human element. People have to learn it. They have to learn not to be afraid of it. They have to learn that it can be beneficial for them as individuals and as, you know, in the case of being a leader or manager and in those circumstances.

And so that's what's emerged for me is I would just say, hey, get after it, you know, get after it with alacrity. But but don't try and, you know, jump over and say we're going to transform our entire firm, you know, from the ground up by reinventing everything that I feel like is unnecessary tumult. In addition to your internal work, you also have a significant investment in and collaboration with OpenAI. Can you tell us a bit about how the organizations connect on their technology and strategy?

Our relationship with OpenAI is one of the most significant relationships that we have had for many years. It has allowed us to create a partnership that I think has pushed the envelope of AI in ways that otherwise wouldn't be possible. They had amazing ideas. They turned those ideas into amazing technology. But the reality of it is they needed real computing resources to see if their ideas and technology could be scaled in a way that would change the world.

And that's where I think Microsoft and OpenAI have been such a good combination. Building on top of what we think of as the world supercomputer with Azure, they've been able to scale their ideas in a way that have demonstrated first with the introduction of ChatGPT, wow, there's something here. This is not the AI that we've seen incrementally progress over the last couple of decades. This feels like a discontinuity, like something new has happened.

And we're particularly proud of what we see from OpenAI as they continue to push the limits of innovation. We saw in 2022, as an example, that they realized that people wanted to now start to use chat GPT in particular domains. And so they created the idea of GPTs, you know, which was in some ways the app equivalent of

to the iPhone moment. You know, it was this idea of if you have a platform, how do you start to build on that platform for very specific scenarios and use cases? So I, as I work with them, as I see us using their technology and them using what we have to provide through infrastructure, it just feels like a really natural complementary relationship. And that's what we're working to make sure that it continues to be for us.

Sure. We spent a lot of time on the productivity advantages and the amazing use cases that will be in the future. Of course, there's also risks and there's ethical considerations and so forth. So I know that's an enormous set of topics, but can you share a little bit about how your team is approaching these questions towards the office workplace and what's kind of the role of Microsoft versus industry-wide and government in the future here?

AI is incredibly powerful, but we tend to think of it as a neutral technology, meaning it can be used for good and it can be used for bad. And so we are trying to take steps in the form of what we call our responsible AI initiatives to ensure that we are really doing everything we can to have it be used for good.

So a couple of areas that are important to us. In responsible AI, for instance, we have provided a number of tools that allow us to be much more transparent about the training and about essentially kind of the substance of the foundational model. So you can take a look. People are often worried about bias in those foundational models. They're trained on particular aspects.

sources of information, a particular body of information, and we want them to feel good about what's there. And so we do a lot of work broadly with different aspects of society to do that. We also take special precautions within, for us, our instantiations.

of these AI assistants. So in particular with Copilot, you can kind of think of Copilot as being an orchestrator of powerful, large language foundational models, and then kind of the inputs that we're getting from the business. And in that orchestration layer, we've been doing a lot of work to make sure that we're catching, trapping for things that we just don't think are appropriate. So as an example, in a meeting, Copilot can answer questions about, you know, what did what

what did Sally say in this meeting? What were her key points? But you can certainly ask co-pilot questions like, do you think Sally's smart? Do you think Sally knows what she's doing? Should I listen to Sally? And we trap for those types of value-oriented questions so that we're making sure we just catch them and say, hey, I'm an AI assistant. You're going to have to make that determination on yourself. Here are some of the facts coming from the interaction that I can give you as you think about

So that orchestration layer for us is very important. We try to do work both at the LLM layer and at that orchestration layer. And we're trying to engage with every facet of society. We've done some work with organized labor that's just kicking off. We certainly are very engaged with governments all around the world. And we understand it's very important to allow every facet of society to have their input with this very powerful technology.

I'm sure some of our listeners would love to hear just a couple more sentences about the link to or pilots with organized labor. Tell us about that. Well, organized labor, of course, is in this place where they're always interested in the evolution of the tools that affect the labor pool, the workforce. And I think what we have seen over the years is sometimes innovation kind of

ignores the fact that humans have certain skills, that they do real jobs today. Innovation tends to say, hey, over the long run, it all works out for us.

And at Microsoft, we're really trying to take an approach and say, yeah, over the long run, for sure. But what about in the short run? There certainly will be displacement. There will be the need for skilling. And instead of trying to treat that as something that we do after the fact, we've actually proactively gone to work with the AFL-CIO, as an example, and put in place agreements that say, here are the types of things we'd like to work with you on. So some of those

Some of the initiatives include, for instance, making sure that we take time from our product groups to get input from them, show them the technology and get input.

and also take time to work on skilling with them as well. Ultimately, if we do this right, we definitely think, we think of this as very accretive to society. Even in the short term with displacement, there is so much goodness, so much work that can be done. But we think the best approach, like the best model for us is engagement. Let's engage, you know, particularly with labor in the case of your question so that we can work on it together.

Microsoft already has, of course, a number of certifications around its products that it offers. And as we think about all the technology change ahead, do you anticipate that those types of micro-credentials will become more in play? Something that a number of vendors are going to be trying to use as ways of helping workers make the next jump?

Absolutely. I think there's a bigger conversation around what's happening to education all across the world. Snackable content in kind of business settings has definitely become the norm. And while that's become the norm and, you know, the economic context, we still see education largely as these big set piece, you know, courses.

and semesters and schools. So there's definitely room, I think, for disruption here. If we just zoom in for a moment on AI, it will be imperative from my perspective to make sure that materials are just in time, that they're delivered in a way that's very consumable. So I've been given this assistant, this co-pilot, what do I do with it? What's the first thing I do with it?

And the good news is our consumer lives have shaped our expectations about how we consume content. And so rather than fight that, our perspective, our approach is that we're going to go right with that. We're going to adapt to those models and patterns.

In addition to that, one of the really important things that we found is the product itself needs to be able to help users learn how to use it. And so creating ways for the product to say, hey, it looks like you asked this question, but did you know if you did this, you'd get a better outcome? That may seem simple, but you can teach people how to use it. And one very simple example of that is some work that we're doing with almost a Mad Lib style of prompt engineering.

prompt engineering and telling people they need to write sophisticated prompts or queries for an AI assistant is one thing, but we're finding that everybody knows how to do Mad Libs. So we essentially can put, you know, a three to five sentence prompt there with blanks to fill in and allow people to fill in those blanks and then see it work across their own data. You only have to do that one or two times before you realize, okay, I got this. I can figure out how to do this.

So it's a very simple example, you know, of how we take simplicity, weave it into the experience, and then we think we can drive the right outcome. That's amazing. I want to maybe end us with one kind of

thought or projection of yours going towards the future. You've been at Microsoft almost 20 years, which would have had you land there right about a year or two before the iPhone and all the smartphone revolution was going to take off. As you think about all the change that's happened over the last roughly 20 years versus what's going to happen towards the future, do you think what we're looking at ahead is

the same as the last 20? Is it more? How do you envision the next 20 years unfolding?

I think it's much more significant. In fact, I'd say this, if you go look at what economists have been studying is almost a little bit of an enigma related to productivity. They have wondered, gosh, we put so much information technology into firms, into the macroeconomic environment. Why don't we see productivity just through the roof? Why hasn't it changed things? That's been one of the really big questions for economists over the last couple of decades.

My answer would be because we're getting ready for this moment right now. A lot of the information technology we put in place has been about getting our data state in order. It's been about starting to work in digital ways. Even during the pandemic, we saw an incredible increase in human interactions, essentially being what we call digitally mediated. You know, it's a fancy way of saying we now meet online. All of that matters, though, because it lays the foundation for AI to work across, to reason across that data.

So in so many ways, when I'm meeting with leaders of all types across the world, I say, man, everything we've done over the course of the last 50 years in information technology is working up to this moment. And this moment is literally the industrial revolution for information work. That's what we'll see. We will reconfigure firms. We will, with this technology, reconfigure the way organizations work. And because of that, we'll reconfigure entire economies.

I don't know how it's going to play out, but we do believe that it will have that much impact. And so we're excited to work together with everyone, every facet of the society to invent the future together. Gary, this has been an amazing conversation and covered so much from the very micro of what might be in our inboxes in the near future to the macro. Thanks so much for joining us today. My pleasure. Thanks for having me, Bill.

We hope you enjoy the Managing the Future of Work podcast. If you haven't already, please subscribe and rate the show wherever you get your podcasts. You can find out more about the Managing the Future of Work project at our website, hbs.edu forward slash Managing the Future of Work. While you're there, sign up for our newsletter.

Thank you.

If you have any suggestions or just want to say hello, email us at coldcall at hbs.edu. Thanks again for joining us. I'm your host, Brian Kenney, and you've been listening to Cold Call, an official podcast of Harvard Business School and part of the HBR Podcast Network.