Deep Dive
Shownotes Transcript
Give me an example of the sexiest, the most complex, advanced agent that you've ever built. And my answer is, sure, we can do that. But actually, you want the most boring, the most repeatable, the most low-hanging fruit agent to start with. I know people who are hiring AI managers, people whose role is just simply to manage the agents. The thing about agentic AI that is...
just so incredible. It can think for itself. Nobody wants to be a headline for the wrong reason or have a lawsuit. Designing experience between an agent and a human is another flavor of a skill that is very new. Critical component of that operating model is that business and IT partnership. Turning on tech is the easiest part.
It requires constant monitoring, updating, and kind of having a pulse on where it is and where it needs to go. When I talk to my innovation team, I'm constantly blown away. How they take capability of Salesforce AI product and turn it into the solution that elevates customer experience to a completely new level. I'm like, what? We can do that? This is not science fiction.
Hello, everyone. Welcome to Experts of Experience. I'm your host, Lauren Wood. Today, we are diving into a very, very big topic, agentic AI and what it means for businesses, for workforces, for leaders, and really humans at large. And I'm going to be talking about
And we are joined by Irina Gutmann, the global leader of AI professional services at Salesforce, who is here for the second time because so much has happened since we talked to her six months ago. And we're really going to dive into what is happening in AI when it comes to the agentic AI layer.
How are businesses implementing it? How are organizations adapting? And really, how can we powerfully leverage this technology safely?
So we're going to be talking about all of that today. I am so excited for this episode. I've literally been like giddy for it. Irina, thank you so much for coming back. So great to have you. Thank you, Lauren. Super excited to be here and share with you what's happening in AI world. And as you mentioned, a lot had happened. So in preparation for this episode, I was actually reflecting and thinking, we didn't talk like that long ago.
And when we first started and talked through different types of AI and evolution of AI, I was mentioning that agentic technology is somewhere there on the horizon, but let's just not talk about it yet. Literally a few months later, not only we're talking about it, it is the primary focus for Salesforce as well as many tech companies. So very happy to be with you and unpack this very relevant topic.
Amazing. I want to make sure that everyone is caught up to what agentic AI is, because it is different from the generative AI and the predictive AI. So can you just really quickly define those different types of AI and where we are today?
Absolutely, yes. Let's get down to the definition and kind of bring everyone up to speed. Let's start with predictive AI, which probably been around for longer than all the other AIs. And
The definition is literally in the name for all those technology categories. Predictive AI focuses on making predictions based on data, based on rule, and based on the structure that we provided with. A good example in using Salesforce example would be lead sales lead qualifications.
will give all the necessary rules, information, and data to predictive AI. Using that information, it would categorize and qualify the leads and put it in the priority category for salesperson to focus on. Or case qualification, when again, using a set of rules, AI would qualify and classify various cases for the service agent to work on. Now,
Now moving on to a generative AI. Again, the definition is in the name. It generates dynamic content. It still uses the data, but it is using natural language models to generate dynamic content based on the request or what we call as prompt or basically what you put in ChatGPT because ChatGPT is the most common example of generative AI.
I want to stop you right there and just make sure, because this is the LLMs. Generative AI is LLMs. So it's when we can ask a question and it can pull from many different sources where predictive was really, here's the box we want you to operate in. Generative can go further than just a simple set of find me, for example, a sales lead that has this revenue in this industry with this number of employees. Okay, great. Correct. Correct. You're absolutely correct.
So now we're going to go one step further and talk about agentic technology, which is the topic of today's conversation. It sees the components of everything that we talked about before, but elevates it to the next level. It still has rules. It follows the rules, but it makes a decision how to act based on those rules. It still uses data.
But it uses the data to almost reason on what action to take and how to interact. And yes, it interacts to us based on the instruction we're provided with, but it uses natural language processing and converses with us in the natural language.
So to summarize, we still have rules, we still have data, we still have actions and instructions, but this agent uses this information to reason, to make decision, and to converse with us in natural language.
One of the examples of an agent, and we will talk further about different type of agent, but the funnest example that I can give is if you've been to Bay Area, they have VAMOS, which is the self-driverless car. This is an example of autonomous technology that operates without a human driver.
It makes decision of how to switch lanes. It takes the information provided to it, instruction provided to it, as well as the data that it takes from the road and makes decision how to switch lanes, how to accelerate, when to stop, etc.
Mm-hmm. I love that you use Waymo as an example, because if you are in a city that has Waymo and you see them driving around, it's this crazy experience. I mean, I'm still like, I can't believe we are here, but we are. They are driverless cars. They are following the rules of the road. And when you think about it, there are pretty strict guidelines for how to drive. But then there is also this element of
intuition that is required of, oh, I feel like that car is coming at me really fast. Maybe I should slow down and let it pass or anything like that. And the thing about agentic AI that is just so incredible, which we're going to dive into so much more, but the thing about it that is so incredible is that it can think for itself. All right. Kind of creepy, but here we are. And there's so much, there's so much opportunity. Yeah.
We refer to it as a reasoning engine, which kind of sort of an evolution of thing. One step further, it also has memory. So it can tap in into the previous data and information and make decision not only on the data provided at this time, but also from the learnings of data that it has available to it before. So we refer to it as it has memory in addition to reasoning. Great. We love this.
I want to make sure that we ultra define this for all the folks in customer experience, because I do hear...
chatbots and AI agents being intermingled. And can you just now apply this definition to the difference between a chatbot and an AI agent? Perfect. So let's keep in mind what we just learned about the genetic AI, right? Components of reasoning, memory, conversation, being able to infer action based on the information that's available to it and compare it to the chatbot.
Chatbot follows a very prescribed process flow. It is pre-programmed how to intake information and how to respond based on that prescribed information. It cannot deviate. It cannot change a course of action or interaction based on that process flow. And it does not understand natural language. For example,
Let's say we're using a chatbot. And don't take me wrong, chatbots are super effective and super fast for addressing repeatable processes with very few deviations. And in fact, if you have that instance that it's literally a repeatable process that has prescribed process flow with very few deviations, maybe chatbot is an effective technology.
However, if we, let's say, ask a chatbot, where is my order? You have to ask it in a certain way and provide an order number. Otherwise, poor chatbot might be lost. If you interact with an agent, you say, hey, the thing that I wanted from you last week, what's up with that?
An agent literally should be able to interpret the slang, the various variation of the language, and come back to you saying, oh, you're looking to find out the status about your order. Let me get that for you. Would you mind providing an order number? Agents should be able to infer based on the information provided to it.
what the next step should be and what response should be to a human. Agent could pick up on the tone and change the way if I speak in slang, maybe in an interaction or two, agent will start responding to me in some slang versus a formal language. Chat cannot deviate from that.
Every company using agent, they might want that agent to reflect the tone of this company to begin with. More formal, more casual, cool versus some other type of a flow representing company's brand. You have that flexibility with an agent. Chatbot would not be able to do that. I think from a consumer standpoint, it is, I'm just so excited as a consumer because we've all had that experience.
where we're on with a chatbot and it is not understanding what we are trying to say. We are, you know, maybe there was a spelling mistake and it's just like taking us a totally different direction and it's just downright frustrating. But with an agent, I feel so much more comfortable
willing and able to actually solve my issue with that agent. Like before I felt, oh, chatbot, you're just deflecting me because you don't want to talk to me. You don't want to hear what my problem is. And now with an agent, I feel seen and heard and my problems getting solved faster. So it's really like this incredible balance between
an actual human who can be slower at answering these things at times. And, you know, the chatbot that just wasn't solving my problem at all. So yeah, I'm really excited about this. So last time you were on, we talked a lot about human in the loop.
But this is now advancing. How are you working with agents now? If you can kind of give us an overview of that. Absolutely. And by the way, yes, you are allowed to make spelling mistakes with agents, which is awesome. Great, because I make them all the time. But going back to your question, we did refer to a previous iteration of generative AI as a human in the loop, meaning that human makes the checks and final decisions. Right.
Now you have a technology that almost operates agentically. And we change our phraseology from human in the loop to human plus AI. We now have AI augmenting and expanding humans' capability with an assistant of this digital, let's say, assistant called agent.
Humans still have a decision power and we provide agent with instructions what to do or not to do. But once those instructions are provided, that agent is able to assist us in a way that literally translates to human plus AI versus human is in the loop or in the hell making decisions for AI. So I think the thing about this is it's much more analogous to AI
an actual human or an employee or a customer experience agent. If we think about here's the guardrails, here's the types of things we want you to answer. Here's how we want you to answer them. Now, instead of saying, yes, answer it, we're saying, okay, you answered this. I'm going to give you feedback so you can do it better.
Next time, because it's really being like trained through those types of interactions of feedback where it's operating on its own. We don't have to say, yes, go do that. It's doing it.
but we now need to work with it in helping it to grow. Absolutely. And before we talk about agent us, potentially when people refer to it as digital labor, I think it would be helpful to understand main components of an agent because that will help us have conversation about agent being that digital employee. Agents have five components. We're going to start with role.
Just like a human will have a job description, an agent is going to play a specific role. For example, let's say this is going to be a customer service agent. And the job of this customer service agent is to answer a specific set of customer questions. So we just define the role of an agent. Next step is action. What is it actually going to do?
Well, we want it to answer FAQ, frequently asked questions, and we want it to do certain things based on the instructions that we're going to do, that we're going to give it. And we also don't want it to do certain things. So actions are based on instructions that provide the agent.
And it always includes things that we wanted to do and absolutely things that we don't want it to do. Which brings me to our next component. It is called guardrails.
Guardrails are super, super critical. When you're putting autonomous technology in front of a customer, and by the way, we'll talk about an agent being assistive internal to the company versus customer-facing agent. But let's say in our example, we're talking about a customer-facing agent that is playing a role of the customer service representative. We define its role
We know it needs to take specific action, but now we're going to give it guardrails to tell it what it's not allowed to do. For example, it can only answer questions about the order and whether shipping is free or you have to pay for based on some zip code.
But if it goes beyond a certain set of questions that agent knows how to answer, it has to hand off to a human. That's where human plus AI comes in, is that always, always, always as part of those instructions or guardrails, there is an instruction of when to hand off to a human.
The next component that we need to discuss is data. It can't take any of the actions or follow any instruction unless it has information, the knowledge to make those decisions and take those steps on. So data is the fourth component. And the last component is called channel. How is it interacting with us? Is it living on the customer's website?
Is it an employee agent that lives in Slack? Channel is how this agent interacts with a human being or human beings being a customer facing or internal agent. So these five components of an agent really help us understand of how we're starting to make this transition from a technology to almost a digital assistant or digital employee. Mm-hmm.
Thank you for explaining that. It really helps me to start thinking about how do I start onboarding an agent? Because I just cannot wait. I cannot wait until I have my own AI agent assistant. I'm like, if you know of any tools, please let me know because I'm so ready. We can build one for you. Okay, perfect. I can't wait. I really cannot wait. But so you, as leading the professional services team,
department at Salesforce, you are helping organizations to implement these AI agents. And I'd love to spend some time talking about how do you really approach that? If you can walk us through a little bit of the practices or the frameworks or the structures that you utilize to help leaders and teams think about where are we bringing these agents in and how are we training them
With everything, say goodbye to chatbots and say hello to the first AI agent. AgentForce Service Agent makes self-service an actual joy for your customers with its conversational language anytime on any channel. To learn more, visit salesforce.com slash agentforce. You just shared.
Absolutely. And when I meet with customers, sometimes customer would tell me, give me an example of the sexiest, the most complex, advanced agent that you've ever built. And my answer is, sure, we can do that. But actually, you want the most boring, the most repeatable, the most low hanging fruit agent to start with.
Because what you want to start with is autonomous technology in the area where you want to augment humans. We want to leave creativity to humans. We want humans to address more complex scenarios. And we want to augment humans' capacity in the areas when there is repeatable, there is low risk.
And those are the questions that human may not want to answer. Or if we're looking at the assistive agent, the agent that lives inside the company and helps its employees, the same logic applies. You would want it to answer the question and help with those frequently asked problems or frequently occurring problems that humans are facing.
I spoke with the client recently from regulated industry, from financial services, and this particular customer was from the bank. And she said, well, you know how hard it is to implement agentic technology in the bank.
And I said, absolutely. But if I ask you to make a list of questions that your customers call and ask that always, always, always end up in pretty much the same answer with very minimum variability. What products does this bank offering? How do I open account? I bet you'll be able to provide me a list mile long. She said, absolutely. I said, that's the list that we give to the agent. Yeah. Yeah.
So we start with identifying the, it's going back as to any other project, identifying the business objectives, identifying pain points and saying, can this technology help address those pain points? We also want to start small, incremental and low risk. In addition to that, we need to identify data that will support that type of a use case.
But, and it's becoming more and more relevant with AI and agentic technology, turning on tech is the easiest part. The hardest part to tackle, and we talked about it last time at Guess What did not change, is organizational readiness. Yeah.
I just had an executive actually awesome event. It's called Empowering Women in AI. And we had a room full of women executives from all over tri-state area. And I am local to New Jersey. I wanted to say New York, but local to New Jersey. But we did have an event in New York. And
And what we talked about is exactly that. How do we get started? What are some of the blockers within organization? And we had absolutely incredible discussion that was focused on organizational readiness, on the new operating model, way more than about the tech. If I could boil it down to the successful approach,
I think we'll talk about five phases. So first, just general readiness assessment. Kind of checking off all the points. Do we have minimum required criteria? Check, check, check. Just enough data. Identify use cases. Enough people that support this initiative and wouldn't run away. So that is checked. Second,
incremental implementation and unlocking of the capability. You don't want it to be a year-long transformation program where you don't see any outcome until we implement all complex agents in the world. Three, foundational technology. We need to have data and integration capabilities in place in order to unlock agente capabilities and broader AI capabilities.
And that we can also do as an iterative approach, identifying just enough minimum criterias from a data and integration perspective, but also building out that holistic roadmap for the end state.
Tell me a little bit more about that foundational tech. What are some of the key components that are needed there? From a Salesforce infrastructure perspective, and I'm going to talk about Salesforce, we have two key elements. We have data cloud, which is critical for agent force, which is the technology component that Salesforce refers to. That's our version of agentic technology. We call it agent force.
Data Cloud, which allows for data syncing and console synchronization across various sources. And MuleSoft is our integration layer that allows to connect any data from anywhere. Combination of Data Cloud and MuleSoft unlocks any data needs, any communication needs, being it structured data, unstructured data within Salesforce or outside of Salesforce. Mm-hmm.
Okay, great. So it's really having the technology to store and process the data that is then going to be used by AI, by your agents. Okay, great. Thank you for answering that. What's step four? Absolutely. Absolutely. Next step is what we talked about, organizational readiness. Do we need to build that new operating model? What are the new roles? And by the way, new roles are required.
We're now talking about agent owners and agent monitors. Some people go as far as saying, do we need agent HR? So I don't know how many agents who has, but with one, you probably don't need an HR department. Totally. I know people who are hiring AI managers, people whose role is just simply to manage the agents, which is a, it's a, it's a management position, but it's very different. Yeah.
Exactly. And there are variations of those new roles. Those new roles need to fit within organizational operating model. Standing up center of excellences that help manage that, building out new operating model, defining that process of introducing new roles, enabling new roles, upskilling people is the next absolutely critical component. And we talked about it before. If some of those
things were considered optional with more traditional technology, AI and especially agentic technology truly changed it and made it, as we said, as foundational as probably as data. Because if your organization is not ready to take responsibility for this technology, no one's going to use it. Change management is even more and more critical.
Last piece is the roadmap. We don't want to stop with one agent. Understanding what that ultimate North Star and how organizations are going to progress in maturity of introducing AI, agents, and data is the last piece of the puzzle that I would like to introduce. Awesome. I want to go deeper into the new operating model. I think a lot of leaders are thinking about
And if they're not, they probably should be thinking about what is my organization going to look like? Like you said, what are the roles, the skills, the governance that we need to implement? And I'm curious to know a little bit of, you know, any examples you've seen of this being done really well. And I know it's a lot of this is future thinking. We don't even fully know what it's going to look like in six months from now. But yeah, what have you seen?
You're right. Absolutely. This is very new and future thinking. And a lot of my more mature customers like, do you have an example? And I'm kind of say, well, you're the first one. So let's figure out together. Yeah, totally.
But more and more, we have instances when some of the leading customers are being trailblazers in this area. But let's talk about kind of some foundational components of that operating model. It is absolutely critical. So when we're talking about bringing in new operating model, it is probably the easiest way to think in the form of standing up some kind of
body, meaning a center of excellence or practice focusing on that. In fact, Forbes magazine says that the best way to handle emerging technology is to have some kind of a center of excellence around it.
There are debates, oh, center of excellence can be obsolete, they're that ivory tower. Not if you design it right. If you design it right, there's a way to make center of excellence really foundational and a way to help control and also drive that innovation. Let's discuss some of the key elements. We need to align on strategy and objectives of what it is.
We need to get visible and actionable leadership support, understanding that it's not an extracurriculum activity, that it's actually critical for the organization and everyone is on board. What's that common vision that we're working towards and what's going to be the charter of this particular center of excellence? For short, I'm going to refer to it as COE. And let's assume it's an agenda or AI-focused COE. Do you have any examples of what that vision is?
The vision might be is to have agent first mindset and to prepare organization to intake agentic technology and to elevate, I don't know, our customer service and to the next level with the help of agents. Granted, I made it up on the fly. So no, all good. All good. I just wanted to understand because it can be so big.
broad. But also the vision is so important because it at least starts to give us a little bit more focus in what is this body of work about? It's the headline. We need to set that headline to kind of guide everyone towards something. What's the headline, but also what is company trying to achieve? Where is the company trying to get to? And then we can articulate how is that particular body within organization is going to help company to get there.
That's where it comes. Vision is supported by strategy, by objectives, right? Then we need to define this center of excellence itself. That would entail operating model, meaning how is it going to be structured? Is there going to be an overarching leadership committee? How is business and IT are going to work together in that operating model?
We talked about it during AI. It is even more critical with a genetic technology. IT alone, unfortunately, so sorry everyone who works for IT, but you can make that decision alone. It has to be in super strong partnership with the business because guess what? AI is augmenting most likely a business employee. Therefore, IT becomes innovator, capability provider.
But business needs to be in the room saying which capabilities they need, which business problems need to be addressing. So critical component of that operating model is that business and IT partnership. And then there are kind of discipline going across such as governance, risk mitigation, and responsible AI usage. Another critical element that I would say is a fun element is innovation.
I wouldn't recommend having a genetic COE without explicit focus on innovation to drive that culture of innovation, but also to drive the culture of experimentation.
to give a specific focus area where people can freely try new things, try new technology, fail quickly, improve upon, and adjust it per that company's need. Another important area is education. We just talked about all the new skills that we will need to define that will go into that operating model.
You said agent manager. I said agent owner. Some people say agent supervisor. I've heard roles that is a content creator, and we'll talk about that, why it is important. Prom builders, et cetera, et cetera. There are roles ranging from the ones that you need to manage agent to the roles that you need to have to build agents.
such as I said, content creators or prom building, because guess what? Articles that we human consume need to be written differently for consumption of AI. AI consumes information differently from humans. All of those roles need to be upskilled
trained and then continuously trained as new innovation or new capabilities come about. And with AI, it is faster than we've ever seen before. So that continuous upskilling needs to be a pretty robust process. And that is why we need to have explicit focus on it as well. Where can leaders go to upskill their teams? Because like you said, it's happening all
So fast. And if we try to build our own systems of education, I assume we're going to be a little bit slow. Is there any resources or things that you would suggest people go to look at or learn from?
No, this is definitely a hard one. And I can tell you where Salesforce are currently addressing that situation as well. As you know, my team is explicitly focused on AI and we're working very closely with product organization to stay ahead of the curve. But Salesforce Professional Services, 10,000 people, Salesforce Partner Ecosystem,
is multiply that by a hundred. All of those experts need to stay up to date on the latest capabilities and technology. Yes, sure, there are million classes in very respectable institution that will give you foundation and to give you a core knowledge.
but then it goes to the next level. So there are multiple upskilling, right? Understanding the technology, understanding the essence of this particular, you know, what agents are, how they constructed, how you do prompt engineering, and that skills could be gained from any respectable known institution. There are classes, there are courses there, but then there's a next level of specific to that product. And that I would suggest getting from,
partnering, being at a Salesforce, or if you're working with a different technology provider, working with that particular company to know the nuances of that product.
And that's what we're doing. We're working closely with product. We're putting new programs in place that combine propagation of knowledge and propagation of experience across entire professional services. And we came to the realization that there's no longer an option to spend three months putting a training program together.
Materials and information needs to be available very fast. And we need to build into those enablement program constant updating and changes. Because sometimes the answer I provide you today might be obsolete and weaker too. It's happening so fast. I mean, the good news is that AI can help us put those training programs together really quickly, which is great. But what's coming to mind for me right now is the fact that the vendors, for example, Salesforce,
You now have another role of education that goes beyond simply this is how you use our tool, but this is how you actually bring your tool into your organization, which is exactly what you do. And I think it's something I tell my clients all the time that if you want to start using AI, first look at the vendors you're already utilizing.
and go and see what technology they have rolled out and then ask them to help you learn how to bring it inside. And I just, I think that there's this new role almost for SaaS companies to not only say here's technology, but here's actually how you can utilize it in the way that's going to be most effective. And it goes beyond just simply like click these buttons, but actually like-
You know, here's the roles and the education your team needs and all of that, because that's really essential to using AI. No, you're absolutely right. And that's why it folds all into those components of the organizational readiness worksheet, because this is exactly what it will focus on in addition of what this tool and technology is, is
how it's actually going to be used and how companies are going to assume responsibility for it and continuously work with it because AI is ever evolving. It's not once and done. And that's what makes it unique and super uncomfortable for a lot of us.
Because if you build a system, unless somebody goes and changes the code, it's pretty much going to be operated as expected. With AI, with its learning capability, with its reasoning capability, it is going to evolve.
the data that it's grounded on is going to change. So there's a lot of variability in this technology and therefore it requires constant monitoring, updating, and kind of having a pulse on where it is and where it needs to go. Yeah. The other thing I'm really curious about, and I'd love to even hear this from like a personal perspective from you, is what are the skills we need to be learning?
As we look towards the new AI age, like what are the human, what's the human advantage that we can really like focus on? And I'm curious, even just from a personal perspective, what have you been learning and trying to upskill yourself on?
What's foundational to all AI is how we interact with it. Because AI is grounded or is based on natural language models or natural language processing. Yes, when you are a user of AI, it will respond to you. And as I said, use example of broken English or slang. But in order for it to do it, somebody had to tell it to do that.
And that's where we get to prompt engineering. Prompt engineering is absolutely a foundational skill set to have. And when your agent is not actually answering the questions the way you want to, one of the first things we look at is how the prompt is constructed.
Sentence structures makes a difference. Punctuation makes a difference. Word choice makes a difference. Like you can be free with the words when you interact with the finished product, but when we construct an agent, all those things are critical. Prompt engineering is a new skill with a new requirement. I also mentioned that AI consumes information differently.
When we refer to knowledge articles, like, oh, yeah, we have a million knowledge articles. Sometimes we see that those knowledge articles need to be restructured for AI consumption. So there are roles, there are skills to restructure unstructured data. I know it's a mouthful.
for AI consumption. So AI interprets that information, make it kind of easier for AI to interpret this information that it could provide a better answer. And another fun one, and you mentioned in the beginning that sometimes you want to scream at the bot and say, what are you doing to me? Are you trying to torture or deflect me? That talks to customer experience. And we want agents to provide amazing customer experience.
Experience designer is a known role. Now we want them to elevate their skills to the next level, almost like conversation designers. Designing experience between an agent and a human is another flavor of a skill that is very new.
And that is from a person who is responsible for the team developing the agent. Those are some of the technical skills. I don't need to be an expert in those skills, but I do need to have foundational understanding.
However, what I was focusing on is what we were talking about is understanding how that organizational ecosystem needs to change. How do we approach change management differently? What are all those nuances of things that we've done before from standing up those organizational operating model, from defining all those departments and new roles, from integrating all of that new stuff into organization? What needs to be nuanced?
for a genetic technology? And how is that different from more of a traditional change management? A lot of it is the same, but still this nuance is important. Yeah, we're dealing with a very different technology than other technology rollouts. And it's also the nature of how it changes needs to be built into how we're operating. And the thing I also wonder is like,
Now that, you know, learning how to use AI, learning how to design our organizations around AI, everything we've been talking about is table stakes. But then I think about how are organizations creating a competitive advantage and how
The thing I think about, and I love your thoughts on this, is that it's really like in the connections and the way that we're listening to what it is that the customer really wants and needs. And it's almost like now that AI is dealing, handling a lot of these tasks that we've typically been bogged down with, what's next? Yeah.
I'd love your thoughts on that. Like now, what can we focus our attention on? Well, we still need to focus our attention on what's happening now. I'm getting ahead of myself. Um,
And what's next? It's really, actually, it's funny you ask because it really depends on people's personality and readiness for change and readiness to embrace this technology. When you're talking about, okay, we're good with AI, we're good with agentic arena, what's next? Some of my customers are like, really? I can turn in email generation? Yeah.
And no one's going to come after me. So we have a full spectrum of maturity when it comes to AI. But I think what's next and very near or already here is the agent, multi-agent.
That is the most logical next evolution is that most of the companies even now have multiple agents. When we talk about agentic technology, the natural progression for a lot of companies is internal assistive agent or multiple internal assistive agent, meaning they're not customer facing. They're still autonomous, but they are within the company and they're interacting with employees and they're there to help employees do their job.
Then you have customer-facing agent.
And then you have agent to agent collaboration. When you have what we call the multi-agent network, and then you have agents. And it doesn't mean from the same company. It could be Microsoft agent talking to a Salesforce agent. So that is broad spectrum of multi-agent collaboration, I think is the next wave. And I can't tell you what's going to happen after. It's really way too fast even for me. Yeah.
I mean, I'm just I just can't wait until I have my own agent who's been doing all my customer service chats. That's for me being like, oh, I need to order a new dress size or can you this thing showed up broken. Can you send me another one?
So Saks actually has an agent like that. Her name is Sophie and she handles this type of questions for Saks. They were one of our first agentic customers. And the demo that was shared at AgentForce, sorry, at Dreamforce last year, all the forces. The demo that was shared at Dreamforce last year blew my freaking mind. It's on YouTube. If you want to find it, the Saks example, it was great. And I'm really excited for that.
My question for you is, what are you the most excited about? You're in such a unique place where you can see what is happening in this space. What are you the most excited about? So I'm going to use an example of a session, as I said, that we conducted this week for women in AI. And we started the session with the exercise exercise.
We ask participants to go to a word cloud space and put one word that comes to mind when they think about agentic technology and digital labor.
And the responses that came back were mixed. Some were optimistic, word opportunity popped up, but some of the words were job replacement, scary, new. Then we conducted this discussion when we talked about how companies get started. What is the incremental and iterative approach? What are some of the foundational elements that you need to put in place?
how we partner with the customers to take it through the journey. And after only two hours, we did this exercise again.
Oh my God, such positive word. I did not see a single negative word on the screen. People were excited. Opportunity still was the hardest word, highest word, but people were excited. They said they use words as innovation, new roles, new opportunities. So very, very forward looking in just two hours. And that is what I'm most excited about is the chance to partner with my customers and
to help them understand and embrace this technology and to help them start taking advantage of this technology. Because if it's done right, and I love your attitude, I love that my company can build agents for you to help you with all of that. But-
Those that are a bit more of a skeptics or have a more of like an enterprise or industry where it's harder to introduce, to help them handle those objectives and show them how this technology could be really helpful is
is very rewarding. The other thing that I'm excited about is when I talk to my innovation team, I'm constantly blown away. I'm like, what? We can do that? This is not science fiction. You actually build it? And it is amazing how they take capability of Salesforce AI product and turn it into the solution that elevates customer experience to a new level. Mm-hmm.
I mean, the opportunity, like you said, that word came up a lot. The opportunity at play is immense, but it is almost so big that
it is scary of, I mean, it's a lot of change and it's a lot of change happening really quickly. And I could not agree with you more that there's, you know, the, the skeptics in, in the room. I mean, we should all be skeptical to some extent. I think it's healthy to have a little bit of that. But, but you should talk to my responsible AI team. There are a lot of skeptics there. I love it. I love it. But,
This is changing whether we like it or not. So let's jump on. Let's jump on the bandwagon. Actually, now that you bring up responsible AI, since we do have a couple minutes left, I do want to talk just to touch on that a little bit around what does it mean to be responsible with AI? Because it's a little bit feeling like the Wild West out there. And how can organizations approach this in a responsible way? So, you know, they don't have a lawsuit on their door next week.
No, thank you for bringing it up. It is a very, very important topic. When we're designing agents, we have a dedicated and explicit exercise called guardrails and risk management definition. What we're advising our customers to do as they define the role of the agent and what job it needs to do,
We asked them to define risk in categories that are very familiar to them. People, process, data, and technology. Okay.
So we work with the customer to list all the possible risks, all the possible violation that this agent could cause in those four categories. And then we take each of those items and we say, okay, how are we going to mitigate it? Are we going to mitigate it with people? Are we going to mitigate it with process, with data and technology?
Yes, the risk is inherent. There's some variability, vulnerability inherited in AI, but it does, as you said, it's not an option. This technology is here to stay. So what we're doing is developing framework, just the one that I explained to you, of how companies can go systematically forward.
from defining an agent to defining risks, to defining risk mitigations, and then turning them into what I referred to earlier in our conversation as guardrails, which is instructions that we give to the agent what it's not allowed to do. The other thing that we do to make sure we do
stress testing for bias and toxicity. Salesforce already has a trust layer that is very, very robust and pretty much nothing can get in past us, nothing bad from the outside. However, we put agent through additional stress
of bias and toxicity testing. When we overload the system, we talk to the agent in bad language, profanities, using racist slurs,
On purpose. It's not a pretty testing, but it's absolutely necessary because as you said, nobody wants to be a headline for the wrong reason or have a lawsuit. So that type of testing is absolutely critical. And my responsible AI team developed a methodology of how to do that kind of testing. And we set thresholds really low, like 1% or less than 1%. And I'm proud to say that our agents have done.
Amazing. Just for anyone who's thinking about utilizing, which we all should be, how are we utilizing agent technology, agentic AI? We have to have those conversations. What are the risks at play? Just put them all out on the table. Let's make sure we're thinking about the edge cases because they are there. And when we are not hand in hand with our AI the whole way through, when the AI is operating independently,
we need to make sure that we've thought about all the risks at play. So thank you so much for sharing that, Irina.
As per usual, we always like to ask this question to end our show. I'd love to hear about a recent experience that you had with a brand that left you impressed. Why was it amazing? So I thought about that. I think I'm going to go with Starbucks. I know it's simple and I know everyone's familiar with it, but you know what? When I'm getting off New Jersey Transit, headed to my, on my way to the office, I click on my Starbucks app.
It allows me to set my favorite stores. There are a few. I have one set up for San Francisco because Salesforce headquarters are in San Francisco. There is a Starbucks across the street from our office. So it has my different locations. It has my favorite stores per location. I click. It remembers what I've ordered before. It displays my frequent orders. I usually go with caramel macchiato, so it comes up. I click.
click and when I get to the store, I can see on Tableau saying, hey, Irina's order moved from received to in progress. I know next it's going to be on ready. It is simple. It is simple to use, but it has just enough key elements to make my life easy and
And also from a customer service perspective, I went to the store that happened to be backed up. I had an urgent meeting. I came to the assistant and I said, I'm so sorry, but this is the situation. Is there any way you can help me? She looked and said, in fact,
You are next in queue. Let me move you up so they start processing your order. So it's not only visible to me as a customer. There is an option for an assistant to see where I am, to do the overrides and to address my needs. And this is how human plus technology working together provides amazing experience.
I love this example. Speed, efficiency, thinking about what you need. You're in the morning getting off the train. You don't want to stand in line. And the key thing that you shared is that it's also enabling the staff to support you, to jump in and intercept like a team. Absolutely. So thank you so much, Irina. This has been such a wonderful episode. I'm sure I'll see you in a couple of months.
as things keep changing. And we'll have something new to talk about, I'm sure. I'm sure we will. Well, I wish you a wonderful day and we'll talk soon. Thank you so much. Thank you for having me.