How to Setup AI Agents That Your Customers Will Love
41:18 Share

We were, I think, the first customer in Europe to go live with AgentForce. It's learning by doing. It's a completely revolutionary technology. We're in at the very beginning of it. And it's something that you need to take as a long-term investment. I applaud you for being the guinea pig on such a new technology. This is not really the moment to

do it yourself. And I know Mark Benioff talked about this at Dreamforce this year. He's like, don't DIY your AI. Let's leverage the expertise here. It took longer than expected. And one other challenge was reporting on return on investment. How successful in general was the handling of the conversations with our customers? It's not a quick win deflection journey.

It's actually investing into relationship you have with your customers and tackling everything that you can take away from humans. I'm sure you notice that the reality is far more complex than just turn it on and it's magic. If you read through some of the transcripts, it is impressive how seamless it is and how the customer didn't even realize they were talking to an AI agent if we weren't declaring it at the beginning of the conversation.

Hello, everyone, and welcome to Experts of Experience. I'm your host, Lauren Wood. The AI revolution in customer experience is everywhere in the headlines and especially on LinkedIn. But what's happening behind the scenes is perhaps another story. What does the implementation actually look like when you move past the buzzwords?

buzzwords into the real world application. Today, we are speaking with Laura Miske, the customer experience manager at Secret Escapes, who has recently rolled out Salesforce's agent force. And we're going to talk about what that implementation actually looked like and get her perspective on what really makes it work.

In this conversation, we're going to explore how AI is reshaping customer experience. Of course, it's our favorite topic on this show and really key strategies for a successful rollout and what opportunities are at play here. Laura, thanks so much for coming on the show. Thank you for having me. Hello, everyone.

So AI and customer experience is often talked about as an efficiency tool. It can reduce agent workloads, it can increase automation, and of course, deliver a seamless service. But

Secret Escapes, as you rolled this out, I'm sure you noticed that the reality is far more complex than just turn it on and it's magic. Tell me a little bit about why you decided to utilize agentic AI in your customer experience and how did that implementation go?

Yeah, so the reason why we started the journey with AgentForce was mainly around trying to deflect, or at least the initial thought was around deflecting the contacts for the highest reasons for contact that we were getting through our live chat channel. I have to say that partially one of the reasons why we got on this journey is also because we wanted to be at the forefront of the new technology. So it wasn't just about...

the efficiency side and the big promises around AI, but also we wanted to be on the AI wagon for everyone else. So we started the pilot with Salesforce. We were, I think, the first customer in Europe to go live with AgentForce. So it felt really good to be at the forefront of something completely new.

unknown and exciting like AI, but also it was a learning curve and excitement turned into realization that it wasn't exactly what we expected, but also the excitement is not over and there is a long way. It's a long-term investment. Of course. So...

Yeah, it's really, really fun to be working on this project where the technology is so new and it feels new for everyone. So you don't feel left out in a way. You're not alone. Totally. Well, I applaud you for being the guinea pig on such a new technology, but also a technology that has so much potential and also immediate potential. We're going to get into all of that stuff. But I think that, you know, with any project,

organization, with any CX org, the sooner the better, right? The sooner we dive in, even if there's going to be challenges, at least we're learning and we're growing.

right from the outset? Probably AI is the new internet smartphone technology revolution. So it's like being in the midst of something that is probably life-changing. So it's exciting and scary at the same time. Of course. Let's talk a little bit about the challenges that you faced. And I know so many people who are embarking on their own AI journey. I have a

There there's often like, oh, I didn't expect that to be a challenge. Tell us a little bit about what you faced as you started rolling out agent force. So as I said, we started with setting up use cases for the usage of agent force based on the highest reason for contact, which is.

In hindsight, it was the only way we could do this being so unknown and, you know, unknown territory for everyone. But in hindsight, I think we probably chose use cases that were a little too complex. So it took a little longer to build the functionality as it wasn't just answering FAQs questions. We started off by asking the virtual agent to actually perform some actions.

So it took longer than expected. And one other challenge was reporting on return on investment, which fell short of what we

felt a bit more complicated than what I expected within the functionality itself. So it took a little bit of a DIY solution to try to get an understanding of how successful the deflection rate was for the agent and how successful in general was the handling of the conversations with our customers.

Yeah, I would say those two. And probably the setup process, it's fairly easy, especially if you're looking into setting up a use case that is an FAQ-based type of use case. But the time that it takes for testing and quality assessment of the interaction, it's quite extensive.

And that's something I completely, not completely overlooked, but I kind of did not take into account that much when we started off this journey. Yeah, for sure. For sure. I mean, I think the nature of...

And I want to just ground everyone. And that is what we are talking about is agentic AI. It goes beyond a chatbot. It goes beyond a LLM. This is really a almost a virtual employee that we are training to act remotely.

on your team's behalf within a certain set of guardrails. And for anyone listening, if you want to go and hear more about Agentic AI and Salesforce's journey of Agentic AI, check out our episode with Bernard Slowey, who's the SVP of Digital Customer Success at Salesforce.

But I think it's really fascinating to hear your take on it because it is different for every company, the types of things that you run into. And what you were just sharing about how it was harder to show the ROI. I'd love to go a level deeper on that piece. And why is that? Because everyone's saying, okay, let's roll this out because it's going to drive cost savings.

And there's, it is yes and. And so I'd love to hear a little bit about what was challenging about that. It was mainly the fact that the deflection rate we were expecting to achieve has not been as high as we expected. Partially, it could have been an unreasonable expectation to have, because as you said,

You know, we thought it would be much more immediate, whereas this is something completely brand new and you kind of have to learn by doing. So it's part of the challenge and part of the beauty of this is that you just learn as you go. So that was the challenge. And yeah, because the deflection rate is not what we expected, we kind of have to try different ways of achieving and maximize the power of the technology that we have.

And that's also why this is such an interesting journey, because you have to kind of learn while you're on the ground. There isn't much more we could have planned around this apart from the choice of use cases. So I think it's also about changing the perception on the metrics that you want to look at to measure success of a technology like this, probably ROI, right?

is not the best matrix to give it justice. People could consider also the impact on CISA and satisfaction of customers in the interaction with the virtual agent.

The easiness of how it changes from a menu-based bot, which is what we had before, to an LLM that can literally pick up your tone and can have a conversation with you like a human. So it can pick up the context, it can resend the information. So it's also shifting the focus from ROI to a range of wider matrix options.

It's possibly something companies have to take into consideration when deciding to embark into an AI journey. Yeah, I love that you say that. I think a lot about how the real benefit

to AI, of course there are those cost savings, but we need to play with it and tweak it and find, you know, how can we use this in a way that's both preserving the quality of our customer experience and allowing us to speed things up. And it takes experimentation. It's not an out of the box, you just save 20%. It's, we need to work on it a little bit. But the real benefit here is the customer lifetime value. Yeah.

If we think about creating a great experience using AI and the trust it can build with the customer,

I'm more interested in that metric, in what drives customer lifetime value. How do we extend that? How do we improve that through AI technology? So I think you're totally right. ROI isn't the straight up metric that we should be looking at. There's more to it that goes beyond ROI.

the simple return on investment. In another context or flipping the script a little bit, were there any unexpected benefits that you found when

when rolling out agent force? I was impressed by how easy it is to set up the actual facility. If you're not asking the agent to perform any actions, it's literally like typing a narrative and it's like you're talking to a human, explaining a new employee what they're supposed to be doing and telling them what they should and shouldn't do.

and how easy it gets on with it in a way, how fast it learns. And if you read through some of the transcripts of communications with our customers, it is impressive how seamless it is and how the

Probably the customer didn't even realize they were talking to an AI agent if we were declaring it at the beginning of the conversation. So that's something that I knew was going to happen. But every time I see it, I'm like super impressed. Wow, this is happening. Yeah. So the easiness of setting it up, I was really impressed with it being easy.

you know, used to coding and in the backend, you have to be an engineer and developer to set this up or to even test. This was kind of my bread and butter because you actually need someone that knows the

and how the customers are asking questions in order to set this up effectively. So I felt it was a very rewarding adventure for someone like me that is not a technical person, but got out

of it quite a lot and learned a lot about this technology. - Amazing. - Say goodbye to chatbots and say hello to the first AI agent. AgentForce for service makes self-service an actual joy with its conversational language anytime on any channel. To learn more, visit salesforce.com/agentforce.

Yeah.

What did you learn as you started that adventure? Well, it takes a lot of time to quality check the interactions and it takes longer than I expected, I have to say. It's quality checking and testing and troubleshooting and changing the instructions as you go. In terms of training it, it's

I feel the key here is to use historical interaction, which is something that is coming up with Salesforce. So being able to base the testing and the quality assessment of the interaction with the virtual agent with historical interactions, it will be a game changer because

now it was a lot of manual work for a human, funnily enough, to be able to be on top of the quality of the interactions and tweaking what needed to be tweaked in the back end to be able to improve the quality of the interactions. So what did you use to initially train?

the agents? So we wrote up the use cases and we did a lot of testings based on different iteration of the same questions. And we used previous interactions with customers that we entered into the system and tried to test the reaction of the LLM. Okay.

But then how did you consistently improve it? Tell me a little bit about your QA process. So at the moment, our QA is human driven, meaning that we review a certain number of interactions per day and we have put together a form where we score the results.

Quality of the interaction, if there were any issues, if the issue needed to be escalated and if it was escalated correctly. And then whenever there is an issue flagged via this QA process, us and the development team that we have worked together with to set this up.

We'll go in and try to see if it's an actual bug or whether it's just a tweak of an instruction, whether we need an additional KV article, whether we need to change an existing KV article.

So there is a lot of trial and error. And at the moment, there is no functionality to bulk test or bulk QA, which I think in the future is coming and it will be very handy, especially for a complicated industry like ours, where even the same theme in terms of type of query can be asked in 20 different ways. Yeah, for sure. Yeah.

How has your team changed as a result to this technology, if anything? It's funny you ask because it's very much a current discussion that we're having because my role, of course, I'm mainly responsible for the quality and performance of our external vendor that does the frontline support. And then I got involved into the AI project, but it really...

takes focus to be able to work on this and improve it in a short amount of time. So having a pool of experts that is focused on this full time, or at least the majority of their working time, I think it's key to get the best results in the fastest way.

This might be a sticky question, but do you think that from what you've seen so far that you'll be able to hire less people or outsource less and let the AI handle a vast majority of the tickets?

My take on this is rather than seeing the AI as a way to cut jobs is I think a way to scale up the company without growing the workforce so much, but not cutting what we have. So it's bringing the company to the next level without having to add, um,

more resources, but definitely having what we have focusing on the more interesting side of the interaction with customers. My goal is to take away all the repetitive and admin-y kind of tasks that agents have to perform. It's not just

in the realm of repetitive queries and easy queries, but it's also workload management and what they have to do to flip through a case. I think AI is key, summarizing a case, case logging, all these things that take up

Two, three minutes per interaction. We could be spending more time on the phone with a customer, helping them upselling, giving an advice of where to go next for their holiday. So yeah, investing more time in building that long-term relationship is the way I look at this. AI should be taking away time.

the redundant tasks and giving the agents more time to do what humans are good at. So building relationship. Completely. It's such a big opportunity for improving our customer relationship. I mean, I think historically, I mean, not I think, I think we can all agree. Historically, no one really wants to deal with customer service. You know, having to reach out is just like, oh, such a pain. What if it's not?

What if it's actually a great experience that greatly adds value to your interaction with this company or your purchasing from this company or whatever the case may be? I think there's such a massive opportunity for us to really move from that cost center to a...

of insights and relationship building with our customer. And I know pretty much everyone, everyone I know in customer experience wants to be doing that. Yet so often we're bogged down by, like you said, those experiences

doing summaries or just moving through multiple tools or just this like technical debt that we are often so burdened by, we are now being freed up to do the things that we really know are going to make an impact for our customers. Yeah, definitely. And I think when it comes to, you know, a

near future and thinking of using AI on voice, for example, the way I would approach this is not saying, oh, now the machines are just going to handle all our conversations and we will deflect everything. It's actually the opposite. It's kind of understanding actually what is it that the virtual agent can do and

And filter that out and make whatever the humans are needed, you know, at closer reach so that members, and we call our customers members, can, you know, be closer to the agents when an agent is actually needed rather than trying to funnel everything through the AI and probably frustrating customers because there is...

There are certain things that AI will not be able to help our customers with. But the rooting of those questions, picking up the tone of someone, whether a customer is escalated or angry on the phone and immediately passing it through the right type of agent with the right set of skills. That's how I think we should be looking at AI.

helping us making good connections between customers and agents. Mm-hmm.

How did you go about deciding what the AI would handle versus what a human needed to handle? Yeah, so the way we approached this was very efficiency driven. As I said, we looked at the highest reason for contacts and we came up with a set of use cases. What we tried to do in the instructions and scope and description of the use cases was

was to give it enough boundaries so that we have enough control on what it could and couldn't do. But there is a fine line between putting boundaries and between putting limits to it. So it took a lot of refinement, but it's basically based on those use cases that we picked at the beginning. In hindsight, probably we could have...

opted for less complicated use cases, as I said, and start from simply FAQs, question and answer kind of use cases. However, our industry, unlike, for example, retail, there is much less that you can actually resolve in an FAQ kind of way.

So the personalized kind of question about a specific booking, about a specific deal where you actually need to action something, it's much bigger than the simple FAQ environment. So I think we did the best thing we could do. And we were quite brave in exploring also the actions and make it available to the virtual agent to be

able to do something and not just answering based on the knowledge base. And that's something we want to leverage on. That's how we picked it. But it's a mix of type of queries and type of interaction that you can have with the virtual agent that we got to, but quite unconsciously, like I wasn't aware of conscious that we

we were mixing up FAQs and the personalized type of questions and actions. Now I realize it in hindsight, when I look back and look at the different type of use cases we have, and when you go to conferences and you see, you know, that we just should start from FAQs only and then move forward. I was like, okay, we were quite brave and, you know, mixed things up a little bit. But that distinction between...

Easy tickets, let's call them the FAQs, the Q&As, the simple, I have a question, here's the answer tickets versus the more complex. I'm assuming you're in the travel industry. So people are asking questions about their trip and where to go or what happens next, which just is a more complicated, which is more complicated in its nature. Yeah. Let's talk a little bit about data.

Because I know that this is such a big piece when it comes to AI implementation. How did you approach preparing the data to train the AI model? Because of the use cases that we picked, we had to have a knowledge base that could sustain the FAQs type of questions.

We were already using a knowledge base and we had articles that belonged to a menu-based bot that we retired when we implemented the agent force agent. So we used a little bit of that, of course,

It might be in need of a cleanup, but we were already at a very advanced stage when it comes to having the knowledge required for this project inside Salesforce. Some of the use cases required more personalized information, as I said, information about your flights or luggage or stuff.

your cancellation terms for your specific bookings. So we had to push that type of data inside Salesforce for us to be able to then have the virtual agent look at the data depending on the question that he was asked and being able to refer it back to the customer. So it took a little bit of preparation. And I would say the more data that is in Salesforce, the better chance

It is for you to start the agent force journey. It's all about the quality of the data you have. I am hearing that time and time again, every single person we've had on the show talking about AI implementation, especially agentic AI implementation. It really comes down to your data. And I think that as any team is thinking about their rollout, we have to consider the upfront work of agents

gathering your data, improving your data, ensuring that you have everything in the right place before you turn it on. And I'm curious to know your thoughts on like, how much time did it take your team to prepare? Like, I imagine there was a quite a bit of upfront work. Were you prepared for that? Would you do it differently next time?

So we worked with a development company that has helped us in the past with Salesforce development. And I strongly recommend companies to assess that possibility because it was really helpful for us to be able to count on them when it comes to the prep work. So it took, as I said, longer than we expected, but we were able to go live with the first complex

use case in the space of two months since we started the whole

operation on this. And then it continued for the next three months. So it took longer than the six weeks everyone goes on about, but it was because we picked quite complex use cases, as I said, and the preparation in terms of data was quite extensive. Yeah. Okay. So we need to think about

either carving out our own capacity if we're trying to do it all ourselves, but really are there experts that we can help, that we can bring on to expand our capacity and actually move through all the work that needs to be done as quickly as possible.

Exactly. Especially because within companies, it's kind of a new role. So there isn't anyone that has done this before. Whereas if you start speaking to a development consultancy company, they have worked with our other clients and kind of bring their knowledge and good practice. So they can spare you a lot of time. Yeah, completely. This is not...

really the moment to do it yourself. And I know Mark Benioff talked about this at Dreamforce this year. He was like, don't DIY your AI. Let's leverage the expertise here. It's new. We can learn from others.

Exactly. And, you know, there are many companies out there where if your values align, then it's an extension of your own team. And it's very easy to work with them because they understand the business and they also understand the technology much better than you as the company. Totally, totally. I'm curious to get your thoughts on

and actually gathering conversations

customer data that you're using to create insights internally. Have you found that this has been helping or hurting your ability to really listen to the customer and understand their challenges? One of the surprises or like welcome surprise, I was hoping this was going to happen, was that the negative CES score on how easy it is to get in touch with us

via live chat has increased massively. Before, around 15% of the negative entries when it comes to negative CES were about how clunky it was to get in touch with us via live chat because mainly of the menu-based bot. That was my baby. I liked working on it, but it wasn't the best when it comes to smoothness.

Whereas now that type of negative feedback is virtually dissolved. This is incredible. Customer effort score, CES, just in case everyone isn't familiar with that term, is one of the most important metrics in customer experience. If you think about reaching out to a company and it is difficult to get in touch, it is difficult to solve your problem, you...

Think less of that company. You trust that company less. You want to do business with them less. We need to be thinking about customer effort. And it is one of the most beautiful results of AI is that we can really help to improve the or I guess decrease the effort that it takes.

in order to reach out. So I love to hear that. And sorry, I'll let you continue. It was great to see that, especially because we were struggling with it so much in the menu-based bot era. Of course,

The virtual agent is a little bit more sensitive to escalating the issue to humans because it can pick up the tone. A menu-based bot where you click through the options, it can't really feel your sentiment and tone of voice. So you have to be

careful in tuning the instructions of the agent to avoid escalating as much as it would naturally do or organically do. Our escalation percentage in terms of interactions that are passed to humans has increased. But as I said, overall, I can see a better perception of the interaction with the virtual agent versus the bot.

And the number of repeated contacts from the same customer has decreased, meaning that the customer is not giving up on us on chat and reaching out via phone. But this new journey allows them to stay in the channel of choice and hopefully get to a nice resolution. And I think, you know, you mentioned that you've actually increased the number of interactions a human is having with your customer. Yeah.

And sometimes people think like, oh, that might be negative, but we want to talk to our customers. If we can talk to our customers about real meaningful things, that's an opportunity for us to build trust with that customer. If we're able to solve those really quick issues,

and the AI can handle it, great. Get those off our plates. But let's actually, speaking to a customer can actually be really beneficial. So I always say to people, don't be afraid to talk to your customer. It's a goldmine of insights. Yeah.

Exactly. And also an increase in escalated interactions. And by escalated, I mean passed over to a human means a decrease in abandoned interactions. So as I said, you don't want your customers to give up on you. You want them to be able to reach a resolution and do it fast. So if the AI can do it, just interacting with the agent,

Fine. If not, it's fine to pass it over to a human. What you don't want is abandonment.

So customers giving up on your agent, virtual agent and choosing to phone us or email us. Yeah. Or just give up altogether. Or give up and never come back. Exactly. Exactly. And so going back to what we spoke about earlier, customer lifetime value, if they're giving up because they can't get through your bot, right?

You're losing that customer for life versus if you can really support them and bring them in, you're extending the life cycle of that customer potentially. An AI agent that your customers actually enjoy talking to? Salesforce has you covered. Meet AgentForce for service. The AI agent that can resolve cases in conversational language anytime on any channel.

To learn more, visit salesforce.com slash agentforce.

So what is next, Laura? What are you looking forward to as you move forward in this journey? I am trying to find different ways to leverage the FAQs side of things, as I said. So to increase and improve our knowledge base for the agent to have additional grounding.

But also exploring more of, you know, different type of grounding so that we could expand that side of action and support that the agent can provide to our customers when it comes to.

to helping them about their specific booking, whether they're looking at a specific holiday or whether they have issues with their own specific account. So I'm looking at different strategies we could take, but these are the main two avenues that I would like to explore.

And I have to say that I'm also looking at some of the new functionalities that will be coming up in Salesforce when it comes to workload management and co-piloting for agents when it comes to case management and workload management, as I think that would be key for us to free up time from the agents. So case summaries and case logging, analyzing reasons for contacts,

taking that away from humans. There's so much opportunity. And just in you sharing this story, I think it really explains how early we are in understanding

our agentic AI journey collectively and how much possibility there is. I mean, just hearing, when did you first roll out AgentForce? Like when was the first date you turned it on? So we went live with the messaging sessions in September. And I think we went live in November with the first use cases. Okay. Okay.

So you are early, early on that. It is April 2nd for everyone's context. So they're only a couple months in and so many learnings have been taking place. So I'm very excited to keep in touch and hear how all of this is progressing and what new use cases you're coming to find. Yeah. Thank you.

So Laura, we have two last questions that we'd like to ask all of our guests. And the first is, I'd like to hear about a recent experience you had with a brand that left you impressed. Why was that experience amazing? Definitely, it would have to be Vito. It's a furniture company based in the UK. I was interested in getting a shelving unit from them.

And so I contacted them. And instead of pushing me to a product page on their website, they set up a call with me to look into the design options. They wanted to see my space. So it felt really refreshing for a company that is quite big, historical. It's kind of an icon of design to take the time and discover the interaction with the customer and

They kind of understand that what you are buying from them is a long-term investment. It's not cheap.

And therefore, it needs to be the best for you and for your future and for your space. So, yeah, I really appreciated that. And it reminded me how important sometimes it is not just to be efficient and quick, but taking the time and being intentional and being very human when it comes to the interaction with customers. It really makes a difference.

So much so that I've ordered the shelving unit and I'm going to order another one for another room in my flat because I liked it so much. Amazing. And so they leaned into

To your interaction versus leaning out and trying to just solve it quickly. Exactly. And I think that is really bringing it back to the AI conversation, the opportunity here. There's such a big opportunity for us to free up time and space and actually lean into those customer interactions versus leaning out. Yeah.

Absolutely. Like people can really feel when they're being sold on rather than actually supported. And that's the key, right? So the key to achieve the best customer experience in general, and it's easier said than done, is to really listen and really try to help.

as that customer will stay with you for a very long time rather than a quick win. They can perceive how genuine you are in the support that you provide. Definitely. Let's think about the long-term. It's a long-term game here in customer experience. We all know it and it's so important. It's the same thing with the AI. It's a long-term investment, I think, and that has to be

the mindset when you start this kind of journey. It's learning by doing. It's a completely revolutionary technology. We're in at the very beginning of it. And it's something that you need to take as a long-term investment and something that will stay with you for a very long time. Exactly. So my last question for you, I'm going to put a little twist on it. If anyone listens to this show frequently, we're going to, we're going to do a spin. Okay.

What piece of advice should every customer experience leader who is rolling out AI hear? It's start simple.

And make sure you have good data. Make sure you have a good foundation of knowledge base. Make sure you lean on the experts out there. Don't DIY this. And make sure you start the journey with AI, with the creation of long-term relationship with your customers in mind. It's not a quick win-win.

journey, it's actually investing into relationship you have with your customers and tackling everything that you can take away from humans that is not as valuable and that focusing your teams on what humans are best at. Let's focus on the human advantage.

Yeah. Thank you so much, Laura. This has been a masterclass in what leaders need to be thinking about as we implement AI in our customer experience and the opportunities at play. I really appreciate you coming on the show to share all of this with us. And I can't wait to see where it goes. Thank you. Thank you.