EP 524: Agentic AI Done Right - How to avoid missing out or messing up.
18:33 Share

This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. AI agents are all the rage. I literally just left one of the sessions and it was standing room only. But I think one thing that business leaders are constantly thinking about when it comes to agentic AI is,

getting it right. And you know, you can either mess up or miss out or you can do it correctly and really see a new level of productivity for your enterprise that you maybe haven't experienced in a very long time. So that's what we're going to be talking about today and a lot more on Everyday AI. What's going on, y'all? My name is Jordan Wilson. I'm the host of Everyday AI. This is your daily live stream podcast and free daily newsletter, helping us all not just learn what's happening in the world of AI, but how we can leverage it to grow our companies and our careers. And

And if you're joining us on the live stream, you probably see this is quite a different setup. I'm here at the IBM Think Conference. Very excited to partner with IBM to be able to tell some of the stories. And the story has definitely been so far agentic AI. So that's what we're going to be talking about today. How you can not

miss out on it. So I'm very excited for our guest, Dr. Mariam Ashuri, who is the Senior Director of Product Management at Watson X. Mariam, thank you so much for joining the Everyday AI Show. Thanks for having me. Yeah, the special edition here at IBM Think. But before we get into all the new announcements, agentic AI, all that, can you tell us a little bit about what your role is at Watson X and IBM?

Absolutely. I'm the head of product for WatsonX.ai and the past 24 months have been super exciting. Like every day a new piece of technology is coming to the market and mid-year last year, we saw the excitement around LLMs taking actions as agents. It's been revolutionizing every

corner of businesses. And we are excited with the new features and capabilities that we are announcing to roll out as part of WatsonX.ai. Yeah, so we're going to talk a little bit more about all of the agentic AI and all the new announcements, but a little bit on your day to day, you know, and maybe for some of our audience that isn't super familiar with everything that WatsonX has to offer. Can you tell us a little bit about the different, you know, products and services that IBM has just for those that aren't aware?

It's about AI. We are building AI, but also we are consuming AI. So we have the platform that is helping enterprises customize their AI solutions, but every solution that we are designing, also we use them to enrich a series of our software products. We have a

series of products like intelligent, environmental intelligence suite that are powered up and enriched with the foundation models that we are delivering. We have some new products like Watson X code assistant that are powered up by the granite code models. And we are also having a series of services that are helping together with the customers. So look into their problems and see how AI can benefit from them. So we are looking at the various

wide spectrum of how AI can help businesses through the platform, through the services, and through the products. So speaking of solving problems, right, that's ultimately what this is all about. You know, new products, new services, new techniques to solve customer problems. What would you say with everything that was announced? And there's a ton that was announced here at IBM Think. What would you say is the biggest problem

solution for those enterprise customers that are already maybe on the WatsonX platform? You know, what are they able to maybe accomplish now that maybe last year at this time they weren't able to accomplish? So the market is still experimenting with agents. They are still looking for a wow factor and aha moments. But what we are designing is for production and the scale. As the

Enterprises go through the journey to production. They soon realize the path to success is not straightforward. There are major challenges there that are amplified with agents. And I tell you why. Let's start with ensuring a responsible implementation of AI. All the limitations that the LLMs historically had, now they are carried forward to agents because agents are powered up by LLMs.

But at the same time, these agents are taking actions. They can access data, they interpret code, they connect to external services, right? They can leak data potentially if not designed well. So the transparency and the traceability of actions is essential for agents. Observability is a challenge number one. Challenge number two, optimization.

When you're looking for a value factor, the larger the model, the more capable the model is. But we all know that the larger the model, it also requires larger compute. That translates to an increased cost. That translates to an increased latency. That's your response time. That translates to an increased carbon footprint and energy consumption. So the pattern that we are seeing in the market is moving toward getting, grabbing much smaller LLMs, even for powering up agents.

fine-tune it on proprietary data of the enterprise that the data value users. That's their domain-specific data to create something differentiated that delivers the performance they need for a fraction of the cost for their target use case, right? At the same time, why is it amplified by agents? Because this was the story of LLMs. You know agents. They have advanced planning capabilities. They have chain of thoughts reasoning, inference time scaling data.

That translates to additional compute. So think about the scale of enterprise, the cost adds up and that brings it back to optimization, cost performance optimization and why custom enterprises should pay attention to this. These two has been the guiding principles for basically everything that we announced.

I think thinking about agent lifecycle, managing the lifecycle all the way from building it to deploying it and monitoring the performance of the agents is what we've been talking. So, you know, building, deploying, monitoring, it seems like even those three steps have improved a lot, right? On the front end building, you know, now you have the agent catalog, you have the build your own agent, you know, you can use them as templates. On the back end, you know, being able to trace and monitor a little bit

better. And I love seeing like the, the, the chain of thought reasoning and an agent that you build for traceability. It's huge. What would you say from everything that was announced here? You know, whether you want to pick one of those three areas, but which one do you think is the area where enterprise leaders should first focus on? You know, are they, should, should they

try to rebuild a different way? Should they monitor what's already working, going wrong and adjust? What is the best next step to make sure agents actually work? Yeah. In order to deliver these agents in production, they need all of them. They need to fill the agent, they need to deploy the agent, and they need to monitor the performance of the agent. Right?

If you are in highly regulated environments like finance or insurance, they have serious guidelines in terms of monitoring the agents. So, for example, making sure the agent behavior is...

adhering to the policy of the company, as an example. Or they are monitoring the tracing of what happened, the agent behavior, not just for the purpose of logging, but auditability, right? So they have to pay more attention on that. But you said if you pick one, I'm going to pick the one in the middle. There we go.

The deploy one, right? Enterprises in average, the developers in enterprises are spending 18 hours in deploying and scaling a generic application. 18 hours. We don't want the developers to spend 18 hours. We want them to deploy their agents as a matter of seconds and scale it as a matter of minutes, right? That has been one of the examples that we've been focusing on. The deployment service that we just announced and released in the market,

gives developers a single click deployment from the UI or single command deployment from the command line,

is a few as a stress dot just apply and it's designed for the scalability of enterprise let's say that you're an enterprise you want highly available agents if one of the instances fail you don't want to fail your workloads right the other one automatically to load balancing comes up so easily as a matter of let's say two minutes the one that I tried yesterday in three three you you can increase the scale and instances of your agents

The last factor that I like to highlight here is access control. Enterprises are very concerned about security. Even for some of them, like some of the telecommunication companies that we work with, they have very unique security requirements.

We have designed these deployment services in a way that the access control is managed by projects and spaces. So you have full control over who can access this agent under what circumstances to do what, which is essential for enterprises.

One big, I guess, mindset shift that we're seeing a lot with enterprise leaders is they've been looking at the past maybe two years since large language models became popularized. And they're like, OK, we probably made some mistakes along the way. And that's with our smartest humans in control. But when we talk about now multi-agentic orchestration and these interoperability

agents that are actually so easy to get out, right? Less than five minutes, but then they're so powerful. You know, there is this fear of maybe messing up. So how can companies not miss out and also not mess up and kind of get it right when these models are, and these agents are so powerful and so capable? Yeah. I would say that they should focus on the problem they are solving versus, hey, there is an agent. How can I use that agent? Right? Because when you have an

a problem, you know exactly what are the expectations from this agent and then if the technology delivers or not. If the technology delivers perfect, if the technology doesn't deliver, you can mitigate with everything in-house or the existing workloads that you have mixed and matched.

Then look into the sensitivity of the workloads. For some of the workloads, the risk is just too high that you need to make sure a human is in the loop. But for some of the low stakes, like the example that I'm using is like if I'm using agents to provide recommendations for dinner, I probably don't care if there's a human in the loop or explainability of why I arrived at that decision. So the stakes matters, right?

And the third one is just the industry. What are the regulations? And think about the future, not the regulations for today. So just bringing that together, human in the loop, understanding the problem and the stakes, like what is the use case? What are the requirements for that? And then the last one was designing for a responsible implementation of these agents. All of these capabilities that

I even look at now that with everything that's been announced here at IBM Think, I'm like, wow, this really not only changes what's possible, but it also changes maybe how work gets done, right? Because if you would have asked me two and a half years ago when I started the show and said, hey, today you can connect your enterprise data with an agent that can reason, and it's a non-technical person that can put it together, I would have been like,

Okay, what does that mean for both technical people who would generally be building these things and the non-technical people that maybe wouldn't usually be taking advantage of all these capabilities? So how do all of these new capabilities just change the way that developers work and non-technical people taking advantage of it all?

It has already started changing every single one of us life. Like we ran a study with thousand developers across the states, the developers that are building AI applications. And we asked them, are you using AI assisted coding for development? The majority of them, the answer was yes. We said, how much time saving are you getting? Most of them, they said one to two hours a day. Just think about it. The additional value that you can create

by that two extra hours per day. That translates to acceleration in the speed of creation. That translates into freeing up the time of developers, or it's not just developers, every single one of us to do higher value work. And I feel like that's really where the opportunity lies and where I'm personally excited about because I feel like collectively as humans now we have way more time in our hands to do more higher valuable work.

Yeah, you know, one piece of advice that you gave, which I think is great, is don't go out there and try to use agents. Go out there and find a problem to solve and find the right agent that aligns with it. You know, one thing we've talked about and we've heard is, you know, IBM had this massive, right, $3.5 billion in savings because of AI and automation. So, you know, when business leaders are seeing all of these new announcements from Watson X and everything else that IBM

IBM has going on and they're like, okay, where do I go? Where do I go to save time? Right. Where should businesses be looking? Because it's almost like there's so many different agents. There's so many different places you can apply. Where should they be looking? Two things. The first one is looking to LLMs itself.

and how they can help businesses. The most common use cases for LLMs are content grounded question and answering. Customer care is a very good example of that. Code generation or content generation, classification, information extraction, summarization. So basically anywhere in your business that you have these workloads,

they can be accelerated by gen i but then the opportunity that agents represent is bring that all into every single corner of your enterprise blend them these two words together through function calling and tool calling so literally all of that acceleration can be mapped to even your legacy

systems in enterprise. And I think that's where the opportunity lies. So I would start with LLM application itself and then looking to one, how can I bring that acceleration to every single corner of my business? And two, focus on problems, workflows,

Can I use agents to automate some of them? If the answer is yes, go for it. If the answer is like explore and build your own and watch and see how the market evolves to solve your problem, then that's the path forward.

So I'll even ask you, so, you know, how might your work change in your department, your team's work change with everything that you've just announced? I know, I'm sure your team has already been, you know, testing it out for some time. But, you know, I think maybe our audience can learn a little bit about how even your work might change without

with all of the new tools and features that we have available now. That's actually fascinating. I run a team of product managers, and my product managers are wipe coding. When we think about a new feature, an idea, they are showing me the fully functional prototype that they had coded, and they are like, "Marianne, this is it." And I'm like, "Is it real or..."

What am I looking at? So I feel like this is literally changing everything. Like the way that we are thinking about technology, the way that we are thinking about solving problems, our problem solving processes,

It's already changed. Yeah, that's amazing. And I always think, okay, our internal presentations and internal long rollouts are those that thing of the past when you can just go in. I know there's the new code assist that you all updated. Is that just going to be a thing of the past where it's just like, no, I'm just going to go solve the problem first and then talk about it and see how we can use it? Is that going to happen?

Well, back to start with your problem. Don't get distracted with the technology because it keeps changing. All right. So we've talked about a lot here. I wish we could talk for hours. But, you know, as we wrap up today's conversation and hopefully advising, you know, business leaders on the right way to take advantage of agentic AI and do it the right way. What is your one most important piece of advice or the one step that business leaders need to take in order to not mess up on agentic AI?

Know your limits and lines. It's like, what are the risks associated with your use cases that can't be jeopardized? Understanding the risks gives them a true and good lens to assess the technology.

And align these lines is don't limit your people. Like closing your eyes doesn't erase the problem. It just lets you not be able to solve it and sit on it. So I would say that understand the risk, provide guidelines, establish the guidelines, go talk to the experts in the field to understand how can you mitigate those risks and be open to that and make it accessible to your staff and trust your workforce to find solutions

the right way and help them and empower them to move forward as the AI moves forward. I think that's great advice and some great practical next steps for business leaders that are looking at all of these new agentic AI capabilities and they're like, I don't want to miss out. I don't want to mess up. Now you have the blueprint. So if you missed anything, don't worry. We're going to be recapping today's conversation and sharing a ton more both at what was at the IBM ThinkCon conference

convention, and a lot more. So if you haven't already, please go to our website at youreverydayai.com. Sign up for the free daily newsletter. Thanks for tuning in. We'll see you back tomorrow and every day for more Everyday AI. Thanks, y'all.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.