Ep 524: Agentic AI Done Right – How to avoid missing out or messing up.

Topics Covered in This Episode:

1. Agentic AI Benefits for Enterprises
2. watsonx's New Features & Announcements
3. AI-Powered Enterprise Solutions at IBM
4. Responsible Implementation of Agentic AI
5. LLMs in Enterprise Cost Optimization
6. Deployment and Scalability Enhancements
7. AI's Impact on Developer Productivity
8. Problem-Solving with Agentic AI
9. 
   

Keywords:

Agentic AI, AI agents, Agent lifecycle, LLMs taking actions, WatsonX.ai, Product management, IBM Think conference, Business leaders, Enterprise productivity, WatsonX platform, Custom AI solutions, Environmental Intelligence Suite, Granite Code models, AI-powered code assistant, Customer challenges, Responsible AI implementation, Transparency and traceability, Observability, Optimization, Larger compute, Cost performance optimization, Chain of thought reasoning, Inference time scaling, Deployment service, Scalability of enterprise, Access control, Security requirements, Non-technical users, AI-assisted coding, Developer time-saving, Function calling, Tool calling, Enterprise data integration, Solving enterprise problems, Responsible implementation, Human in the loop, Automation, IBM savings, Risk assessment, Empowering workforce.

Podcast Transcript

Jordan Wilson [00:00:16]:
AI agents are all the rage. I literally just left one of the sessions and it was standing room only. But I think one thing that business leaders are constantly thinking about when it comes to agentic AI is getting it right. And, you know, you can either mess up or miss out or you can do it correctly and really see a new level of productivity for your enterprise that you maybe haven't experienced in a very long time. So that's what we're gonna be talking about today and a lot more on Everyday AI. What's going on, y'all? My name is Jordan Wilson. I'm the host of Everyday AI. This is your daily livestream podcast and free daily newsletter, helping us all not just learn what's happening in the world of AI, but how we can leverage it to grow our companies and our careers.

Jordan Wilson [00:00:59]:
And if you're joining us on the livestream, you probably see this is quite a different setup. I'm here, at the IBM Think conference. Very excited to partner with IBM to, be able to tell some of these stories, and the story, has definitely been so far Agentic AI. So that's what we're gonna be talking about today, how you cannot miss out on it. So, I'm very excited for our guest, doctor, Maryam Ashoori, who is the senior director of product management at watsonx. Mariam, thank you so much for joining the Everyday AI Show.

Dr. Maryam Ashoori [00:01:28]:
Thanks for having me.

Jordan Wilson [00:01:29]:
Yeah. The special edition here at IBM Think. But, you know, before we get into all the new announcements, agentic AI, all that, can you tell us a little bit about what your role is, at watsonx and IBM?

Dr. Maryam Ashoori [00:01:39]:
Absolutely. I'm the head of product for WatsonX.ai, and the past twenty four months have been super exciting. Like, every day, a new piece of technology is coming to the market. And mid year last year, we saw the excitement around LLMs taking act actions as agents. It's been revolutionizing every corner of businesses, and we are excited with the new features and capabilities that we are announcing to roll out as part of Watsonx.ai.

Jordan Wilson [00:02:06]:
Yeah. So we're gonna talk a little bit more about all of the agentic AI and all the new announcements, but little bit on your day to day, you know, and maybe for some of our audience that isn't super familiar, with everything that WatsonX has to offer. Can you tell us a little bit about the different, you know, products and services that IBM has just for those that aren't aware?

Dr. Maryam Ashoori [00:02:24]:
It's about AI. Like, we we are building AI, but also we are consuming AI. So we have the platform that is helping enterprises customize their AI solutions. But every solution that we are designing, also we use them to enrich a series of our software products. We have a series of products like intelligent environmental intelligence, suite that are powered up and enriched with the foundation models that we are delivering. We have some new products like watsonxcode Assistant that are powered up by the Granite Code models. And we are also having a series of services that are helping together with the customers to look into their problems and see how AI can benefit from them. So we are looking at the very wide spectrum of how AI can help businesses through the platform, through the services, and through the products.

Jordan Wilson [00:03:14]:
So speaking of solving problems, right, that's ultimately what this is all about. You know, new products, new services, new techniques, to solve, customer problems. What would you say with everything that that was announced and there's a ton that was announced here at IBM Think. What would you say is the biggest, solution for those enterprise customers that are already maybe on the watsonx platform? You know, what are they able to maybe accomplish Yeah. Now that maybe last year at this time they weren't able to accomplish? Yeah.

Dr. Maryam Ashoori [00:03:44]:
So the market is still experimenting with, agents. They are still looking for a wow factor and moment. But what we what we are designing is for production and escape. As the enterprises go through the journey to production, they soon realize the path to success is not straightforward. There are major challenges there that are amplified with agents, and I tell you why. Let's start with ensuring a responsible implementation of AI. All the limitations that the LLMs historically had, now they are carried forward through agents because agents are powered up by LLMs. But at the same time, these agents are taking actions.

Dr. Maryam Ashoori [00:04:22]:
They can access data. They interpret code. They connect to external services. Right? They can leak data potentially if not designed well. So the transparency and the traceability of actions is essential for agents. Observability is a challenge number one. Challenge number two, optimization. When you're looking for a WAF actor, the larger the model, the more capable the model is, but we all know that the larger the model, it also requires larger compute.

Dr. Maryam Ashoori [00:04:51]:
That translates to an increased cost. That translates to an increased latency. That's your response time. That translates to an increased carbon footprint and energy consumption. So the pattern that we are seeing in the market is moving toward getting, grabbing a much smaller elements even for powering up agents. Point unit on proprietary data of the enterprise that the data value users. That's their domain specific data to create something differentiated that that delivers the performance they need for a fraction of the cost for their target use case. Right? At the same time, why is it amplified by agents? Because this was the story of Alaa Labs.

Dr. Maryam Ashoori [00:05:30]:
You know agents. They have advanced planning capabilities. They have chain of thoughts reasoning. Inference time scaling, that translates to additional compute. So think about the scale of enterprise, the cost adds up, and that brings it back to optimization, cost performance optimization, and why custom enterprises should pay attention to this. These two has been the guiding principles for basically everything that we have knows, I think. Thinking about agent lifecycle, managing the lifecycle all the way from building it to deploying it and monitoring the performance of the agents is what we've been talking.

Jordan Wilson [00:06:07]:
So you you know, building, deploying, monitoring, it seems like even those three steps have improved a lot. Right?

Dr. Maryam Ashoori [00:06:14]:
Oh, yeah.

Jordan Wilson [00:06:14]:
On the front end building, you know, now you have the agent catalog. You have the build your own agent. You know, you can use them as templates. On the back end, you know, being able to trace and monitor a little bit better. And I love seeing, like, the the the chain of thought reasoning in an agent that you build, for traceability is huge. What would you say from everything that was announced here? You know, whether you wanna pick one of those three areas, but which one do you think is, the area where, enterprise leaders should first focus on? You know, are they should should they try to rebuild a different way? Should they monitor what's already, you know, working, going wrong, and and adjust? Like, what is the the the best next step, to make sure agents actually work?

Dr. Maryam Ashoori [00:06:52]:
Yeah. In order to deliver these agents in production, they need all of them. They need to build the agent, they need to deploy the agent, and they need to monitor the performance of the agent. Right? If you are in highly regulated environments like finance or insurance, they have serious guidelines in in terms of monitoring the agents. So for example, making sure the agent behavior is adhering to the policy of the company, as an example. Or they are monitoring, the tracing of what happened, the agent behavior, not just for the purpose of logging, but auditability. Right? So they have to pay at more attention on that. But you you said if you pitch one, I'm going to pitch I'm going to pitch them one in the middle.

Dr. Maryam Ashoori [00:07:39]:
There we go. The deploy one, right? Enterprises in average, the developers and enterprises are spending eighteen hours in deploying and scaling the GenAI applications. Eighteen hours. We don't want the developers to spend eighteen hours. We want them to deploy their agents as a matter of seconds and scale it as a matter of minutes. Right? That has been one of the examples that we've been focusing on. The deployment service that we just announced and released in the market gives developers a single click deployment from the UI or single command deployment from the command line. It's a fuse.

Dr. Maryam Ashoori [00:08:16]:
It's just just deploy, and it's designed for the scalability of enterprise. Let's say that you're an enterprise, you want highly available agents. If one of the instances fail, you don't wanna, fail your workflows. Right? The other one automatically load balancing comes up. So easily, as a matter of, let's say, two minutes, the one that I tried yesterday, in three three, you you can increase the scale and instances of your agents. The last factor that I like to highlight here is access control. Enterprises are very concerned about security. Even for some of them, like some of the telecommunication companies that we work with, they have very unique security requirements.

Dr. Maryam Ashoori [00:08:59]:
We have designed this deployment services in a way that the the access control is managed by projects and spaces. So you have full control over who can access this agent under what circumstances to do what, which is essential for enterprises.

Jordan Wilson [00:09:15]:
You know, one big, I guess mindset shift that we're seeing a lot with enterprise leaders is, you know, they've been looking at the past maybe, you know, two years since large language models became popularized and they're like, oh, okay. We probably made some mistakes along the way and that's with our smartest humans in control. Right? But when we talk about now multi agentic orchestration and, you know, these these, agents that are actually so easy to get out. Right? Less than five minutes, but then they're so powerful. You know, there is this this this fear of maybe messing up. So how can companies not miss out and also not mess up and kind of get it right when these, you know, models are and and and these agents are so powerful and so capable?

Dr. Maryam Ashoori [00:09:58]:
Yeah. I would say that they should focus on the problem they are solving versus, hey, there is an agent, how can I use that agent? Right? Because when you have your problem, you know exactly what are the expectations from this agent, and then if the technology delivers or not. If the technology delivers, perfect. If the technology doesn't deliver, you can mitigate with everything in house or the existing workflows that you have mixed and matched. Then look into the workloads that you have mixed and matched. Then look into the sensitivity of the workloads. For some of the workloads, the risk is just too high that you not need to make sure human is in the loop. But for some of the low stakes, like the example that I'm using is like, if I am using agents to provide recommendations for dinner, I probably don't care if there's a human in the loop or explainability of why I arrived at that decision.

Dr. Maryam Ashoori [00:10:42]:
So the stake matters, right? And the third one is, like, just the industry. Like, what are the regulations? And and think about the future, not the regulations for today. So so just bringing that together, human in the loop, understanding the problem and the stakes, like what is the use case, what are the requirements for that, and then the last one was the designing for, a responsible implementation of these agents. Mhmm.

Jordan Wilson [00:11:09]:
You know, all of these, you know, capabilities that I even look at now that, with everything that's been announced here at IBM Think, I'm like, wow. This really not only changes what's possible, but it also changes maybe how work gets done, right? Because I if you would have asked me, you know, two and a half years ago when I started the show and said, hey. Today, you can connect your enterprise data with, an agent that can reason and it's a nontechnical person that can put it together. I would have been like, okay. What does that mean for both technical people who would generally be building these things and the nontechnical people that maybe wouldn't, usually be taking advantage, all of all these capabilities. So how do all of these, you know, new capabilities just change the way that, you know, developers work and non technical people, taking advantage of it all?

Dr. Maryam Ashoori [00:11:56]:
It has already started changing every single one of us life. Like, we we we ran a study with thousand developers across The States, the developers that are building AI applications. And we asked them, are you using AI assisted coding for development? The majority of them, the answer was yes. We said, how much time saving are you, are you getting? Most of them, they said one to two hours a day. Just think about it. The additional value that you can create by that two extra hours per day. Mhmm. That translates to acceleration in the speed of creation.

Dr. Maryam Ashoori [00:12:32]:
That translates into freeing up the time of developers, or it's not just developers, every single one of us to do higher value, work. And I feel like that's that's really where the opportunity lies and where I'm personally excited about because I feel like collectively as humans now, we have way more time in our hands to to do more higher value, well, work.

Jordan Wilson [00:12:54]:
Yeah. You you know, one piece of advice that you gave, which I think is is great, is don't go out there and try to use agents. Go out there and find a problem to solve and and and find the right agent that aligns with it. You you know, one thing we've talked about and and we've heard is, you know, IBM had this this massive, right, $3,500,000,000, in savings because of AI and automation. So, you know, when business leaders are seeing all of these these new announcements, from from from watsonx and everything else that, IBM has going on and they're like, okay. Where do I go? Where do I go to save time? Right? Where should businesses be looking? Because it's almost like there's so many different agents. There's so many different places you can apply.

Dr. Maryam Ashoori [00:13:33]:
Yeah.

Jordan Wilson [00:13:33]:
Where should they be looking?

Dr. Maryam Ashoori [00:13:34]:
Two things. The first one is look into LLMs itself and how they can help businesses. The most common use cases for LLMs are content grounded question and answering. Customer care is a very, good example of that. Code generation or content generation, classification, information extraction, summarization. So basically, every anywhere in your business that you have these workloads, they can be accelerated by GenAI. But then, the opportunity that agents represent is bring that all into every single corner of your enterprise, blend them these two words together through function calling and tool calling. So literally, all of that acceleration can be mapped to even your legacy systems in enterprise.

Dr. Maryam Ashoori [00:14:23]:
And I think that's where the opportunity lies. So I would start with LLM application itself, and then look into one, how can I bring that acceleration to every single corner of my business? And two, focus on problems, workflows. Can I use agents to automate some of them? If the answer is yes, go for it. If the answer is, like, explore and build your own and watch and see how the market evolves to ans to to solve your problem, then that's the path forward.

Jordan Wilson [00:14:52]:
So, I'll even ask you. So, you know, how might your work change in your department, your team's work change, with with everything that you've just announced? I know I'm I'm sure your team has already been, you know, testing it out for some time. But you know, I I I think maybe our audience can learn a little bit about how even your work might change with all the, you know, all of the new tools and features that we have available now.

Dr. Maryam Ashoori [00:15:16]:
That's that's actually fascinating. I run a team of product managers and my product managers are wipe coding. When we think about a new feature, an idea, they are showing me the fully functional prototype Yeah. That they had coded. And they are like, Mariam, this is it. And I'm like, is it real or Or what am what am I looking at? So I feel like this is this is literally changing everything. Like, the way that we are thinking about technology, the way that we are thinking about solving problems, our problem solving processes is already changed.

Jordan Wilson [00:15:50]:
Yeah. That's amazing. And, you know, I I always think, okay, our internal presentations and, you know, internal, you know, long rollouts, are those a thing of the past when you can just, like, you know, go in? I know there's the new, you know, code assist that that that you all updated. Like, is that just gonna be a thing of the past where it's just like, no. I'm just gonna go solve the problem first and then talk about it and see how we can use it? Is that it? Like, is that gonna happen?

Dr. Maryam Ashoori [00:16:16]:
Mobax do start with their problems. That's Don't get distracted with a technology because it keeps changing.

Jordan Wilson [00:16:22]:
Alright. So we've we've we've we've talked about a a lot here. I wish we could talk for hours. But, you know, as we wrap up today's conversation and and hopefully advising, you know, business leaders on the right way to take advantage of agentic AI and and do it the right way. What is your one most important piece of advice or the one step that business leaders need to take in order to not mess up on agentic AI?

Dr. Maryam Ashoori [00:16:48]:
Know your limits and lines. It's like, what are the risks associated with your use cases that can't be jeopardized? Understanding the risks gives them a true and good lens to assess the technology. And align these lines is don't limit your people. Like, closing your eyes doesn't erase the problem, it just lets you not be able to solve it and sit on it. So I would say that understand the risk, provide guidelines, establish the guidelines, go talk to the experts in the field to understand how can you mitigate those risks, and be open to that. Mhmm. And make it accessible to your staff and trust your workforce to find the right way and help them and empower them to move forward as the AI moves forward.

Jordan Wilson [00:17:39]:
Mhmm. I think that's that's great advice and some great practical, next steps for business leaders that are looking at all of these new agentic AI capabilities and they're like, I don't wanna miss out. I don't wanna mess up. Now you have the blueprint. So if you miss anything, don't worry. We're gonna be recapping today's conversation and sharing a ton more, both at what what was at the IBM Think Concert, convention and a lot more. So if you haven't already, please go to our website at youreverydayai.com. Sign up for the free daily newsletter.

Jordan Wilson [00:18:08]:
Thanks for tuning in. We'll see you tomorrow and everyday for more everyday AI. Thanks, y'all.