Ep 536: Agentic AI – The risks and how to tackle them responsibly

Topics Covered in This Episode:

1. Responsible AI: Evolution and Challenges
2. Agentic AI's Ethical Implications
3. Multi-Agentic AI Responsibility Shift
4. Microsoft’s AI Governance Strategies
5. Testing Multi Agentic Risks and Patterns
6. Agentic AI: Future Workforce Skills
7. Observability in Multi-Agentic Systems
8. Three Risk Categories in AI Implementation

Keywords:

Agentic AI, multi agentic AI, responsible AI, generative AI, Microsoft Build conference, AI governance, AI ethics, AI systems, AI risk, AI mitigation, AI tools, human in the loop, Foundry observability, AI testing, system security, AI monitoring, user intent, AI capability, prompt injection, Copilot, AI orchestration, AI deployment, system governance, Entra agent ID, AI education, AI upskilling, AI workforce integration, systemic risk, AI misuse, AI malfunctions, AI systemic risk, AI-powered solutions, AI development, AI innovation, AI technology, AI security measures.

Podcast Transcript

Jordan Wilson [00:00:16]:
Responsible AI used to be much more straightforward. I don't think it was ever simple necessarily, but with the rate of change when it comes to generative AI and everything we've seen from, big tech companies everywhere with agentic AI, and not even that, multi agentic AI. I think it changes responsible AI drastically. Right? Because it used to be, hey. One human goes in, talks to an AI chatbot, and you could probably more accurately understand what guardrails that organizations need to put in place in order for this thing to work. But what about now, when we talk about agentic AI, how does that change the ethics, the governance, the responsibility that we as business leaders need to have in order to make this thing work the right way. And then when it comes to multi agentic AI, when agents are talking among themselves, divvying tasks up and executing on our behalf, how does that change things? These are big questions. I don't have all the answers, but today I have a fantastic guest who does.

Jordan Wilson [00:01:31]:
Alright. I'm excited for this conversation. I hope you are too. What's going on y'all? My name is Jordan Wilson and welcome to everyday AI. This is your daily livestream podcast and free daily newsletter, helping us all not just keep up with AI, but how we can use it to get ahead to grow our companies and our careers. So if that sounds like what you're trying to do, it starts right here with this livestream podcast. But where you need to go is our newsletter. We're gonna be recapping the most important insights from today's conversation, so make sure you go sign up for that free daily newsletter at youreverydayai.com.

Jordan Wilson [00:02:07]:
We'll also have everything else that's happening in the world of AI. So most days before we get the show started, we go over the AI news. Not gonna do that today, so you can just go make sure to check out the newsletter for that. Alright. Enough of me. Let's bring on our guest for today. This is one I'm very excited for, multi agentic systems and how we do it responsibly. Believe it or not, it's something I think about probably every day.

Jordan Wilson [00:02:34]:
And I have so many questions, and maybe you do too, and I'm happy to have someone now that can help us answer those. So, live stream audience, please help me. Welcome to the show, Sarah Bird, the chief product officer of responsible AI at Microsoft. Sarah, thank you so much for joining the Everyday AI Show.

Sarah Bird [00:02:52]:
Thanks for having me. I'm so happy to be here.

Jordan Wilson [00:02:54]:
Alright. I'm excited. But before we dive into this topic, could you please just let everyone know, like, what the heck do you do as the chief product officer of Responsible AI at Microsoft? It seems like a gigantic, job, right, in terms of what it could cover, but what do you do?

Sarah Bird [00:03:10]:
Yeah. You know, it involves doing a lot of different things, but at the core, we look at kind of risk we see emerging in new AI systems and then figure out how do we actually go address those risks. What does it take to test them? What does it take to mitigate them? And then how do we make it easy for everyone to do that? And so my team builds tools and technologies once we figure out those patterns that allows everyone to do this successfully.

Jordan Wilson [00:03:33]:
And I kind of talked about it a little bit here in the beginning about how AI has changed so much, obviously, right, over the last, couple of years. But how has your role changed, right, from, you know, maybe five years ago when we're first getting glimpses of generative AI technology to, you know, Copilot's been out now for almost three years, I think. Right? How has your role changed in the products that Microsoft has been building around this technology? How drastically has it changed the last couple of years?

Sarah Bird [00:04:05]:
Yeah. I think, you know, in some ways, it's pretty similar. We're trying to figure out how to make sure that AI matches our principles, that we can build it responsibly, that people can use it responsibly. But I think the big thing that's changed is people's awareness of how important this is and level of engagement. So I feel like before, generative AI really took off, I was working in this space and I would meet with, you know, Microsoft's customers and share what we were doing. And they're like, it's so great that Microsoft is doing this, but we're really early in our AI journey. So we have to get a lot more sophisticated before we even think about responsible AI. And now the first thing that people do is ask about responsible AI before they even get started with AI, which is excellent.

Sarah Bird [00:04:45]:
And so I really did not expect the field to grow so much in maturity and understanding. And, actually, I credit, media and organizations such as yourself who are helping get the message out there so that people understand the risk and why this is important. But it's a, you know, it's a big change.

Jordan Wilson [00:05:02]:
Yeah. I think, you know, risk and change are, you you know, two topics that are on any business leader's mind right now. Right? And, hey, as a as a reminder to our livestream audience, if you have any questions, for for Sarah, now's a great time, to get them in. But, you know, let's even just look at what was just announced. Right? What Microsoft just announced, at, its its build conference, which last week, which seems like so long ago now. You know, one thing in particular is is the agentic AI and everything around it. It just seems like it's everywhere now within the Microsoft, ecosystem and within, Copilot. But, you know, how has even just the growth of agents, how has that changed what responsible AI even means?

Sarah Bird [00:05:50]:
Yeah. I think that, you know, the thing that's amazing about agents and why I think we're seeing so much growth is they really are, I think, a more complete fulfillment of the promise of this technology. Right? I don't want a system that I just chat with. I want a system that's gonna go and complete task for me so that I don't have to think about it. And so, it's not surprising that we're seeing this, you know, huge excitement and people really starting to get value from these systems. But the challenge is that now if an agent is actually able to go and do task on your behalf, then there's more that can go wrong because it can actually take an action. And that can be bigger surface area, that can be higher implications because of the action it's taking. And we've also lost kind of one of our most important mitigations, which is having the human just directly in the loop having oversight.

Sarah Bird [00:06:41]:
Because now you're gonna have agents working for longer periods of time without a human in the loop. And so it really kind of changes the game. And there's a couple of ways we're we're thinking about how we go and address this, which is, first, agents are a little bit like an application, a little bit like a user, but not exactly like either. And so, we need to adapt our systems to manage these new entities and secure and govern them in the way that we do users and agents or users and applications and devices and all sorts of things today. But we also need to address those new types of risks that agents bring in terms of being able to go off task or accidentally leak sensitive data. And so it's a pretty exciting time, I think, with this new technology coming out and the potential of it, but also pretty fun to think about how we really do this well.

Jordan Wilson [00:07:28]:
And and maybe help our our our viewers and listeners here better even understand what this means. Right? Maybe if you could break it down. So, specifically, these these multi agentic systems. Right? It just sounds like we just add, like, another buzzword. Right? Like like, every couple of quarters, it's it's AI, then it's agentic AI, then it's multi agentic AI, right, with orchestration. But, like, what the heck does that mean? So whether it's at, you know, Copilot Studio, I I think this is also maybe in, the Azure AI Foundry. But, like, how does that actually work? Multi agentic orchestration.

Sarah Bird [00:08:02]:
Yeah. I think, you know, there's a lot of exciting visions where agents are just inventing things and talking to each other, and you have these really crazy multi agent systems. I think what we're seeing in practice right now is, something much simpler, but still extremely powerful. And I think, much easier to deal with from a responsible AI point of view, which is that people are really using multi agent systems to break their task into a bunch of subtasks. And what's really great about that is that you can make each individual agent really good at the smaller task that it is doing. And so you can test it specifically to do that task. We can put guardrails around it that ensure that it's only doing that task. And then you can have them coordinate and work together to complete a bigger picture, but every single thing is, you know, a component like in an assembly line doing what it needs to do.

Sarah Bird [00:08:52]:
And so, I think that, actually, I'm really excited about this multi agent pattern. It's something that we think people can be really successful with.

Jordan Wilson [00:09:01]:
I think the opportunity and the upside of multi agent orchestration is pretty pretty obvious. Right? How does the the the risk change, right, when you're not just working with one agent? Right? And you kind of mentioned how human in the loop changes, but how does the risk change when you're working with a a series where you're orchestrating multiple agents versus just working one on one with a single agent?

Sarah Bird [00:09:27]:
Yeah. You know, I think the probably the biggest thing is exactly that you're going to have a more complex system that you're trying to govern. And so you do need to break it into these components so that you're still governing each individual agent and not just the system as a whole because you still need visibility into what's happening in there. One of the areas that we, released that build was foundry observability, and this is exactly giving you a monitoring system for agents so you can see, did the agent go off task? Is it struggling to, find the right tool for the job and all that. And so we still need visibility at the individual agent level. You don't wanna look at just the multi agent system and the boundaries, or it's gonna be much more difficult to, debug it. It's gonna be difficult to ensure that you're doing important security practices like, least privileged access and everything. And so, even though they're multi agent systems and they're combining together, I don't think that that has to look different than a single agent working with a human.

Sarah Bird [00:10:27]:
Right? You can have different type of entities in the system, and you wanna make sure that you're governing each of them and you're having guardrails around each of them.

Jordan Wilson [00:10:35]:
And and, you know, haven't even started to to talk about the underlying models. Right? So, you know, this is another thing that, you know, it seems like everything is kind of happening at once because, you know, now these models that are powering the agents can can reason, and they can plan ahead. Right? And then when you combine that with this multi, a agentic formation, you you like, again, just the the the potential and the challenges are just jumping out. But, you you know, one other thing, that, you you know, I kind of heard you say there is is being able to work for longer. Right? So, I think we've always been trained, right, at least, you know, in the first, like, year or two of generative AI. It's like, okay. I sit. I talk with an AI.

Jordan Wilson [00:11:16]:
I I I look. I see what it sends back. But now it, you know, might be many minutes or multiple hours, right, in in the very near future where these agents are going and doing work. So how does that change? Right? You talked about the, observability and, you know, in boundary observability. But how does it change kind of the human role, when now these agents are gonna be working longer and deeper, together for a much, longer duration?

Sarah Bird [00:11:45]:
Yeah. The the way we think about it is, you know, in the the previous era, we had, humans really in the inner loop. And it's like you did a small task and then the human checked, and you did a small task and the human checked. And so what happens now is we're really moving humans more to the outer loop. And that can still be extremely powerful, but the tools that users need to use then are a little bit different. And so, for example, you wanna test your agent system a lot more before you deploy it so you know that it's really working well, that it works well on the task that you're expecting. But then you also need to be able to monitor it. So if it's going off task, then a human can come in and intervene.

Sarah Bird [00:12:26]:
And so that's some of the new technologies that my team has been developing are specific monitoring inside of foundry observability or guardrails, that look and say, okay. How well is the agent doing at staying on task? How well is the agent doing in understanding the user intent? How accurate is it in picking the right tool for the job? And if any of those seem to be not performing well, our system in Foundry is going to detect that issue and go and alert the the human, either the human administrator or the human user depending on the application and what makes sense there. And so then the human knows that they should come in and intervene. And they may wanna come in and intervene for a specific task, or they may wanna come in and intervene and say, oh, I need to make adjustments to my system overall so that it's doing the series of task well. And so we still have to build these same human in the loop mechanisms. It's just where the human goes in the loop is different now, and it's much more in the predevelopment setting what you actually care about and appropriate testing and in the post deployment monitoring and administration of the system.

Jordan Wilson [00:13:35]:
Yeah. And I love how you described it there, kind of the the inner loop versus the outer loop and and how even, observability is is changing a lot. But one other thing that you just talked about there, Sarah, was was testing, right, which, you know, I think, unfortunately, some organizations at times can glaze over that part in my experience because when you get these new capabilities, it almost like it's it's like getting a new toy, right, in Christmas. It's like, you know, if you get a Super Nintendo in in 1992, the last thing you're doing is reading the manual. You're putting the game in and you're playing it. How should organizations be testing, you know, just multi agentic systems? It seems like a a a herculean, you know, task.

Sarah Bird [00:14:21]:
Yeah. You know, the the we're gonna test the same way that we are testing anything, which is you test the components and then you test the system. Right? And you build up testing paradigm. I think, what's different is that what we need to test for are different behaviors and different types of risk. And so, a lot of what we've been building and the part of my job about making it easy for everyone to do this is building, testing systems inside of Foundry that people can build on top of and people can use to test for these new types of risk. So I mentioned some of the categories that are specific of did the system understand the user intent. Right? That's a test that we can run. But also, is the system vulnerable to new types of prompt injection attacks, or, is the system producing copyright material? Right? These are all different things that you wanna test for in your application, and you're gonna have application specific things you wanna test for.

Sarah Bird [00:15:14]:
If you're doing this in a financial setting or a health care setting, there's gonna be specific tests that you wanna run to see, is this fit for purpose? And so we've built the testing platform and many built in evaluators to make it easy for people to test for this, but also for the ability for them to customize so that they're really getting that what's important for their application so that they can trust the system. But I completely agree with your point that, one of the things we've seen is most organizations just don't realize how much they should be testing right now in this space. And so, they get to the end, and they're like, okay. We're ready to ship this thing. We're so excited. And then someone points out it does something that they don't quite like, and they're, how do we know it's not doing this all over the place? How do we know that this is ready to go? And then that's when they start going and investing and testing. And so, you know, we hear from a lot of customers exactly that that they, that the thing that is sort of delaying them into getting into production is building that trust. And, you know, some of that comes through testing.

Sarah Bird [00:16:12]:
And so one of the things that we are trying to do and, you know, I try to do is educate people earlier because you should start this, like, when we're building an AI system, we start with what is the system supposed to do, and we build the testing right alongside with the development of the system. So we don't wait till the the final last mile and then test and find out we have an issue. We're codeveloping, you know, looking at the different risks, looking at the quality of the system in every single iteration. And so, you know, I hope that other organizations do that as well. But we're all on a a learning journey on how to do this well.

Jordan Wilson [00:16:41]:
So a a lot of these, you know, capabilities that that we've been talking about here, when it comes to, you know, multi agentic orchestration, you know, they're they're fairly new for the general public at least. Right? But, you know, I'm I'm curious. What have you learned or maybe what were you even surprised by, on the responsible AI side, you know, as you've been building out, you know, this, these systems, which I'm sure you've been, you you know, testing them internally for many months or, you know, multiple years. But, you know, maybe specifically when it comes to responsible AI and agentic AI, in in your own internal testing, what has been maybe the biggest, surprise or or learning that you think would be helpful for business leaders to understand? Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on GenAI. Hey. This is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft, and NVIDIA have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use GenAI.

Jordan Wilson [00:18:08]:
So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too, just like some of the biggest companies in the world do. Go to your everydayai.com/partner to get in contact with our team, or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on GenAI.

Sarah Bird [00:18:39]:
You know, I think that when we, like, when we switch when, like, generative AI started and we were in the era of the chatbots. Right? I think that a lot of the focus and responsible AI was just, is this system producing harmful content? Can my user jailbreak it? Did I accidentally produce copyright material? And so a lot of the energy for us was about developing guardrails and testing for these kind of new AI specific risk we're seeing. I think, when it comes to agents, agents are, as I was saying earlier, basically a new entity that you're deploying in your system. And, and people are pretty excited about this. Our work trends index, for example, showed that I think 81 of employers are looking at deploying agents, alongside their workforce in the next eighteen months. Right? And that's a really different paradigm if you're starting to have users in your system and you have agents and you have applications. And so, once you have a new entity, they're saying, okay. We've already figured out how to govern, users.

Sarah Bird [00:19:43]:
Right? We have, for example, Entra, which gives every user an ID and helps you have access control on that. And we have Defender that's monitoring your systems and making sure that, you know, you don't see threats coming in on your devices. And, and the, you know, the first question people start asking is, well, how do we secure and govern agents in that same way? And so, a lot more focus actually on not just the novel risk that we see with AI, but just being able to secure and govern AI like any other thing. And, it sounds really basic, but I think there was just much less energy in that before agents came along. And so a lot of what we released at Build, for example, is a new Entra agent ID so that agents can be tracked in your system just like anything else and making sure that we're connecting this governance and security plane with what the developer is doing so that when I build a develop when I build an agent in Foundry or in Copilot, it just has an identity already attached. So I've done the right thing for my organization, and my organization can govern it the way it needs to. And so, I don't think, you know, maybe twelve months ago, I expected that I would be spending as much time kind of learning about how all of these, you know, traditional sort of security patterns work today. But that's that's where we're at, and I think that's one of the most important things with agents.

Jordan Wilson [00:21:01]:
Yeah. And and, a a kinda kinda related question here from Cecilia that I think maybe a lot of people, are are are thinking. So she's asking here on LinkedIn, does this human in the loop model create a different level of users with higher skill levels to understand hallucinations and derail it? Yeah. It's it's a great question. Like, as the the the agentic systems become more capable, how do the the the human in the loops need to maybe change their skill set in order to better, monitor and observe these agents?

Sarah Bird [00:21:32]:
Yeah. I I love this question because I think that's exactly right. If you're in a different point in the loop, you are are doing a different job. Right? And so, we mentioned that one of the things that you want to look at is testing. Well, testing is not looking at a single example and saying, did the system do the right thing? Yes. I'm, you know, gonna approve this. You're often then looking at aggregates and looking at numbers overall and saying, okay. You know, if 99.8% of the time it does the right thing, is that good enough for my task? And so you are making decisions with, like, different type of information.

Sarah Bird [00:22:06]:
And, and so, you know, I think the answer to the question is we're still figuring it out, and this is a place where, I'd actually love to see a lot more innovation. And it is this, human AI interface and how we design it for the world of agents where humans are farther out in the loop. And so, actually, one of the things we released at Build coming out of Microsoft Research is a, system that is called the Magentic UI. And it is a, it's a research system for people to play with and experiment with different interfaces for users to, interact with agents, basically. And so you can try different interaction patterns for all of us to learn which ones are really working, what is the best way for the human to intervene in a meaning in a way that's meaningful for them for the the skill set that they have. And, this isn't a new problem though, because, like, one of the things that's exciting about, this especially this recent wave of technology is it's so much better at coding. And so you have people now who can essentially complete coding task who are maybe not experts at coding. And, we had, you know, when we inside of Microsoft, you know, teams come forward and say, we built this great thing.

Sarah Bird [00:23:19]:
It's for, people that can't code, and it's gonna code for them. But if there's a bug, they're just gonna find the bug. And you're like, how are they gonna find the bug? The whole thing is that they don't code. And so, we have to be more thoughtful about how can the human speak a you know, and a way that they have, the ability to and govern in the way they have the ability to and still kind of play their part. So one of the things is that's where AI is a really helpful tool for this. So one of the things we've built, for example, is a system that looks at, does it understand the user intent and then alerts you if it seems to be confused about what you want. And so you can focus on just really specifying your intent, and our system's gonna check if the agent is confused. And so you don't have to go look at that.

Sarah Bird [00:24:03]:
And so where we can provide tools that bridge the gap between what the AI system is doing and what the user or the administrator wants to specify, then we can help make those interfaces actually feel natural and AI and humans work together.

Jordan Wilson [00:24:19]:
Speaking of of humans and AI working together, I think a big part of responsible AI is being able to understand and maybe even, categorize the different risks that businesses are facing. Sarah, could you walk us through what are the big, you you know, maybe risk categories, that that we need to understand, when it comes to, proper agentic AI implementation and just doing it responsibly.

Sarah Bird [00:24:43]:
Yeah. I so I really like and, you know, we've been using these at Microsoft, the, categorization that came out in the international safety report that came out at the AI, action summit in Paris. And, it has three categories. So the first is malfunctions, and that is the AI system, you know, doing something that it's not supposed to be doing. And, that could, for example, it be, you know, producing harmful content, or that could be it getting confused and going off task. It could be that it's, leaking sensitive data accidentally. Right? And those are some of the the big ones we see with agents. It's vulnerable to prompt injection attacks.

Sarah Bird [00:25:23]:
Right? Those are all types of malfunctions. The next category is misuse. And you can see kinda two types of misuse. I might misuse an use an AI system because I don't understand what it does. I don't understand if it's appropriate for my task, and that we address through things like education and thoughtful user interfaces so that people really understand what the AI system does and what it doesn't do. Then, of course, unfortunately, we live in a world where people are also gonna intentionally misuse these systems. And there, you know, we look at where we can have guardrails and monitoring and defenses and traditional security approaches for that. And the last risk, and then something that's very top of mind for me, is the, systemic risk with, AI.

Sarah Bird [00:26:06]:
And so, for example, with agents, I mentioned that people are gonna deploy these alongside, you know, their workforce. And, that's really exciting because people are gonna get to focus on much more interesting task and have agents do the very undifferentiated task in their work. But that is a different way of working, and so preparing the workforce to actually be be ready for this new skill set and collaborate with these tools, that's, you know, some a systemic type of risk that we need to go and address. And so when we think about response to AI, we have to look holistically across all of these, but it's pretty different tools that we're using to solve each of these risk with malfunctions getting much more technical. And then with systemic risk, we're looking much more at policy and education and upskilling programs in a very different type of of work to address those risks.

Jordan Wilson [00:26:55]:
That's one of my favorite topics to just, think about is, you know, this concept of, you know, getting the workforce ready and and and upskilling. Right? Preparing for the future. How should business leaders be doing that? Because it it's one thing I even struggle with because even agency, it seems like, is changing so much. Right? As these models, you know, now they can, they can think and they can plan and they can reason on top of, you know, you know, working with each other. And a lot of times, you know, business leaders are like, this is what I've spent, you know, twenty or thirty years doing. Right? Like, this is my agency. Right? This this critical thinking. Right? So how do we need to get the workforce ready, for working hand in hand with Agentic AI?

Sarah Bird [00:27:42]:
Yeah. So first, I think it's it's motivating people to want to do it. I mean, as you said, if you've been doing your job the same way for twenty years, depending on your personality, you might not wanna just go randomly pick up new tools. You might not even know that new tools are available. And so, you know, important part is the leadership and the cultural element of incentivizing people and making people excited to try new tools. Now, I personally have found that AI is a huge boost in my work. We I most of the systems I told you that we were developing to address these different risks, those are AI powered. Those are all things we couldn't have done five years ago without generative AI.

Sarah Bird [00:28:22]:
And so my job and what we're able to do and build has totally changed because of AI. But we have to and so I'm very excited to pick up the latest and greatest new AI thing, but we need kind of everyone to have that moment where they see how it changes their job in a good way, and then then that makes you want to try more. And so one is definitely incentivizing it. Number two, I think, is education. The tools work well for some things. They don't work well for everything. And so one of the things that, you know, we do even within my own team at Microsoft is have learning sessions where people share, look, I built an agent that did this and isn't it cool and it worked really well. And, you know, I've tried this and I'm struggling to get it to work.

Sarah Bird [00:29:00]:
And so having people learn from each other about the patterns of what's working and what's not and really having this be a shared learning journey and not something that just everyone is on in their own. And so I think those are some of the the important parts of this. And, you know, as the technology gets more mature mature and we have sort of more standard patterns, then it will be easier to say, you know, this is how you use it. Go do that. But in the earlier days right now, it's a lot of also getting people to experiment and and find the things that are gonna work best for them in their job.

Jordan Wilson [00:29:31]:
I think I think that's a a great, way to to look at this, as as we all grapple, with what this means. But, you know, Sarah, as as we wrap up, the the show here, I mean, we've we've covered a lot when it comes to responsible AI. Right? We've talked about new capabilities with this multi agentic orchestration, the the increase, you know, risk and responsibility and opportunity. And also, we we dove in a little bit on the future of human agency and preparing the workforce for a more agentic future. But as we wrap here, what is maybe the one most important takeaway that you have for business leaders when it comes to understanding the risk of agentic AI and doing it responsibly?

Sarah Bird [00:30:13]:
Yeah. I am you know, I think you have to go into it eyes open. Like, you need to know that there are risk and understand the risk and, pick use cases and things appropriately. And you have to invest. Like, the idea of you're just gonna go ship these systems without investing in testing, I just really don't recommend that. And so, you know, really going in with an intentional plan for response play, that's the organizations that we see the most, being the most effective.

Jordan Wilson [00:30:39]:
I love that. Being intentional, investing, and testing. I think that's a great way to end, today's show. I love it. So, thank you so much, Sarah, for your time and for helping us all understand a little bit better the risk and how to tackle them responsibly when it comes to agentic AI. Thank you so much for your time and joining the Everyday AI Show.

Sarah Bird [00:31:01]:
Yeah. Thanks for having me.

Jordan Wilson [00:31:02]:
Alright, y'all. That was a lot. We just got, so much ins like, so many insights just dumped on our heads. If you didn't catch it all, don't worry. We're gonna be recapping it all in today's newsletter. So if you haven't already, please go to your everydayai.com. Sign up for the free daily newsletter. Thank you for tuning in.

Jordan Wilson [00:31:18]:
We'll see you back for more everyday AI. Thanks, y'all.