Episode Categories:
Resources:
Join the discussion: Got something to say? Let us know on LinkedIn and network with other AI leaders
Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineup
Connect with Jordan Wilson: LinkedIn Profile
Leveraging AI to Transform Business Conversations into Tangible Revenue
Every organization is awash in conversations—calls, meetings, webinars, and spontaneous discussions. Traditionally, these invaluable exchanges were locked away in unstructured formats: audio files, chat logs, or meeting minutes that rarely saw systematic use. Yet, recent advancements in artificial intelligence make it possible to tap into this "offline dataset," extracting and converting conversational value into real-world business outcomes.
This article draws upon technical, practical, and strategic insights shown in a recent podcast episode that explores how businesses can use AI—specifically large language models and agentic architectures—to convert day-to-day conversations into direct financial growth. The discussion goes beyond vague promises and focuses on specifics: secure experimentation, structured data extraction, scalable automation, accurate sentiment analysis, and responsible deployments.
Structured Data Extraction: Turning Meetings and Calls into Actionable Intelligence
The foundation for successful AI deployment in communications is data structure. Unstructured conversations, historically overlooked because they don’t fit easily into spreadsheets or dashboards, are now prime material for business advantage. Using AI-powered communications platforms, entire libraries of interactions—voice, video, chat—can be brought online, tagged, and cleaned. Rather than treating meetings and calls as fleeting moments, businesses can extract precise topics, questions, resolutions, and repeated pain points.
Teams using this approach no longer speculate about what happens in customer service or sales calls. AI models can identify patterns: frequently asked questions, recurring objections, or sentiment shifts during key moments. Specific use cases include auto-generating knowledge bases, instantly surfacing relevant product documentation, and reducing manual note-taking with real-time AI transcription and analysis.
AI-Powered Automation: Identifying and Scaling High-ROI Workflows
Businesses benefit not just from understanding conversations, but from acting automatically on common patterns. Contact centers, for example, handle immense volumes of customer inquiries that fall into predictable categories. Modern AI solutions, tuned via domain-specific fine-tuning processes, don’t just record questions—they categorize them and recommend or trigger direct automated responses.
Instead of workers manually pulling up answers or transferring calls, AI agents supply contextually appropriate suggestions, automate routine follow-ups, and triage cases for human intervention only when necessary. The impact: reduced costs, faster resolutions, and measurable improvements in customer satisfaction, as proven by deploying AI-powered customer satisfaction scoring across every call, not just manually surveyed samples.
Sentiment Analysis and Deep Context Linking: Unlocking New Operational Insights
Sentiment is more than a buzzword—it is a crucial metric that AI systems analyze with precision. By connecting sentiment data across multiple interactions (calls, emails, chats), companies can refine their customer knowledge. Instead of relying on biased post-call surveys, AI ensures every exchange is measured, presenting a more accurate view of satisfaction, friction points, and opportunities for upsell or intervention.
The utility stretches further when organizations use AI for deep research capabilities, connecting dots between conversations that would have remained siloed. For example, models can flag recurring negative sentiment tied to specific products, channels, or even agent behaviors—surfacing insights previously invisible to management. These connections lead directly to smarter decisions: targeted retraining, product updates, or strategic pivots based on aggregated conversational evidence.
Secure Experimentation: Sandboxing AI Projects to Accelerate Adoption
Many internal AI initiatives fail due to IT concerns about production risk, halting innovation before anything reaches users. AI sandboxes—a secure, drag-and-drop experimentation environment—solve this by letting business units prototype, AB test, and compare results across models/agents, all within governed boundaries. This reduces shadow IT risk and enables even non-technical staff (using no-code or low-code tools) to participate in building AI-powered solutions.
By deploying these platforms, departments collaboratively iterate on workflows with full accountability and security. Winning prototypes can be seamlessly migrated into production, maximizing speed without jeopardizing governance.
Responsible Automation: Evaluation and Red-Teaming for Trustworthy AI
With automation comes the challenge of reliability. Successful organizations rarely implement AI in a “set and forget” manner. Instead, they commit to rigorous evaluation: saving every automated output for ongoing review, using multiple models to cross-validate responses, and tasking internal teams to probe for edge cases and failures (so-called “red teaming”).
Such diligence ensures automation delivers ROI without introducing unacceptable risks. Evaluation frameworks evolve as models improve, with test cases and safeguards updated regularly to match new capabilities and regulatory requirements.
Integration Readiness: Laying the Groundwork for Scalable AI Deployment
Even the most advanced AI is powerless if it can’t connect with existing tools and workflows. Before scaling conversational AI, organizations assess their technology landscape—point-of-sale systems, proprietary workflows, and third-party vendors—to guarantee seamless integration. Crucially, they choose platforms that are model-agnostic, avoiding vendor lock-in and future-proofing their investments against continual model improvements and market shifts.
Conclusion: Actionable Roadmap for Conversation-Driven Revenue
By transforming everyday conversations into structured, actionable, and automated processes, businesses unlock substantial revenue and operational efficiency. The roadmap is clear:
Structure and analyze conversational data in real time
Automate high-volume, repeatable processes safely
Implement multi-level sentiment and context analysis for strategic insights
Empower secure, rapid prototyping across business units
Evaluate models continuously, with rigorous, adversarial testing
Align internal systems for seamless AI integration
The organizations that act on these specifics—not promises—stand to extract genuine business value from the conversations happening every day.
Topics Covered in This Episode:
- AI-Powered Communications Platforms Explained
- Enterprise Data Security in AI Sandboxes
- Turning Business Conversations into AI Insights
- Large Language Models for Conversation Analysis
- Leveraging Unstructured Meeting Data with AI
- Real-Time Sentiment Analysis for Revenue Growth
- AI Automation in Contact Centers and Sales
- Evaluating and Fine-Tuning Language Models Safely
- Responsible AI Automation and Red Teaming Practices
Keywords:
AI-powered communications platform, Dialpad, large language models, business automation, customer conversations, conversation analytics, structured data, unstructured data, conversational AI, voice data, meeting insights, sales optimization, customer service AI, agentic AI, data integration, enterprise AI adoption, data security, AI model evaluation, sentiment analysis, call transcription, AI in business, automation use cases, secure sandbox environment, drag and drop AI builder, no-code AI tools, low-code AI tools, AI for business leaders, AI driven revenue, compliance in AI, voice to revenue, knowledge extraction, customer satisfaction score, AI CSAT, sentiment detection, business process automation, AI feature development, AI hackathon, internal AI tools, optimizing machine learning, AI red teaming, responsible automation, AI integration strategy.
Podcast Transcript
I talk to hundreds of enterprise leaders a year, and do you know what kills most AI projects? Teams can't experiment without IT freaking out about production risks. That's where Aria comes in. They built a secure sandbox environment where your teams can prototype, test different models, and even run AB tests between agents all before anything touches production. Drag and drop interface works with any AI model you choose, and when you're ready, you move to production on the same platform. So check out today's show notes or see our website for a free trial of Aria. Go to airia.com, because innovation needs a safe place to fail fast. When it comes to working with large language models and AI, we all know that data's everything. We need clean data.
We need structured data because that's what's ultimately gonna help us leverage AI to hopefully grow our companies. But I've been talking about for a very long time. There's a source of legit gold that your company is sitting on that you've probably doing nothing with. Because it's not a spreadsheet. It's not something structured that you would feed into a large language model. Normally it's conversations, it's meetings, phone calls, jumping on with other people, maybe webinars, anything. Right? But when we are talking and when we are engaging with others, there's so much valuable insights that come out of that that we should be capturing, collecting, cleaning, and using, in whatever AI model that you are leveraging at your company. So that's what we're gonna be talking about today and just how we can use AI to turn those conversations into revenue.
And we're gonna be getting a guide from an a leader doing it at a global scale. So I'm excited for today's show. I hope you are too. Welcome to Everyday AI. My name is Jordan Wilson. If you're new here, we do this thing every dang day, at least Monday through Friday, bringing you an unedited, unscripted look at the realist in AI with a daily livestream podcast and free daily newsletter. So if it's your first time, welcome. It starts here.
But if you wanna take it to the next level, make sure you go to our website at youreverydayai.com. There, we're gonna recap the highlights from today's show as well as keeping you up to date with all of the other AI news. So I'm excited for today's conversation. Enough of me chit chatting. Let's bring on someone that's been doing this for a long time and really knows how to take and extract the value of our conversations, and better use that with AI. So, livestream audience, if you could help me, welcome to the show, Jim Palmer. He is the, chief AI officer at Dialpad. Jim, thank you so much for joining the Everyday AI Show.
Thank you, Jordan. Thank you for the invite. I one quick comment. You're gonna have to start working weekends because the way that AI is accelerating is it's hard
to keep up with, but it's so exciting. It's true. Yeah. Every weekday AI just didn't really roll off the tongue, quite quite the same as everyday AI. But, you know, I'm I'm obviously very familiar with Dialpad. But maybe for those, of our audience that aren't, tell everyone a little bit about what it is that you all do.
Dialpad is an AI powered communications platform. We have all of the different types of channels and, tools and essentially platform for our customers to communicate with their customers. It's the business to business use cases, business to consumer use cases, And we try to make this as seamless and as easy to use and navigate and grow and scale with your business to better communicate with your customers, better understand your customers. And, we've been at it for quite some time. I think we just hit our fifteen year anniversary, and we've also got a a lovely, slogan here, first in AI, best in agentic. And first in AI in the sense that we, acquired, actually the the company that I, cofounded and, started pretty many years ago, almost eleven years ago. But we had been acquired by Dialpad, seven years ago to directly integrate AI, you know, state of the art, AI at the time, and I will admit that it's, grown and changed and evolved a considerable amount since then. But it's been a a a huge investment for Dialpad for a great many years to have that be basically seamless AI and and being able to bring, you know, seamless insights to help our customers better communicate.
You know, obviously, the generative AI, kind of boom of large language models over the past couple of years, has kind of changed everyone's viewpoint on who should be using AI or how, we should be using AI. Obviously, at Dialpad, you're you're very tuned into, you know, conversations, at least on the front end. You've mentioned the agentic side. But talk a little bit. How has the landscape changed, in your view over the last couple of years?
It well, it's changing hour by hour, minute by minute. It it's at a it's just keeps accelerating. And I remember even ten years ago lamenting at what was coming through even just with academia and the papers, you know, the PhDs, postdoctoral, all the advancements were moving faster than anyone's even a small team could really keep up with. It it was amazing. And then now I I can't even tell you the multitude of how much more it's advanced, but a couple of really big shifts happen in in a very short period of time. And I'd liken it to yeah. With the ebb and of, large language models, there was a huge push two, three years ago that basically showed the world what what's actually possible. If you invest the right amount of capital, the right amount of data, the right amount of compute to be able to generate an LLM that is capable of things that that, yeah, there might have been a a a few folks in academia that might have said this might be possible, but we haven't seen it before.
So there was just a huge shift ultimately. And what had been building up all the way up until that time about three years ago and is that we're starting to get integration for all of the data to go into large language models. Right? And now there's a lot of advancements on top of that, not just language as in, like, digital communications, emails, text messages, chats, that kind of thing. But now we've got audio. Now we've got video and meetings and webinars. But what what essentially happened was, I think, the biggest shift because we can start even as a a casual user, someone at home, or, me having built this AI team in a a large company like Dialpad and building AI. We found ways to not only leverage that and add it as functionality for our end customers, but also we use it internally too. And I think there's, like, Jordan, I there's so much to talk about here, but I think what's really important is calling out the there's there's the AI that that you can build for your customers as part of your product to sell to your customers or or AI that you can use in in conjunction or that complements other vendor software that you're using to build just your business, but there's also AI that you can use for your team, AI that you can use internally, AI that you can use in the home.
Right? And I I think there's a lot of really great stories even with what, the success that that I've had and with, what Dialpad's had of the AI that we built for our customers and the AI that we use to to better improve the way that we're building things, the quality of what we're building, the speed and execution, how how we basically scale it internally, and to help us scale our business.
So I wanna maybe zoom out a little bit here and go, thought leadership y on this one. You know, I kinda started off the the show today talking about how much value that there is in just conversations. Right? And it seems like everyone's, you know, rushing to the next, data source or data set to improve, you know, AI, implementation at their at their company. How much just how much value is there in the conversations that we're having every single day? And maybe how much of it would you say is just is is just being wasted, thrown away, and and and not leveraged? Discarded.
But, there's a lot of things. We gotta do this we have to do this safely. But we are getting so much better about leveraging that data and especially with voice. Let me just go off a little bit of a tangent too because that was part of my original pitch that that pitch of, hey. Here's the idea is where we want to take the last offline dataset and bring it online. And there's a lot now that we kinda bucket this to bringing it online. We wanna add structure to it. We all wanna extract the meaningful information and the insights.
And that that that was kind of the pitch of starting, TalkIQ. And that carries through even still to today because we wanna leverage the important parts of those kinds of conversations, whether it's, you're talking to another business, you're talking to another customer, or you're talking you know, there's every use case that that kind of involves the way that humans communicate. And I think what's what there's there's still a massive opportunity in that because, yeah, look at the lead up for the last two, three years where, there's a lot of, training for massive machine learning models being built on things that we're able to get for free on the Internet. Mhmm. But it only has so many modalities or, you know, it's really only digital form. And now now we're starting to find that there's there's there's the there more data sources. There's the there's the speech. There's the things that are happening in even in private scenarios.
Right? So there's, like, that kind of that last data frontier has just been, like, pulling trying to get access to that information, but do so securely. So I think that's that's something that is really part of I'd love to say, hey. Our differentiation with Dialpad is we're helping not only build the communications to, well, better communicate, but we're also getting getting safe and secure access for our customers to tap into all of the ways that they communicate with their customers and be able to build off of that. You know, bring the structure in, train off of, and, of course, we go through all the security we go through with so many different steps and protections and and compliance to be able to to do this, but it it does help because what what happens is you start to see a lot of patterns emerge. And I'm not to claim I'm a linguist by any means, but I think this really gets the linguist excited in the room in the sense that they're they're and especially in a business use case, they're they're they're actually they're we're saying a lot of the same things. Mhmm. And I can I just go off on a on a tangent just with, like, a a really good use case?
That way, you see all of them.
You know, dial dial pad. So we like to sell our, contact center part of our solution, and they sell we sell to teams that support or use Dialpad to support their product. So they've got a pool of agents that are able to, you know, get, respond to their customers as they're calling in, And there is such a high level of common questions, basically asking the same question with a few words that are different. And then that that right there is so right for not only being able to extract that, you know, what was this question about, what was the answer, but also automation. So there's so many of these use cases, these common use cases that are, like, that's where I see this massive acceleration and especially in the business use case. And then you start to think of even outside of that. How am I using that internally for our business? How am I using that potentially at home? You know? There's a lot of the smart home movement, etcetera. So but just, like, a lot of those the the the common questions, the common answers, the common automations that you could potentially build.
So that's why it's just so so exciting because we're we're we've we've kind of proven that there is a lot of that commonality, and we've been able to build up a lot on the accuracy, the speed, the real time, the real value, like, the real ROI ultimately when it comes down to
for our customers. It's a problem I hear all the time. The gap between the AI champions and everyone else in your organization is sizable. You might have half a team that wants to fine tune models by hand and the other half doesn't know what an API is. How do you get them working together on AI that moves the needle without creating a security nightmare? That's where Aria really shines. They built one platform with three ways to work. Your developers can go full pro code and build custom agents with Python. Your business analyst can use the low code tools or your domain experts who've never coded, they can use the drag and drop no code builder.
Everyone's building in the same secure governed environment. No shadow IT, no security gaps, and because they're model agnostic, you're not locked into one vendor's ecosystem with Aria. You can even a b test different agents against each other. Try different models, and when you're ready, deploy to production on the same platform. Your AI strategy should unite your team, not divide them. Check out Aria in today's show notes or on our website for a free trial. Go to airia.com. Get rid of the AI gap and move forward with a more resilient AI ecosystem.
So, yeah, you gave you gave some great examples there. Right? And I think that there's a lot of direct lines that can be drawn, right, between, you know, analyzing conversations and, you know, customer service, sales, right, and seeing certain phrases, certain words, maybe push things in a different direction. Right? Very direct line between conversations and revenue. But I'll ask you this. Maybe outside of of, you know, using a certain product or or service from Dialpad, how or maybe why should the average, business leader out there, pay attention to conversations, pay attention to meetings, pay attention to phone calls, or at least start, thinking about capturing that and turning it into knowledge. Why is it worthwhile, or is it worthwhile?
I definitely think it's worthwhile. I mean, even just me personally. What did I miss in that meeting? What I you all I I take notes all the time, but I've started using other tools to help me take notes. And I could list off a a great number of startups and even open source and totally free software to help you do that. And it has helped. I've seen the immediate effect. I can go back to that information. I've built a gym, you know, graph of knowledge.
I've got chatbots for myself. I've it it it's it's painfully obvious how much it's helped. And if you could think about that, even if you're able to kinda have that breakthrough on a personal level, trying these tools and doing so safely. Right? And not just absolutely trusting whatever you might get from, the biggest large reasoning model out there, but also just being, you know, finding where it works and where it doesn't work is so much easier to do right now for AI. So I think that's that's the big accelerator. Right? Is that people are finding whether they're working for a company and they're forced to use AI. No. No.
No. They're they're finding ways that it's actually helping them in their whether it be their personal life or they're they're hearing about a friend that might have tried this or they're listening to your podcast, Jordan, and then they go, this this person mentioned something that sounded really I don't know. I might try that out. And what we're basically building on right now is this massive kind of show and tell. And there's, obviously, the socials are are picking this up. We have our own internal show and tell for, hey. We have engineers. We have customer support.
We've got all these different business units that are excited to be continuing to try AI. Right? Not just try it and it doesn't work and then they give up on it, but more they try it, they take notes, and then they'll find time to try it again for maybe the same use case or another use case and see. Now now it works. And so the show and tell is showing what works, what doesn't, and we're getting much better even internally at Dialpad at at being able to know, hey, where we would what's a what's a higher level of accuracy, a high level of trust for doing these certain things versus others that we can, you know, keep trying on. And so now it's this feedback loop internally even where we're all kind of challenging each other. How can we use this better? How can we use this more intelligently to to to build a better product, to have a better customer experience? And then what also is is kind of a neat part about this is it's almost like a a a hackathon for the engineers in the room or the folks that, have have worked at companies that have done this where Mhmm. Some of the best features, some of the best tools, and sometimes even the best, like, spin off companies have come at a hackathon projects where you get to sit back and say, I just I really wanna have time to solve this problem. And now AI can help with that.
And we've had I I could talk about our our hackathon and our we have, hackathon we do internally for purely product facing. Now we have an a hackathon just for for AI specific, and it's been a huge accelerator for us, not only just in productivity, but it also potential features that we've been adding to the product. So that my whole point with this is, like, I I I see this and, yeah, I'm I'm fully entrenched in the middle of the the AI, AI world, but, you know, I'm I'm also seeing, and I'm I'm looking, and I'm hungry to see where people are. Are they pushing back? Are they skeptical? What are they skeptical about? Where have they seen it not work? Right? And and I think we all are growing to this this how how we can use this. Right? It is it a calculator or is it something more? And so we're just it's becoming far more real. That's what's so exciting.
So, I do you tease something out there. You said some of the tools, that you're using some startups, some open sourcing. So I wanna follow-up on that, but, just have to take a quick, quick little break here for a word from our partners first. It's a problem I hear all the time. The gap between the AI champions and everyone else in your organization is sizable. You might have half a team that wants to fine tune models by hand and the other half doesn't know what an API is. How do you get them working together on AI that moves the needle without creating a security nightmare? That's where Aria really shines. They built one platform with three ways to work.
Your developers can go full pro code and build custom agents with Python. Your business analyst can use the low code tools. Or your domain experts who've never coded, they can use the drag and drop no code builder. Everyone's building in the same secure governed environment. No shadow IT, no security gaps. And because they're model agnostic, you're not locked into one vendor's ecosystem with Aria. You can even a b test different agents against each other. Try different models, and when you're ready, deploy to production on the same platform.
Your AI strategy should unite your team, not divide them. Check out Aria in today's show notes or on our website for a free trial. Go to airia.com. Get rid of the AI gap and move forward with a more resilient AI ecosystem. Alright, Jim. So walk us through. You teased us there a little bit. And I love, especially people who are chief AI officers, who have been in AI, you know, a long time way before the the chat g b t moment.
What are some of those those tools or techniques that you're even using personally, you know, to, maybe leverage more value from your your conversations?
Oh, boy. How much time do we have? It's, my my favorite, and this is just more on a personal level. Like, what do I do at home to to grow, and how am I leveraging this? And how do I get more access? I really wanna see what's kinda under the hood. Right? It's almost like I I wanna get a a certain type of car that I can work on as opposed to something that's just gonna you know, I don't need a flying car. But when when it comes down to it, I I wanna bring models in house. I wanna get models in on on hardware in my house. I wanna work with those. I wanna try and tune those, fine tune.
There's so much that you can do, but having it run essentially locally. And it it but you're always having enough compute. I mean, there's a huge investment that that it takes. It's you it's not gonna just run on your cell phone and be even remotely as accurate as, anything you might get as the the top foundational model providers out there right now because the those models don't fit on this this type of computer. I'm pointing down here because my phone is right there. So it's like, what's that balance? And and this this all kinda comes back to, my my team has done so much work over the years. We've had, basically peer reviewed published papers at academic conferences, and academic conferences have started to kinda open up to to industry, you know, companies that are making money and ROI and everything, yeah, actually starting to contribute back in this academic world. But what we're what I love the fact that my my team has been able to show, hey.
You can take something and optimize it like a machine learning model and optimize it so so much so you can get the scale. You can get the real ROI. Your cogs don't go. You know, your costs don't go just absolutely out of out of control. And you you you have so much more control over all of that as well as accuracy. And this all comes full circle to what I was saying a little bit earlier with there's a lot of the same things that are being said in a lot of conversations. And it's the same thing even for for this the AI that I wanna have running in my house kind of thing Mhmm. Where you can is it fine tuning is a kind of a a very popular, but also very difficult, process to basically start to optimize these models because what you see in a lot of business context.
And I think this is a really important walk away is that that the amount of things that are said are the goalposts, like the the the differences between what's said and what's not. It is very small compared to what a lot of the, a lot of the the AI for general use. Right? Like, general use, foundational models out there. They're trying to be as accurate as they can for every possible use case, every single language, every single, voice, you know, modality as as some like to call it. So you you with that that being able to shorten the things, you know, increasing the accuracy for the amount of things that are that are said or the things that you wanna try and extract from conversations gives you so much so much of an advantage from an accuracy standpoint and everything else. So that's the stuff I love doing is how do we optimizing this? And then what's the next phase of that? Where where the other advancements? How do all these things tie together? So yeah. Like, I think home automation is a little bit of okay. It's terrifyingly fun, I will admit.
But it's like, I like to do it and on on my own. But I know there's there's so many other tools, services, tie ins, integrations. But it's like finding those opportunities for where I I wouldn't immediately think, like an LLM might help me. I'm starting to find that there are opportunities that where that could that could help.
You know, one thing that, and I love the use case of, you know, talking about, you know, even, like, personal models. Right? And I'm I'm big on the, the small language models of the future and and how good those will be for voice and just actually powerful edge dictation. Right? But one thing I'm always still doing is is using AI to know a little bit more of the unknown. Right, kind of the, Johari's window type, exploration. Right? Obviously, I have way too way too much, content, that I've captured conversations, voice, right, and being able to find connect dots that I maybe didn't know I should be connecting. Right? And understanding, and maybe tying together some, sentiment that could improve my podcast or could, improve, you know, trainings that I do or something like that. I'm wondering, is there maybe a use case on the sentiment side that you found, that maybe you're like, hey. Once we looked at, all of these calls, all of these conversations, maybe we're able to connect some dots Oh, right.
Revenue or save time after the fact that we didn't know.
It's a good lead out. This is great. Okay. It's one of my favorite, features because it's not so much of the, oh, we've created a whole new, field of science, but more this is an immediately obvious use case. We have an AI CSAT. We're generating a customer satisfaction score on every single call. And it's at face value, it's just a it's a percentage. Was this something good or was this something bad? All of us on this call have, hey.
Would you like to stay on two minutes and enter the survey? And you have to press a 1 through 41 through 5 to I and and it's it's horribly biased in the sense that the only people are gonna stay on it. I'm really mad. That's a very negative score. So here we are building this this we can gauge the sentiment in so many different ways. We can add more context to that, whether it's prior conversations, other channels that we're communicating with that customer, other inputs. Right? And and it's a kind of a classifier type of problem. And large language models, small language models, massive reasoning, MOE models can all do classification very, very well. But it's it's that kinda input that you don't just oh, hey.
I run it against the model I got. Oh, it has a low sentiment, and then I've I've shuffled it off. No. It's just it's yet another input into something like, if you're you're saying, hey. I wanna use the the biggest and the best that I can to give me other insights that I might not have known about too. But if you have things like a sentiment that you can add to, the maybe the transcript from another podcast where to to link to, say, you find another person, you know, I wanna talk to them. Right? And getting all of that context, yeah, that's still extra information that you can bring in, extra context that you can add to another another model. And I I use everything is is basically my my point.
Not just, oh, hey. Bring it in house like I was talking about and use your models that you're trying to run on your cell phone or in house, but try everything. The the the idea that I think, has been kinda marketed as deep research is I'm gonna admit, it's that's amazingly, amazingly powerful. I use it. I'd say most of the business functions and go to market that I've seen successfully that that almost feels like table stakes for a lot of the go to market teams. Anybody who's trying to sell anything uses a a deep research like feature, and I'm gonna admit that's it's absolutely amazingly powerful. We also have to think about the limitations of it. Where is it getting all of its information from? It doesn't know everything.
So the more that you can add to it based off of prior conversations, other inputs, that's that's kind of the the, I think, the breakthrough as users. Right? As you try to find, you know, new content for your podcast or us says, hey. What's a new potential revenue stream we haven't even thought of or a way to kind of, you know, sell the specific feature to a a an end customer because they've talked about it in another context, that's massive. You're starting to connect all those dots. So I use use everything you can try it out, but okay. Here's what I wanna come back to. Because we're we're going off on this, you know, Jim, what do you recommend? Here's one thing that has persisted through as long as I've been in this. Going back to kinda your earlier question of, you know, Jim, you've been doing this for a long time.
The one thing that persists ultimately is how are we evaluating? How are we just not just sending a prompt to, an LLM and getting a completion and saying that looks great. What are we doing to to build up the trust? Right? And it's obviously, it's been improving the accuracy so much about it, screams we should trust this. But if you're a builder and a practitioner of AI like we are, like I am, or if you're a company who's using a vendor who's promising all of this amazing agentic functionality, I would still implore everybody to find a way to evaluate it, to test it, and not just one quick, we refer to it as smoke test. Like, is it smoking or is it not kind of thing? But more start to build up those test cases. Right? Like, you're Jordan, I I assume you're building up this massive treasure trove of all this structured data from all your prior podcasts and all of your research. Right? And and how are you gonna if you're asking an LLM or, you know, the reasoning model to do something, you you you do it. You pay for that completion back. You save that.
And then there are ways that you can build a test on top of that. Mhmm. And you can use other LLMs to test other LLMs and start to build up this whole network. So just coming back to this, if if you're just starting in using AI, if you're already deeply entrenched and have 12 different vendors that all kinda do it really, really, really well, invest in a way to just evaluate it because of there's a lot of things. It's getting better. And in some cases, you know, it it's changing. It might get better or it might get worse. I I can't give any promises.
But the fact is that it's it changes. And my I I I'd say just looking back even over the last three years, it's definitely changed for the better. It's definitely improving in a lot of accuracy, and all the players are are it's amazing. But just keep updating those tests, those evaluations. And I could spend another hour talking about the ones that, you know, that that we've used and I like, some that are really hard and constantly have to maintain, but just find a way to to store and to allow you to to test that over and over again so that it doesn't become, you know, too much work to do that. So that that's yeah. Alright. Sorry.
I could keep going on this whole, spin, but No. No.
I mean, it's you bring up some some great points, and I love even, you know, the concept. And I think it's a good practice of, you know, understanding evaluations and how you evaluate and and why and, you know, what role the human plays in in augmenting, you you know, with a certain LLM, in that evaluation process, obviously, extremely important as well. So, you know, Jim, we've covered a lot in today's conversation, but, you know, as we wrap up, I'm not gonna ask you to predict the future. Right? But I will ask you this. Right? Because we're already starting to get into this, you know, year end mode, and, you know, 2026 is right around the corner. Aside from using Dialpad. Right? But, aside from that, how, should business leaders be, looking at the, relationship between meetings, conversations, and using AI to tie it to revenue? What is your most important takeaway? Or, you know, if you were to advise someone, hey. In 2026, you need to be doing this.
What is that thing?
Oh, you're gonna everybody's just gonna say, oh, he's gonna say agentic. Yeah. I'm gonna say that, but I'm gonna say there's a couple of things. And it's not what it it would be it would be also kind of exercising, not caution and not trying to put fear, but responsible automation. Right? And it's what are those use cases that you are confident you can automate? Because right now and and even going back to doing this AI thing for a long time, and and I've been trying to sell AI dream and trying to make it a reality. And it's definitely it's it's a lot more real now. But before, it was all about time savings and how can we more efficiently close the gaps, the knowledge gaps, those kinds of things. But right now, there's this huge leap into that that full blown automation because that that's how you can kinda prove this is how you save money.
But there still is no there's no guidance on how does it actually work and work 100%. So I think it kinda comes back to the, you know, the evaluations and everything like that. But but coming in as a business and to say, hey. I'm gonna get ready for 2026. I'm gonna spend some time and some money, not just diving into a solution just yet, but figuring out what can I automate and what should we not? What protections do you need to put in place? And I'm not here to, you know, fear monger or anything, but that those are the kinds of things that if you if you jump in and you oh, if this works amazing, it is an edge case. Just a random if this one thing happened, is that gonna cause harm in some way, shape, or form? And and there's so many things that I could, you know, advise on this. And I this this is a part of my top track for years, but I use the concept of red teaming, and and not going back to, like, cold wear at cold war esque. But just think about it if you're testing something.
And even if you're testing someone home or a new model you got on your phone or the new massive, you know, video video generation thing, try to break it. And I've always loved that. Right? And red teaming is basically like adversarial testing. Try and find how it doesn't work. Because if you find where it doesn't work, you're also gonna find where it does work. And so it's a win win situation. And so they part of this is if you if you're gonna go into 2026 is figure out what it is that you can automate safely, and then what are those integrations? Because the AI, I think we're there's there's the investments there. The momentum is there, but it's also the integrations.
Are the point of sale systems gonna be able to keep up? Are the that you've been using and invested in and there's no you you know, you're you're you're basically this is what you're you you have to use. If you have a proprietary system, what changes need to be made? Are your vendors gonna be, you know, AI first or AI native at some point in time? Right? It's like, make sure that all the the scaffolding is built up. The foundation is strong for being able to say, we're gonna do AI, and we're gonna do it right.
Mhmm. Jim just hit us with the the quad facta ending. Some of my favorite things, evals, breaking things, responsible automation, and integrations. Love it. So thank you, Jim, so much for taking time out of your day to join the Everyday AI Show. We really appreciate your time and insights.
Yeah. Thank you, Jordan. That was fun. Alright.
It's pretty good. If if you miss anything, don't worry. We're gonna be recapping it all in today's newsletter. So if you haven't already, go to youreverydayai.com. Sign up for the free daily newsletter. Thanks for tuning in. Hope to see you back later for more Everyday AI. Thanks, y'all.
As someone that covers AI every day, no one knows which AI model will be best in six months. That's why betting everything on one vendor is dangerous, and that's why Aria can solve the LLM FOMO issue. They're completely model agnostic. Use GPT for one agent, Claude for another, your own fine tune model for something else. Switch anytime without rewriting code. Their intelligent routing automatically sends requests to the right model based on your rules. Check out today's show notes or our website for a free trial of Aria. Go to airia.com because vendor lock in is a terrible long term bet.