Resources:
Join the discussion: Got something to say? Let us know on LinkedIn and network with other AI leaders
Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineup
Connect with Jordan Wilson: LinkedIn Profile
The True Business Value of AI in 2025: Three Industry Myths Debunked
Every year, studies surface in the AI industry that dominate news cycles, sway enterprise investments, and shift the narrative for business leaders. In 2025, three high-profile claims about artificial intelligence gained widespread traction—but closer inspection reveals that each was built on shaky ground, with significant implications for business decision-making.
This article isolates the three most viral AI misconceptions of 2025—drawn from Menlo Ventures, Graphite, and MIT—and translates actionable insights for business leaders to ensure data-backed, strategic decisions.
AI Enterprise Adoption: Misinterpreting Market Share Data
A midyear report from Menlo Ventures claimed that Anthropic had surpassed OpenAI in enterprise market share, citing a shift from OpenAI’s historical lead to Anthropic owning 32% of the market compared to OpenAI’s 25%. This narrative flooded industry commentary across LinkedIn and other media.
Critical analysis shows the following issues that matter for business strategy:
Selection Bias: The entire claim stemmed from a single survey of 150 technical decision-makers, all of whom were part of Menlo Ventures’ portfolio or close network. As a venture capital firm with $1.2 billion invested in Anthropic and a direct interest in accelerating Anthropic’s adoption through its Anthology Fund, the incentives and sample were heavily biased.
Survey Scope Limitation: The methodology only considered API usage, notably excluding enterprise adoption through products like Microsoft Copilot—which, until recently, relied on OpenAI models for the majority of its AI functionality. Enterprise clients using Copilot were not counted as OpenAI users, artificially deflating OpenAI’s actual business penetration.
Neglecting Fastest-Growing Sector Data: No reference was made to ChatGPT Enterprise, which reached over a million business users in record time, further discrediting claims of Anthropic’s market leadership.
Real Industry Benchmarks: Independent reporting from Ramp (October 2025) showed 93% of organizations in the generative AI category use OpenAI, with 95% adoption among enterprise companies—directly contradicting the Menlo report.
Key takeaway for leaders: Scrutinize the source and participant selection of any AI adoption research. Valid enterprise AI metrics require independent, comprehensive, and methodology-transparent data.
AI Content Detection: Misleading Narratives About Internet Quality
Graphite’s October 2025 study claimed that 57% of new content on the internet was AI-generated “slop.” This assertion was drawn from testing 65,000 URLs with Surfer SEO’s AI content detector and counting any article flagged as over 50% AI as entirely AI-generated.
Risks and limitations from a business perspective include:
Reliability of Detection Tools: AI content detectors for text are deeply unreliable, with no industry-standard method. Surfer SEO’s tool, like all current detectors, only analyzes statistical word patterns—not authorship—and can classify anything, including classic literature, as AI-generated. For example, passages from the Book of Genesis were flagged as 86% AI.
False Flagging Impact: These tools have a long history of flagging non-native English writing as AI, creating reputational and operational risk for multinational enterprises.
Watermarking Gaps: While video and image AI outputs may embed watermarks, no reliable, invisible watermark exists for text, and textual content is quickly and easily manipulated post-publication.
Educational and Compliance Risks: Stanford studies found 61% of ESL student work falsely flagged as AI; major universities like Vanderbilt have discontinued faulty detection tools due to hundreds of false accusations.
Actionable advice: Avoid basing hiring, compliance, or content strategies on AI content detection claims. Instead, build internal expertise to evaluate content authenticity and shift policies to mitigate risk from unreliable third-party tools.
AI Pilots and ROI: Flawed Failure Rate Statistics
In August 2025, MIT released a report claiming that 95% of enterprise AI pilots resulted in zero measurable ROI, catalyzing skepticism and pausing major initiatives across the business world.
A detailed review identifies several pitfalls:
Tiny, Selective Sample: The headline figure was based on only 52 interviews, not a large-scale quantitative analysis, with the report emphasizing that its findings were only “directionally accurate.”
Unrealistic Success Criteria: To qualify as a “success,” AI pilots were required to prove a profit-and-loss (P&L) statement impact within six months—a standard nearly impossible for any new enterprise software rollout, AI or otherwise.
Misclassification of Value: Processes that saved time, enhanced productivity, or prevented failed rollouts were still deemed “failures” if not immediately visible on P&L statements.
Promotional Motive: The research report ultimately funneled readers to consider Nanda—MIT’s own proprietary AI agent platform—as the solution to these high “failure rates.”
Alternative, credible benchmarks:
Boston Consulting Group found that 75% of employees see real business value from AI.
McKinsey reported 92% of companies planned increased AI investment.
97% of senior leaders in a broad study reported positive ROI from their AI investments.
Return-on-investment rates in broad, methodologically sound studies consistently fall in the 70-90% range, diametrically opposed to the MIT claim.
Strategic implication: Demand full data transparency and benchmarking in all AI ROI studies. Red-flag research that sets impractical success metrics or is released with sales funnels for proprietary solutions.
Best Practices for AI Research Evaluation
Leaders making AI procurement or implementation decisions must establish due diligence procedures for research validation:
Investigate Financial Incentives: Clarify who funded the study and what financial gain they may realize.
Examine Methodology: Assess whether sample size, scope, and success criteria align with the realities of enterprise IT rollouts.
Cross-Reference Independent Findings: Validate viral claims with third-party, unbiased industry analyses.
Avoiding reliance on misleading or agenda-driven studies is mission-critical as AI adoption accelerates. Requiring evidence-based insights and reading beyond the headline can prevent costly missteps, ensuring business value creation from sound technology strategies.
Topics Covered in This Episode:
- Three Viral AI Lies of 2025 Debunked
- Menlo Ventures Anthropic vs OpenAI Study Critique
- Anthropic Enterprise Adoption Market Share Myth
- Graphite's "57% AI Content" Internet Claim
- AI Content Detector Inaccuracy Exposed
- MIT 95% GenAI Failure Rate Study Audit
- AI Study Bias and Marketing Manipulation
- Best Practices for Evaluating AI Research
Episode Transcript
Jordan Wilson [00:00:15]:
You've been lied to about AI a lot in 2025. It's not your fault. There's so much money to be made in these money AI waters. Even reputable organizations have squatted to all time lows to basically lie to you and try to make a dollar. Why? Well, because with a single viral study, markets can move and billions of dollars can be made. So on today's show, I'm going to break down the three biggest AI lies you were told this year that you may have believed. And I'm gonna slap a few of these silly studies around with facts and stats and let you decide. Were these three organizations who created these viral AI studies telling you the truth or intentionally trying to mislead enterprise leaders in order to cash blank checks? Let's dig in and find out.
Jordan Wilson [00:01:09]:
I'm excited for today's episode. I hope you are too. Time for some hot take Tuesday y'all on everyday AI. What's going on? My name is Stuart Wilson. Welcome to Everyday AI. If you're new here, we do this every single day, Monday through Friday at least. It's a daily unscripted, unedited, live stream podcast, and free daily newsletter helping everyday business leaders like you and me make sense of all this AI because it's coming at us nonstop. And what can you even believe anymore? Alright.
Jordan Wilson [00:01:36]:
And I help you decipher it and grow your company and your career with the actual facts and stats that you need to know. So if that's what you're trying to do, awesome. It starts here. But to take it to the next level, make sure to go to our website at youreverydayai.com. Sign up for the free daily newsletter. We're gonna be recapping today's show, but also on that very website, you can go listen to for free hundreds of hours. Yeah. Six sixty plus episodes, from some of the brightest minds in AI interviews with people from all over the place, all on our website for free.
Jordan Wilson [00:02:09]:
All right. And also if you want today's AI news, make sure to check out today's newsletter. So, yeah, you've been told a lot of lies, about AI in 2025. And aside from the money factor, because literally billions of dollars can be made with a single AI study or a single, article that maybe is false or is cherry picking, certain facts to make a certain trillion dollar company look a certain way or to say, hey. This, AI startup's product is better than the other. It can literally move markets. It's not your fault though, because not only are the AI waters muddy because no one or I would say hardly anyone, is an expert in the generative AI era. Right? So unless you were an early, employee at in, you know, 2015 or 2014 at Google or OpenAI, hardly anyone has any real long standing expertise in AI.
Jordan Wilson [00:03:08]:
So when you see all these studies from what looks like reputable organizations, You just kinda blindly believe them, and so does the media and so do people on social media. Right? So, essentially, these, viral headlines just get trumpeted and, you know, regurgitated, and everyone just believes them. But it's not the truth. I think, unfortunately, AI has become very polarized. It's become binary. Right? It is, hit or miss. It is, doom and gloom or miracles, and there's often nothing in between. So on today's show, we're gonna audit the $100,000,000 conflict of interest behind the viral open AI is collapsing narrative.
Jordan Wilson [00:03:51]:
We're gonna deconstruct the faulty science claiming that 57% of the Internet is now AI slop and expose the hidden definitions that label successful innovation in AI as 95% failure. Yeah. You know which one that is MIT. Alright. So here are the three myths that we're to be tackling today. So myth number one, that anthropic has overtaken OpenAI in the enterprise. Yeah. That was a study from Menlo Ventures.
Jordan Wilson [00:04:19]:
Doesn't make sense. Myth number two, the dead Internet, right, is 57% of the web really AI generated? That was from graphite. Spoiler alert. There's no way to prove it. And myth number three, not that you needed to hear me talk about this again, but y'all, I still am seeing these posts on social media and even new articles referencing this all the time. The 95 failure rate, of gen AI, pilots, study from MIT. Right? So, yeah, if if, 95% of, AI pilots are failing, why are the smartest and biggest companies in the world, spending hundreds of billions of dollars? I don't know. So this, I think, and I think it's important to talk about this here, you know, as we are in December in the last month of the year because it's important to reflect on, the conversation, the overall conversation of AI and so many times.
Jordan Wilson [00:05:16]:
Right? It's it's, predicated or based around a couple of things. Right? Usually, new model releases benchmarks, that's one thing. And then the other is just the narrative. And like I said, it is binary. It is either, you know, AI is the best thing ever or AI is absolutely terrible. And usually, there's some weaponized studies that go either way. Right? And they all start with real data. The data gets, massaged and, cherry picked obviously, and, you know, try to put into the best light possible.
Jordan Wilson [00:05:47]:
So we're gonna break down, some of those, three biggest AI studies or research papers that have dominated a lot of the conversation in 2025, but they're wrong. Alright. So, I think it's important because we need to find the middle ground, right? In AI there's, there's, there's so few things that are binary, right? Even as myself, someone that talks about AI every single day, I don't think AI can do everything. Right? But I also know that AI is very capable. So I think it's important to have a balanced, look, but, you know, laced with straight hard facts, because it's it's every day. It's, hey, AI changes everything to, oh, AI is a scam. AI is a bubble. Right? And the truth is actually just much messier.
Jordan Wilson [00:06:34]:
It is nuanced. And, well, unfortunately, if you write a study that is, factual and you try to give it a headline that is balanced, it's not gonna get clicks. Right? It's not gonna drive the narrative. I'm a former journalist. Right? There's a there's a story, or or, a saying that if it bleeds, it leads. Right? You don't find the, the median, of your studies findings. You find the outlier and then you slap it with the sexiest headline possible. Right? That you're like, hey, this is gonna be the the headline that sells this newspaper.
Jordan Wilson [00:07:08]:
This is gonna be the headline that gets us clicks, that gets us ad revenue, that puts us as a thought leader in this space, right, that gives us sales. So, let's look at lie number one or, not truth number one. Right? We'll say that, Menlo Ventures, their report that claimed that Anthropic is beating OpenAI in the enterprise. Oh my gosh. People believe this. Smart people. Literally, I saw dozens of smart people that I once respected, you know, tweet this out or put this out on LinkedIn or write long medium articles about this. And I'm like, some of these articles were almost as long as this study, yet they just glossed over the fact that no one should even, like, look at this, almost as bad as the MIT study, which I'll obviously get to.
Jordan Wilson [00:07:59]:
But this is laughable, y'all. Anyways, Menlo Ventures, right, a venture fund, they in July, they had their midyear LLM market update, and it said Anthropic overtook OpenAI in enterprise market share. Right? So that, Anthropic now holds a 32% and OpenAI is 25%. And, you know, people started talking about this flippening. Right? Because before OpenAI was in the lead and now Anthropic's in the lead. Oh my gosh. And everyone just literally copy paste repeated this stat. I swear humans don't like to use their brains anymore.
Jordan Wilson [00:08:32]:
You know, it's it's it's the social media fication of our brains. Our brains have literally become, rot, because all all people care about, they they they look at a graph and a headline, and then they'll just repeat it. Use your brains. Alright? This this study is is garbage, and apologies to people in garbage. I have friends in garbage who are brilliant. So to call this study garbage is doing a disservice to those people. So let's look at this. The the the big, kind of graphic that caught everyone's attention.
Jordan Wilson [00:09:04]:
Right? It it's it's labeled here. So, livestream audience, you'll see this on my screen. Podcast audience, nothing super visual in today's show, but you can always go watch the video version on our website, youreverydayai.com. So, this graphic showed in 2023. Obviously, you know, the enterprise large language model API market share. You know? So, in 2023, Anthropic was at 50% or sorry. OpenAI was at, just over 50%, and Anthropic was, at, like, 12%. Right? And then in 2025, according to Menlo Ventures, it flipped.
Jordan Wilson [00:09:42]:
Right? So, what was the exact stat? It was, now Anthropic holds 32% versus OpenAI's 25%. So the headlines all read, Anthropic overtakes OpenAI in enterprise, but no one read it. Alright. Also, did no one care to bother to see like, oh, when did these companies launch? Right. OpenAI obviously launched in 2022, Anthropic in, mid twenty twenty three. So, yeah, of course, by default, in Tropic's market share is gonna be much smaller when their product wasn't even out the entire year. Anyways, facts would hurt our brains apparently. But here's the important part.
Jordan Wilson [00:10:22]:
This Menlo report is not neutral. It's promotional. We need to look at who Menlo Ventures is. Well, they're a venture fund, and they have invested billions billions, into, anthropic. Let me get the the exact math here. Let me fact check myself here in in real time. So okay. No.
Jordan Wilson [00:10:44]:
Sorry. Over, reportedly over $1,200,000,000 So billion plus. Alright? But regardless, Memo Ventures is a huge financial supporter in Anthropic. So they obviously have a lot to gain. And Memo Ventures has publicly declared that Anthropic is the single largest investment in the firm's history, and they led Anthropic's series C, D, and E funding round, which was billions of dollars. So this obviously is just an investor talking up their own book of business. And not only that, but Menlo and Anthropic actually co created the Anthology Fund specifically to fuel anthropic ecosystem adoption. So startups in this fund receive $25,000 in free anthropic credits and a $100,000 in AWS credits.
Jordan Wilson [00:11:36]:
And guess what? It's really easy to use on AWS and especially in 2023 and 2024. Oh, anthropic. So essentially they're paying companies in this anthology fund to adopt Claude and guess who they surveyed. You'll never guess. People in their fun, in their joint fun. Alright? Yeah. So, this entire claim came from a survey, a single survey where they talked to a 150 yeah. 150.
Jordan Wilson [00:12:08]:
150 technical decision makers, but they were pulled from Menlo's own venture capital network and portfolio ecosystem. So, yeah, it's it's selection bias so obvious. It's embarrassing that anyone even reported on this, or talked about it because this is clearly marketing. Alright. Bad marketing, by the way. But people fell for it. And here's a couple important details. Right? Aside from the the the fact, selection bias at its finest or at its worst, and the fact that it was a 150 people, well, it only looked at API usage.
Jordan Wilson [00:12:48]:
Right? It sounds super comprehensive, but it's not because it completely ignores Microsoft Copilot, which guess what? Up until, four months ago was built almost entirely on OpenAI's models. So they really overlooked the entire enterprise at the time when this study came out. You know, Microsoft Copilot had the largest slice of the enterprise, so they, ignore the fact that OpenAI's models power Microsoft Copilot. Obviously, they, use Anthropic in some instances now, and they just cherry picked API calls only. Oh, and the fact that, ChatGPT Enterprise is one of the, fastest growing software products ever, and they have more than a million business user users, which Anthropic obviously does not. So, again, they rigged the result of this before it was even produced. So let's just go ahead and say how this is disguised as market research, but it's just marketing. So Menlo invests, more than a billion dollars in anthropic.
Jordan Wilson [00:13:52]:
It then creates a $100,000,000 fund to accelerate cloud adoption. They give startups a bunch of money to use cloud, and then they survey those same people, and then they announce, oh, Claude is winning. Come on. This is trying to make the market not measure it. Alright. And talk about revenue. Right? OpenAI has more revenue than Anthropic. They have more customers.
Jordan Wilson [00:14:15]:
They have more, users. So, no, there is no flippening. No. Anthropic did not overtake OpenAI, in any measurable, overall scope of enterprise usage. It's not even close. So, just FYI, if you believe this, if you saw this, again, it's not your fault. But if you saw anyone talking about this, go ahead, unfollow them, delete them. Right? Stop.
Jordan Wilson [00:14:38]:
You need to stop paying attention to people that don't want to read and don't want to understand facts, because independent data and anyone with a brain, knows that OpenAI is still leading overall enterprise adoption. So enterprise from, sorry, data from ramp, from October showed that 93% of organizations who have a vendor in the GenAI category use OpenAI, and that OpenAI maintains the highest adoption among enterprise companies at 95%. So the truth is surveying a 150 people from your own network is has absolutely nothing to do, with actual real enterprise AI usage. Right? When billions of people use generative AI, you don't talk to a 150 people in your backyard who you know how they're gonna answer because you're paying them to use that product. Menlo's just, promoting their fund, and not conducting real research. Alright. Line number two. Here's one I don't think I've actually talked about a lot on this show.
Jordan Wilson [00:15:37]:
The graphite study that claimed 57% of the Internet is now AI slop. Alright. So this was a study in October that analyzed 65,000 URLs, via the Common Crawl database, and they claimed Graphite claimed, that over half of these new articles are AI generated. So these were, you know, quote unquote new articles from common call. And then the headlines all said that the internet is now slop or, you know, data internet theory is officially here and there's more, AI content on the web, but then human written content. So let me just say this first. I don't necessarily disagree, right with the the, underlying premise here, from the graphite study. Right? I'm someone that's been, a writer my whole life.
Jordan Wilson [00:16:27]:
Right? I not my whole life, but I've been getting paid to write for twenty four years. Right? So, wow. Saying that out loud is, that hurts to say. Right. But I've been getting paid to write for almost a quarter century. So I know how much the Internet has shifted to just becoming a lot of slop now. And even before, Chat GPT, there was plenty of generative AI, content going back to 2020. And then before that, you know, there's spintax and all these other, you know, just there's pre AI slop and now it's AI slop.
Jordan Wilson [00:17:01]:
So don't get me wrong. I agree with the premise that probably, more than half of what's written now is AI slop. However, the methodology is garbage. There's no way to decide this, right? So, here we had this, this graphic that kind of went viral that showed, over time, right, since, the, early twenty twenty, the percentage of human written content was, you know, near a 100. AI written content was, you know, in the single digit percentage. And then now, today, AI content has overtaken, or surpassed human content. So here's the problem. They used Surfer SEO's AI content detector to classify these 65,000 articles.
Jordan Wilson [00:17:50]:
And if the detector flagged more than 50% of an article as AI, they label the whole thing as AI generated. So this entire half the Internet AI claim depends on one commercial detector being accurate. Guess what? It's not. No, but this isn't anything against Surfer, right? Surfer AI. It's actually a fantastic product. I've been using it for many, many years before I even started everyday AI. So Surfer has a great product, but any company that has an AI detection, it it it there's no such thing. There's literally no such thing as, AI content detectors for text.
Jordan Wilson [00:18:26]:
Right? Because there's so many different things that you can do, with AI content. Right? You can put it in a quote unquote humanizer. Right? No. I like, I'm a dork. So before ChatGPT came out, I literally every single AI content detector at the time, because before chat g p t, they had their g p t technology, to dozens of different companies. And so pre chat g p t, there were even AI content text detectors. And at the time, I spent, I don't know, thirty hours, going through and looking at human written articles versus AI written articles and putting them through these detectors. Yeah.
Jordan Wilson [00:19:02]:
They're they're garbage. You know, one AI content detector, could say, hey, this piece is 99% AI and another one could say it's 1%. Alright. They, all these AI content detectors do is they measure statistical patterns in word choice, not actual authorship. So it's a pseudo science, not in actual science, and the error rates are obviously massive. So here's an example. Right before, you know, starting the show, I just grabbed, I don't know, first part of the Bible. Right? Genesis from the Bible.
Jordan Wilson [00:19:37]:
Copy and pasted, some of that. And, yeah, 86%. Right? 86% AI generated. So yeah. Hey, biblical writers. You should have used your brain, in divine intervention and not God GP. Alright? Yes. Yeah.
Jordan Wilson [00:19:58]:
These AI content detectors are all garbage. There's no such thing. Right? Even right. You like, you can make an argument like, oh, anyone that includes the, you know, follow-up response from chat g p t, that's a telltale sign. Is it? Maybe. Yeah. But I could write that as a human to try to throw you off, couldn't I? Yeah. I could.
Jordan Wilson [00:20:17]:
So, a lot of people point back to the, time when even OpenAI had their own AI content detector. And this, unfortunately, I wish OpenAI never did this. And I said this at the time before OpenAI ever took it down. I'm like, this is dumb. It does not work. OpenAI needs to take this out. They eventually did. And the reason why, well, it's because it only got it right 26% of the time, worse than a coin flip.
Jordan Wilson [00:20:42]:
So detectors cannot detect. So if the company that built chat GPT can't detect AI text, why should we trust any content detector? Well, we shouldn't. And it's actually bad. AI content detectors are bad because they target, unfortunately, non native English speakers. So detectors flag simple structured sentences as robotic because that's how non native speakers write. So a Stanford study found that 61% of ESL or English as a second language student essays were falsely flagged as AI generated. And a Vanderbilt, university actually disabled Turnitin's AI detector to avoid hundreds of false accusations per year. Hey.
Jordan Wilson [00:21:25]:
If you are working at a college university listening to this and you are still using one of these in the year 2025, stop. Tell someone. Right? I will personally get on the phone with your president of the university and tell them that this is stupid. Right? I I I won't even use, better words than that. I will say this is dumb. You are robbing, you know, whether it's for entry essays, homework, whatever. Yes. I know that it's a problem.
Jordan Wilson [00:21:53]:
Right? Students are just copying and pasting everything from ChatGPT. They're not writing a single word, probably. A single word of anything that they're turning in, especially if it's a hybrid or an online class where you just turn in a paper, they're not writing it. AI is, but you can't use these, AI content detectors. It's not a thing. So here's the truth about line number two. Well, there's no concrete way to prove text is AI generated. Yes.
Jordan Wilson [00:22:16]:
On the video visual side, there's synth ID. There's all of these things, but as quickly as any, embedded watermark in whether we're talking about in text, in video, in photos, etcetera, as soon as anything pops up, there's already minutes or hours later. There's already multiple, you know, watermark removers. Right? I remember when Sora came out or Sora two came out, and it had these watermarks. Literally that same night, there is already multiple services that had watermark removers. Text is very easy, to manipulate, change things. Right. But again, there there's no embeddable, you know, invisible watermark in text like there is in video and photos.
Jordan Wilson [00:23:01]:
Alright. Line number three. I'm not gonna go long on this one because I've already given it plenty of airtime, but the MIT NANDA study that claims ninety five percent of AI pilots deliver zero ROI. Let's be honest, many, many, many people fell for this one, and you shouldn't have. So this was an August 2025 report from MIT. Yes. That MIT that stooped so low, they would have won a limbo contest. Claim that only 5% of AI pilots reach production with measurable business impact.
Jordan Wilson [00:23:36]:
Yeah. We don't, even when MIT comes out with a study now, we just don't share it in our newsletter because I don't view MIT as a credible source of AI information anymore, which is crazy to say. And that statistic obviously went mega viral, right? When you can slap something on that says ninety five percent of AI pilots fail, you know, and then you get into this whole, you know, AI is a scam, or AI is a miracle. This binary, nothing in between. Right? It went mega viral, and this actually spooked markets. Yes. Markets moved billions of dollars because of this piece of marketing masquerading as a study. And this became the go to proof for skeptics claiming enterprise AI is a bubble that's about to burst.
Jordan Wilson [00:24:22]:
FYI for the all the people, you you know, cold emailing, trying to get a guest on the show, or just trying to sell me some AI product, if you lead with the MIT 95 study, I don't read it. Right? Because I I I don't know. To me, that just means that, you need to do some some reading and some thinking. So, yeah, it was called the state of AI in business 2025. Right? And and here was the telltale sign this thing was a dumpster fire before it got there. Again, sorry to my people in the trash fields. You couldn't get this report. You had to fill out an application just to see the report.
Jordan Wilson [00:24:58]:
Yeah. There was a Google form. I didn't get it, but plenty of other people got it and then shared it online. And it was an absolute disaster. But when it came out, it was completely viral. And part of the reason, well, it was the media and social media. Right? And that's why this show, the whole point is like, you gotta be careful what you read and what you believe, because all of these articles that came out, I remember being an an overworked journalist. I I I was a journalist, for, I don't know, full time for, like, seven years.
Jordan Wilson [00:25:28]:
It's hard. Right? You're overworked. You're underpaid. You're always trying to not get scooped by someone else. You gotta get the story out faster, faster, faster. Right? So So everyone just essentially copy and paste the same headline. Right? 95% of AI pilots have no return. So if if you want my full, hot take on this one, go listen to episode five ninety seven because I'm gonna wrap this one up quick.
Jordan Wilson [00:25:52]:
Alright. So that 95% failure from MIT came from just 52 interviews. Oh my gosh. We thought the, we thought the one from, Menlo was bad with a 150. This is 52 people. So that's 52 conversations. Alright? And this is clickbait dressed up as academic research. And here's the thing.
Jordan Wilson [00:26:13]:
This was just a vibe study because even those 52 conversations, those were just directionally accurate. So in the paper that hardly no one read in one of the little footnotes there, it talked about the research limitations and it said these figures are directionally accurate. All right. Based on individual interviews rather than official company reporting. Sample sizes vary by category. How many you have categories when you only talk to 52 people? Sorry. What? There's like three people in each category, and success definitions may vary across organizations. In other words, this was a vibe study.
Jordan Wilson [00:26:49]:
It was directional accuracy only. It wasn't even a true, like, quantifiable study. These were conversations, and then, you know, someone over there is like, yeah. Seems like they've this this failed. Right? Yeah. Seems like it. No. This is garbage.
Jordan Wilson [00:27:07]:
I I need to get a better word. What's a better word than one word calling something garbage? I don't know. Rubbish, but that's just another word for garbage. Dumpster fire garbage. Why is everything just say something's bad garbage? I need to, expand on that in 2026. Alright. So, well, one of the reasons why. Well, because MIT had a preconceived definition of success that they knew would come out with nothing but failures.
Jordan Wilson [00:27:30]:
And don't worry, it gets better because they turn it into an infomercial because only they can help you. Just wait. So success for their study, for their criteria required measurable p and l. Yes. Actual measurable, movements on a profit and loss statement within six months of deploying a jet AI pilot. My gosh, there's no such thing. Like nothing, no major it, no major it, pilot has success in, six months on a p and l statement. Are you serious? Come on.
Jordan Wilson [00:28:07]:
Yeah. Because most, enterprise transformations need at least twelve to eighteen months to even show financial results, even ones that are obvious. Right? Not even AI. Like, hey, let's, I don't know, get our people computers. Right? Takes years. So by definition, any new enterprise software initiative would fail by design out of the box. Nothing they can do about it. That's not the worst part.
Jordan Wilson [00:28:30]:
So the pilot, successfully prevented a bad deployment would get counted as a failure. So, the study just completely misclassified what a failure actually meant. So productivity improvements that don't immediately hit the P and L failure. Right? I don't know. If something if you can now get something done in half the time, but you can't prove that on the p and l failure. Right? So, early stage products that needed more time to mature counted as a failure even if they were showing signs of success, saving time, increasing revenue didn't matter. But here's the thing. Here's the infomercial at the end.
Jordan Wilson [00:29:04]:
And, like I see, am I, MIT stooped to an all time low on this one. And when I read this, I kid you not, I literally put my face in my palm when I read this, because MIT was literally trying to put themselves with their Nanda product, which stands for networked agents and decentralized AI. Right? So essentially, they're building new AI agent protocols. So they publish that 95% of current AI fails, to essentially position their product as a solution. Yeah. Marketing again. So Nanda at the MIT Media Lab, if you didn't know because no one knows, because no one's heard of MIT's Sanda. Right? Even me who covers AI every day, I had barely heard of it by the time this study came out.
Jordan Wilson [00:29:49]:
I had read about it, like, once. Right? But, they offer that solution to businesses for about $250,000. Yeah. Covering their expertise tools and possible bespoke implementation. Yeah. So, don't worry. Literally MIT in this report said one of the reasons is, well, we need to start thinking agentic. And they put themselves in this Nanda product, you you know, in the same paragraph as some of the biggest names in agentic AI as if, like, you know, it's it's it's almost like, you know, how the bottom shelf is, like, the cheap generic stuff, and then the top shelf is the nice stuff.
Jordan Wilson [00:30:30]:
It's like they took something from the below the bottom shelf, and they snuck it on the top shelf thinking no one would notice. Yeah. We noticed. Like, you you sorry. MIT NANA is not in the same conversation as real, enterprise agentic AI. Right? It's not. So classic marketing funnel. Step one, publish an alarming study claiming that current AI has a ninety five percent failure rate.
Jordan Wilson [00:30:54]:
Step two, you identify the learning gaps in static tools. Right? They're saying that's the reason why AI is, failing. It's because these static tools. And then, oh, don't worry. Step three, that's our solution, Nanda, because it's a learning capable system in new protocols. That's the solution, and you gotta buy it. So, obviously, other studies looking at AI and ROI tell a completely different story. I did this on the, previous show, but, yeah, real studies done by real companies that are unbiased, mostly unbiased.
Jordan Wilson [00:31:27]:
Right? But with tons of people show something completely different. So the Boston Consulting Group, AI at Work, surveyed 10,000 plus employees, and they said that 75% of employees see value in momentum in AI. Study from McKinsey and Company, 1,300 participants, said that 92% of companies plan to increase their investment in generative AI. If you want just strictly ROI, well, the study, of 500 senior US leaders, but an actual study, not just a vibe conversation. 97% of senior leaders who invested in AI reports experiencing positive ROI, and then a snowflake in ESG report of a thousand business and IT leaders said that 92% of early adopters are seeing a positive return on investment of AI. So, yeah, any legitimate, study with sound methodology, obviously, shows a positive return on investment of AI. Right? So here's the truth about line number three, it's clickbait. Right? From a lab looking for customers.
Jordan Wilson [00:32:34]:
Right? Looking to make a name for themselves. You know, 52 interviews with a six month P and L requirement is not rigorous research methodology. It's a joke. No one, which is again which is why I know half the people half of the media people that, wrote about this study did not read the whole thing. Right. They probably read the synopsis, the first page overview, the last page overview, maybe ran it through an AI and said bullet point this. Right? Because if you have a brain and you read that and you're saying, oh, six month has to show success on a p and l. And, I did this in in the dedicated episode.
Jordan Wilson [00:33:10]:
Even their selection their selection bias was ridiculous. Right? But 52 interviews is what this is based on directional accuracy. Right? It's like I talked to you for a little bit and at the end, I'm like, yeah, I decided that your, you know, your AI probably doesn't meet our criteria. Next 52 y'all. I already said this. I can go on a parking lot at a conference, or I can go anywhere in an hour and come up with a, better study with, more sound methodology, than this piece of marketing. Yeah. Because studies with 20 times, 50 times larger samples show, at least seventy, eighty, 90%, are showed seeing a positive ROI versus if you look at the MIT study, it's 5%.
Jordan Wilson [00:33:54]:
Right? So, here's the pattern right across these three AI lies that you've been told. So, Menlo used 150 people from their own paid ecosystem while being Anthropic's largest investor ever to say, Anthropic has overtaken OpenAI. Graphite used pseudo scientific detection tools that the FTC has literally sued companies over and MIT used 52 interviews with an impossible success bar while building a competing technology to sell as the solution for the ninety five percent failure rate. Do you see the trend here? Yeah. Bad methodology, bad studies disguised as marketing. So here's what I want you to do in 2026 and beyond. Number one, read. Number two, unfollow anyone that's been sharing these, AI lies.
Jordan Wilson [00:34:49]:
But I want you to ask yourself these three questions. Any viral study. Alright. Number one, who funded the research and what do they financially gain from the conclusion? In all three of these cases, right? Graphite, a lot of people don't know. They essentially sell SEO. Right. So they come out with this study and, oh my gosh, you know, Google hates, AI. Right.
Jordan Wilson [00:35:10]:
Because there are some other stats that said, you know, 80 some percent of, you know, what ranks on Google is is human written. So it's like, oh, you know, you need our SEO services. Right. But all three of these companies had a very direct, financial gain from cherry picking bad stats and selling it to you. So number one, who funded this research and what do they financially gain from the conclusion? Number two, does the study have sound methodology that an unbiased person with a brain would agree with? Alright. Gosh, it's it's so sad that the the bar is this low, but it is. This is the truth. And then number three, what are similar studies from unbiased third parties saying about the same subject? Right.
Jordan Wilson [00:35:53]:
Because yeah, if you look at, studies that have sound methodology, that talk to a lot of people, and if you look at multiple studies, you will start to see the trend. And the trend is, well, it's undeniable. At least when, you know, ending on study number three, AI adoption is here, and you need to be paying attention to it. And you also need to be paying attention where you get your information from. Alright? So I hope today's show was helpful and, well, I hope you continue, to get your information from everyday AI the rest of 2025 and 2026 and beyond. We spent a lot of time, trying to cut through the nonsense, the BS, the marketing, the hype, and cutting it to you straight. So I hope this is helpful, but I think this episode was important because I got so many messages, emails about these types of studies all the time. And I'm sorry.
Jordan Wilson [00:36:42]:
I suck at emails. I, I, I don't, I can't get to them all right without having an AI answer. And I don't want that. But let me tell you this. I feel bad because so many well intentioned business leaders saw some of these studies and it probably derailed either some of their AI progress or for those that were unfortunately still kind of sitting on the entire AI implementation fence in 2025, this caused them to sit on it for even longer. You cannot believe these types of studies because they're not real. Right? You need to use your own brain. You need to read, right? Literally I talk to people who made huge multimillion dollar AI decisions on some of these studies.
Jordan Wilson [00:37:33]:
And clearly they did not read it. Right? They did not read past the headline or the one cherry picked graphic. We all need to do better. All right. AI is is here to stay. It's driving the conversation, whether you want it to or not. Alright. So please in 2026, let's all do better.
Jordan Wilson [00:37:52]:
Not everything is binary. Let's actually read. Let's use our brains. Let's put the human, keep the human in AI. Right? Alright. So I hope this is helpful. If so, please go to youreverydayai.com. If you could share this show, repost this because I think more people need to hear this, and then go to our website at youreverydayai.com.
Jordan Wilson [00:38:11]:
Sign up for the free daily newsletter. Thanks for tuning in. See you back tomorrow and everyday for more everyday AI. Thanks, y'all.