Comment by irjustin
6 days ago
FWIW, these studies are too early. Large orgs have very sensitive data privacy considerations and they're only right now going through the evaluation cycles.
Case in point, this past week, I learned Deloitte only recently gave the approval in picking Gemini as their AI platform. Rollout hasn't even begun yet which you can imagine is going to take a while.
To say "AI is failing to deliver" because only 4% efficiency increase is a pre-mature conclusion.
> Rollout hasn't even begun yet which you can
If rollout at Deloitte has not yet begun... How on earth did this clusterfuck [0] happen?
> Deloitte’s member firm in Australia will pay the government a partial refund for a $290,000 report that contained alleged AI-generated errors, including references to non-existent academic research papers and a fabricated quote from a federal court judgment.
[0] https://fortune.com/2025/10/07/deloitte-ai-australia-governm...
Because even if an organisation hasn't rolled out generative AI tools and policies centrally yet, individuals might just use their personal plans anyway (potentially in violation with their contract)? I believe that's called "shadow AI".
Correct. Where I work we are only "allowed" to use AI since December 2025.
But obviously people were copy/pasting content to ChatGPT and Claude long before that.
1 reply →
Its a $400+k report to a government, where the references either weren't audited, or only audited by the AI system that regurgitated them.
That requires more than a single person's involvement.
Haven't even read the source, but I like how it's "a partial refund". The chutzpah to deliver absolute nonsense[0] and then give a partial refund!
[0]: If it contains references to nonexistent papers and fabricated quotes, the conclusions of the report are highly doubtful at best.
290 out of 440 isn't that bad, when Deloitte claim the AI stuff is only secondary supporting claims and had zero effect, and they did hand over a corrected report (with real references) without asking for more.
... And they did also get blacklisted for the next report.
Exactly, my company started carefully dipping their toes in to org wide AI mid last year (IT has been experimenting earlier than that, but under pretty strict guidelines from infosec). There is so much compliance and data privacy considerations involved.
And for the record I think they are absolutely right to be cautious, a mistake in my industry can be disastrous so a considered approach to integrating this stuff is absolutely warranted. Most established companies outside of tech really can’t have the “move fast break things” mindset.
I'm not sure this is even measuring LLMs in the first place! They say the definition is "big data analytics and AI".
Is putting Google Analytics onto your website and pulling a report 'big data analytics'...?
Meanwhile, "shadow" AI use is around 90%. And if you guess IT would lead the pack on that, you are wrong. It's actually sales and hr that are the most avid unsactioned AI tool users.
Agreed. We've been on the agentic coding roller coaster for only about 9-10 months. It only got properly usable on larger repositories around 3-4 months ago. There are a lot of early adopters, grass roots adoption, etc. But it's really still very early days. Most large companies are still running exactly like they always have. Many smaller companies are worse and years/decades behind on modernizing their operations.
We sell SAAS software to SMEs in Germany. Forget AI, these guys are stuck in the last century when it comes to software. A lot of paper based processes. Cloud is mainly something that comes up in weather predictions for them. These companies don't have budget for a lot of things. The notion that they'll overnight switch to being AI driven companies is arguably more than a bit naive. It indicates a lack of understanding of how the real world works.
There are a lot of highly specialized niche companies that manufacture things that are part of very complex supply chains. The transition will take decades, not months/weeks. They run on demand for products they specialize in making. Their revenue is driven by demand for that stuff and their ability to make and ship it. There are a lot of aspects about how they operate that are definitely not optimal and could be optimized. And AI provides plenty of additional potential to do something about it. But it's not like they were short of opportunities to do so. It takes more than shiny new tools for these companies to move. Change is invasive and disruptive for these companies. And costly. They take the slow and careful perspective to change.
There's a clean split between people that are AI clued in and people working in these companies. The Venn diagram has almost no overlap. It's a huge business opportunity for people that are clued in: a rapidly growing amount of people mainly active in software development. Helping the people on the other side of the diagram is what they'll be mostly doing going forward. There's going to be a huge demand for building AI based stuff for these people. It's not a zero sum game, the amount of new work will dwarf the amount of lost work.
Some of that change is going to be painful. We all have to rethink what we do and re-align our plans in life around that. I'm a programmer. Or I was one until recently. Now I'm a software builder. I still cause software to come into existence. A lot of software actually. But I'm not artisanally coding most of it anymore.
Looking at the study, +4% is what they get when they chose to adopt AI, not overall.
I think people want to read how AI is not working , so those are the articles that are going to get traction.
Personally, I don't think the current frontier models would help the company I work for all that much. The company exists because of the skill in networking and human friendships. The company exist in spite of technological incompetence.
At some level of ability though, a threshold will be reached and a competitor will eat our lunch whole by building a new business around this future model.
It is not going to be a % more productive than our business. It is like the opposite of 0 to 1. The company I work for will go from 1 to zero really quick because we simply won't be able to compete on anything besides those network ties. Those ties will break fast if every other dimension of the business is not even competitive and really in a different category.
Yes I was recently talking to a person who was working as a BA who specializes in corporate AI adoption- they didn’t realize you could post screenshots to ChatGPT
These are not the openclaw folks
What does it even mean to specialise in something and know so little about it? What exactly is this BA person doing?
Genuinely confused, I don't get it
The “corporate” in “corporate AI” can mean tons of work building metrics decks, collecting pain points from users, negotiating with vendors…none of which requires you to understand the actual tool capabilities. For a big company with enough of a push behind it, that’s probably a whole team, none of whom know what they are actually promoting very well.
It’s good money if you can live with yourself, and a mortgage and tuitions make it easy to ignore what you are becoming. I lived that for a few years and then jumped off that train.
1 reply →
What do you mean? Deloitte has been all in on Microsoft AI offerings for quite some time, people have access to a lot of AI tools through MS.
Did they communicate this from the top or just turn a blind eye to it?
They had official trainings on how to use Copilot/ChatGPT and some other tools, security and safety trainings and so on, this is not some people deciding to use whatever feature was there from Ms by default.
As a counter-point, someone from SAP in Walldorf told me they have access to all models by all companies to their choosing, at a more or less unlimited rate. Don't quote me on that, though, maybe I misunderstood him, it was in a private conversation. Anyway, it sounded like they're using AI heavily.
Yeah. We are only just beginning to get the most out of the internet, and the WWW was invented almost 40 years ago - other parts of it even earlier. Adoption takes time, not to speak of the fact that the technology itself is still developing quickly and might see more and more use cases when it gets better.
> We are only just beginning to get the most out of the internet
The Internet has been getting worse pretty steadily for 20 years now
Its worse but we are more dependent on it than ever.
> We are only just beginning to get the most out of the internet
"The Internet" is completely dead. Both as an idea and as a practical implementation.
No, Google/Meta/Netflix is not the "world wide web", they're a new iteration of AOL and CompuServe.
OpenAI is buying up like half of the RAM production in the world, presumably on the basis of how great the productivity boost is, so from that perspective this doesn't seem any more premature than the OpenAI scaling plan. And the OpenAI scaling plan is like all the growth in the US economy...
4% isn’t failure! A 4% increase in global GDP would be a big deal (more than what we get in a whole year of progress); and AI adoptionis only just getting started.