Comment by fidotron
9 hours ago
But humans have been confusing pattern matching against known solutions for intelligence for a hundred years!
Seriously though, it ends up looking like that. To take a stupid example a couple of weeks ago I asked an agent to look at porting my hand written WebGL renderer (+ shaders etc) to WebGPU. It estimated a human would take 6-10 weeks, and I would agree. (Which is why I hadn't done it). 24 hours later it was deployed and live. This is classic tedious, difficult, low level if quasi mechanical work (rather like cracking an enigma message), and LLMs absolutely fly through it.
Time estimations by LLMs are hilariously incorrect all the time. It estimates very simple things that a human could do in an hour to take days or weeks and other things that are genuinely tedious and time-consuming it estimates taking a few hours. LLMs have no understanding of time and no world model that even allows them to make correct time estimates. They will always fail to provide decent time estimates unless the task is well-known, in the training data and they can extrapolate that with a simple math script.
My favourite story is of an elderly coworker with no AI experience who innocuously asked GPT-5-mini (at the time his framework's default model) to translate a 50 page engineering spec, and it kept him at arm's length for about a week about how that task would take just another 24-36 hours more. He kept asking like, you finished yet? and it just made up excuses, "so sorry, I got distracted, give me one more day", and he went "please finish, I need this", and 5-mini came back "I totally understand -- let me get to work immediately, I'll report back ASAP", end of conversation. He got annoyed but never even suspected anything wrong, because this is the kind of conversation he's used to. I wonder how long this would have continued, if I hadn't intervened by chance.
> It estimated a human would take 6-10 weeks
You do understand this is intentionally trained into recent models for marketing purposes? "Wow, it saved me months of work in a day! This is the most amazing technology ever!!!!"... is what it intends to evoke by underpromising and overdelivering. I routinely have it helpfully suggest it will take something like "three engineer-months" to do something I do by hand without any LLM assistance in a day. The estimates may be accurate if you have literally never touched a computer in your life before and are starting to learn from there.
In the games industry I was tech lead of teams of hundreds of devs and had to deal with their estimates of this sort on a daily basis. 6-10 weeks for a total renderer rewrite is on the low end.
As an indie dev who built their own WASM-capable engine that I've shipped in real games, I've built both WebGL and WebGPU renderers from scratch myself in significantly less time than that. Sounds like typical corporate dysfunctionality. If you have hundreds of devs you're going to get bogged down by having a share who spend 90% of their time at the company on Reddit, another share who write actively bad code, and then maybe 10% of the employees who have a clue what they're doing dealing with the overhead of communication, meetings, other people not upholding their assigned responsibility, etc. slowing them down 10x what they could actually do.
3 replies →