Comment by tim333
6 hours ago
The author is a bit of a stopped clock that who has been saying deep learning is hitting a wall for years and I guess one day may be proved right?
He probably makes quite good money as the go to guy for saying AI is rubbish? https://champions-speakers.co.uk/speaker-agent/gary-marcus
a contrarian needs to keep spruiking the point, because if he relents, he loses the core audience that listened to him. That's why it's also the same with those who keep predicting market crashes etc.
Well the same can be said about non contrarians ...
Well..... tbf. Each approach has hit a wall. It's just that we change things a bit and move around that wall?
But that's certainly not a nuanced / trustworthy analysis of things unless you're a top tier researcher.
Indeed. A mouse that runs through a maze may be right to say that it is constantly hitting a wall, yet it makes constant progress.
An example is citing Mr Sutskever's interview this way:
> in my 2022 “Deep learning is hitting a wall” evaluation of LLMs, which explicitly argued that the Kaplan scaling laws would eventually reach a point of diminishing returns (as Sutskever just did)
which is misleading, since Sutskever said it didn't hit a wall in 2022[0]:
> Up until 2020, from 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling
The larger point that Mr Marcus makes, though, is that the maze has no exit.
> there are many reasons to doubt that LLMs will ever deliver the rewards that many people expected.
That is something that most scientists disagree with. In fact the ongoing progress on LLMs has already accumulated tremendous utility which may already justify the investment.
[0]: https://garymarcus.substack.com/p/a-trillion-dollars-is-a-te...
I like how when you click the "key achievements" tab on this site it just says
> 1997 - Professor of Psychology and Neural Science
I thought the point though was that Sutskever is saying it too.