Comment by tim333
2 months ago
The author is a bit of a stopped clock that who has been saying deep learning is hitting a wall for years and I guess one day may be proved right?
He probably makes quite good money as the go to guy for saying AI is rubbish? https://champions-speakers.co.uk/speaker-agent/gary-marcus
Well..... tbf. Each approach has hit a wall. It's just that we change things a bit and move around that wall?
But that's certainly not a nuanced / trustworthy analysis of things unless you're a top tier researcher.
> expert in human language development and cognitive neuroscience, Gary is a futurist able to accurately predict the challenges and limitations of contemporary AI
I'm struggling to reconcile how these connect and he has been installed as Head of AI at Uber. Reeks of being a huckster
I didn't know the Uber bit, but googling:
>...held the position briefly after Uber acquired his company, Geometric Intelligence, in late 2016. However, Marcus stepped down from the directorship in March 2017,
which maybe fits your hypothesis.
Indeed. A mouse that runs through a maze may be right to say that it is constantly hitting a wall, yet it makes constant progress.
An example is citing Mr Sutskever's interview this way:
> in my 2022 “Deep learning is hitting a wall” evaluation of LLMs, which explicitly argued that the Kaplan scaling laws would eventually reach a point of diminishing returns (as Sutskever just did)
which is misleading, since Sutskever said it didn't hit a wall in 2022[0]:
> Up until 2020, from 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling
The larger point that Mr Marcus makes, though, is that the maze has no exit.
> there are many reasons to doubt that LLMs will ever deliver the rewards that many people expected.
That is something that most scientists disagree with. In fact the ongoing progress on LLMs has already accumulated tremendous utility which may already justify the investment.
[0]: https://garymarcus.substack.com/p/a-trillion-dollars-is-a-te...
I thought the point though was that Sutskever is saying it too.
a contrarian needs to keep spruiking the point, because if he relents, he loses the core audience that listened to him. That's why it's also the same with those who keep predicting market crashes etc.
Well the same can be said about non contrarians ...
The same can be said about hucksters of all stripes, yes.
But maybe not contrarians/non-contrarians? They are just the agree/disagree commentators. And much of the most valuable commentary is nuanced with support for and against their own position. But generally for.
I like how when you click the "key achievements" tab on this site it just says
> 1997 - Professor of Psychology and Neural Science