Comment by xtreak29
10 hours ago
https://xcancel.com/JeffDean/status/2085034604172603724
> Announcing Discovery Loop!
> I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.
I think their move makes a lot of sense. Frontier models can probably not advance much further with the datasets we have currently available. Most of the text that goes in is online chatter, images and some scientific texts. That's great for chatbots, knowledge retrieval and programming. But with that database genuine discovery is hard to do. I think for the next step in intelligence the models need to have access to much more data: Data from physics, chemistry, biology experiments - and so on. And they need the data in much higher fidelity than you can currently access. If we just feed AI with all the knowledge we've acquired it's much harder for it to become smarter than us - it basically needs it's own eyes, ears, nose and so on.
> Data from physics, chemistry, biology experiments
There are PetaBytes of important scientific data locked in archival file formats. The first step is to make this efficiently readable.
https://news.ycombinator.com/item?id=46659254
> Frontier models can probably not advance much further with the datasets we have currently available
I’ve been reading this sentiment on HN since GPT4o, yet models got better and better
There’s a difference between things getting incrementally better and a step function.
LLMs will obviously keep getting incrementally better, but in order to get the kind on jump that LLMs themselves were, the sentiment is that we need something more.
1 reply →
That’s because a significant portion of the work and spending going on at frontier labs is generating new, more curated data, in select domains like software engineering and now biology[1].
[1] https://www.synbiobeta.com/read/anthropic-is-hiring-biologis...
2 replies →
It's quite interesting that those events are happening at the same moment. I wonder whether they are related
I think there are multiple separate events here, timed to coincide in order to minimise disruption. It's hard to be sure whether they share overlapping causes, but I'd guess that there's at least an element of that.
Yes. In public relations you never want to trickle out bad/concerning the news, you want to "rip off the band-aid and preferably follow it up with some positive/distracting news like 3.5 Pro model release or price drops.
Isn't it "just" a matter of Jeff Dean (& co) leaving, Hassabis replacing Dean, and Kavukcuoglu replacing Hassabis ?
A massive shake up for sure, but why do you see it as separate events merely timed to coincide ?
Some do-nothing EVP probably just got control over the Gemini org
Gemini had come under Hassabis, being co-led by Shazeer and Vinyals, who are now both gone. Gemini will now come under Koray Kavukcuoglu, reporting direct to Pichai as SVP of DeepMind.
Big shake up for Gemini it seems.
Is it _that_ Jeff Dean?
https://github.com/LRitzdorf/TheJeffDeanFacts
Fun fact: the Jeff Dean Facts was started by Kenton Varda (kentonv around here [1]), as a joke inside Google. He has expressed regret that it eclipsed Sanjay Ghemawat, with whom Jeff Dean basically did pair-programming with.
[1] also known for open-sourcing ProtoBufs v2, creating CapnProto and Sandstorm, and commenting on the recent Cloudflare OS post https://news.ycombinator.com/item?id=49182996
> When Jeff Dean goes on vacation, production services across Google mysteriously stop working within a few days. This is actually true. (TRUE)
Uh oh...
Basically, he had a server that did code indexing running on his workstation. If he didn't login and run a command to refresh his prod credentials every day, the server would lose its ability to talk to prod (as it should) and fail. Much of the company depended on that service. Eventually it was moved to prod.
That's a pretty poor reflection on him if true.
I've noticed that Gemini has been timing out yesterday & today !
Yes, him.
"Once, in early 2002, when the index servers went down, Jeff Dean answered user queries manually for two hours. Evals showed a quality improvement of 5 points."
ahahah golden.
Will be following their journey
Is the subtext that Demis was trying to block what Dean etc wanted to do and so they had to leave Google in order to do it?
Is Jeff also leaving Google?
I am sure VCs are fighting to invest and they will get 4B investment immediately with this team
They are leaving Google, but Google is an investor in the new company. So a split, but not exactly a divorce. Or something.
A "Maria's not an asset to the abbey" situation for you Sound of Music enjoyers.
4 replies →
> I am sure VCs are fighting to invest and they will get 4B investment immediately with this team
This is probably the only time some prominent VCs would be active during August, only for Jeff Dean & Co's company.
I can see them now frantically negotiating, responding and firing off emails right now during their holidays to get an allocation in Jeff's new company.
Would love to hear the pitch: We never had the lead in AI and now definitely lost it, despite the gazillions dollars and engineering resources of Google, but now will be different?
The "lead" in AI, even with google's TPU advantage, may not make economic sense for a company. Can you make back the training investment on a competitive frontier model, before everyone switches to a newer frontier model within a year?
Turns out you can't pay em as much as VC will, haha
its a public benefit corp
Don’t worry they will just limit them to a 10x return but then later on just convert to for profit and fahgetabout the cap!
1 reply →
So is Anthropic...
1 reply →
Funny how that works, so are Anthropic and OpenAI apparently. Maybe there is something we are missing here!