Comment by whimsicalism
2 years ago
> DeepMind now merged with Google Brain seemingly with a product focus, FAI
I don’t really agree that training massive causal LMs is a “product focus”.
I agree that there is an increasing product focus in orgs like OAI, but a lot of that is coming from new growth rather than trading off with base research.
> I don’t really agree that training massive causal LMs is a “product focus”.
I'd hope that Google DeepMind's mission statement is a bit broader than that, but certainly Gemini seems to be the focus. It seems to me there's a world of difference between DeepMind's original goal of developing AGI (via whatever means - as a research objective), and now being told they have to build LLMs.
If we compare Google to Meta, it seems it used to be that DeepMind was equivalent to FAIR as a pure research organization, and Brain equivalent to Meta's product focused ML group(s), but now the joint Google DeepMind is more akin to Meta's GenAI group, and there is no unfettered research group at Google left free to pursue AGI in any way other than hoping it can be developed out of LLMs. However, FAIR also seems constrained now that they have been moved into a product-focused part of the organization (under CPO Chris Cox).
Did you work in this field? I just don’t find that a fair characterization. They are all still doing blue-sky research.
There is a ton of further research to be done in deep models. A lot of it will incorporate LLMs because that is currently the most powerful primitive you have. Research is a lot more closed now but I would not take that to mean research is no longer happening.
No - I don't work in the ML field, or for any of these companies (but I have built a Torch-like NN framework in C++ from scratch, and followed the early transformer development closely, so do understand the tech).
I'm just writing about the changes to these organizations, and their corporate governance, as reported in the press. I don't think you need to be an insider to appreciate the difference between, say, DeepMind as an independent entity pursuing AGI anyway they saw fit (RL), and now as part of Google DeepMind apparently tasked with developing SOTA LLMs. No doubt this is still a research vs pure engineering endeavor, but hard to call it blue sky when the research direction and goal is so proscribed. I personally don't believe that LLMs (or RL for that matter) are the path to AGI, but at least DeepMind used to have the flexibility to pivot and pursue whatever lines of research they felt were most promising. Do they still have that flexibility today?
2 replies →