Comment by HarHarVeryFunny
2 years ago
No - I don't work in the ML field, or for any of these companies (but I have built a Torch-like NN framework in C++ from scratch, and followed the early transformer development closely, so do understand the tech).
I'm just writing about the changes to these organizations, and their corporate governance, as reported in the press. I don't think you need to be an insider to appreciate the difference between, say, DeepMind as an independent entity pursuing AGI anyway they saw fit (RL), and now as part of Google DeepMind apparently tasked with developing SOTA LLMs. No doubt this is still a research vs pure engineering endeavor, but hard to call it blue sky when the research direction and goal is so proscribed. I personally don't believe that LLMs (or RL for that matter) are the path to AGI, but at least DeepMind used to have the flexibility to pivot and pursue whatever lines of research they felt were most promising. Do they still have that flexibility today?
> I have built a Torch-like NN framework in C++ from scratch
Mm, care to share? I am skeptical when people make claims like this, even though it is very achievable. Many people are simply "playing house" when it comes to ML tech.
I would not believe everything you read even in the tech aligned press, it is very often false. Google Deepmind is not exclusively researching LLMs.
I never released my framework, and don't intend to (abandoned this a good few years ago), but it was more than "playing house" ... It was complete enough to build/train a convnet that worked with CIFAR-10, supporting both GPU via cuDNN and CPU via my own Tensor class with MKL/IPP BLAS/etc acceleration. The API was Torch-like where you build the graph (create nodes, then connect them), then run it. I was in process of writing a version 2 with support for RNNs and auto migration of tensors from CPU to/from GPU, but gave up since PyTorch had since appeared (obviously a better approach) and it became increasingly obvious how ridiculous it was for a one-man project to attempt to catch up to SOTA!