Comment by WarmWash
9 days ago
I was just thinking earlier today how in an alternate universe, probably not too far removed from our own, Google has a monopoly on transformers and we are all stuck with a single GPT-3.5 level model, and Google has a GPT-4o model behind the scenes that it is terrified to release (but using heavily internally).
This was basically almost real.
Before ChatGPT was even released, Google had an internal-only chat tuned LLM. It went "viral" because some of the testers thought it was sentient and it caused a whole media circus. This is partially why Google was so ill equipped to even start competing - they had fresh wounds of a crazy media circus.
My pet theory though is that this news is what inspired OpenAI to chat-tune GPT-3, which was a pretty cool text generator model, but not a chat model. So it may have been a necessary step to get chat-llms out of Mountain View and into the real world.
https://www.scientificamerican.com/article/google-engineer-c...
https://www.theguardian.com/technology/2022/jul/23/google-fi...
> some of the testers thought it was sentient and it caused a whole media circus.
Not "some of the testers." One engineer.
He realized he could get a lot of attention by claiming (with no evidence and no understanding of what sentience means) that the LLM was sentient and made a huge stink about it.
He was unfairly labelled as a lunatic early on. I'd implore anyone reading this thread to see what he had to say for yourself and form your own opinion: https://youtube.com/watch?v=kgCUn4fQTsc
He had a history of causing noise at Google’s weekly leadership Q&A.
Now think about how often the patent system has stifled and stalled and delayed advancement for decades per innovation at a time.
Where would we be if patents never existed?
Who knows? If we’d never moved on from trade secrets to patents, we might be a hundred years behind.
Is that really the case in the last few years/decades?
My understanding is that any company that can (read: has enough money for good lawyers), will prefer to use trade secrets for a combination of reasons, a big one being that competitors cannot use that technology after 10 years/when the patent expires.
Admittedly this was from my entrepreneurship classes in a European uni, so I'm not sure how it is in different places in the world.
1 reply →
To be fair, Google has a patent on the transformer architecture. Their page rank patent monopoly probably helped fund the R&D.
They also had a patent on map/reduce.
It would have been nice for me to be able to work a few more years and be able to retire
will your retirement be enjoyable if everyone else around you is struggling?
What does that mean? Everyone was going to struggle because I still had my 9 to 5 middle class job?