Comment by lkbm
2 years ago
It's worth remembering that AI is more than LLMs. DeepMind is still doing big stuff: https://deepmind.google/discover/blog/millions-of-new-materi...
2 years ago
It's worth remembering that AI is more than LLMs. DeepMind is still doing big stuff: https://deepmind.google/discover/blog/millions-of-new-materi...
I just want to underscore that. DeepMind's research output within the last month is staggering:
2023-11-14: GraphCast, word leading weather prediction model, published in Science
2023-11-15: Student of Games: unified learning algorithm, major algorithmic breath-through, published in Science
2023-11-16: Music generation model, seemingly SOTA
2023-11-29: GNoME model for material discovery, published in Nature
2023-12-06: Gemini, the most advanced LLM according to own benchmarks
Google is very good at AI research.
Where it has fallen down (compared to its relative performance in relevant research) is public generative AI products [0]. It is trying very hard to catch up at that, and its disadvantage isn't technological, but that doesn't mean it isn't real and durable.
[0] I say "generative AI" because AI is a big an amorphous space, and lots of Google's products have some form of AI that is behind important features, so I'm just talking about products where generative AI is the center of what the product offers, which have become a big deal recently and where Google had definitely been delivering far below its general AI research weight class so far.
> Google is very good at AI research. Where it has fallen down (compared to its relative performance in relevant research) is public generative AI products
In such cases, I actually prefer Google over OpenAI. Monetization isn’t everything
17 replies →
They publish but don't share. Who cares about your cool tech if we can't experience it ourselves? I don't care about your blog writeup or research paper.
Google is locked behind research bubbles, legal reviews and safety checks.
Mean while OpenAI is eating their lunch.
The researchers at all the other companies care about the blog write-ups and research papers. The Transformer architecture, for example, came from Google.
Sharing fundamental work is more impactful than sharing individual models.
3 replies →
this. google is like academia; you publish a lot but others who implement your work are the ones making money off of it.
6 replies →
Indeed, I would think the core search product as another example of ai/ml...
The question is whether greater use of AI correlates with the declining quality of search results.
I think the real underlying cause is the explosion of garbage that gets crawled. Google initially tried to use AI to find "quality" content in the pile. It feels like they gave up and decided to use the wrong proxies for quality. Proxies like "somehow related to a brand name". Good content that didn't have some big name behind it gets thrown out with the trash.
I think the bottom line (profit) inversely correlates with the quality of search results. I've been using phind.com lately and it seems there can be search without junk even in this age.
Google has lots of people tagging search rankings, which is very similar with RLHF ranking responses from LLMs. It's interesting that using LLMs with RLHF it is possible to de-junk the search results. RLHF is great for this task, as evidenced by its effect on LLMs.
3 replies →
Web has grown by 1000x over years. The overall signal to noise ratio has been worsen, around by 100x and SEO has been become much more sophisticated and optimized against Google. A large fraction of quality content has been moving toward walled gardens. The goalpost is moving (much) faster than technologies.
2 replies →
I recently google searched "80cm to inches" and it gave me the result for "80 meters to inches". I can't figure out how it would make this mistake aside from some poorly conceived LLM usage
1 reply →
It would be fun to see modern Google ran against a snapshot of the old web.
Maybe the declining quality of internet content has something to do with the declining quality of search results.
There's a constant arms race between shitty SEO, walled gardens, low-quality content farms and search engines.
This does highlight the gap between SOTA and business production. Google search is very often a low quality, even user hostile experience. If Google has all this fantastic technology, but when the rubber hits the road they have no constructive (business supporting) use cases for their search interface, we are a ways away from getting something broadly useful.
It will be interesting to see how this percolates through the existing systems.
I am at first just saying that search as PageRank in the early days is a ML marvel that changed the way people interact with the internet. Figuring out how to monetize and financially survive as a business have certainly changed the direction of its development and usability.
Yes, it is very successful in replacing useful results with links to shopping sites.
This is because their searches are so valuable that real intelligence, i.e. humans, have been fighting to defeat google's AI over billions of dollars of potential revenue.
We are just seeing remnants of that battleground.
8 replies →