Comment by gregw134
7 months ago
Ex-Google search engineer here (2019-2023). I know a lot of the veteran engineers were upset when Ben Gomes got shunted off. Probably the bigger change, from what I've heard, was losing Amit Singhal who led Search until 2016. Amit fought against creeping complexity. There is a semi-famous internal document he wrote where he argued against the other search leads that Google should use less machine-learning, or at least contain it as much as possible, so that ranking stays debuggable and understandable by human search engineers. My impression is that since he left complexity exploded, with every team launching as many deep learning projects as they can (just like every other large tech company has).
The problem though, is the older systems had obvious problems, while the newer systems have hidden bugs and conceptual issues which often don't show up in the metrics, and which compound over time as more complexity is layered on. For example: I found an off by 1 error deep in a formula from an old launch that has been reordering top results for 15% of queries since 2015. I handed it off when I left but have no idea whether anyone actually fixed it or not.
I wrote up all of the search bugs I was aware of in an internal document called "second page navboost", so if anyone working on search at Google reads this and needs a launch go check it out.
Machine learning or not, seo spam sort of killed search. It’s more or less impossible to find real sites by interesting humans these days. Almost all results are Reddit, YouTube, content marketing, or seo spam. And google’s failure here killed the old school blogosphere (medium and substack only slightly count), personal websites, and forums
Same is happening to YouTube as well. Feels like it’s nothing but promoters pushing content to gain followers to sell ads or other stuff because nobody else’s videos ever surface. Just a million people gaming the algorithm and the only winners are the people who devote the most time to it. And by the way, would I like to sign up for their patreon and maybe one of their online courses?
I think a case can be made that the spam problem can be traced all the way back to Google buying Doubleclick.
Its really easy to spot the crap websites that are scaping content-creating websites ... because they monetize by adding ads.
If Google was _only_ selling ads on the search results page, then it could promote websites that are sans ads.
Instead, it is incentivised to push users to websites that contain ads, because it also makes money there.
And that means scraping other sites to slap your ads onto them can be very profitable for the scammers.
They hired people who introduce Jack Welch methods.
This is like in that Steve Jobs video about product people being kicked out and exchanged by ones who dont care about product:
https://m.youtube.com/watch?v=P4VBqTViEx4
They will not make good search. That is not their priority.
1 reply →
We need a Reverse Google search that will weed out the garbage.
13 replies →
A bit chicken-and-egg. Another perspective: Google’s system incentivizes SEO spam.
Search for a while hasn’t been about searching the web as much as it has been about commerce. It taps commercial intent and serves ads. It is now an ad engine; no longer a search engine.
Best exercise bike articles, and such, are what lots of people people actually search for. There is no incentive to provide quality work which answers these queries hence the abundance of spam and ads.
If you want to purchase consumer products at your own expense and offer an impartial opinion on each of them then you will have no problem getting ranked highly on google. You will lose a lot of money doing so, however, and will also be plagiarized to death in a month. The sites you want to be rid of will outrank you for your own content, I have been there and have the t-shirt.
1 reply →
Absolutely this. I don't think many people consider how odd it is that the largest internet advertising company in the world and the largest search engine company in the world are one and the same, and just how overt a conflict of interest that is, so far as providing quality service goes. It would be akin to if the largest telephone service company in the world was also the largest phone maker in the world. Oh wait, that did happen [1] - and we broke them up because it's obviously extremely detrimental to the functioning of a healthy market.
[1] - https://en.wikipedia.org/wiki/Breakup_of_the_Bell_System
For me what killed search was 2016, after that year if some search term is "hot news" it becomes impossible to learn anything about it that wasn't published in the last week and you just get the same headline repeated 20 times in slightly different wording about it.
After that I only use search for technical problems, and mouth to mouth or specific authors for everything else.
Yes, this is a thing I find really frustrating about Google. Especially as I often search for old news stories to find out what people were saying on a topic a few years ago in order to give some context to more recent stories.
Most of the problems I complain about are not related to SEO spam but to Google including sites that does not contain my search terms anywhere despite my use of doublequotes and the verbatim operator.
As for SEO spam a huge chunk of it would have disappeared I think if Google had created the much requested personal blacklist that we used to ask them for.
It was always "actually much harder than anyone of you who don't work here can imagine for reasons we cannot tell or you cannot understand" or something like that problem, but bootstraped Kagi managed to do it - and their results are so much better that I don't usually need it.
I've heard this argument again and again, but I never see any explanation as to why SEO is suddenly in the lead in this cat-and-mouse game. They were trying ever since Google got 90%+ market share.
I think it's more likely that Google stopped really caring.
Well yeah, it's in the article - at some point, they switched completely to metrics (i.e. revenue) driven management and forgot that it's the quality of results that actually made Google what it is. And, with a largely captive audience (Google being the default-search-engine-that-most-people-don't-bother-or-don't-know-how-to-change in Chrome, Android, on Chromebooks etc.), they arguably don't have to care anymore...
Well, it's in the name. SEO is a fancy name for trying to game whatever heuristics Google employs to form their SERPs. It's just that at some point those heuristics shifted from rewarding "quality content" as defined by the disgruntled towards enshitification.
There are various kinds of SEO - internal: technical, on-page and external. A long time ago Google had an epiphany that instead of trying to make sense out of sites themselves they could offload that effort to website administrators and started ranking pages how well they implement technical elements helping Google index the web. For a very long time that was synonymous with white-hat SEO. Since Google search was in part based on web-of-links, various shady tactics to inflate number of indexed backlinks and boost rankings. That was black-hat SEO.
These days Google search puts tremendous focus on on-page SEO. So much that as long as the internal structure of a site is indexable (no dead links, internal backlinks, meta info) it is typically better to hire copywriters spitting out LLM-like robotic mumblings than to try and optimize further.
Massive media companies finally caught on and started churning out utter shit because it's wildly profitable.
When the 'trusted websites' caught on and embraced the game, Google was apparently helpless to stop it.
I don't know, but Youtube seems to have a more solid algorithm. I'm typically not subscribed to any channel, yet the content I want to watch does find me reasonably well. Of course, heavily promoted material also, but I just click "not interested in channel" and it disappears for a while. And I still get some meaningful recommendations if I watch a video in a certain topic. Youtube has its problems, of course, but in the end I can't complain.
I don't think youtube is trying that hard to desperately sell stuff to you via home screen recommendation algorithm. And I agree its bearable and what you describe works cca well, albeit ie I am still trying to get rid of anything related to Jordan Peterson whom I liked before and detest now after his drug addiction / mental breakdown, it just keeps popping back from various sources, literal whack-a-mole.
I wish there was some way to tell "please ignore all videos that contain these strings, and I don't mean only for next 2 weeks".
Youtube gets their ads revenue from before/during video, so they can be nicer to users.
What I don't understand about this explanation is that Google's results are abysmal compared to e.g. DuckDuckGo or even Brave search. (I haven't tried Kagi, but people here rave about it as well.) Sure, all the SEO is targeting googlebot, but Google has by far more resources to mitigate SEO spam than just about anyone else. If this is the full explanation, couldn't Google just copy the strategies the (much) smaller rivals are using?
Have you read the article this thread is about?
To summarize it: Google reverted an algorithm that detected SEO spams in 2019.
(Note that I never work for Google and I don't know whether it's true or not. It's just what this article says.)
3 replies →
When a large search engine deranks spam websites, the spam websites complain! Loudly! With Google they have a big juicy target with lots of competing ventures for an antitrust case; no such luck for Kagi or DDG.
3 replies →
I've been using Kagi for a while, and I find that it delivers better results in a cleaner presentation.
spam didn't kill search. Google willingness to promote spam for ads killed Google. Google is not search.
Machine learning is probably as much or even more susceptible to SEO spam.
Problem is that the rules of search engines created the dubious field of SEO in the first place. They are not entirely the innocent victim here.
Arcane and intransparent measures get you ahead. So arcane that you instantly see that it does not correspond with quality content at all, which evidently leads to a poor result.
I wish there was an option to hide every commercial news or entertainment outlet completely. Those are of course in on SEO for financial reaesons.
>I wish there was an option to hide every commercial news or entertainment outlet completely.
There's alway plugins or you can subscribe to Kagi, although I don't think there's any blocklist preconfigured for "all commercial news websites"
Hard disagree. As another reply mentions, just compare the alternatives such as Kagi that aren’t breaking search by pursuing ad growth.
Kagi isn't amazing, it's just not bad and it really makes plain how badly Google has degraded into an ad engine. All it takes to beat Google is giving okay quality search results.
These search companies should have hired moderators to manually browse results and tag them based on keywords instead of leaving tagging up to content and info creators. The entire results game became fixated on trending topics and SEO spam that it became a game of insider trick trading, that's what makes results everywhere so terrible now.
In a bid for attention, only the fraudsters are winning, well, the platforms are winning lots of money from selling advertising, I guess that's why they're perfectly fine with not fixing results and ranking for many years now. I'm not sure there is a way back to real relevance now, there's no incentive for these large companies to fix things, and the public has already become used to the gamified system to go back to behaving themselves.
This explodes for search terms dealing with questions related to bugs or issues or how to dos. Almost all top results are YT videos, each of which will follow the same pattern. First 10 secs garbage followed by request for subscribe and/or sponsorship content then followed by what you want.
SEO Spam didn't kill search so much as Google failed to retain Matt Cutts or replicate his community involvement https://www.searchenginejournal.com/matt-cutts-resigns-googl...
What did he used to do ? Your comment seems contradictory cutts seem to be on anti spam but your comment implies seo did not kill search . Is seo not part of spam?
2 replies →
It's like "Do some SEO magic and Tada!"
And who forgot the recent Reddit story.
Could you link it please? I have unfortunately no idea what you are referencing
Much agreed, and this is prompting me to experiment with other search engines to see if they cut off also the interesting humans sites. With todays google I feel herded.
[flagged]
This is the correct insight. Google has enough machine learning prowess that they could absolutely offload, with minimal manhours, the creation of a list ranking a bunch of blogspam sites and give them a reverse score by how much they both spam articles or how much they spread the content over the page. Then apply that score to their search result weights.
And I know they could because someone did make that list and posted it here last year.
1 reply →
> where he argued against the other search leads that Google should use less machine-learning
This better echoes my personal experience with the decline of Google search than TFA: it seems to be connected to the increasing use of ML in that the more of it Google put in, the worse the results I got were.
It's also a good lesson for the new AI cycle we're in now. Often inserting ML subsystems into your broader system just makes it go from "deterministically but fixably bad" to "mysteriously and unfixably bad".
I think that’ll define the industry for the coming decades. I used to work in machine translation and it was the same. The older rules-based engines that were carefully crafted by humans worked well on the test suite and if a new case was found, a human could fix it. When machine learning came on the scene, more “impressive” models that were built quicker came out - but when a translation was bad no one knew how to fix it other than retraining and crossing one’s fingers.
21 replies →
I've heard AI described as the payday loan (or "high-interest credit card") of technical debt.
I think - I hope, rather - that technically minded people who are advocating for the use of ML understand the short comings and hallucinations... but we need to be frank about the fact that the business layer above us (with a few rare exceptions) absolutely does not understand the limitations of AI and views it as a magic box where they type in "Write me a story about a bunny" and get twelve paragraphs of text out. As someone working in a healthcare adjacent field I've seen the glint in executive's eyes when talking about AI and it can provide real benefits in data summarization and annotation assistance... but there are limits to what you should trust it with and if it's something big-i Important then you'll always want to have a human vetting step.
21 replies →
This is why hallucinations will never be fixed in language models. That's just how they work.
mysteriously with a helping of random too!
that's not something ML people would like to hear
Is ML the new SOAP? Looks like a silver bullet and 5 years later you're drowning in complexity for no discernible reason?
17 replies →
Well, it depends on the ML person. I work on industrial ML and DL systems every day and I'm the one who made that comment.
Same here with YouTube, assuming they use ML, which is likely.
They routinely give me brain-dead suggestions such as to watch a video I just watched today or yesterday, among other absurdities.
For what it's worth, I do not remember a time when YouTube's suggestions or search results were good. Absurdities like that happened 10 and 15 years ago as well.
These days my biggest gripe is that they put unrelated ragebait or clickbait videos in search results that I very clearly did not search for - often about American politics.
22 replies →
YT Shorts recommendations are a joke. I'm an atheist and very rarely watch anything related to religion, and even so Shorts put me in 3 or 4 live prayers/scams (not sure) the last few months.
26 replies →
I think it's probably pushing pattern it sees in other users.
There's videos I'll watch multiple times, music videos are the obvious kind, but for some others I'm just not watching/understanding it the first time and will go back and rewatch later.
But I guess youtube has no way to understand which one I'll rewatch and which other I don't want to see ever again, and if my behavior is used as training data for the other users like you, they're probably screwed.
1 reply →
This is happening to me to, but from the kind of videos it's suggested for I suspect that people actually do tend to rewatch those particular videos, hence the recommendation.
Install "Unhook" chrome extension. That changed my life.
Thanks for writing this insightful piece.
The pathologies of big companies that fail to break themselves up into smaller non-siloed entities like Virgin Group does. Maintaining the successful growing startup ways and fighting against politics, bureaucracy, fiefdoms, and burgeoning codebases is difficult but is a better way than chasing short-term profits, massive codebases, institutional inertia, dealing with corporate bullshit that gets in the way of the customer experience and pushes out solid technical ICs and leaders.
I'm surprised there aren't more people on here who decide "F-it, MAANG megacorps are too risky and backwards not representative of their roots" and form worker-owned co-ops to do what MAANGs are doing, only better, and with long-term business sustainability, long tenure, employee perks like the startup days, and positive civil culture as their central mission.
What's odd to me is how everything is so metricized. Clearly over metricization is the downfall of any system that looks meritocratic. Due to the limitations of metrics and how they are often far easier to game than to reach through the intended means.
An example of this I see is how new leaders come in and hit hard to cut costs. But the previous leader did this (and the one before them) so the system/group/company is fairly lean already. So to get anywhere near similar reductions or cost savings it typically means cutting more than fat. Which it's clear that many big corps are not running with enough fat in the first place (you want some fat! You just don't want to be obese!). This seems to create a pattern that ends up being indistinguishable from "That worked! Let's not do that anymore."
Agree you have to mix qualitative with the quantitative, but the best metrics systems don't just measure one quantity metric. They should be paired with a quality metric.
Example: User Growth & Customer Engagement
Have to have user growth and retention. If you looked at just one or the other, you'd be missing half the equation.
2 replies →
Oh god. The blind faith in reductive, objectivist, rationalist meritocracy that somehow "everything can be measured perfectly" and "whatever happens is completely unbiased as proscribed by a black-and-white, mechanical formula". No, sorry, that's insufficiently holistic in accounting for intangibles and supportive effort, and more of a throwback ideology that should've died in the 1920's. Some degree of discretion is needed because there is no shortcut to "measuring" performance.
The hard part about starting worker owned co-ops is financing. We need good financing systems for them. People/firms who are willing to give loans for a reasonable interest, but on the scale of equity investment in tech start ups.
The problem is risk —- most new businesses will go under. Who’s going to take on that unreasonable risk without commensurate reward (high interest loan rate, if any, or equity).
Co-ops could go the angel/VC route for funding if they don’t give up a controlling share.
I formed a worker co-op - but it's just me! And I do CAD reverse engineering, nothing really life-giving.
I would love to join a co-op producing real human survival values in an open source way. Where would you suggest that I look for leads on that kind of organization?
Let's start with replacing Google. Count me in.
While DDG, Brave, Kagi etc are working generously to replace Google search. The other areas that I think get less attention and needs to be targeted to successfully dismantle them and their predatory practices are Google maps and Google docs.
Maps are hard because it requires a lot of resources and money and whatever but replacing docs should be relatively easier.
6 replies →
I would imagine GitHub and technology social media
I guess it depends on how much equity you own as to what is better (to your first paragraph), and how large the paycheck is (to the 2nd paragraph.
Problem is, worker owned co-ops would still require money to do anything even remotely competitive to existing businesses.
So... people go walk up for handouts from VCs....and the story begins lol.
> There is a semi-famous internal document he wrote where he argued against the other search leads that Google should use less machine-learning, or at least contain it as much as possible, so that ranking stays debuggable and understandable by human search engineers.
There's a lot of ML hate here, and I simply don't see the alternative.
To rank documents, you need to score them. Google uses hundreds of scoring factors (I've seen the number 200 thrown about, but it doesn't really matter if it's 5 or 1000.) The point is you need to sum these weights up into a single number to find out if a result should be above or below another result.
So, if:
Are there any humans out there who want to write a traditional forward-algorithm to tell me which result is better?
You do not need to. Counting how many links are pointing to each document is sufficient if you know how long that link existed (spammers link creation time distribution is widely differnt to natural link creation times, and many other details that you can use to filter out spammers)
> You do not need to.
Ranking means deciding which document (A or B) is better to return to the user when queried.
Not writing a traditional forward-algorithm to rank these documents implies one of the following:
- You write a "backward" algorithm (ML, regression, statistics, whatever you want to call it).
- You don't use algorithms to solve it. An army of humans chooses the rankings in real time.
- You don't rank documents at all.
> Counting how many links are pointing to each document is sufficient if you know how long that link existed
- Link-counting (e.g. PageRank) is query-independent evidence. If that's sufficient for you, you'll always return the same set of documents to each user, regardless of what they typed into the search box.
At best you've just added two more ranking factors to the mix:
So I ask again:
- Which document matches your query better, A or B?
- How did you decide that, such that not only can you program a non-ML algorithm to perform the scoring, but you're certain enough of your decision that you can fix the algorithm when it disagrees with you ( >> debuggable and understandable by human search engineers )
3 replies →
> spammers link creation time distribution is widely differnt to natural link creation times
Yes, this is a statistical method. Guess what machine learning is and what it actually excels?
For a few months last year every time I searched for information about a package related to software available in homebrew, the first few results were to a site that clearly just had crawled all of the links in homebrew, and templated out a site of links corresponding to each package name. and thats about it. It would have been nice if the generated pages contained any useful information, but alas it did not.
There's got to be a better way.
[dead]
Amit was definitely against ML, long before "AI" had become a buzzword.
He wasn't the only one. I built a couple of systems there integrated into the accounts system and "no ML" was an explicit upfront design decision. It was never regretted and although I'm sure they put ML in it these days, last I heard as of a few years ago was that at the core were still pages and pages of hand written logic.
I got nothing against ML in principle, but if the model doesn't do the right thing then you can just end up stuck. Also, it often burns a lot of resources to learn something that was obvious to human domain experts anyway. Plus the understandability issues.
i worked on ranking during singhal's tenure, and it was definitely refreshing to see a "no black box ML ranking" stance.
I was there from 2015-2023 and, although I didn't work in Search, I remember a lot of the bigger initiatives designed at improving Search for users, like the project to add cards for the top 500 most commonly searched medical terms/conditions, using content from Mayo and custom contracted digital art (for an example, here's a sample link: https://www.google.com/search?q=acl+tear ). There were a lot of things like this going on at any point in time, and it was terrific to see. Then I discovered the manually curated internal knowledge graph, that even included many-language i19n. And then that it was possible for any googler to suggest updates/changes/additions.
Point being, there's a lot of amazing stuff that folks on the outside never would have seen, and it would be a shame for beancounters to ruin it all with decisions actively not "respecting the user".
That amazing internal knowledge graph you're talking about folks on the outside never seeing? That is very ironic because that knowledge graph used to be Freebase.com and a lot of the data came from the open data community who volunteered their efforts and expertise. Then Google bought Metaweb and shut down Freebase.
simplicity is always the recipe for success, unfortunately, most engineers are drawn to complexity like moth to fire
if they were unable to do some AB testing between a ML search and a non-ML search, they deserve their failure 100%
there are not enough engineers blowing the whistle against ML
> most engineers are drawn to complexity like moth to fire
Unfortunately, Google evaluates employees by the complexity of their work. "Demonstrates complexity" is a checkbox on promo packets, from what I've heard.
Naturally, every engineer will try to over-complicate things just so they can get the raises and promos. You get what you value.
I've heard a similar critique for Google launching new products and then letting them die, where it's really driven by their policies and practice around what gets someone a promotion and what doesn't.
1 reply →
I definitely think the ML search results are much worse. But complexity or not, strategically it's an advantage for the company to use ML in production over a long period of time so they can develop organizational expertise in it.
It would have been a worse outcome for Google if they had stuck to their no ML stance and then had Bing take over search because they were a generation behind in technology.
Engineers love simplicity but management hates it and won’t promote people that strive towards it. A simple system is the most complex system to design.
@gregw134 Thank you for sharing! I've never worked at Google, but really curious what the engineering context is when you say "needs a launch" in the last line.
Guessing: perhaps this means, if someone needs credit for shepherding an improvement to search quality into production, here is a set of known improvements waiting for someone to take ownership.
Exactly. The main way to get promoted at Google is to claim that you launched something important. Results in a lot of busywork and misaligned incentives.
I'm glad you shared this.
My priors before reading this article were that an uncritical over-reliance on ML was responsible for the enshittification of Google search (and Google as a whole). Google seemed to give ML models carte blanche, rather than using the 80-20 rule to handle the boring common cases, while leaving the hard stuff to the humans.
I now think it's possible both explanations are true. After all, what better way to mask a product's descent into garbage than more and more of the core algorithm being a black box? Managers can easily take credit for its successes and blame the opacity for failures. After all, the "code yellow" was called in the first place because search growth was apparently stagnant. Why was that? We're the analysts manufacturing a crisis, or has search already declined to some extent?
[flagged]