Comment by JohnFen

As someone who worked in rules-based ML before the recent transformers (and unsupervised learning in general) hype, rules-based approaches were laughably bad. Only now are nondeterministic approaches to ML surpassing human level tasks, something which would not have been feasible, perhaps not even possible in a finite amount of human development time, via human-created rules.

Yes, but I think the other lesson might be that those black box machine translations have ended up being more valuable? It sucks when things don't always work, but that is also kind of life and if the AI version worked more often that is usually ok (as long as the occasional failures aren't so catastrophic as to ruin everything)

yes, who exactly looked at the 70% accuracy of "live automatic closed captioning" and decided Great! ship it boys!

But rule-based machine translation, from what I've seen, is just so bad. ChatGPT (and other LLM) is miles ahead. After seeing what ChatGPT does, I can't even call rule-based machine translation "tranlation".

*Disclaimer: as someone who's not an AI researcher but did quite some human translation works before.

I think NM translation was broken all along. Not in the neural network part but in choosing the right answer. https://aclanthology.org/2020.coling-main.398.pdf

Perhaps using a ML to craft the deterministic rules and then have a human go over them is the sweet spot.

> I hope, rather - that technically minded people who are advocating for the use of ML understand the short comings and hallucinations.

The people I see who are most excited about ML are business types who just see it as a black boxes that makes stock valuation go vroom.

The people that deeply love building things, really enjoy the process of making itself, are profoundly sceptical.

I look at generative AI as sort of like an army of free interns. If your idea of a fun way to make a thing is to dictate orders to a horde of well-meaning but untrained highly-caffienated interns, then using generative AI to make your thing is probably thrilling. You get to feel like an executive producer who can make a lot of stuff happen by simply prompting someone/something to do your bidding.

But if you actually care about the grit and texture of actual creation, then that workflow isn't exactly appealing.

I’m not optimistic on that point: the executive class is very openly salivating at the prospect of mass layoffs, and that means a lot of technical staff aren’t quick to inject some reality – if Gartner is saying it’s rainbows and unicorns, saying they’re exaggerating can be taken as volunteering to be laid off first even if you’re right.

> technically minded people who are advocating for the use of ML understand the short comings and hallucinations

really, my impression is the opposite. They are driven by doing cool tech things and building fresh product, while getting rid of "antiquated, old" product. Very little thought given to the long term impact of their work. Criticism of the use cases are often hand waved away because you are messing with their bread and butter.

> but we need to be frank about the fact that the business layer above us (with a few rare exceptions) absolutely does not understand the limitations of AI and views it as a magic box where they type in

I think we also need to be aware that this business layer above us that often sees __computers__ as a magic box where they type in. There's definitely a large spectrum of how magical this seems to that layer, but the issue remains that there are subtleties that are often important but difficult to explain without detailed technical knowledge. I think there's a lot of good ML can do (being a ML researcher myself), but I often find it ham-fisted into projects simply to say that the project has ML. I think the clearest flag to any engineer that this layer above them has limited domain knowledge is by looking at how much importance they place on KPIs/metrics. Are they targets or are they guides? Because I can assure you, all metrics are flawed -- but some metrics are less flawed than others (and benchmark hacking is unfortunately the norm in ML research[0]).

[0] There's just too much happening so fast and too many papers to reasonably review in a timely manner. It's a competitive environment, where gatekeepers are competitors, and where everyone is absolutely crunched for time and pressured to feel like they need to move even faster. You bet reviews get lazy. The problems aren't "posting preprints on twitter" or "LLMs giving summaries", it's that the traditional peer review system (especially in conference settings) poorly scales and is significantly affected by hype. Unfortunately I think this ends up railroading us in research directions and makes it significantly challenging for graduate students to publish without being connected to big labs (aka, requiring big compute) (tuning is another common way to escape compute constraints, but that falls under "railroading"). There's still some pretty big and fundamental questions that need to be chipped away at but are difficult to publish given the environment. /rant

> SOAP

Argh. My PTSD from writing ONVIF drivers just kicked in.

ML is somewhere between the new SOAP and the new cryptocurrency.

So... obviously SOAP was dumb[1], and lots of people saw that at the time. But SOAP was dumb in obvious ways, and it failed for obvious reasons, and really no one was surprised at all.

ML isn't like that. It's new. It's different. It may not succeed in the ways we expect; it may even look dumb in hindsight. But it absolutely represents a genuinely new paradigm for computing and is worth studying and understanding on that basis. We look back to SOAP and see something that might as well be forgotten. We'll never look back to the dawn of AI and forget what it was about.

[1] For anyone who missed that particular long-sunken boat, SOAP was a RPC protocol like any other. Yes, that's really all it was. It did nothing special, or well, or that you couldn't do via trivially accessible alternative means. All it had was the right adjective ("XML" in this case) for the moment. It's otherwise forgettable, and forgotten.

Don't forget about that expensive GPU infrastructure you invested in.

ML is a quite well adopted technology. iPhones has ML bulit in since about 2017. It has been more than 5 years.

15 years ago, I used to keep many tabs of youtube videos open just because the "related" section was full of interesting videos. Then each of those videos had interesting relations. There was so much to explore before hitting a dead-end and starting somewhere else.

Now the "related" section is gone in favor of "recommended" samey clickbait garbage. The relations between human interests are too esoteric for current ML classifiers to understand. The old Markov-chain style works with the human, and lets them recognize what kind of space they've gotten themselves into, and make intelligent decisions, which ultimately benefit the system.

If you judge the system by the presence of negative outliers, rather than positive, then I can understand seeing no difference.

I do remember when Youtube would show more than 2 search results per page on my 23" display.

Or when they would show more than 3 results before spamming irrelevant videos.

Or when they didn't show 3 unskippable ads in a 5 minute video.

Or when they had a dislike button so you would know to avoid wasting time on low quality videos.

YouTube seems to treat popular videos as their own interest category and it’s very aggressive about recommending them if you show any interest at all. If you watch even one or two popular videos (like in the millions of views), suddenly the quality of the recommendations drops off a cliff, since it is suggesting things that aren’t relevant to your interest categories, it’s just suggesting popular things.

If I entirely avoid watching any popular videos, the recommendations are quite good and don’t seem to include anything like what you are seeing. If I don’t entirely avoid them, then I do get what you are seeing (among other nonsense).

Long long time ago; youtube "staff" would manually put certain videos on the top of the front page when they started. Im sure there we're biases and prioritization of marketing dollars but at least there was human recommending it compared to poorly recorded early family guy clips. I dont know when they stopped manually adding "editors/staff" choice videos but I recall some of my favorite early youtubers like CGPGgrey claim that recommendation built the career.

It all depends on your use case but a lot of people seem to be in agreement it fell off in the mid to late 10s and the suggestions became noticeably worse.

Similarly, Google News. The "For You" section shows me articles about astrology because I'm interested in astronomy. I get suggestions for articles about I-80 because I search for I-80 traffic cams to get traffic cam info for Tahoe, but it shows me I-80 news all the way across the country, suggestions about MOuntain View because I worked there (for google!) over 3 years ago, commanders being fired from the Navy (because I read a couple articles once), it goes on and on. From what I can tell, there are no News Quality people actually paying attention to their recommendations (and "Show Fewer" doesn't actually work. I filed a bug and was told that while the desktop version of the site shows Show Fewer for Google News, it doesn't actually have an effect).

YT Shorts itself is kind of a mystery to me. It's an objective degradation of the interface; why on earth would I want to use it? It doesn't even allow adjustment of the playback speed or scrubbing!

I imagine my blocked channels list is stress testing YouTube at this point from the amount of shit Shorts results it's fed me after 2 years. Lol

Besides the religious crap, ill randomly get shit in India in hindu, having had not watched anything Indian and not even remotely Indian.

Just because you're an atheist doesn't mean you won't engage with religious content though. YT rewards all kinds of engagement not just positive ones. I.e. if you leave a snide remark or just a dislike on a religious short that still counts as engagement.

Prayers for the unbelievers makes some sense.

But I associate YouTube promotions with garbage any how. The few things I might buy like Tide laundry detergent are entirely despite occasional YouTube promotion.

[flagged]

A simple "rewatch?" line along the top would make this problem not so brain dead bad, imho. Without it you just think the algorithm is bad (although maybe it is? I don't know).