Comment by bbor

I don’t have much of a response but this comment is very well written and exactly what I would say if I could write clearly. It’s an exciting time! Hopefully it turns out more like the internet and less like… idk I’m struggling to think of a bad invention haha. Quadcopters I guess

Very true :). It doesn’t help that this isn’t exactly a little blog post, it’s a popular New Yorker feature…

Very true, but it is hard not be frustrated reading the constant stream of confidently incorrect information about this topic. I left /r/programming similar reasons and it is sad to experience the same on HN.

If you did then the LLM might stop hallucinating so much.

obligatory https://xkcd.com/386/

When my pedantic keyboard warrior gears start turning, I think about the same xkcd a sibling commenter posted.

But I've been struggling with the recent tsunami of openly anti-intellectual, alt-everything pseudoscience in the US. I recently pleaded with my sister to read The Demon-Haunted World by Sagan.

There's something important in patient advocacy of truth.

The most potent use case of the current implementation of ChatGPT is plagiarism.

Businesses who need blog posts for content marketing would hire copywriters who would paraphrase my content and publish it as their own. Now they can do it faster and for free. Blogspam got cheaper and more effective. This applies to other kinds of spam too: email spam, forum spam, comment spam... Spam spam spam. I'm afraid that we'll soon drown in even more noise.

Again, this is just the current implementation, which has not reached its full potency. We're already seeing this combined with voice and video generation. Then it's students doing their homework with it, and even professionals.

This is a game changer. You can argue that it makes the game better or worse, but you can't deny that it's changing the rules.

its no fad. I use to spend an hour going over an email to management. make it less technical, make it sound nicer / polite, etc. now I can take a sentence and say 'make this more succinct. ok, but take that and make it more polite, great thanks'.

I even used it for project proposals. 'write me a 1 page document for this idea'. and then I just use the scaffolding from there. its a huge time saver.

also you are notn seeing the bigger picture. "I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. " do you realize how this is going to make writing (books, songs, tv shows, movies) so much easier?

I had a dream, all I did was give chatgpt the basics of the dream and I said make this into a short story. the results weren't bad, its definitely something one can work with. I think content creation on platforms is about to explode.

Want niche use case? I feed chatgpt the whole warstuff one page rule miniature combat system, and I can ask it to generate thematic units.

I generated a wide variety of content from LOTR to ships.

it even created new trait to complement the existing one when needed by special units, along with explanation of the mechanics of the trait.

It doesn't quite understand positioning, but it will simulate round of combat between unit when asked.

I have openly used LLMs to build a custom prompt engine that allows me to make systemic refactors across an entire open source codebase of mine (of 250 files) with a bash command, in parallel.

There are big changes on the horizon.

You’re missing an important point - it’s not /trained/ on live internet content, it /reads/ that content at runtime. I mean it is trained on the internet but please try to separate the concerns. Remember that the goal of this model is language, not learning facts about the world - they could’ve trained it completely on fictional novels if there was a big enough corpus.

The only way that LLM-enhanced search returns misinformation is if the internet is full of misinformation. So yeah we’re still in trouble, but the inclusion of the LLM isn’t going to affect that factor either way IMO

EDIT: this is completely separate from using LLMs to, say, write political statements for Facebook bots and drown out all human conversations. That’s obviously terrifying, but not related to their use in search engines IMO.

Hmm. I’d say definitely yes. I mean we are on the internet right now, presumably across many km of distance. Do you disagree?

In my view it’s very simple, which is what makes it so exciting. Summarizing a design doc that I imagine Microsoft and google are both spending millions of dollars of manhours/day working on their versions of:

1. User enters query.

2. LLM augments query if necessary, adding extra terms or clauses.

3. Normal search pipeline returns ranked links, just like it does now.

4. LLM reads the content of the first 100 links and decides which are the best based on your stated preferences and past behavior, uses that to augment the ranking a bit.

5. LLM generates various summaries depending on the type of query, such as laying out a few common answers to a controversial political question or giving a summary of a technical Wikipedia article tailored to your expertise level in that field.

6. Finally, for a tiny subset of queries, maybe the user wants to converse with the AI in a chat-like format, where it cites all of its claims with direct links.

It’s gonna be awesome :)

Wouldn't a connection to Cyc be a million percent more useful ?

Very true - I’m stealing that rule!

Well, 30 years later and they still can't. At least I know of no one who has a house cleaning robot. (I wouldn't classify Roomba as 'clean my house').

> It's nothing like described in the article and I don't understand why people who should know better don't call out the bullshit media reporting more.

I'm kind of assuming you didn't read the article, but if you did then I'm kind of assuming that you've never done machine learning, but if you have: how did you manage to do that without ever noticing that you were doing approximation?

Objectively, neural networks are approximators. Like, truly objectively, as in, the literal objective function, objectively minimizes approximation error. We call them objective functions and minimizing the approximation error is typically the objective of these objective functions. This isn't bullshit. It isn't. If you think it is, you are deeply and profoundly mistaken.

The article advances this view of language models. This is a reasonable view of language models for the same reason that machine learning papers exploring neural networks describe them as universal function approximators.

> We've had GPT3 for ages, it's not like most of us only tried it since Chat GPT3 came out, right?

For myself, I tried GPT models and read the Attention Is All You Need paper before ChatGPT. I also read analysis, for example, from Gwern, about capability overhangs and underestimation of present capabilities in these models. In many cases, I found myself agreeing with the logic. I found that it was very possible to coax greater capabilities out of the model than many presumed them to have. I still find this to be the case and have, in recent memory demonstrated this is true in present models: for example, I posted a method of coaxing the solving of some puzzles by prompting to include the representation of intermediate states in order to successfully solve problems related to reasoning puzzles of a 'can contain' nature, which was a capability that someone claimed these language models lack, despite them gaining that capability when appropriately prompted, which suggests that they always had that capability in their weights, but that it wasn't exercised successfully - the capability was there, but not used, rather than absent, as claimed by the people who claimed it was absent.

That said, I don't think it matters much what most people did or didn't do with regard to this experimentation and, as you imply, ages really did past - I would feel trepidation, not hope, about the quality of my ideas compared to the people who came later. Historically, the passing of ages tends to improve, not diminish. So if I was experimenting with, for example, flying machines in the 1700s, but then ages past and someone who did not do that experimenting was talking to me about flying machines in the early 2000s, I would suspect them to be more informed, not less informed, than I was. They, as a matter of course in casual classroom settings, have probably done better than my best experiments including high effort costly experiments. Their toys fly. A generation ago, we would talk about planes, but now we can also talk about their toys. It is that normal to them. They have so much better priors.

> Anyone who's played around with it knows that it's fun but it's not a search engine replacement and it doesn't know nor understand things.

I've noticed that some people have been talking as if the view is nonsense in the direction you imply it is nonsense, but I think the argument for a lack of sensory ties is much stronger when stated in the other direction.

This fails when arguing against the straw man position, but of course, drinking water also fails when addressing the weakest possible justification for it - clearly drinking water is bad because if you drink water you die of osmosis, though technically true, isn't really reflective of the actual predictions made by the people advancing the claim that we ought to drink more water than we do.

So I'll give the argument for language models replacing search engines not being nonsense, but the position that no one could arrive at the belief actually being nonsense.

Lets start with people not arriving at that belief being nonsense. I think it is nonsense in two ways. One way is that because some people have used them, then claimed they would be replacing search engines for some queries, it follows that you deny your senses with regard to the existence of these people. This is not sensory tied. So it is non-sensical. The second way is your belief anticipates experiences, but this experience you are now having in which someone disagrees with you is in contradiction to the anticipated experienced supposed by the belief. So it fails to predict past experiences and fails to predict current experiences, so it probably fails to predict future experiences. There probably will be someone in the future who, after interacting with a language model, thinks it can serve as a replacement for some search engine queries.

Now in contrast, their claims were not non-sensical when they thought they could replace some search engine queries with language model queries. By that I also mean that they were not non-sensical in two respects. The first is that they arrived at the belief after replacing queries to search engines with queries to language models. Then, after having found value in doing so, they announced a belief congruent with that value, intimately connecting their belief to their senses. The second is that, having arrived at their belief, their belief paid rent in anticipated experience: it successfully predicted three noteworthy events: the internal code red status at Google, you.com's language model as enrichment for search, and bing.com's use of language model as enrichment for search. So it successfully predicted past experiences, successfully predicted current experiences, and may or may not predict future experiences - I think most people who hold this view tend to think there will be some refinements on the current generation of language models, in particular, that they will further refined at query time with relevant data, like fact databases, to help correct for some of the existing approximation error. This is my belief. I anticipate this happening. You can judge whether I anticipate successfully by watching reality. I should note that part of the reason I think this will happen is because I've already seen it happen. So I'm not really making a bold prediction here, but I suspect you will later see this happen, because I've seen this happen.

Anyway... the belief that language models can replace some search queries is not a non-sensical belief, like beliefs in fairies is, but the belief that no one arrives at a belief that they could be is a non-sensical belief, because people do arrive at such beliefs, and so therefore the belief that they don't is fanciful and not reflective of reality.