← Back to context

Comment by oblio

3 days ago

LOL. Tolkien invented multiple languages. There are/were probably 100 000 languages on this planet. That's without including dialects.

We use existing languages not because we love to copy, it's because languages are a medium for communication. They're only useful if they're understood by others.

If you wanted to prove a point: writing is much, much harder to invent, yet it was invented independently at least 3 times in history. By definition LLMs can't invent something truly novel like writing. Because nobody trained the first writers. There was no "training set".

It's fine, LLMs can still be useful. Science is looking at the gaps and most of the gaps that are useful are small and LLMs can explore those faster than us. That could potentially improve many lives. Though not in this corporate LLM world, most likely.

10 comments

oblio

Reply
criley2  3 days ago

I don't think the invention of writing is as awe-inspiring as presented or as difficult/impossible for an LLM to achieve as is commonly believed. The invention of writing was a long series of micro-improvements over common every day things. Let's keep track of count by writing a line in the sand. Let's cut it into a tree. Let's push the line into clay. Let's use two lines touching. These are the kinds of things an LLM can also reason through and achieve.

I reject this idea that an LLM couldn't come up with something novel that isn't in the training data. We know it can come up with sentences that aren't in there. We know it can come up with mathematic proofs that aren't in there. So could it feasibly create something as fundamental as writing or language? Would we even be able to understand what it had done, or would it be so foreign to us that we wouldn't even realize?

Those early writers did have a training set of human knowledge that led them to pushing stones into clay. And our agents may very well have a set of knowledge that leads it to pushing their proverbial stone into their proverbial clay too.

  • skydhash  3 days ago

    It can come up with something novel, but only by combining whatever present, which can be granular enough to be completely different. But to truly nudge it, you need precise prompting and a good harness. But I’m more likely to credit those and the people creating them. As well as the data that goes in the training. I won’t praise Fable or Opus. It would be like praising the pencil or the hydraulic press.

    I think the Turing Machine, the Von Neumann Architecture, …, Unix, …, The IBM PC,… as more awe-inspiring because those has truly helped humanity. I still can’t see the positives of LLM technologies.

    • jason_oster  2 days ago

      > It can come up with something novel, but only by combining whatever present

      That's what artists do. Long before LLMs, Kirby Ferguson made this observation in "Everything is a Remix" [1] in the context of Copyright and IP disputes. He has since updated it with a chapter explaining how GenAI works on the same principles. In short, "combining whatever present" is the only source of novelty.

      [1]: https://www.everythingisaremix.info/watch-the-series/

bbor  3 days ago

1. Languages are used for much, much more than communication, and there is good reason to think that capacity didn't evolve for communication, but rather for metacognitive problem solving.

2. The idea that Tolkein --one of the most reknowned writers in our entire canon-- did something isn't relevant to the philosophical discussion of creativity and its obvious bounds. Even Tolkein was just painting within the lines set by the linguists he had read and the obscure languages he adored, not to mention the fundamental limits set by our capacity for language in the first place.

For anyone who's curious about this kind of thing, I cannot reccomend the infamous debate b/w Chomsky & Foucault enough; it ranges across topics a bit, but chapter markers should help you skip to the core of it (creativity) if you prefer. Video here: https://www.youtube.com/watch?v=3wfNl2L0Gf8 , some old summaries on /r/AskPhilosophy here: https://www.reddit.com/r/askphilosophy/comments/vgz1vb/what_...

jefftk  3 days ago

Are you saying your bar for being impressed by an LLM is something equivalent to the initial invention of writing?

  • oblio  3 days ago

    LLMs can be quite impressive with much more mundane accomplishments.

    The bar for what OpenAI, Anthropic, Google & co are trying to sell me, all of us, yes, for sure. Inventing writing is even too low a bar.

  • teiferer  2 days ago

    > your bar for being impressed by an LLM

    Please don't move the goal posts. If you read my post again you will notice that I explicitly state that I'm impressed. This is not about whether somebody is impressed but about whether "it created its own Xyz" is a suitable description.