Comment by gabriel666smith

I understand that point of view.

Perhaps this is contextually useful - when writing prose fiction, one technique I've played with recently which I found interesting is generating a really broad spectrum of 'next tokens' halfway through a sentence, via multiple calls to different models on different temp. settings, etc.

It's fascinating to see the expected route for a sentence, and (this is much harder to get LLMs to output!) the unexpected route for a sentence.

But seeing some expected routes, per the LLM, can make the unexpected, surprising, or interesting routes much more clear in the mind's eye. It makes sentences feel closer to music theory.

You are right that this does create a more 'editorial' relationship between yourself and the work.

I'd stress that this isn't a negative thing, and has heavy literary precedence - an example that comes to mind is Gordon Lish's "intuitive structuring" principle, in which you just write the best-sounding next word, and see what the story becomes by itself, then edit from there - a totally sonic approach.

My example here with "arrays of next tokens" is a super granular, paintbrush-type example, but I want to be clear that I'm not at all advocating for the workflow of 'write a prompt, get a piece of art'.

I do however think that there's a vast middleground between "write me a whole book" and "show me the expected next token", and that this middleground is absolutely fascinating.

Not least because it makes literature (an artform previously more resistant to mathematics than say, music, or painting) more in touch with its own mathematics, which were previously very hidden, and are only currently being discovered.