← Back to context

Comment by dTal

1 month ago

The output tokens are the only information that is carried forward through each inference pass, so "more room to think" is incompatible with "basically meaningless". Perhaps one could imagine it somehow stenographically encoding information in its precise choice of meaningless throat clearing, but there are only so many variations on that theme - word choice is heavily constrained, so it doesn't feel like you could store a whole lot of information there without it starting to read froopiliciously.