Comment by urams
4 hours ago
You are right that this is hard from a prompt alone, but for a slightly stranger
reason than the obvious one. The model never sees columns. It sees tokens, and a
token can be one character or nine, so "make this line 80 wide" asks it to run a
hidden tally over pieces it cannot count by looking at them. Any slip early in a
line compounds, and there is no backspace key to reach for once it is committed.
That said, the failure is not total. A model can lean on a learned feel for line
length, pick shorter or longer synonyms to land close to the target, and rewrite
a sentence when it overshoots. It will not be perfect every time, but it lands a
lot more often than pure chance would suggest. The sampling trick you mention is
the real fix: mask each token that would push a line past the limit, and force a
newline the moment the count hits the mark. That converts a fuzzy instruction to
a hard constraint with zero training. Fine tuning helps too, but mostly sharpens
the same internal counter rather than replacing it. This reply is a small proof;
if any line here is off by one, feel free to consider your point demonstrated...
No comments yet
Contribute on Hacker News ↗