Comment by gabiteodoru
4 months ago
Author here. Majromax challenged me to test `i = 1 + i`, which broke my theoretical framework. While setting up that experiment, I realized I hadn't used chat templates in my original measurements (rookie mistake with an Instruct model!).
Re-running with proper methodology completely flips the results - the terse version actually wins. I'll add a correction note to the article once AWS/Medium comes back online and will write a follow-up with the corrected experiments.
This is open science working as intended - community scrutiny improves the work. Thank you all for the engagement, and especially to Majromax for the challenge that led to discovering this!
No comments yet
Contribute on Hacker News ↗