{"id":49707528,"user":"erickhill","time":1789444699,"time_ago":"8 hours ago","type":"comment","content":"

With emoji for the bullets, natch.","comments":[{"id":49707955,"user":"TomK32","time":1789448762,"time_ago":"7 hours ago","type":"comment","content":"

Just put "Don't use emojis" into the prompt.","comments":[{"id":49708598,"user":"tacomagick","time":1789454347,"time_ago":"5 hours ago","type":"comment","content":"

Pass it as a memory or a system prompt and watch it forget 3 agent messages later.","comments":[{"id":49708930,"user":"mxmilkiib","time":1789456760,"time_ago":"5 hours ago","type":"comment","content":"

what frontier models are this bad these days?","comments":[{"id":49709068,"user":"tacomagick","time":1789457830,"time_ago":"4 hours ago","type":"comment","content":"

I can't name one but I can tell you these models keep getting stupider as they keep existing and their system prompts get inflated with what not to do. I have experienced this happen with Qwen 3.8 Max Preview where for the early days it was blasting through Rust code fine and then before the release it silently got replaced and started to suck hard. Claude still has the "blue button" problem so it is probable that a casual chat with it could have this exact outcome of forgetting what to do. I saw some models also have this thing where when you tell them a text restriction they first ignore it in thinking, creating the full text and then apply them in their thinking rewriting it fully causing your output tokens to be inflated.","comments":[{"id":49710471,"user":"fennecfoxy","time":1789469376,"time_ago":"an hour ago","type":"comment","content":"

Well yes, there are only so many attention heads (well whatever magical variant frontier models are using these days) that can attend to the context and so as the context grows attention becomes spread thin.

But with reasoning enabled I find that even with a large context that induces mistakes most of the time a decent model realises and corrects itself before output.

And it's always been known that prompting what it should do is far better than what it shouldn't, since just introducing "DON'T do X" into the prompt means that the tokens for X are present and can be paid attention to in the wrong way.

But even then I've used plenty of "Do X, not Y" recently, especially for tools "Use this for x, don't use this for Y" and models perform like 90% of the time.

I would be interesting to experiment to see performance curves given a restriction on reasoning tokens allowed to n% of context tokens and see if there's some magic number of "reasoning should be at least n tokens for a context of length p" even ignoring the complexity of instructions in the prompt itself.","comments":[],"comments_count":0,"level":4,"url":"item?id=49710471"}],"comments_count":1,"level":3,"url":"item?id=49709068"}],"comments_count":2,"level":2,"url":"item?id=49708930"}],"comments_count":3,"level":1,"url":"item?id=49708598"}],"comments_count":4,"level":0,"url":"item?id=49707955"},{"id":49707563,"time":1789445032,"time_ago":"8 hours ago","type":"comment","content":"[deleted]","deleted":true,"comments":[],"comments_count":0,"level":0,"url":"item?id=49707563"}],"comments_count":6,"url":"item?id=49707528"}

Comment by erickhill