Comment by Powdering7082
10 hours ago
This was using llama-serve on my local which by default gave it current time & unhelpfully "get_info" which is specific to the browser
The full request was
``` {"messages":[{"content":"What animal lays the biggest egg?","role":"user"}],"return_progress":true,"sse_ping_interval":1,"stream":true,"tools":[{"function":{"description":"Returns the current local date and time in ISO 8601 format, with the IANA time zone name","name":"get_datetime","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"},{"function":{"description":"Get runtime info (OS name), may call when user asks about local files or shell commands","name":"get_info","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"}],"reasoning_format":"auto","reasoning_control":true,"backend_sampling":false,"timings_per_token":true} ```
Disabling tools removes the bit about mentioning instructions.
Just an interesting detail I noticed.
Do y'all have any penalty/regularization w.r.t. reasoning traces or is it pure SFT -> RL?
interesting, I didn't see it on the playground or on my local llama-server.
SFT -> RL. In our fine-tuning notebooks, we typically say that rollouts that go past the max context length (too much reasoning) get 0 reward, so there is slight pressure for shorter reasoning, but I don't think we have strong regularization. I think the agreed-upon recipe is to first make the model smarter (longer reasoning), then teach it to reason faster/denser.