Comment by unbrice

2 hours ago

I asked Gemini to review the repo. Apparent issues include:

  - Corrupted/dropped Gemini thought_signature (reasoning_details) in tool loops
  - Blinded model.api.id = "contestant" bypasses gemini.txt and serves Claude's prompt (default.txt)
  - Anthropic-only prompt caching

The full report is here, if you're interested: https://docs.google.com/document/d/1-HqujXuDRJSWJIqEvmctcINr...

Prompt was:

> I read on hacker news the following, which does not match my expectations. Check if something in the harness or bench is hurting gemini perf.

followed by your post.

1 comment

unbrice

seizethecheese 1 hour ago

Wow, cool, thanks!

> anthropic only prompt caching

Anthropic has custom prompt cache requirements, but we've verified all providers are working well.

> Blinded model.api.id = "contestant" bypasses gemini.txt and serves Claude's prompt (default.txt)

This is wrong. We're serving default.txt which is opencode's default prompt. We are not serving a custom prompt for any model, including Gemini.

> Corrupted/dropped Gemini thought_signature (reasoning_details) in tool loops

We blind the reasoning details in the frontend because this would leak information. Some models summarize thinking while others provide raw thinking. I'll check if we're corrupting the thought signature that's actually sent to the backend.

So far this sort of looks like an example of Gemini being bad. Was this Argon?