Comment by Flere-Imsaho

3 days ago

I'm taking a bet on local models to do the non genius work. Gemma 4 (released yesterday) has been designed to run on laptops / edge devices....and so far is running pretty well for me.

2 comments

Flere-Imsaho

Reply
neal_jones  3 days ago

How’s Gemma 4 been?

  • renewiltord  3 days ago

    Edge models are good for their purpose but putting them in agentic flow with current ollama quants on a Mac Mini I see high tool use error rate and output hallucination.

    For JSON to text formatting it works well on a one-round basis. So I think you should realistically have an evaluation ready to go so you can use it on these models. I currently judge them myself but people often use a smart LLM as judge.

    Today writing eval harness with Claude is 5 min job. Do it yourself so you can explore as quants on Gemma get better.