← Back to context

Comment by MrBuddyCasino

1 day ago

Yeah I was thinking about giving Luna a go with my PDF data extraction, but I think I‘ll stay on Gemini. It does a very good job.

Gemini is still my top choice within production software for typical data extraction from unstructured data. Gemini Flash Lite feels like a cheat code for speed, and it's really cheap.

Some other Chinese models are also fast and cheap, but a harder sell in a U.S. production environment.

  • Speaking from experience here, flash lite models have amazing price, speed, and perform far above their size, but are susceptible to very bad instruction following and recall when either complexity or context size inch up. They’ll just forget to apply your instructions to portions of the input, and repeat parts of the input that should be returned verbatim as direct quotes but with subtle changes (breaking urls, for example).

    • Yes you have to continuously tune the prompts ever so subtly. 3.5 is a lot better than than 3.1 tho.

      Important to remember that json schema instructions take precedence over the normal prompt, so move as much into property descriptions as possible.

      1 reply →

  • Yeah Gemini 3.5 Flash Lite is really good. Which Chinese models can you recommend?

Gemini is honestly an excellent LLM with many capability strengths.

For example, 3.7 Flash is #1 on MMLU Pro and AA’s agentic spreadsheets/docs benchmark, etc. Yes, beating Fable.

Agentic coding is only one dimension.

  • Anecdotally, Gemini Flash is the leader for a particular use case of mine and has been since at least version 2.5. But now there's also Luna as the first real competitor thanks to the price cut.

    My worry is that this is a zero-sum game and when Gemini catches up on coding, it'll regress to the mean in other areas.