Recently tried the pelican test on GPT-OSS which was probably one of the best local models of 2025. So cool to see how models have improved in the SVG pelican!
I have been overly critical and arguing in bad faith about your writing in the past. As well as negative towards you, which in turn was breeding a bad environment.
While I dont really enjoy LLMs, you did help me realize my unreasonable feelings as well as realize the occupation (and the joys I got from it) is essentially dead from it’s previous iteration and that I should let go and just join in the “I’m doing it for the money and attention” crowd. I will still just hand code my own projects and not use LLMs when I can.
I think it’s cool you started the pelican meme however useful it really is even if only aesthetically.
Do people really use 100B+ models for writing? I am no writer but to me it seems like writing is one of the easiest tasks with barely any logic or reasoning and as long as its not longer than a handful of pages I expect even 8B models to perform great.
Curious how people feel about this compared to DS4 Flash, given they are pretty close in size. Also curious how well it holds up to heavy quantization.
DS4 Flash can currently run reasonably well on systems with ~96gb+ RAM, I wonder if Hy3 can compete there.
I was playing with Hy3 via openrouter yesterday (and I've also been using DS4 Flash/Pro as a daily driver since I cancelled my Anthropic sub a week ago).
I've found DS4 Flash to be very temperental (via Claude Code). The speed is great, but it often builds a completely wrong mental model and charges off down the wrong path. I find myself needing to rein it in regularly (and also compact the history, which undercuts the whole cache price advantage).
Hy3 isn't as fast, but so far it seems to stay on track much more reliably than DS4 Flash. It also doesn't seem to degrade as much with longer context. I'm not sure what the real pricing is, but I feel like it's a very competitive model.
As an aside, I also nabbed a 50m token pack for LongCat 2.0 to give it a whirl. Not free, but it's so cheap they're basically giving it away. Very impressed too - seems roughly on par with Hy3. Not frontier-level intelligence, but a dependable workhorse that can navigate a codebase well and can reliably execute what you tell it to do.
One thing that might not be obvious about about DSV4 is how much innovation the Deepseek team implemented in its architecture. When llama.cpp fully supports its lightning indexer, the full 1M context will only require about 6G of RAM. So even though they are similar in size, I believe Deepseek will be much more efficient in that regard.
> I wonder if Hy3 can compete there
Highly depends on how well Hy3 is resilient to quantization. DSV4 is useful even at 2-bit quants.
Having heavily evaluated both antirez’s ds4 flash and Qwen 3.6 27B at FP8 and Q8: it depends. The quantised Flash is better in a number of tasks despite running much slower on my DGX Spark-alike.
27B is amazing for its size but has some surprising limits when used for longer agentic coding sessions, especially if you’re using tools that are outside the stock standard web tech stuff: it really isn’t good at Relay, for example.
I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work).
Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well.
Isn't Q8 way overkill these days? I see many graphs showing Q4 or Q5 having less than %1 deviation. Nvidia's NVFP4 Qwen quantization should be even better due to its better training methods.
Hy3 lacks the DSv4 architecture's KV Cache efficiency.
Whereas I can run DSv4 Flash on a pair of DGX Sparks and have enough memory left over for 3M tokens of KV cache, with Hy3 (quantized to FP4), there is only room for ~130K tokens of KV cache.
Lower context window notwithstanding, Hy3's coding benchmarks hold their own against DeepSeek v4 Pro & MiMo v2.5 Pro. That's quite something for a model priced like DeepSeek v4 Flash & MiMo v2.5 (for non-cached tokens), which are 3x cheaper than their respective Pro variants.
This model is shockingly small for how capable it is. its a little bit bigger than deepseekV4 flash but around as capable if not more on some benchmarks than V4 pro, i wouldnt be surprised if this becomes a popular local model.
I've been wondering about that. GLM-5.2 is also half the size of DeepSeek V4 Pro. (But costs roughly twice as much.)
I looked into DeepSeek's architecture a little bit and the main focus was how can we save as much money as possible. They did a lot of cost cutting with the attention mechanisms. This allowed them to offer an insanely cheap price even on massive contexts, but seems to have come at the cost of performance?
At least, that's my guess, when I see smaller models costing more and outperforming, I think, "they must have denser attention?"
The current Deepseek V4 Pro is still just their initial preview AFAIK, with the "real" model release rumored to come later this month. GLM-5.2 might be outperforming simply because it's had more post-training on top of the GLM-5 base.
Yeah i shouldve been more clear, a model of this size could run on 2 dgx sparks so out of the range of a lot of the typical consumer sure, but I think there is definitely a market for that size
I tried out the model it's pretty great, better than ~~gpt5.4~~ gpt-5.4-mini perhaps, atleast close enough to sonnet 5 in performance that I didn't notice much of a gap.
Not really at gpt 5.5 tier though, and probably below glm 5.2...
But most of all it just works for me for most things I tried and it's exceedingly cheap so there is no reason not to use it, if you need a foss model.
A lot of contaminated benchmarks in the blog post about Hy3, needs real testing though I have a distinct feeling it's benchmaxxed like a lot of Chinese models.
I feel like I'm taking crazy pills with hy3, it's either benchmaxxed to hell and back or skill issue on my part but I'd rather use dense gemma. I don't think there's a single model that's wasted more of my time in recent memory.
The Hy3 preview has been a mediocre performer in my benchmarks of security auditing with models, and yes, it is outperformed by Gemma 4 (31b soundly beats it, the MoE does slightly better, even at 4-bit quantization when using the QAT version). Qwen 3.6 27b also beats it.
I'll try it again now that it's out of preview and has been updated with more post-training. It presumably can't be worse, so maybe it's better enough to compete with a 31b model.
What we really need is a breakthrough in inference or LLM architecture to allow running GLM-5.2-level models at the size of Qwen 3.6 27b or smaller on consumer devices like a 48GB Macbook Pro, and at least at 100 tokens/second. My hypothesis is that a smaller, less capable but faster model paired with a good harness can run for longer and brute force its way out to solve problems that the bigger models can one-shot.
Why not! And free hot showers during the summer. Just looking at the progress made since 2023, I don't think the LLM architecture we have today is the most efficient. We need creative game developers to start making LLMs.
Been using this and GLM 5.2 back and forth. I like the speed of Hy3. Also seems very happy to follow instructions. Still haven’t found any open models that follow instructions as good as Mimo v2 pro though
MiMo v2.5 Pro is very spiky, in my experience. Sometimes excellent, sometimes mediocre. Weirdly high non-deterministic behavior. Run the same task three times, get three different results. I mean, they're all rolling dice for the next word, but MiMo seems to run hot on the randomness dimension in my benchmarks.
But, it performs very well for its size. I just looked it up, and it's much smaller than I thought it was when I was testing it. 310B A15B is tiny for how well it performs. I guess that explains why it's so cheap.
Quite interesting to see them and Meta and others release before OpenAI supposedly is to release GPT 5.6 today, would it be better to release it before or after? Calm before the storm type of thing?
It's a very good model for this size and price. I tried it with a couple of small tasks - just an year ago this would be the level of the leading models.
If they can't tell me what it is or how to use it then they can't explain to me how to install it or anything else properly so forget it. I'm not wasting my time trying to figure it out.
I'm sorry but what on earth is going on with that bar chart, the bars are not consistent. E.g., in the frontierscience-olympiad chart Hy3 preview scores the same as DeepSeek (70.0) but Hy3 preview's bar is visibly lower.
Got really excited for a minute that the long-standing [Hy](https://hylang.org) project had had a release, but it's just some confusingly-named LLM. Shame.
Pelican from a few days ago: https://simonwillison.net/2026/Jul/6/hy3/ - I was using the free tier on OpenRouter, which expires on July 21st.
I tried the preview model 41 days ago and got a pelican with a "change pelican color" button: https://static.simonwillison.net/static/2026/hy3-preview-pel...
Recently tried the pelican test on GPT-OSS which was probably one of the best local models of 2025. So cool to see how models have improved in the SVG pelican!
Curious why TFA calls out "Tencent in China".
Is there a Tencent AI lab elsewhere (MiniMax have some association with Tencent, for example)?
I have been overly critical and arguing in bad faith about your writing in the past. As well as negative towards you, which in turn was breeding a bad environment.
While I dont really enjoy LLMs, you did help me realize my unreasonable feelings as well as realize the occupation (and the joys I got from it) is essentially dead from it’s previous iteration and that I should let go and just join in the “I’m doing it for the money and attention” crowd. I will still just hand code my own projects and not use LLMs when I can.
I think it’s cool you started the pelican meme however useful it really is even if only aesthetically.
Tony the Tiger on a bicycle selling addictive items to children is not new.
A month ago I wrote a blog post about how Hy3 was topping the OpenRouter rankings despite no one talking about it: https://openrouter.ai/deepseek/deepseek-v4-flash
I had to stop using it because I was getting rate limited like crazy. Probably why it has dropped. Seemed like they couldn't keep up with demand.
That was the preview model right? This one appears to be significantly better.
I mean it's still a small model, but at least the benchmark scores (incl. on DeepSWE) went up significantly.
It costs as much as Flash, but the benchmarks are on par with Pro (or above in some cases).
Of course, benchmarks are mostly meaningless -- the only real benchmark is the actual work you give it :)
It is really slow on openrouter and I ran into many http errors.
Writes pretty engaging prose, finetunes well, now MIT licensed... what's not to like?
Oh and very good world knowledge for the size: better than than DS4 Flash
Do people really use 100B+ models for writing? I am no writer but to me it seems like writing is one of the easiest tasks with barely any logic or reasoning and as long as its not longer than a handful of pages I expect even 8B models to perform great.
6 replies →
Novita is offering free Hy3 on OpenRouter until July 21st
https://openrouter.ai/tencent/hy3:free
https://x.com/novita_labs/status/2074158304159510819
Curious how people feel about this compared to DS4 Flash, given they are pretty close in size. Also curious how well it holds up to heavy quantization.
DS4 Flash can currently run reasonably well on systems with ~96gb+ RAM, I wonder if Hy3 can compete there.
I was playing with Hy3 via openrouter yesterday (and I've also been using DS4 Flash/Pro as a daily driver since I cancelled my Anthropic sub a week ago).
I've found DS4 Flash to be very temperental (via Claude Code). The speed is great, but it often builds a completely wrong mental model and charges off down the wrong path. I find myself needing to rein it in regularly (and also compact the history, which undercuts the whole cache price advantage).
Hy3 isn't as fast, but so far it seems to stay on track much more reliably than DS4 Flash. It also doesn't seem to degrade as much with longer context. I'm not sure what the real pricing is, but I feel like it's a very competitive model.
As an aside, I also nabbed a 50m token pack for LongCat 2.0 to give it a whirl. Not free, but it's so cheap they're basically giving it away. Very impressed too - seems roughly on par with Hy3. Not frontier-level intelligence, but a dependable workhorse that can navigate a codebase well and can reliably execute what you tell it to do.
> given they are pretty close in size
One thing that might not be obvious about about DSV4 is how much innovation the Deepseek team implemented in its architecture. When llama.cpp fully supports its lightning indexer, the full 1M context will only require about 6G of RAM. So even though they are similar in size, I believe Deepseek will be much more efficient in that regard.
> I wonder if Hy3 can compete there
Highly depends on how well Hy3 is resilient to quantization. DSV4 is useful even at 2-bit quants.
That's a 2-bit quant of DS4 flash. You're probably better off running Qwen3.6-27B at Q8.
Having heavily evaluated both antirez’s ds4 flash and Qwen 3.6 27B at FP8 and Q8: it depends. The quantised Flash is better in a number of tasks despite running much slower on my DGX Spark-alike.
27B is amazing for its size but has some surprising limits when used for longer agentic coding sessions, especially if you’re using tools that are outside the stock standard web tech stuff: it really isn’t good at Relay, for example.
I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work).
Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well.
2 replies →
For most coding or agentic tasks, Qwen 3.6 27B likely outperforms, yes.
For 'general intelligence', DS4 Flash seems to be a noticeable step up still.
I suspect it would depend on the task. DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent.
qwen 27b at q8 is slower and worse than ds4 at q2 in my experience.
Isn't Q8 way overkill these days? I see many graphs showing Q4 or Q5 having less than %1 deviation. Nvidia's NVFP4 Qwen quantization should be even better due to its better training methods.
3 replies →
[dead]
Hy3 lacks the DSv4 architecture's KV Cache efficiency.
Whereas I can run DSv4 Flash on a pair of DGX Sparks and have enough memory left over for 3M tokens of KV cache, with Hy3 (quantized to FP4), there is only room for ~130K tokens of KV cache.
Lower context window notwithstanding, Hy3's coding benchmarks hold their own against DeepSeek v4 Pro & MiMo v2.5 Pro. That's quite something for a model priced like DeepSeek v4 Flash & MiMo v2.5 (for non-cached tokens), which are 3x cheaper than their respective Pro variants.
1 reply →
DS4-Flash is not only "significantly" smaller, it will also benefit from a lot more speed thanks to DSpark
299B for Hy3 vs 284B* for Flash
Edit: fixed, got bad info
2 replies →
I don’t like DS4 in my experiences with it I still prefer qwen locally and glm on api
This model is shockingly small for how capable it is. its a little bit bigger than deepseekV4 flash but around as capable if not more on some benchmarks than V4 pro, i wouldnt be surprised if this becomes a popular local model.
I've been wondering about that. GLM-5.2 is also half the size of DeepSeek V4 Pro. (But costs roughly twice as much.)
I looked into DeepSeek's architecture a little bit and the main focus was how can we save as much money as possible. They did a lot of cost cutting with the attention mechanisms. This allowed them to offer an insanely cheap price even on massive contexts, but seems to have come at the cost of performance?
At least, that's my guess, when I see smaller models costing more and outperforming, I think, "they must have denser attention?"
The current Deepseek V4 Pro is still just their initial preview AFAIK, with the "real" model release rumored to come later this month. GLM-5.2 might be outperforming simply because it's had more post-training on top of the GLM-5 base.
1 reply →
hardly, its still quite big unless by "local" you mean people that spend many thousands on rigs :)
Yeah i shouldve been more clear, a model of this size could run on 2 dgx sparks so out of the range of a lot of the typical consumer sure, but I think there is definitely a market for that size
> Hy3 has 295B parameters in total. To serve it on 8 GPUs, we recommend using H20-3e or other GPUs with larger memory capacity.
I would.
I tried out the model it's pretty great, better than ~~gpt5.4~~ gpt-5.4-mini perhaps, atleast close enough to sonnet 5 in performance that I didn't notice much of a gap.
Not really at gpt 5.5 tier though, and probably below glm 5.2...
But most of all it just works for me for most things I tried and it's exceedingly cheap so there is no reason not to use it, if you need a foss model.
Edited: gpt-5.4-mini not the base gpt-5.4
Hy3 DeepSWE - 28%
GPT5.4 xhigh DeepSWE - 52%
A lot of contaminated benchmarks in the blog post about Hy3, needs real testing though I have a distinct feeling it's benchmaxxed like a lot of Chinese models.
I think you’ve got the models wrong…gpt-5.4? I doubt there is any open source mode matching it. Maybe in a year
Yeah I meant gpt-5.4-mini, but GLM 5.2 is pretty close to gpt-5.4 base, and much better than it when it comes to design stuff.
GLM 5.2 already matches GPT-5.4 easily.
I feel like I'm taking crazy pills with hy3, it's either benchmaxxed to hell and back or skill issue on my part but I'd rather use dense gemma. I don't think there's a single model that's wasted more of my time in recent memory.
The Hy3 preview has been a mediocre performer in my benchmarks of security auditing with models, and yes, it is outperformed by Gemma 4 (31b soundly beats it, the MoE does slightly better, even at 4-bit quantization when using the QAT version). Qwen 3.6 27b also beats it.
I'll try it again now that it's out of preview and has been updated with more post-training. It presumably can't be worse, so maybe it's better enough to compete with a 31b model.
For context Hy is 295B, roughly on par with DeepSeek V4 Flash (284B).
I haven't tested it yet so I cannot comment on the quality (nor the comparison with 10x smaller (!) models)
2 replies →
I was expecting a new release of the Hy language (https://hylang.org).
Yes, I was gonna say... Today, I am this old.
What we really need is a breakthrough in inference or LLM architecture to allow running GLM-5.2-level models at the size of Qwen 3.6 27b or smaller on consumer devices like a 48GB Macbook Pro, and at least at 100 tokens/second. My hypothesis is that a smaller, less capable but faster model paired with a good harness can run for longer and brute force its way out to solve problems that the bigger models can one-shot.
That would be great during the winter months
Why not! And free hot showers during the summer. Just looking at the progress made since 2023, I don't think the LLM architecture we have today is the most efficient. We need creative game developers to start making LLMs.
1 reply →
im more expecting the harness to be a literal LLM, Like how you put vibration dampeners on all kinds of mechanical structures
This site can’t be reached https://hy.tencent.com/research/hy3 is unreachable.
Been using this and GLM 5.2 back and forth. I like the speed of Hy3. Also seems very happy to follow instructions. Still haven’t found any open models that follow instructions as good as Mimo v2 pro though
MiMo v2.5 Pro is very spiky, in my experience. Sometimes excellent, sometimes mediocre. Weirdly high non-deterministic behavior. Run the same task three times, get three different results. I mean, they're all rolling dice for the next word, but MiMo seems to run hot on the randomness dimension in my benchmarks.
But, it performs very well for its size. I just looked it up, and it's much smaller than I thought it was when I was testing it. 310B A15B is tiny for how well it performs. I guess that explains why it's so cheap.
Er, actually, Pro is a big 1T model (which makes more sense given how well it does, sometimes). Regular MiMo is small, and I haven't tested it.
Quite interesting to see them and Meta and others release before OpenAI supposedly is to release GPT 5.6 today, would it be better to release it before or after? Calm before the storm type of thing?
It's a very good model for this size and price. I tried it with a couple of small tasks - just an year ago this would be the level of the leading models.
Congrats, the trial chat is QR locked. The AI companies are spoiled and get crazier every day (not only this one, US companies as well).
I would never use any product that can't explain on its own front page what it is and why I should use it.
It's like a computer, but you can talk to it.
If they can't tell me what it is or how to use it then they can't explain to me how to install it or anything else properly so forget it. I'm not wasting my time trying to figure it out.
That UI demo page is… really quite janky.
Very impressive model for its size
I'm sorry but what on earth is going on with that bar chart, the bars are not consistent. E.g., in the frontierscience-olympiad chart Hy3 preview scores the same as DeepSeek (70.0) but Hy3 preview's bar is visibly lower.
Got really excited for a minute that the long-standing [Hy](https://hylang.org) project had had a release, but it's just some confusingly-named LLM. Shame.