Comment by spwa4
4 days ago
https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B...
816B model, A42B active parameters
Unsloth and Meta have guides on how to run the model, but allow me to summarize: You & Me won't be running this one.
4 days ago
https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B...
816B model, A42B active parameters
Unsloth and Meta have guides on how to run the model, but allow me to summarize: You & Me won't be running this one.
I do not know about you, but I will probably also run this model, even with a mini-PC like an ASUS NUC, albeit very slowly.
Even if for your main LLM you want a speed high enough for interactive use and fast iterations, it is still useful to also run a much bigger LLM from time to time, even if that requires many hours for completing a task, due to the slow inference speed when the weights are read from SSDs.
There are tasks where the quality of the result is more important than the time needed to produce it. For example you might run all day a great number of iterations of a fast small LLM, and then run overnight a big LLM to check the end results of that work.
Therefore we need both small open-weights LLMs that can be run from the GPU RAM or at most from the CPU RAM, but also the biggest open-weights LLMs, even if the latter can be run only from SSDs by most people.
> The 2-bit quant fits on devices with 256GB of RAM, VRAM or unified memory.
Ouch. I wonder how it compares to Qwen 3.8 running with 4-bit quant on 10 times less VRAM.
Not at an appreciable token rate. I wonder if these large open weight models are considered safer to release for that reason? I'm thinking no because any small to medium size criminal organization or state funded group will be able to cobble together the resources to achieve some kind of runtime. Even at 10 tok/sec, making bioweapons at 10 tok/sec is concerning.
You do realize that a high school Chemistry course actually contains all the details you need on both bioweapons used in WW2, right? Plus several others you could use. Those books contain the answer to why those 2 people died in the RV in Breaking bad. And they should. They also contain information on Mustard gas and Zyklon-B, including preparation instructions, but of course, you'll have to puzzle it together. That these substances can be used as bioweapons is heavily deemphasized, but you just can't reasonably avoid mentioning and showing them.
Also these books contain maybe not the solution to making thermobaric explosions, but certainly the principle and show you exactly what problem you'd have to solve to make those weapons.
Just about the only thing they stopped teaching is that if you combine that knowledge with which plants contain what ... there's an incredibly easy way to make pudding with cyanide that's safe to touch, for example. That they suppressed.
But at least for Europe, the number of lethal plants just growing along essentially every road or in every forest numbers in the dozens. Including a very famous one they teach every kid about.
Of course, in both cases, the real challenge is making them and deploying them without killing yourself.
We are not safe from bioweapons because nobody knows how to make them. That's not it.
It seems like you're talking about chemical weapons.
Which is fair, they are very easy to make without the assistance of an LLM, and I don't hear about that kind of thing in the news.
I was talking about biological weapons like gain of function and that kind of thing.
The implied question was 'Should companies like Meta publish open weight models when they could theoretically be subverted by malicious users?'. So far as I'm aware there are few people worrying about this kind of thing. Bill Gates for one. But apparently it's off topic and/or not interesting, or not welcomed as a discussion point.
2 replies →