Comment by Aurornis
12 hours ago
These are small enough that you can run them entirely in the browser https://huggingface.co/spaces/webml-community/ternary-bonsai...
Remember to clear the downloaded weights afterward.
Like the last model, it's amazing they work as well as they do. Use it for any longer task and they fall apart spectacularly and in interesting ways.
So you have some fun examples?
Sadly crashing on Pixel 9 Pro, but I guess phone GPU with 16GB RAM total wouldnt be enough anyway.
Runs on my 16GB M2 Air (firefox). ~7 tok/s
It should be though.
Even native AI gallery uses smaller Gemma models. I guess whatever Chrome is using for WebGPU compute on Android is just adding too much overhead.