Comment by Arcuru
3 days ago
It's good to see this getting some continued development. I looked into it last year[1] and I thought it showed a lot of promise so I've been very disappointed that I never saw a newer model.
3 days ago
It's good to see this getting some continued development. I looked into it last year[1] and I thought it showed a lot of promise so I've been very disappointed that I never saw a newer model.
I think this approach is not so interesting because it's just quantization of a full precision model. So it speeds up inference (at a quality penalty) but not training. It would be more interesting to train an actually binary model directly, without any floating point multiplication, like in this paper: https://proceedings.neurips.cc/paper_files/paper/2024/hash/7...