Comment by WoodenChair
18 hours ago
I guess I don't understand the point of this. Of course a set of digital logic circuits can be reduced to a set of weights. Am I missing something?
18 hours ago
I guess I don't understand the point of this. Of course a set of digital logic circuits can be reduced to a set of weights. Am I missing something?
Apparently it's news to some people. It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.
Doesn't help much
> It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.
Reality says otherwise, especially from those who run LLMs in production at a large scale. Hacks such as baking in the weights with tiny experiments doesn't disprove anything.
The same LLM boosters also continue to repeat one of the biggest misconceptions: "sEt TeMperaTurE tO zERo tO aChiEvE fUlL dEterMinIsm."
The accurate statement is that LLMs are not inherently non-deterministic. From a mathematical perspective, they’re obviously deterministic. But choices in the way we deploy them introduce non-determinism.
Sometimes projects have no point other than to show what can be done – and the pure thrill of doing is an end in itself … but I agree with you in the "so what" sense here … I feel like with this project there is meant to be a point (other than play for the sake of play and art for the sake of art and coding for the sake of coding) but I'm not seeing it either?
Because it's fun
This work, and the original qwenputer work it builds on, are both technically uninteresting slop. HN eats it up though.
HN loves slop meetoo projects without questioning the details. With close inspection it is an overly complicated version of this paper [0] on neural turing machines.
This concept was hyped up (because DeepMind) for a while at the time and in reality it amounted to nothing. This is just the second reincarnation of a similar concept but with added extra saturated slop and hype on top.
[0] https://arxiv.org/abs/1410.5401
That paper is great work. This qwenputer stuff is slop. The problem is that HN can't tell the difference.
Not a climber then?