Comment by GenerWork
1 day ago
As soon as a local LLM that can match Claude Codes performance on decent laptop hardware drops, I'll bow out of using LLMs that are paid for.
1 day ago
As soon as a local LLM that can match Claude Codes performance on decent laptop hardware drops, I'll bow out of using LLMs that are paid for.
I don't think that's a realistic expectation. Sure, we've made progress wrt smaller models being as capable as larger ones three years ago, but there's obviously a lower limit there.
What you should be waiting for, instead, is new affordable laptop hardware that is capable of running those large models locally.
But then again, perhaps a more viable approach is to have a beefy "AI server" in each household, with devices then connecting to it (E2E all the way, so no privacy issues).
It also makes me wonder if some kind of cryptographic trickery is possible to allow running inference in the cloud where both inputs and outputs are opaque to the owner of the hardware, so that they cannot spy on you. This is already the case to some extent if you're willing to rely on security by obscurity - it should be quite possible to take an existing LM and add some layers to it that basically decrypt the inputs and encrypt the outputs, with the key embedded in model weights (either explicitly or through training). Of course, that wouldn't prevent the hardware owner from just taking those weights and using them to decrypt your stuff - but that is only a viable attack vector when targeting a specific person, it doesn't scale to automated mass surveillance which is the more realistic problem we have to contend with.