Comment by skydhash
2 days ago
Or you could just read the PPP RFC [0].
I’m not saying that your approach is wrong. But most LLM workflows are either brute forcing the solution, or seeking a local minima to be stuck in. It’s like doing thousands of experiments of objects falling to figure out gravity while there’s a physics textbooks nearby.
Ironically, I could’ve read all 50 pages of that RFC and still missed the actual issue. What really helped was RFC 1331[0], specifically the "Async-Control-Character-Map" section.
That said, I’m building a product - not a PPP driver - so the quicker I can fix the problem and move on, the better.
[0] https://datatracker.ietf.org/doc/html/rfc1331
I could also walk everywhere, but sometimes technology can help.
There’s no way I could fully read that RFC in an hour. And that’s before you even know what reading to focus your attention on, so you’re just being a worse LLM at that point.
The difference is you’d remember some of the context from reading the thing where an LLM is starting from scratch every single time it comes up.
There are opportunity costs to consider along with relevance. Suppose you are staying at my place. Are you going to read the manual for my espresso machine in total or are you going to ask me to show you how to use it or make one for you?
In any case, LLMs are not magical forgetfulness machines.
You can use a calculator to avoid learning arithmetic but using a calculator doesn’t necessitate failing to learn arithmetic.
You can ask a question of a professor or fellow student, but failing to read the textbook to answer that question doesn’t necessitate failing to develop a mental model or incorporate the answer into an existing one.
You can ask an LLM a question and blindly use its answer but using an LLM doesn’t necessitate failing to learn.
3 replies →
All of the above is true, but between solving quicker, and admitting we gave context:
I do agree with you that an LLM should not always start from scratch.
In a way it is like an animal which we have given the ultimate human instinct.
What has nature given us? Homo Erectus is 2 million years ago.
A weird world we live in.
What is context.