Yes. The main problem is that they can only lead you to familiar territory. The next problem is that it's hard to notice when you are back where you started.
This has been my main struggle using LLMs to soundboard my new idea. They can write an eloquent interpretation of the entire concept, but as soon as we get to implementation, it stumbles right into creating the very system I intend to replace.
So I would say it's the other way around: LLMs are an excellent map, but a terrible compass. Good enough if you want to explore familiar territory, but practically unusable on an adventure.
Yes. The main problem is that they can only lead you to familiar territory. The next problem is that it's hard to notice when you are back where you started.
This has been my main struggle using LLMs to soundboard my new idea. They can write an eloquent interpretation of the entire concept, but as soon as we get to implementation, it stumbles right into creating the very system I intend to replace.
So I would say it's the other way around: LLMs are an excellent map, but a terrible compass. Good enough if you want to explore familiar territory, but practically unusable on an adventure.