Comment by oulipo2
7 days ago
Exactly, LLM is good at "code inpainting" : define clear structures and goals, and it will fill the boilerplate. But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views. But that's integral to the way it's designed and has been trained, to do a kind of "averaging" which limits it's capacity to explore novel designs
> But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views
I disagree. Have a conversation with it about your problem and work through design decisions with it. When I do that, I find it gives me a lot of good ideas.
Disclaimer: I'm not working on anything groundbreaking (like most people)
Yes, but "good ideas" compared to what? If you were aware of the better alternatives, you probably wouldn't be discussing those details with an LLM. You'd find that it just randomly gave you one. It might work, but you don't know how well until you're already entrenched.
Nobody knows everything, so of course LLMs can be useful sometimes. More useful than plain old search, books, or even discussion with real humans? Maybe.
Search can offer a much broader context than an LLM hyperfocused on just generating text. Books may lead you to realize you were asking the wrong questions. Discussions will provide an overall "vibe" of the topic.
These are not competing options. We can and should be using all of them when possible.
> Yes, but "good ideas" compared to what? If you were aware of the better alternatives, you probably wouldn't be discussing those details with an LLM
Even when I already have a good idea of how I plan to do something, I may still ask AI and then find it gave me better idea for some particular thing.
I liken it to using GPS even when you know the route like the back of your hand. It can still steer you around an accident.
To do this effectively I have to drop the idea that I always know better than it does.
6 replies →
Sure but you can also google your problem and check what is industry standard/what is the correct way to do things (imo in less time than it takes to go through a conversation).
But the problem is that when you ask ai to solve a problem on its own, its default plan can suck. You can mitigate that by research and context but it doesn't mean the initial problem is solved. But even that requires skill and human judgement (both ai conversation research or traditional research) and a lot of people want to skip that entirely.
I find I don’t necessarily need or want AI to give me ideas, but I would agree having a conversational back and forth generally yields decent results.
I have found being Socratic in my questions, and trying to get the AI to arrive at my intended design via such conversations supplies the right level of context for properly solving the problem. It’s token intensive, without a doubt, but I find the result is the AI tends to be better equipped to handle the many micro decisions that need to be made along the way.
The contrast to this is I give it a detailed prompt where it then asks questions of me, which also generally works but I find the AI tends to not be as well equipped for decisions it needs to make mid implementation.
It’s not perfect, and maybe not even a good fit for some. I also never know what to think when people tell me their idiosyncratic ways of using AI. Ultimately I think the most effective way is whatever lets you translate the vision in your head into the end result.
When I say "novel ideas" it means something groundbreaking, indeed, not rehashing common "this is the best practice for a simple CRUD backend"