Comment by FeteCommuniste
2 months ago
Yeah, I sometimes use AI for questions like "is it possible to do [x] using library [y] and if so, how?" and have received mostly solid answers.
2 months ago
Yeah, I sometimes use AI for questions like "is it possible to do [x] using library [y] and if so, how?" and have received mostly solid answers.
Or “can you prototype doing A via approaches X, Y, and Z, and show me what each looks like?”
I love to prototype various approaches. Sometimes I just want to see which one feels like the most natural fit. The LLM can do this in a tenth of the time I can, and I just need to get a general idea of how each approach would feel in practice.
> Sometimes I just want to see which one feels like the most natural fit.
This sentence alone is a huge red flag in my books. Either you know the problem domain and can argue about which solution is better and why. Or you don't and what you're doing are experiment to learn the domain.
There's a reason the field is called Software Engineering and not Software Art. Words like "feels" does not belongs. It would be like saying which bridge design feels like the most natural fit for the load. Or which material feels like the most natural fit for a break system.
> There's a reason the field is called Software Engineering and not Software Art. Words like "feels" does not belongs.
Software development is nowhere near advanced enough for this to be true. Even basic questions like "should this project be built in Go, Python, or Rust?" or "should this project be modeled using OOP and domain-driven design, event-sourcing, or purely functional programming?" are decided largely by the personal preferences of whoever the first developer is.
17 replies →
Do you develop software? Software unlike any physical engineering field. The complexity of any project beyond the most trivial is beyond human ability to work with. You have to switch from analytic tools to more probabilistic tools. That where "feels", "smells", or "looks" come in. Software testing is not a solved problem, unlike bridge testing.
1 reply →
For example sometimes you're faced with choosing between high-quality libraries to adopt and it's not particularly clear whether you picked the wrong one until after you've tried integrating them. I've found it can be pretty helpful to let the LLM try them all and see where the issues ultimately are.
2 replies →
> Yeah, I sometimes use AI for questions like "is it possible to do [x] using library [y] and if so, how?" and have received mostly solid answers.
In my experience most LLMs are going to answer this with some form of "Absolutely!" and then propose a square-peg-into-a-round-hole way to do it that is likely suboptimal vs using a different library that is far more suited to your problem if you didn't guess the right fit library to begin with.
The sycophancy problem is still very real even when the topic is entirely technical.
Gemini is (in my experience) the least likely to lead you astray in these situations but its still a significant problem even there.
IME this has been significantly reduced in newer models like 4.5 Opus and to a lesser extent Sonnet, but agree it's still sort of bad- mainly because the question you're posing is bad.
if you ask a human this the answer can also often be "yes [if we torture the library]", because software development is magic and magic is the realm of imagination.
much better prompt: "is this library designed to solve this problem" or "how can we solve this problem? i am considering using this library to do so, is that realistic?"
Just be careful if functionality varies between library y version 2 and library y version 3, or if there is a similarly named library y2 that isn't the same.
You may get possibilities, but not for what you asked for.
If you run to the point where you can execute each idea and examine its outputs, problems like that surface pretty quickly
Of course, by that time i could have read the docs for library y the version I'm using...
1 reply →