Comment by hombre_fatal
8 days ago
I think the upper limit is your ability to decide what to build among infinite possibilities. How should it work, what should it be like to use it, what makes the most sense, etc.
The code part is trivial and a waste of time in some ways compared to time spent making decisions about what to build. And sometimes even a procrastination to avoid thinking about what to build, like how people who polish their game engine (easy) to avoid putting in the work to plan a fun game (hard).
The more clarity you have about what you’re building, then the larger blocks of work you can delegate / outsource.
So I think one overwhelming part of LLMs is that you don’t get the downtime of working on implementation since that’s now trivial; you are stuck doing the hard part of steering and planning. But that’s also a good thing.
I've found writing the code massively helps your understanding of the problem and what you actually need or want. Most times I go into a task with a certain idea of how it should work, and then reevaluate having started. While an LLM will just do what you ask without questing, leaving you with none of the learnings you would have gained having done it. The LLM certainly didn't learn or remember anything from it.
In some cases, yes. But I’ve been doing this awhile now and there is a lot of code that has to be written that I will not learn anything from. And now, I have a choice to not write it.
Ehh, I find that the most tedious code is also the most sensitive to errors, stuff that blurs the divide between code and data.
2 replies →
It depends on how you use them. In my workflow, I work with the LLM to get the desired result, and I'm familiar with the system architecture without writing any of the code.
I've written it up here, including the transcript of an actual real session:
https://www.stavros.io/posts/how-i-write-software-with-llms/
Thanks for writing this up.
I just woke up recently myself and found out these tools were actually becoming really, really good. I use a similar prompt system, but not as much focus on review - I've found the review bots to be really good already but it is more efficient to work locally.
One question I have since you mention using lots of different models - is do you ever have to tweak prompts for a specific model, or are these things pretty universal?
1 reply →
This hits home for me. Lawyer, not developer here. Implementation was never a hard part for me, it was an impossible part. Now that the time/cost needed to experiment with prototypes has dropped to near zero I've been been spending a lot of time doing exactly what you describe (steering, brainstorming). I find it fun but I do it mainly as a bunch of personal side projects. Can understand how it might feel different for users when the stakes are much higher (like when it's part of the day-to-day in a real job).
Right when you're coding with LLM it's not you asking the LLM questions, it's LLM asking you questions, about what to build, how should it work exactly, should it do this or that under what conditions. Because the LLM does the coding, it's you have to do more thinking. :-)
And when you make the decisions it is you who is responsible for them. Whereas if you just do the coding the decisions about the code are left largely to you nobody much sees them, only how they affect the outcome. Whereas now the LLM is in that role, responsible only for what the code does not how it does it.
Hehe, speak for yourself- as a 1x coder on a good day, having a nonjudgmental partner who can explain stuff to me is one of the best parts of writing with an llm :)
I like that aspect of it too. LLM never seems to get offended even when I tell it its wrong. Just trying to understand why some people say it can feel exhausting. Instead of focusing on narrowly defined coding tasks, the work has changed and you are responsible for a much larger area of work, and expectations are similarly higher. You're supposed to produce 10x code now.
1 reply →
> Because the LLM does the coding, it's you have to do more thinking. :-)
I keep seeing this sentiment, but it sure sounds wrong to me.
Coding requires thinking (in humans, at any rate). When you're doing coding, you're doing both coding-thinking and the design thinking.
Now you're only doing one half of it.
I’d love to see what you’ve built. Can you share?
>how people who polish their game engine (easy)
This is such a weird statement. Game engines are among the most complicated pieces of software in existence. Furthermore, a game that doesn't run smoothly increases the chances that your player base doesn't stick around to see what you've built.
Maintenance is the hard part, not writing new code or steering and planning.
You can outsource that to another llm