Comment by consumer451
2 days ago
I use LLM dev tools, and even have Supabase MCP running. I love these tools. They allowed me to create a SaaS product on my own, that I had no chance of creating otherwise as a long out of practice dev.
However, we are nowhere near the reliability of these tools to be able to:
1. Connect an MCP to a production database
2. Use database MCPs without a --read-only flag set, even on non-prod DBs
3. Doing any LLM based dev on prod/main. This obviously also applies to humans.
It's crazy to me that basic workflows like this are not enforced by all these LLM tools as they will save our mutual bacon. Are there any tools that do enforce using these concepts?
It feels like decision makers at these orgs are high on their own marketing, and are not putting necessary guardrails on their own tools.
Edit: Wait, even if we had AGI, wouldn't we still need things like feature branches and preview servers? Maybe the issue is that these are just crappy early tools missing a ton of features, and nothing to do with the reliability and power of LLMs?
The only way LLM-based software development / production management will be trustable is by actually scaling back what it can and cannot do. Put critical operations in "real" code, so that the LLM can only request a release, triggering a human review of, at the very least, the operation that is about to be done.
Then again, this reminds me of the prompts in operating systems whenever something needs root access, most people just blindly okayed it, especially on Windows since Vista did too many of them even for trivial operations.
"What, an human in the loop is slowing down our release? I have just the idea!"
This imo is the biggest issue, LLMs can at times be very capable but they always are unreliable.