Comment by wonnage
5 hours ago
That works for well trod paths, e.g “fix ci” works exceedingly well. “why app slow” obviously doesn’t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly
> “why app slow” obviously doesn’t work because the task is underspecified
Definitely not true and like everyone else is saying, shows how people still underestimate these models.
I have been working on a simple vite + react app lately and commonly ask Gemini/Antigravity to just "improve speeds", "x is running slow, check it out" and have no complaints.
> . “why app slow” obviously doesn’t work because the task is underspecified.
Not always. In my case LLM goes to grafana mcp, pulls metrics/traces/cpu profiles. Figures out what is slow and proposes a solution.
I wouldn't agree. Sota models can do self-directed sampling, profiling, benchmarking, read call trees, etc. to give you a report of the app's bottlenecks and then recommend solutions that can be vetted.
I do this constantly.
As the upstream comment points you, you don't need to specify. Sota models are that good. And by being overprescriptive you can accidentally shut off branches that they would've taken, downgrading the quality of their work.
In my experience if you’re at the point where you have something to sample then the hard part is already done.
In a perfect world everything is covered by distributed tracing and the problems are only in your application code and the agent just needs to find the data
In reality the data is often missing or misleading. “Your observability sucks”? Yeah, but that’s life
> “Your observability sucks”? Yeah, but that’s life
You could start by asking your AI "help me add better observability to our stack"
I use skills. The skills are not typically "how to perform a task in detail" they are more about what relevant tools and knowledge are required to work in a domain. That is I give the LLM the information it needs about the system but not a sequence of how to accomplish a task. I treat it more like a human and less like a computer.
It really doesn’t need to be that much more specified, give it context to the tools and level of analysis you expect then “why app slow” is a reasonable prompt