Comment by limecherrysoda
3 hours ago
"Review this project" is that how you use LLMs lmao
Just toss GBs of file structure: "AI, do your work baby!"
I for one break things down much smaller into very specific tasks involving very particular text. Maybe I'm overdoing it lol.
For me, an AI security review would still take hours or days, it would hardly be a 1-shot prompt like this.
Nah I asked the ai and I said it was fine :p
Depends on the size of the repo, a few files and <~10,000 loc this prompt is probably fine, but as it grows it becomes less effective.
If you have any LLM write code, you'll notice it breaks down at about ~1k lines. Anything under that - simple API endpoint, React component, is fine.
But it quickly loses fidelity as you load more into the context. The context window is supposed to be much larger, but in reality, it loses accuracy and fidelity the more you load in.
If I loaded 10k+ lines of code across files into a RAG db (since that's much too large for LLM context) - which is what the foundation of "an agent" is - I highly doubt that it would be very effective on its own. And it isn't IME, that's why so-called agentic coding isn't very good compared to an expert using an LLM manually, breaking it down into task-specific work.