Comment by chaosprint
1 day ago
The title is a bit exaggerated. The depth of the projects covered in the article is clearly not representative of "all".
In fact, I now prefer to use a purely chat window to plan the overall direction and let LLM provide a few different architectural ideas, rather than asking LLM to write a lot of code whose detail I have no idea about.
It's a play on the name of the paper that jump-started ChatGPT: "Attention Is All You Need:" https://arxiv.org/abs/1706.03762
That's my gist. All of these seem pretty basic apps I would see implemented to demo a new web or REST framework. Comment ranker is cool, but I can't imagine its doing much more than scrape text > call semantic api > modify DOM.
How much of this is buildings versus recalling tutorials in the dataset. For every vibe coded project with 20 lines of requirements, I have a model with 20 different fields all with unique semantic meanings. In focused areas, AI has been okay. But I have yet to see Claude or any model build and scale a code base with the same mindset.
I like using Claude-Code, it can be a real timesaver in certain cases.
But it's far from perfect. Really difficult things/big projects are nearly impossible. Even if you break it down into hundred small tasks.
I've tried to make it port an existing, big codebase from one language to another. So it has all of the original codebase in one folder, and a new project in another folder. No matter how much guidance you give it, or how clear you make your todos, it will not work.
What specifically are its modes of failure? I've never tried doing that, do very curious what the roadblocks are.
I've done something similar for a refactor.
It simply forgets code exists during a port. It will port part of a function and ignore the rest, it will scan a whole file into context and then forget that a different codepath exists.
I would never rely on it for a 1:1 mapping of large features/code transformations. Small stuff sure, but beyond say a few large files it will miss things and you will be scratching your head for why it's not working.
1 reply →
Most harnesses provide this as a "plan" vs. "act" mode now. You first "chat" in plan mode (no access to tools, no instructions to write any code basically), you then can optionally write those plans in a memorybank / plan.md, and then say "now go implement it", and it moves to the "act" mode where it goes through and does it, updating progress in plan.md as it goes.
I've found it very useful to have items like requirements.md, plans.md, or todo.md, in my LLM focused projects. I'll use AI to help take the ideas I have at that stage and refine them into something more appropriate for ingestion into the next stage. So, when I want it to come up with the plans, it is going to base is mostly on requirements.md, and then I'll have it act on the plans step by step after that.
the thing is, it's not working as the default mode, which is not ideal imho