Comment by Neywiny

4 hours ago

I keep running into this. It's nice seeing others here struggle. I guess when all your doing is one-shot simple trivial tasks who cares. I've found they're great for that. But once I need to do real work, everybody makes their own esoteric abandonware that kinda works but kinda doesn't. Stars aren't a perfect indicator, but I haven't seen anything over a few hundred for these things I'm finding on GitHub. Same with downloads of plugins. It was very isolating feeling like I'm the only one not enamored by the state of this.

Same here. I have a feeling that once the AI hype cycle blows over to a degree (not that it will go away, just calm down), we'll be left with an enhanced version of pre-boom development. LLMs are hitting a dead end. I'm not convinced they can be bandaid-fixed to become AGI or SI or whatever the cool new trendy term is. They're good at many things, but sustaining a lean and directed software project's architecture over long periods of time doesn't seem to be one of them. That still remains to be further proven over time, but I've already seen plenty of examples of this.

Also, "esoteric abandonware" is a great way to put it. I keep looking for solutions to problems, finding dead projects that haven't been committed to in months, they have all the indicators of AI slopification (the primary one for me being emoji-filled READMEs). I don't think these types of projects will be around for the long-haul. These projects are just as much about community and people as they are about code. And people are much less likely to care about a short-lived project that doesn't have the backing of people that intuitively understand the internals of it. It's like building on sand.

Personally, I've decided that allowing the "agents" (these buzzwords are quickly becoming pet peeves of mine) to write the code for me is a bad idea, because the faster the LLM constructs architecture than I can, the less I'm absorbing the details of it, and the less likely I am to intuit obvious shortcomings. They do help me work much faster than before though, because I still use them like a research/learning assistant chatbot. Learning what I'm doing wrong faster, while still being the one putting the pieces together, is very great.

I'm also in the boat that says things should be better, but instead of improving the agent I decided to try to implement the agent management (in: https://beolis.com).

It still leverages agents (people are used to them already) but can deal with executing it on a different vm, dealing with git worktrees, etc (not that coding agents themselves cannot be improved, but I think the sw development infrastructure around it has a lot to improve too).

A couple of suggestions from the other side of this:

* Hold on to that isolating feeling. Those of us who feel AI is an incredible accelerator also feel just as isolated as you do. We wonder why you aren't seeing what we are seeing - and that isn't criticism, we just genuinely don't understand where the gap is and what would help other people jump across.

* I wouldn't bother looking at GitHub, or using that as any kind of measure. Personally getting away from all dependencies is my goal, and avoids a lot of supply chain issues.

* I don't use any plugins or external skills. I do use a ton of self built local MCP tools though, and it drives much of my workflow. That's me though.

* You sometimes need to work up to it. I've had periods of hitting a wall this year with LLMs. Once I end up building a framework, and the framework is in place, everything flows. The models can learn from looking at your adjacent projects, and they just fly. That's for coding, but it's similar for solving other problems... record everything, give it a ton of context, etc.

* If the software "kinda works kinda doesn't", then someone isn't doing enough testing. You should be dogfooding your software every day. Use your AIs to help you find edge cases, sure, but they're alien minds. Your human mind will sometimes think in ways they don't, but that your customers will. Sometimes we write software for AI to use, sure, but if you're writing for humans, you still should add some human mind thinking just to double check the model's work. "That's right, it goes in the square hole."

It's okay to struggle, some of us have been building toolkits over the last two years that make AI work so much easier. But don't get sucked into thinking "it doesn't work", especially in the Opus 5.5 era. On my side of the fence, these last two weeks have been the most intense and productive in a long time.

Lastly: some of us have tips we will not share. Some is slight selfishness - if others haven't discovered what the models can do, that is a small moat. But there are also some tips you just have to find on your own. I could tell you playing The Witness is some of the best training you could have for working with AI, and many will think that makes no sense at all. But I think a handful will nod and quietly understand, the lesson can't be taught, it must be learned.

Sorry for the long ramble, but I genuinely hope some of that helps.

  • I can see vague parallels between The Witness and AI prompting/engineering, but I'm not sure I'd describe it as training beyond the simple concept of teaching vs learning. Maybe I'm just in the category of people that think that makes no sense at all? (I completed the primary game and got about halfway through the secondary game)

  • I mean that's the problem though. I'm not paid to spend 2 years fine tuning a framework. I have things that need to get done, accurately and precisely. And to me, to my work, the models just suck without intense prompting. The harnesses are simultaneously too open and restrictive, and annoying. The terms are too overloaded. The models fight with me on highly domain specific knowledge that it's long since compressed or quantized away, until I fill the context with the documentation. It's all a nightmare to get up the learning curve.

    I'm not blaming you, but it's incredible how much open source garbage there is and people spend years making actually good stuff that actually works and nobody's publishing it. So to me, and I'm trying, but to me your point is hearsay at best. You may as well have told me it took 2 seconds and you've always had a perfect time with it. It's all meaningless to me.