Comment by mapontosevenths
1 day ago
> "That almost nobody has the skills these days to implement something like TP is something we could eventually come to regret – or, rather, our grandchildren could. Software development is becoming a semi-skilled industry, and I doubt that the rise of AI will entirely compensate for this."
I agree, but AI also there is hope!
For many years I've worried about the ever-growing depth of abstraction in computing. There are few alive now who actually understand all of the layers on abstraction involved in modern computing, and who can debug the truly strange edge case bugs and vulnerabilities introduced by the interoperation of these many complex layers.
For a time I imagined it to be an unsolvable problem, and that there would always be only a special class of neurodivergent hyper-specialists who still mostly had to work in large teams to even come close to understanding a stack in full. We're reaching the point where a single human lifespan just isn't enough to learn it all, even for those of us with what Dilbert called "The Knack". By the time you're proficient in all of it - you're dead.
Now there is a new kind of intelligence. One that CAN fit it all in a single mind, and one that will outlive us all and continue to learn long after our meat has gone bad.
As long as the AI's stay aligned, and we find ways to build them more efficiently there's a non-zero chance that they actually make it all better instead of worse. Genuinely long-term planning and the ability to TRULY grasp the full stack COULD actually happen. At least if we use these new systems in service of the greater good, and not just to play silly games on the stock market.
AI isn't fundamentally different to us and it does benefit from abstractions. So, I don't think that's going to happen.
Besides, it's an exaggeration. I've been programming for 35 years, and Turbo Pascal was one of my first languages. I wrote OWL code! Seeing that splash screen brought back memories. But I can understand and debug problems that cross almost any layer of the software stack. Anything from some ReactJS library to kernel code to UEFI. I'm now 42 so the idea nobody can understand or debug interactions across layers before they die is for the birds. The stack isn't that deep.
I think it’s easier for us to say when we grew up with the stack. In the old days you could pretty much understand everything on your computer in a relatively short period of time. So as a learner it was all so accessible. These days it’s hard to even know where to start.
Yeah, true, it probably helps to have grown up alongside the tech itself.
I'm not sure it was more accessible though. Learning programming back then was really hard. Nothing was user friendly and achieving even simple tasks like drawing a diagonal line might require you to sit down and learn Bresenham's Algorithm (from the books neither you nor your library had copies of, naturally...). So it was easy to get discouraged. You can get fun and relevant results much faster now.
AI is not trending towards small, minimal code with nice abstractions. It is faster at replicating "big team" code, but the code from a small team or a few authors still wins for elegance and simplicity.
I suppose it depends on what you mean by trending. Some of the recent rewrite news[1] for example suggest that, at minimum, it is the macro demand that is driving the software trends. In this case, issues with memory supply drive attempts at more sensible memory utilization.
[1]https://tech-insider.org/discord-electron-rust-gpui-rebuild-...
>https://tech-insider.org/discord-electron-rust-gpui-rebuild-...
The fact that we are celebrating that a chat app only uses 300MB ram at launch instead of 1GB seems absolutely ludicrous to me
The early versions of Skype used about 30MB RAM while in a video call and by the early 2000s standards it was bloatware. Before that we used WebEx in the 90s which was a browser ActiveX or Java applet and RAM use was about 6MB in a video call. In the early / mid 90s I used CU-SeeMe for video calls and its RAM footprint was just over 2MB on Mac System 7
Something has clearly gone very wrong in the software development process in the last 2-3 decades and while Im encouraged to see that some people are interested in reversing that trend I still find the resource wastage has me shaking my head sometimes
9 replies →
I respectfully disagree. All you describe is a deeeeep reliance on who controls the models, and as the things currently trend I would say it goes more and more into a monolithic direction. Sure, you can run your local model, but hardware is so expensive that it will have many cut corners.
I also think you oversell the complexity of things to preach the superiority of a glorified text predictor
We do have free pascal and it looks pretty good I think.
There’s no substitute for human knowledge. What if the climate crisis gets to the point where we cannot justify boiling the ocean to perform tasks we’ve come to depend on?
What’s our disaster recovery strategy in the event of war and loss of internet?
Or worse yet, a near extinction event and we are forced to rebuild civilisation from memory?
Some of these might seem like extreme edge case scenarios but then history is full of disasters that people didn’t prepare for because they assumed they were too big to fail.
> There’s no substitute for human knowledge.
Would you not say that AI is an expression of the sum of all human knowledge? And as humans continue to generate more knowledge, it too, will be added to the AI's representation of it all?
So there's a common dichotomy in software engineering between the "big ball of mud" and "beautiful diamond" patterns. A big ball of mud is a software system that grows through accretion: it often has tangled flow control, data whales, lots of special-casing, lookup tables, and so on. It evolves, it isn't really designed. A beautiful diamond is a software system that is meticulously designed, consistent, oftentimes with tight interdependencies between how it works. Everything is efficient and easy to understand, but if you take one part out of it, it tends to fall apart. Common Lisp is a big ball of mud, Scheme is a beautiful diamond. C++ is a big ball of mud, K&R C is a beautiful diamond. Windows is a big ball of mud, BeOS is a beautiful diamond. Microsoft VSCode is a big ball of mud, Turbo Pascal is a beautiful diamond.
The big ball of mud structure is often taken to be an antipattern: at least, when I Googled this, Gemini told me it's a "haphazard, tangled anti-pattern" and gave me some advice about how to refactor such a software system into a beautiful diamond. But nearly all software systems that people actually use and maintain are big balls of mud. This is because the big ball of mud tolerates partial failure extremely well. It tolerates changing requirements very well. It tolerates parallel development very well. You can change part of it without throwing away the whole thing.
AI is a beautiful diamond. For the most part, if an AI model malfunctions, you have to blow it away and retrain it. Or I guess you can build layers to fix up the output case-by-case, but then you're really building a big ball of mud around the beautiful diamond, and obscuring it in the process.
That's a little worrisome when it comes to durability. Living, enduring systems usually end up becoming a patchwork of mud. Your ability to do this with LLMs are basically limited to LoRa patches and post-processing steps.
There's much more about humans then there is writing about them, a lot of cultures with oral histories, a lot of nuance in behavior we take for granted, can it all be digitized?
I would say your reply feels like you didn’t read beyond the first sentence of my comment.
If you did, then I apologise. But my point is about accessing that knowledge when AI isn’t available. Not about how AI discovered its corpus to begin with.
7 replies →
That already happened many times in history to many civilizations. Knowledge was discovered and lost, and rediscovered years later or never.
Yes, I hinted at that. And each time that civilisation fell. With the next occupiers of the land left to find it.
Do we really want to put ourselves in the same situation?
2 replies →
>I agree, but AI also there is hope!
It's like a religion isn't it.
My typo didn't help, but no. It is literally nothing like a religion.
You just gotta have faith.
Or we could just ask them to build something less embarrassing?
I’m not sure where he got that idea. All I could learn about TP was like one book sold in one bookstore in my entire city. The barriers to entry back then were astronomically higher than they are today, especially given how AI has democratized access to information, knowledge, and expertise.
Hail the Omnissiah! There is no strength in flesh, only weakness!
[dead]