Comment by throw83288
5 months ago
Not enough resources to get another bachelors, and a masters is probably practically worthless for a pivot. I would have to throw away the past 10 years of my life, start from scratch, with zero ideas for any real skill-developing projects since I'm not interested at all. Probably a completely non-viable candidate in anything I would choose. Maybe only Robotics would work, and that's probably going to be solved quickly because:
You assume nothing LLMs do are actually generalization. Once Field X is eaten the labs will pivot and use the generalization skills developed to blow out Field Y to make the next earnings report. I think at this current 10x/yr capability curve (Read: 2 years -> 100x 4 years -> 10000x) I'll get screwed no matter what is chosen. Especially the ones in proximity to computing, which makes anything in which coding is secondary fruitless. Regulation is a paper wall and oligopolies will want to optimize as much as any firm. Trades are already saturating.
This is why I feel completely numb about this, I seriously think there is nothing I can do now. I just chose wrong because I was interested in the wrong thing.
If you're taking a really high level look at the whole problem, you're zooming too far out, and missing the trees themselves. You chose the wrong parents to be born to, but so did most of us. You were interested in what you were interested in. You didn't ask what's the right thing to be interested in, because there's no right answer to that. What you've got is a good head on your shoulders, and the youth to be able to chase dreams. Yeah it's scary. In the 90's outsourcing was going to be the end of lucrative programming jobs in the US. There's always going to be a reason to be scared. Sometimes it's valid, sometimes the sky is falling because aliens are coming, and it turns out to be a weather balloon.
You can definitely succumb to the fear. It sounds like you have. But courage isn't the absence of fear, it's what you do in the face of it. Are you going to let that fear paralyze you into inaction? Just not do anything other than post about being scared to the Internet? Or, having identified that fear, are you gonna wrestle it down to the ground and either choose to retrain into anything else and start from near zero, but it'll be something not programming that you believe isn't right about to be automated away, or dive in deeper, and get a masters in AI and learn all of the math behind LLMs and be an ML expert that trains the AI. That jobs not going away, there's still a ton of techniques to be discovered/invented and all of the niches to be discovered. Fine-tuning an existing LLM to be better at some niche is gonna be hot for a while.
You're lucky, you're in a position to be able to go for a masters, even if you don't choose that route. Others with a similar doomer mindset have it worse, being too old and not in a position to them consider doing a masters.
Face the fear and look into the future with eyes wide open. Decide to go into chicken farming or nursing or firefighter or aircraft mechanic or mortician or locksmith or beekeeping or actuary.
I think if you believe LLMs can truly generalize and will be able to replace all labor in entire industries and 10x every year, you pretty much should believe in ASI at which point having a job is the least of your problems.
if you rule out ASI, then that means progress is going to have to slow. consider that programming has been getting more and more automated continually since 1954. so put yourself in a position where what LLMs can do is a complement to what you can do. currently you still need to understand how software works in order to operate one of these things successfully.
I don't know if I agree with that and as a SWE myself its tempting to think that - it it a form of coping and hope that we will be all in it together.
However rationally I can see where these models are evolving, and it leads me to think the software industry is on its own here at least in the short/medium term. Code and math, and with math you typically need to know enough about the domain know what abstract concept to ask, so that just leaves coding and software development. Even for non technical people they understand the result they want of code.
You can see it in this announcement - it's all about "code, code, code" and how good they are in "code". This is not by accident. The models are becoming more specialised and the techniques used to improve them beyond standard LLM's are not as general to a wide variety of domains.
We engineers think AI automation is about difficulty and intelligence, but that's only partly true. Its also about whether the engineer has the knowledge on what they want to automate, the training data is accessible and vast, and they even know WHAT data is applicable. This combination of both deep domain skills and AI expertise is actually quite rare which is why every AI CEO wants others to go "vertical" - they want others to do that leg work on their platforms. Even if it eventuates it is rare enough that, if they automate, will automate a LOT slower not at the deltas of a new model every few months.
We don't need AGI/ASI to impact the software industry; in my opinion we just need well targeted models that get better at a decent rate. At some point they either hit a wall or surpass people - time will tell BUT they are definitely targeting SWE's at this point.
i actually don’t think nontechnical people understand the result they want of code.
have you ever seen those experiments where they asked people to draw a picture of a bicycle, from memory? people’s pictures made no mechanical sense. often people’s understanding of software is like that — even more so because it’s abstract and many parts are invisible.
learning to clearly describe what software should do is a very artificial skill that at a certain point, shades into part of software engineering.
1 reply →
I think what's missing is that the amount of training data to effectively RL usually decreases over time. AlphaGo needed some initial data on good games of Go to then recursively improve via RL. Fast forward a few years, and AlphaZero doesn't need any data to recursively improve.
This is what I mean by generalization skills. You need trillions of lines of code to RL a model into a good SWE right now, but as the models grow more capable you will probably need less and less. Eventually we may hit the point where a large corporations internal data in any department is enough to RL into competence, and then it frankly doesn't matter for any field once individual conglomerates can start the flywheel.
This isn't an absurdity. Man can "RL" itself into competence in a single semester of material, a laughably small amount of training data compared to an LLM.