Cool. Can you elaborate at which types of task you are better than SOTA LLMs in context of "being good at SW design and architecture"? Got any examples? Is it at the interview questions? Or real world problems? If so what is the scale of the real world SW design and architecture problems you're better than the SOTA LLMs? Is it FAANG scale or mom and pop shop scale?
And do you consider yourself to be representative of the average developer, above them, or below them?
LLMs don't even need to be better than the average dev, let alone the top performing ones, like you. If they can just be better than the bottom 20% of devs and white collar workers in general(easily achievable when you've been around the block and saw how many useless people just keep warm chairs for high wages in large companies), that's already a huge win for those products.
What I mean, at a previous job I had ran into a memory leak issue in our backend and discovered a colleague pushed a library into prod which came with comments in the source code saying "DO NOT USE IN PROD, IT CAUSES A MEMORY LEAK!". There's cases where human stupidity and carelessness far surpasses whatever issues LLMs cause so maybe the average dev isn't really that much better than the SOTA LLMs.
I'm currently driving my AI coding agent harness through the process of refactoring itself. This is the last big refactor I need done before I can polish and release it as OSS later this month. So its not a large scale project I'm working on at present. The general problem is that mostly add new code and try to minimize editing existing code, which effectively means the code base is grown organically rather than being intentionally architected and designed. A few of specifics:
1) duplication - LLMs are great at generating lots of text, so its faster and easier for them to generate entirely new facilities that overlap heavily with existing ones then it is for them find existing facilities that should be expanded and refactored (note I just said 'find'; actually editing raises the time and difficulty even more). This is fine for a while as the duplicate facilities usually work just fine, up until something needs to be changed across all of them and they miss changing one or more of them, things break, and a bunch of tokens have to be burned tracking down the issue.
2) ever increasing surface area - even when making changes that do expand a facility without much duplication they frequently only add without removing much of anything or changing the overall design of the facility to reduce the amount of state its tracking and the number of branches it has based on that state. I've never seen one decide to split up something large or with too many responsibilities on their own. They will happily create a god class or function and just keep making it bigger.
Cool. Can you elaborate at which types of task you are better than SOTA LLMs in context of "being good at SW design and architecture"? Got any examples? Is it at the interview questions? Or real world problems? If so what is the scale of the real world SW design and architecture problems you're better than the SOTA LLMs? Is it FAANG scale or mom and pop shop scale?
And do you consider yourself to be representative of the average developer, above them, or below them?
LLMs don't even need to be better than the average dev, let alone the top performing ones, like you. If they can just be better than the bottom 20% of devs and white collar workers in general(easily achievable when you've been around the block and saw how many useless people just keep warm chairs for high wages in large companies), that's already a huge win for those products.
What I mean, at a previous job I had ran into a memory leak issue in our backend and discovered a colleague pushed a library into prod which came with comments in the source code saying "DO NOT USE IN PROD, IT CAUSES A MEMORY LEAK!". There's cases where human stupidity and carelessness far surpasses whatever issues LLMs cause so maybe the average dev isn't really that much better than the SOTA LLMs.
I'm currently driving my AI coding agent harness through the process of refactoring itself. This is the last big refactor I need done before I can polish and release it as OSS later this month. So its not a large scale project I'm working on at present. The general problem is that mostly add new code and try to minimize editing existing code, which effectively means the code base is grown organically rather than being intentionally architected and designed. A few of specifics:
1) duplication - LLMs are great at generating lots of text, so its faster and easier for them to generate entirely new facilities that overlap heavily with existing ones then it is for them find existing facilities that should be expanded and refactored (note I just said 'find'; actually editing raises the time and difficulty even more). This is fine for a while as the duplicate facilities usually work just fine, up until something needs to be changed across all of them and they miss changing one or more of them, things break, and a bunch of tokens have to be burned tracking down the issue.
2) ever increasing surface area - even when making changes that do expand a facility without much duplication they frequently only add without removing much of anything or changing the overall design of the facility to reduce the amount of state its tracking and the number of branches it has based on that state. I've never seen one decide to split up something large or with too many responsibilities on their own. They will happily create a god class or function and just keep making it bigger.