Comment by diggan
19 days ago
Interesting that every comment has "Help improve Copilot by leaving feedback using the or buttons" suffix, yet none of the comments received any feedback, either positive or negative.
> This seems like it's fixing the symptom rather than the underlying issue?
This is also my experience when you haven't setup a proper system prompt to address this for everything an LLM does. Funniest PRs are the ones that "resolves" test failures by removing/commenting out the test cases, or change the assertions. Googles and Microsofts models seems more likely to do this than OpenAIs and Anthropics models, I wonder if there is some difference in their internal processes that are leaking through here?
The same PR as the quote above continues with 3 more messages before the human seemingly gives up:
> please take a look
> Your new tests aren't being run because the new file wasn't added to the csproj
> Your added tests are failing.
I can't imagine how the people who have to deal with this are feeling. It's like you have a junior developer except they don't even read what you're telling them, and have 0 agency to understand what they're actually doing.
Another PR: https://github.com/dotnet/runtime/pull/115732/files
How are people reviewing that? 90% of the page height is taken up by "Check failure", can hardly see the code/diff at all. And as a cherry on top, the unit test has a comment that say "Test expressions mentioned in the issue". This whole thing would be fucking hilarious if I didn't feel so bad for the humans who are on the other side of this.
> I can't imagine how the people who have to deal with this are feeling. It's like you have a junior developer except they don't even read what you're telling them, and have 0 agency to understand what they're actually doing.
That comparison is awful. I work with quite a few Junior developers and they can be competent. Certainly don't make the silly mistakes that LLMs do, don't need nearly as much handholding, and tend to learn pretty quickly so I don't have to keep repeating myself.
LLMs are decent code assistants when used with care, and can do a lot of heavy lifting, they certainly speed me up when I have a clear picture of what I want to do, and they are good to bounce off ideas when I am planning for something. That said, I really don't see how it could meaningfully replace an intern however, much less an actual developer.
These GH interactions remind me of one of those offshore software outsourcing firms on Upwork or Freelancer.com that bid $3/hr on every project that gets posted. There's a PM who takes your task and gives it to a "developer" who potentially has never actually written a line of code, but maybe they've built a WordPress site by pointing and clicking in Elementor or something. After dozens of hours billed you will, in fact, get code where the new file wasn't added to the csproj or something like that, and when you point it out, they will bill another 20 hours, and send you a new copy of the project, where the test always fails. It's exactly like this.
Nice to see that Microsoft has automated that, failure will be cheaper now.
This gives me flashbacks to when my big corporate former employer outsourced a bunch of work offshore.
An outsourced contractor was tasked with a very simple job as their first task - update a single dependency, which required just a bump of the version and no code changes - after three days of them seemingly struggling to even understand what they were asked to do, inability to clone the repo, failure to install the necessary tooling on their machine, they ended up getting fired from the project. Complete waste of money, and the time of those of us having to delegate and review this work.
2 replies →
> These GH interactions remind me of one of those offshore software outsourcing firms on Upwork or Freelancer.com that bid $3/hr on every project that gets posted
Those have long been the folks I’ve seen at the biggest risk of being replaced by AI. Tasks that didn’t rely on human interaction or much training, just brute force which can be done from anywhere.
And for them, that $3/hr was really good money.
Actually the AI might still be more expensive at this point. But give it a few years I'm sure they will get the costs down.
>>These GH interactions remind me of one of those offshore software outsourcing firms on Upwork or Freelancer.com that bid $3/hr on every project that gets posted.
This level of smugness is why outsourcing still continues to exist. The kind of things you talk about were rare. And were mostly exaggerated to create anti-outsourcing narrative. None of that led to outsourcing actually going away simply because people are actually getting good work done.
Bad quality things are cheap != All cheap things are bad.
Same will work with AI too, while people continue to crap on AI, things will only improve, people will be more productive with AI, get more and bigger things done for cheaper and better. This is just inevitable given how things are going now.
>>There's a PM who takes your task and gives it to a "developer" who potentially has never actually written a line of code, but maybe they've built a WordPress site by pointing and clicking in Elementor or something.
In the peak of outsourcing wave. Both the call center people and IT services people had internal training and graduation standards that were quite brutal and mad attrition rates.
Exams often went along the lines of having to write whole ass projects without internet help in hours. Theory exams that had like -2 marks on getting things wrong. Dozens of exams, projects, coding exams, on-floor internships, project interviews.
>>After dozens of hours billed you will, in fact, get code where the new file wasn't added to the csproj or something like that, and when you point it out, they will bill another 20 hours, and send you a new copy of the project, where the test always fails. It's exactly like this.
Most IT services billing had pivoted away from hourly billing, to fixed time and material in the 2000s itself.
>>It's exactly like this.
Very much like outsourcing. AI is here to stay man. Deal with it. Its not going anywhere. For like $20 a month, companies will have same capability as a full time junior dev.
This is NOT going away. Its here to stay. And will only get better with time.
11 replies →
I think that was the point of the comparison..
It's not like a regular junior developer, it's much worse.
And yet it got the job and lots of would be juniors didn’t, and it seems to be costing the company more in compute and senior dev handholding. Nice work silicon valley.
> That said, I really don't see how it could meaningfully replace an intern however
And even if it could, how do you get senior devs without junior devs? ^^
What is making it difficult for Junior devs to be hired is not AI. That is a diversion.
The raise in interest rates a couple of years ago triggered many layoffs in the industry. When that happens salaries are squeezed. Experienced people work for less, and juniors have trouble finding job because they are now competing against people with plenty of experience.
Simple, there are always people who are intentionally using the hard way. There is a community programming old 16-bit machines for example, which are much harder than modern tools. Or someone learning assembly language "just for fun".
Some of those (or similar) people will actually learn new stuff and become senior devs. Yes, there will be much fewer of them, so they'll command a higher salary, and they will deliver amazing stuff. The rest, who spend their entire carrier being AI handlers, will never raise above junior/mid level.
(Well, either that or people who cannot program by themselves will get promoted anyway, the software will get more bugs and less features, and things will be generally worse for both consumers and programmers... but I prefer not to think about this option)
Sounds like a next quarter problem (I wish it was /s).
> It's like you have a senior phd level intelligence developer except they don't even read what you're telling them, and have 0 agency to understand what they're actually doing.
Is that better?
Did you miss the "except" in his sentence? He was making the point this is worse than junior devs for all reasons listed.
I was agreeing with him, by saying that the comparison is awful.
Not sure how it can be read otherwise.
1 reply →
This field (SE - when I started out back in late 80s) was enjoyable. Now it has become toxic, from the interview process, to imitating "big tech" songs and dances by small fry companies, and now this. Is there any joy left in being a professional software developer?
Making quite a bit of money brings me a lot of joy compared to other industries
But the actual software part? I'm not sure anymore
> This field (SE - when I started out back in late 80s) was enjoyable. Now it has become toxic
I feel the same way today, but I got started around 2012 professionally. I wonder how much of this is just our fading optimism after seeing how shit really works behind the scenes, and how much the industry itself is responsible for it. I know we're not the only two people feeling this way either, but it seems all of us have different timescales from when it turned from "enjoyable" to "get me out of here".
My issue stems from the attitudes of the people we're doing it for. I started out doing it for humanity. To bring the bicycle for the mind to everyone.
Then one day I woke up and realized the ones paying me were also the ones using it to run over or do circles around everyone else not equipped with a bicycle yet; and were colluding to make crippled bicycles that'd never liberate the masses as much as they themselves had been previously liberated; bicycles designed to monitor, or to undermine their owner, or more disgustingly, their "licensee".
So I'm not doing it anymore. I'm not going to continue making deliberately crippled, overly complex, legally encumbered bicycles for the mind, purely intended as subjects for ARR extraction.
15 replies →
I started coding at a young age, but entered the professional world in 2012, just like you. I feel the same. I just can't come to grips with the fact that the goal is not to write good software anymore, but to get something, anything out the door on which we can then sell by marketing it based on stuff it doesn't do yet (but it will, we promise!) so that we can make more money and fake making something "new" again (putting a textbox and a button, and hooking it up to an LLM api). Software is nowadays assumed to not work properly. And we're not allowed to fix it anymore!
It happens in waves. For a period, there was an oversupply of cs engineers, and now, the supply will shrink. On top of this, the BS put out by AI code will require experienced engineers to fix.
So, for experienced engineers, I see a great future fixing the shit show that is AI-code.
Each time that I arrive at a new job, I take some time to poke around at the various software projects. If the state of the union is awful, I always think: "Great: No where to go but up." If the state of the union is excellent, I think: "Uh oh. I will probably drag down the average here and make it a little bit worse, because I am an average software dev."
So many little scripts are spawned and they are all shit for production. I stopped reviewing them, pray to the omnissiah now and spray incense into our server room to placate the machine gods.
Because that shit makes you insane as well.
No, there is absolutely no joy left.
I've been looking at getting a CDL and becoming a city bus driver, or maybe a USPS driver or deliveryman or clerk or something.
I hear you. Same boat just can't figure out the life jacket yet. (You do fine wood work, why not that? I am considering finding entry level work in architecture myself - kicking myself for giving that up for software now. Did not see this shit show coming.)
1 reply →
[flagged]
A very very small percentage of professional software developers get that.
1 reply →
At least we can tell the junior developers to not submit a pull-request before they have the tests running locally.
At what point does the human developers just give up and close the PRs as "AI garbage". Keep the ones that works, then just junk the rest. I feel that at some point entertaining the machine becomes unbearable and people just stops doing it or rage close the PRs.
When their performance reviews stop depending upon them not doing that.
Microsoft's stock price is dependent on them proving that this is a success.
> Microsoft's stock price is dependent on them proving that this is a success.
Perhaps this explains the recent firings that affected faster CPython and other projects. While they throw money at AI but sucess still doesn't materialize, they need to make the books look good for yet another quarter through the old-school reliable method of laying off people left and right.
What happens when they can't prove that and development efficiency starts falling, because developers spend 50% of their time battling copilot?
7 replies →
I am shaking with laughter reading this phrase. You got me good here. It is the perfect repurpose of "rage quit" for the AI slop era. I hope that we see some MSFT employees go insane from responding to so many shitty PRs from LLMs.
One of my all time "rage quit" stories is Azer Koçulu of npm left-pad incident infamy. That guy is my Internet hero -- "fight the power".
Better yet, deploy their own LLM to close the PRs.
> Interesting that every comment has "Help improve Copilot by leaving feedback using the or buttons" suffix, yet none of the comments received any feedback, either positive or negative.
The feedback buttons open a feedback form modal, they don’t reflect the number of feedback given like the emoji button. If you leave feedback, it will reflect your thumbs up/down (hiding the other button), it doesn’t say anything about whether anyone else has left feedback (I’ve tried it on my own repos).
This whole thread from yesterday take a whole different meaning: https://news.ycombinator.com/item?id=44031432
Comment in the GitHub discussion:
"...You and I and every programmer who hasn't been living under a rock knows that AI isn't ready to be adopted at this scale yet, on the premier; 100M-user code-hosting platform. It doesn't make any sense except in brain-washed corporate-talk like "we are testing today what it can do tomorrow".
I'm not saying that this couldn't be an adequate change some day, perhaps even in a few years but we all know this isn't it today. It's 100% financial-driven hype with a pinch of we're too big to fail mentality..."
"Big data" -> "Cloud" -> "LLM-as-A(G)I"
It's all just recycled rent seeking corporate hype for enterprise compute.
The moment I had decided to learn Kubernetes years ago, got a book and saw microservices compared to 'object-oriented' programming I realized that. The 'big ball of mud' paper and the 'worse is better' rant frame it all pretty well in my view. Prioritize velocity, get slop in production, cope with the accidental complexity, rinse repeat. Eventually you get to a point where GPU farms seem like a reasonable way to auto-complete code.
When you find yourself in a hole, stop digging. Any bigger excavator you send down there will only get buried when the mud crashes down.
> improve Copilot by leaving feedback using the or buttons" suffix, yet none of the comments received any feedback, either positive or negative
Why do they even need it? Success is code getting merged 1st shot, failure gets worse the more requests for changes the agent gets. Asking for manual feedback seems like a waste of time. Measure cycle time and rate of approvals and change failure rate like you would for any developer.
It's like you have a junior developer except they don't even read what you're telling them, and have 0 agency to understand what they're actually doing.
Anyone who has dealt with Microsoft support knows this feeling well. Even talking to the higher level customer success folks feels like talking to a brick wall. After dozens of support cases, I can count on zero hands the number of issues that were closed satisfactorily.
I appreciate Microsoft eating their dogfood here, but please don't make me eat it too! If anyone from MS is reading this, please release finished products that you are prepared to support!
> How are people reviewing that? 90% of the page height is taken up by "Check failure",
Typically, you wouldn't bother manually reviewing something until the automated checks have passed.
I dunno, when I review code, I don't review what's automatically checked anyways, but thinking about the change/diff in a broader context, and whatever isn't automatically checked. And the earlier you can steer people in the right direction, the better. But maybe this isn't the typical workflow.
It's a waste of time tbh; fixing the checks may require the author to rethink or rewrite their entire solution, which means your review no longer applies.
Let them finish a pull request before spending time reviewing it. That said, a merge request needs to have an issue written before it's picked up, so that the author does not spend time on a solution before the problem is understood. That's idealism though.
The reality is more nuanced, there are situations where you'd want to glance over it anyway, such as looking for an opportunity to coach a junior dev.
I'd rather hop in and get them on the right path rather than letting them struggle alone, particularly if they're struggling.
If it's another senior developer though I'd happily leave them to it to get the unit tests all passing before I take a proper look at their work.
But as a general principle, please at least get a PR through formatting checks before assigning it to a person.
>> And the earlier you can steer people in the right direction, the better.
The earliest feedback you can get comes from the compiler. If it won't build successfully don't submit the PR.
This comment on that PR is pure gold. The bots are talking to each other:
https://github.com/dotnet/runtime/pull/115732#issuecomment-2...
"I wonder if there is some difference in their internal processes that are leaking through here?"
Maybe, but likely it is reality and their true company culture leaking through. Eventually some higher eq execs might come to the very late realization that they cant actually lead or build a worthwhile and productive company culture and all that remains is an insane reflection of that.
> How are people reviewing that?
I agree that not auto-collapsing repeated annotations is an annoying bug in the github interface.
But just pointing out that annotations can be hidden in the ... menu to the right (which I just learned).
I'm not entirely sure why they're running linters on every available platform to begin with, it seems like a massive waste of compute to me when surely the output will be identical because it's analysing source code, not behaviour.
or press “a”
Hot take : the whole LLM craze is fed by a delusion. LLM are good at mimicking human language, capturing some semantics on the way. With a large enough training set, the amount of semantic captured covers a large fraction of what the average human knows. This gives the illusion of intelligence, and the humans extrapolates on LLM capabilities, like actual coding. Because large amounts of code from textbooks and what not is on the training set, the illusion is convincing for people with shallow coding abilities.
And then, while the tech is not mature, running on delusion and sunken costs, it's actually used for production stuffs. Butlerian Jihad when
I think the bubble is already a bit past peak.
My sophisticated sentiment analysis (talking to co-workers other professional programmers and IT workers, HN and Reddit comments) seems to indicate a shift--there's a lot less storybook "Ay Eye is gonna take over the world" talk and a lot more distrust and even disdain than you'd see even 6 months ago.
Moves like this will not go over well.
AI proponents would say you are witnessing third stage of 'First they ignore you, then they laugh at you, then they fight you, then you win'
> Butlerian Jihad when
I estimate two more years for the bubble to pop.
> @copilot please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
It’s probably the junior devs that get to review these PRs. That and interns.
> This whole thing would be fucking hilarious if I didn't feel so bad for the humans who are on the other side of this.
Which will soon be anyone who directly or indirectly relies on Microsoft technologies. Some of these PRs, including at least one that I saw reworked certificate validation logic with not much more than a perfunctory “LGTM”, have been merged into main.
Coincidentally, I wonder if issues orthogonal to this slop is why I’ve been getting so many HTTP 500 errors when using GitHub lately.