Comment by xnorswap
19 days ago
> Allegedly it's pretty terrible at C#
In my experience, LLMs in general are really, really bad at C# / .NET , and it worries me as a .NET developer.
With increased LLM usage, I think development in general is going to undergo a "great convergence".
There's a positive(1) feedback loop where LLM's are better at Blub, so people use them to write more Blub. With more Blub out there, LLMs get better at Blub.
The languages where LLMs struggle, with become more niche, leaving LLMs struggling even more.
C# / .NET is something LLMs seem particularly bad at, and I suspect that's partly caused by having multiple different things all called the same name. EF, ASP, even .NET itself are names that get slapped on a range of different technologies. The EF API has changed so much that they had to sort-of rename it to "EF Core". Core also gets used elsewhere such as ".NET core" and "ASP.NET Core". You (Or an LLM) might be forgiven for thinking that ASP.NET Core and EF Core are just those versions which work with .NET Core (now just .NET ) and the other versions are those that don't.
But that isn't even true. There are versions of ASP.NET Core for .NET Framework.
Microsoft bundle a lot of good stuff into the ecosystem, but their attitude when they hit performance or other issues is generally to completely rewrite how something works, but then release the new thing under the old name but with a major version change.
They'll make the new API different enough to not work without work porting, but similar enough to confuse the hell out of anyone trying to maintain both.
They've made things like authentication, which actually has generally worked fine out-of-the-box for a decade or more, so confusing in the documentation that people mostly tended to run for a third party solution just because at least with IdentityServer there was just one documented way to do it.
I know it's a bit of a cliche to be an "AI-doomer", and I'm not really suggesting all development work will go the way of the dinosaur, but there are specific ecosystem concerns with regard to .NET and AI assistance.
(1) Positive in the sense of feedback that increased output increases output. It's not positive in the sense of "good thing".
From a purely Schadenfreude perspective, I’d love to see Microsoft face karmic revenge for its abysmal naming “conventions”.
My impression is also that they are worse at C# than some other languages. In autocomplete mode in particular it is very easy to cause the AI tools to write terrible async code. If you start some autocomplete but didn't put an await in front, it will always do something stupid as it can't add the await itself at that position. But also in other cases I've seen Copilot write just terrible async code.
LLMs are terrible at writing hip-hop because they can only move forward.
Hip-hop is just natural language with extra constraints like rhythm and rhyme. It requires the ability to edit.
Similarly, types and PL syntax have more constraints than English.
Until transformers can move backward and change what they've already autocompleted, the problem you've identified will continue.
I rather suspect that it's bad at C# simply because there's much fewer open source C# code to train on out there than there is JavaScript, Python, or even Java. The vast majority of C# written out in real world is internal corporate apps. And while this is also true for Java, it has had a vast open source ecosystem associated with it for much longer than .NET.