← Back to context

Comment by tedivm

17 hours ago

It really has been remarkable watching GitHub just crumble as an organization. There's a lot of discussion about why: the switch from being independent to being part of Microsoft, having resources pushed to Copilot instead of core service, the organization structure itself, a reliance on vibe coding, etc etc.

Regardless of the reason, it's undeniable that GitHub is facing some serious issues. The unofficial status page[1] tells a horrifying story.

I would absolutely love to get some insider perspective on this (if only to learn how to prevent it from happening anywhere I work), but I think it's clear to anyone who has been paying any attention that GitHub is a sinking ship and the only reason people haven't abandoned it already is inertia. Considering how much else is changing in software right now I don't think inertia is enough to sustain a company.

1. https://mrshu.github.io/github-statuses/

> insider perspective on this

I do not work at MSFT but I don't feel that I need insider perspective to understand what's going on. GitHub is being managed the way other services get managed once they're bought by big companies. Initially fine, then starts to decline, then eventually craters. Everything becomes the numbers game.

Microsoft, Oracle, VMware, CA (where software goes to die), Salesforce, the list goes on. Every once in a great while there's a good M&A team that doesn't fuck it up but that's sadly rare.

  • I feel like MS went out of its way to make a point that GitHub and NPM would be independent orgs that no longer had to worry about making keep-the-lights-on money. It was positioned as a benevolent acquisition for the good of the development community.

    As so often happens, that didn't last long.

    Nest was originally independent. Didn't take long for it to merge with the Google Home brand.

    I'm sure there are countless other examples.

    • The problem IMO is that they filled GitHub with Microsoft folks who just don't have the engineering self-sufficient hacker culture that is required to balance the "attraction park" vibe that GitHub paired it with. So now it's just an attraction park for Microsoft employees to go and do silly work with teams of 100 that should have been done by a skilled team of 5 hackers.

      I was there for a couple years after the acquisition and just couldn't stand seeing it. I felt I was becoming useless working in a mad house that was becoming more maddening everyday. And MSFT just keeps replacing leadership with more and more disconnected people who just don't get it, who just never used GitHub like the OG users did. Two years ago I interviewed again for my old team, largely out of curiosity, and the Microsoft engineering manager asked me some brain teaser question as my interview. The disconnect is just too large.

      They don't take GitHub seriously. It's a toy to MSFT and vibes matter more than the product itself. And they hire for it using MSFT drone logic, fill it with people hired and profiled to be MSFT-lifers, and these two things don't mix.

      Sorry I don't have anything great to say. And of course, many of these MSFT folks were actually damn good, but they were swimming in a sea of MSFT drone.

    • > would be independent orgs that no longer had to worry about making keep-the-lights-on money

      It is honestly so shameful that we keep falling for this gambit. It is nothing more than a rank "but this time is different!"

      Economics is what drives things. It is what drives things in households and it is what drives things in companies.

      Unless times are truly great or the company is truly forward-looking, promises of freedom and independence from the business cycle is just an empty promise of creating a research lab.

      8 replies →

    • GitHub had no reason to sell to Microsoft, they could have remained the bootstrapped company they started as, and rode the SaaS boom, since they were profitable on day 1. Seems a bit unfair to blame Microsoft though, because it was the founders who decided they wanted that sweet VC funding and Andreessen was happy to pay out.

      Not sure if it mattered after that but they had that weird Tom Preston-Werner scandal that got him fired. Since he was the CTO, I kind of suspect that sent them on a collision course with needing to exit the VC round and Microsoft paid out.

    • This happens with almost every acquisition from Red Hat to WhatsApp.

      If companies actually meant it then they’d sponsor these projects instead of buying them. The reason they choose to buy is so that they can make decisions about the direction of that project. If not immediately, then at least at some point in the future.

    • GitHub was independent, and then AI happened.

      All long term business goodwill and reputation is simply there to burn to keep the bubble going.

    • > It was positioned as a benevolent acquisition for the good of the development community. call me a skeptic, but can (and has) such a model existed in a capitalist system?

  • I'm afraid this is a form of reversion to the mean. Successful startups are made of exceptional people: the founders, the initial investors, the first employees, the first clients. But when they get acquired by much larger companies, they are necessarily diluted in pool of people that are more "normal", less exceptional. This includes the customer base that is more "normal" as well. Slowly but surely, the extraordinary product/service the startup has been developing reverts to the mean. This is quite sad, because it feels inevitable. I'd like to know how to avoid it.

    • “I'd like to know how to avoid it.”

      To paraphrase a popular quote from IBM: “Executives and MBAs can never be held accountable: therefore executives and MBAs must not be allowed to make decisions.”

      Slightly less flippant: The only way to stop this is to stop letting companies like MSFT gobble up smaller companies. That doesn’t seem likely in the near future, though. Once the Borg assimilate something, it’s just a matter of time before it’s digested and drained of value.

    • That could be A problem, but to me THE problem is that the larger companies buy these smaller companies for resource extraction, not to make the product better.

      In this frame you can see that making the product worse (paying less for its upkeep) and raising prices are just two sides of the same coin - extract more resources.

      Almost no big company has any reason to shepherd a product in a way that's beneficial to its users because they have so much momentum that even changing their approach either costs too much money or those in power are too insulated from the outcomes (fix it for me or I will fire you while I continue to make bad choices and under fund the product).

    • It's not inevitable that the founders have to sell to big tech. They wanted money more than the excellence of the craft. They got the money, the company got to grow and made way more profit than when it was small scale but excellent. The wheel keeps turning.

  • Big companies usually buy other small companies for their users, once the users move to their platform, things change.

  • This is a general observation, no hard data, but I find there seems to be a wall at 2 years after an acquisition. By 2 years a lot of the best talent leave the company entirely or go somewhere else in the company. Things can cruise along just fine for a bit, but as the institutional knowledge slowly leaves it gets worse and worse. Couple that with the bureaucracy and insanity of a global mega-corporation, the quality fades slowly at first, then it picks up.

    • > I find there seems to be a wall at 2 years after an acquisition.

      It's called a vesting schedule. ;)

      What I've seen is that usually the founders and heavy hitters from the original company are very BS-averse and basically just stay around to collect their money and then jet for a situation that doesn't suck.

      For the rest of the gang, it tends to bifurcate: some folks stay at the big company indefinitely after the acquisition because while they can see the suck, nowhere else pays as well or is as cushy (I know people who have been thinking about leaving for 12 years). Still others excel at big company work and make a happy career out of it for a while but don't stay forever.

    • This is the flipside of MBA-brain. Treating people as replaceable equivalent cogs in a machine, thinking that the company itself, as an abstraction, is where value lies, when it lies just as much in the context and nurishing environment. You can't simply move a company from one place to another like a Lego brick and expect it to go on functioning as before, not as long as people have freedom to leave.

    • > but as the institutional knowledge slowly leaves

      I’d like to offer a different perspective: the “institutional knowledge” often (but not always, of course) are the old timers that have been gatekeeping knowledge in order to make themselves irreplaceable.

      I’ve seen this a couple of times, even in faang-sized companies.

      I’m not sure this is the case of GitHub though.

      It might be due to lower quality code spit out by some llm, reviewed by some llm and shipped to production by some llm-generated pipeline.

      Also, wasn’t github pushed to move to azure?

      Anyways, it surely is a strong signal of engineering culture degrading.

  • It's just beancounters doing what they do best, counting beans and screwing up what was previously alright.

    • It's very profitable in the short term, and later they can just move on elsewhere and do it to another company. It's not mismanagement at all, it's a solid strategy from the external point of view.

  • > GitHub is being managed the way other services get managed once they're bought by big companies. Initially fine, then starts to decline, then eventually craters

    Can you explain what you mean by this? Like what does "fine" mean? What, specifically in the management, is the "decline"? What does "craters" mean?

Wow. According to the current metric (87.25% uptime), github suffers a partial outage 3h/day.

https://onlineornot.com/uptime-calculator/87.25

  • In their defense, they dramatically "over"-report sev-2/3's (things like, avatar urls are not loading in saudi arabia), which makes their cumulative uptime look much worse than it is.

    If you filter for major/critical outages, their uptime of core services in trailing 12 months all have two 9's.

    https://isgithubcooked.com/?severities=major.critical

    Also, a huge part of their cumulatively-bad availability story is copilot, which is a functionality (LLM inference) that most organizations have struggled to get two 9's of availability in for the last 9 months.

    • As someone who relies on it for all of my workflows at a normal job, core functionality issues result in me not being able to get work done on at least a weekly basis reliably at this point, and it's been that way for months.

      The things aren't profile pictures not loading in saudi, they're botching merge jobs, git/api operations being down, pull requests not loading, etc. And that's on top of the plethora of UI bugs that have been pervasive for years that aren't blocking functionality.

      1 reply →

    • Two 9’s? You have to work pretty hard to do that badly. That’s like bragging you graduated with a C average from Harvard after your father endowed a chair to get you in.

      Given GitHub has become a utility service globally this should be frankly worrisome to everyone let alone the developer community actively using it. It’s intertwined into many things now beyond simply source code hosting and PRs. And I am surprised GitHub leadership is ok with the state of things. Having worked at a lot of 5-6 9’s shops, this would have been all hands on deck, all roadmaps paused, figure it out or perish sorts of stuff.

      1 reply →

    • That is a much more reasonable look, I am glad you alleviated our wor-

      Why is the graph more dense on the right side?

Some years ago I wondered how long will it take them to go they way sourceforge went. Once you grow too much without a proper leader, you will fall :(.

  • Sourceforge died in a very different way though. Bundling spyware/crapware in install packages for open source software was a serious breach of trust, and was pretty much the direct reason for mass migration to Github. Github is failing on the technical side, but they at least mostly have their brand value intact. I think it will still take quite a lot for a mass migration of the same scale to happen.

    • Microsoft specializes in taking successful products and pumping them full of malware, spyware, bloatware, and adware once they have a critical mass of users. It is often preceded by quality dropping significantly due to under investment and McKinsey being brought in to find a way to prop up declining revenues - of course the answer is never to invest in making it a superior product again, but monetization strategies.

  • Comparing GitHub and SourceForge as if they were cut from the same cloth is laughable to me. SF has always been a wretched hive of ads and dark patterns.

    • Not always. Before dice bought them they didn't do the ads. I even remember early on when you had to submit a project for approval before you got a CVS repo.

> The unofficial status page[1] tells a horrifying story.

If it weren't bad enough, github often has issues when the unofficial status page doesn't report them, so the actual number is even worse.

Have those outages actually been blocking your work? Somehow I haven't even noticed, just seen complaints on HN. I'm not saying it's not real, just wondering where the gap is.

  • Yes, many times. Roughly once a week this year my team or an associated team can't ship changes because PRs, GitHub Actions, or some other associated mechanism is down.

  • A big part of my job is doing code reviews, and its very common that pages or diffs just don't load. Or PRs literally don't appear in the PR list, even though they exist. It's a daily occurrence to play the 'is my internet down or is GitHub just being shit again?' game.

    Oh, and don't forget the cases where the diff view sometimes misses some files for unknown reasons. Both in the 'new experience' and the 'legacy view'. You just can't trust it as much anymore.

So what? People have to unlearn this kind of brand loyalty. Companies are not people and not your friends. They are in the business of making money. We need to be more aloof and simply use their tools when useful and not get emotionally attached. Most of the managers and likely the devs had a good deal. Good money, and if it collapses, people still have a good resume line and can move on. And we users can also move on. There are plenty of other service providers of code hosting and CI/CD.

> crumble as an organization

I think that is exaggerating things a bit... GitHub is alive and well, and they're hosting more and more projects each month. A few well-known projects leaving every now and then doesn't exactly spell doom for GitHub

All of that is revisionist history at best. GitHub was a pile a shit long before Microsoft bought it has everyone forgotten when it would be a coin-flip on any given day if the site was even functional?

GitHub was in the right place at the right time to be a success despite the fact it's a massively clobbered together mess.

  • While I wouldn't necessarily phrase it this way, there is a chart going around social media that tries to imply that GitHub had basically 100% uptime right up until the MS acquisition. All it takes is either 1) having been there or 2) a cursory search of HN to know that this is a complete fabrication.

They are mostly blaming their shift to vibe coding for their problems. https://github.blog/news-insights/company-news/an-update-on-...

  • Hm. I read that as saying that their users are writing more code with the assistance of LLMs, thus placing more stress on their systems. I do not read it as making any comment about their own practices.

literally zero nines of uptime lmao, do they win an award?

  • Even if you go service by service you're talking about critical things like `git` operations (literally what they're named for) at a single nine, and stuff that's pretty basic like static web hosting as only two nines. They literally can't even keep static webpages up.