I think we need to have a little more sympathy for GitHub. You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figured out, but a little sympathy goes a long way. I personally wish them the best and hope their on-call people can go back to getting normal amounts of sleep soon.
Sympathy? It's a Microsoft company that is being ran with a consistency of a startup in early seed rounds. Their downtime is abhorrent and unacceptable as far as enterprise goes. Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.
This is uncharitable, rude, and pretty baseless. Unless you have a lot of direct, personal information about GitHub engineers, they’re dealing with a huge spike in traffic.
Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.
Their engineers almost certainly are not the ones making the decision to let all the new traffic degrade their service for their existing paid customers.
That seems a bit crass but the underlying sentiment stands. Microsoft has more money than God. 4T valuation. When you say you’re worth that much, no excuses. Figure it out.
Typically when a company gets acquired, they get thrown a bunch of terrible initiatives like “change your cloud provider” that distract from their mission, executives saying “more AI!” and they definitely don’t hear anyone in management say “prioritize technical debt”.
It's a shame that the GitHub org that we use at my job that we pay a lot of money for gets affected the same way my personal nonsense does.
I don't know the architecture or any of that, but I feel like there could be (and it's not like they would've really known this until the last year or two with the massive spike) separate infrastructure for paid users/orgs vs free the same way they make the distinction with enterprise.
I get the massive load changes that they are under over the last two years, but why does a bunch of vibe coded slop take down the same resources that my company pays for every single month and has for years? I imagine properly splitting that out would be an absolute headache and not worthwhile for them vs stabilizing the rest of the service, but damn it sucks when I get blocked at work because GH is down.
I think somewhere down the line you have an answer to problem itself. Its for a while that companies are not into the business of sympathies, we should probably not get into that side for the topic.
1. Github has enterprise users who paid for the service, their day job requires Github to be available and working
2. Github has generous free tier which is the one which is exploring a lot more with the AI generated code.
It is a complexity in itself but the traffic should have been separated, the free users should not be allowed to bring down Github for enterprise customers (Just to clarify, I am free user myself). And if they do not have capacity it would have been perfectly fine to push back or throttle new users/repositories.
I agree that they should reserve capacity for enterprises and paying users.
But I think they’re in a tough spot. GitHub has historically been a huge supporter, proponent, and provider for open source projects. Engineers are difficult customers, to say the least, and the community would likely freak tf out of the segmented traffic.
The logical, pragmatic, and justifiable answer doesn’t always align with your market.
They decided they needed to capture the whole open source ecosystem by turning open source work into social networking... on a proprietary platform (because open source is great, especially when it's others' software). That was before they joined Microsoft.
And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
I would have liked if they had left the open source projects alone and didn't create that FOMO for not using them.
> And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
Ding ding ding, we have a winner. I like AI. I work for an AI company. Still, Microsoft aggressively pushed GitHub users toward Copilot. They don't get to do that and complain about increased volume from AI-generated changes.
No Copilot + reasonable operation: the way things were
Copilot + reasonable operation: Well done!
No Copilot + being overwhelmed by AI commits: Sympathy.
Copilot + being overwhelmed by AI commits: "Where did that petard come from that's hoisting us?"
I have a lot of sympathy for the people working at GitHub trying to keep it going, but I do not have sympathy for GitHub. That's what they're paid for. I have things to get done, and it has no sympathy for me.
Both Microsoft and GH issue is capacity which is being solved with huge capex in DC investment, you can't create a new DC overnight and its even harder in the current climate where everyone assumes the DC is for AI and not just normal workload compute like what GH needs...
If you do the math, their numbers are something like 1000 commits per second. That's not pushes per second either, it's just the total number of commits summed up over a week, not all of which trigger much of anything to happen.
Yes, that is scale. And yes, that's not actual requests per second. But it's the sort of scale that big (and even mid-sized) tech has known how to deal with for decades. Microsoft doesn't have an excuse.
> have a little more sympathy for the billion dollar company
No.
Their leadership went all in on AI and in the last blog post essentially admitted some missing test coverage for a critical path. Time to learn lessons and fix your vibe coded shitslop and stop using "user graph go up" as some kind of excuse.
Yeah, all those 1100 commits per second. Like, is this the standard for 'scale of traffic' that we're impressed by these days? And it's not like this snuck up on them. The graph shows a very consistent trend that gave them months and months to respond.
idk man, we pay stupid amounts of money to microsoft, we are an enterprise customer, expecting better availablity compared to my laptop isnt really a high bar.
For that matter, I admire their relative transparency about their incidents. I can think of other big players who will cheerfully show green statuses across the board while everyone can see that their pants are down...
I strongly suspect that quite a few HN members are busy creating agents that are furiously stuffing GH.
As I said before (and it was not a popular comment), it's easy to be an "armchair quarterback," with these services, as I think the brittleness was already baked in, and just waiting for the right time to crack. The only true way to have a robust platform, is to design something that will scale, from the start, and many startups don't do that, because they are feverishly trying to get out an MVP; even if it is a mess of bubblegum and baling wire.
They always say "We'll get it done right, once we get funding," but that never happens.
But my sympathy is limited by the fact that MS paid a lot of money for this, quite a while ago, and that is one company that knows all about issues of scale. They should have seen this coming.
GitHub literally went all in on doing almost nothing but AI-related feature additions for Copilot. So they built their own prison.
I don't have sympathy for them. They have disrupted my work so much in the past month that it's ridiculous, and my company likely pays them millions of dollars. And you can see from this site's plotting that there is a trend towards more frequent and more critical outages. It has been extremely bad the past month.
Our company pays for GitHub. We're paying for a broken product that stops our work. I don't have sympathy for the companies whom I pay for a product and give me broken shit in return. This is entirely preventable and their own fault.
A restaurant makes pizzas. They suddenly get 100x more popular. They can't make 100x more pizzas. But they are still taking orders from 100x more people. Not only are they not getting enough pizzas delivered that they took orders for, but in their rush to make and deliver more pizzas, they set the kitchen on fire, which makes an even longer wait for pizzas.
When the pizza you ordered doesn't get delivered, do you have sympathy for the restaurant? Or do you tell them to stop taking orders they can't fill and try not to set the kitchen on fire?
Now consider the pizza restaurant has 21 billion dollars in cash, is taking your money, and not giving you pizza.
"sympathy" for one of the worst corporations in history is some insane corp bootlicking logic.
This is a multi billion dollar corporation. History of robbing and stealing from others of their labor or IP. History of enshittifying once great services.
Give me a break. Sympathy? For microsoft? That might have flown when github was like seven people, but they have nearly unlimited resources to make it better. They're just choosing not to. Let's talk contracts and money before we pull the sympathy card.
I used to be on-call in a high-traffic environment where single customers pushed more bits than entire nations. I chose the role. I didn't want people's sympathy, if anything, I wanted them to complain to management.
If it gets too bad they can quit. Maybe that would be for the best, just wear the thing down until it outright fails and no one wants to touch it. One less bullshit service sucking all of the oxygen out.
Given that the outages are caused by record traffic, so far they are only cooked in the Yogi Berra sense of “Nobody goes there anymore, it’s too crowded”
Gets me wondering… if I get used to the fact that some location is busy, I stop going there. Other people might do the same. Could this create a detectable wavy signal over time? People stop going because it’s busy. It stops being busy. People go because it’s not busy. Repeat. Is this a known phenomenon?
Random AI slop code pushers don't pay the bills. Traffic isn't revenue.
I've started suggesting to my CTO that we move to something else because we do pay the Github bills and yet we are also affected by other people's activity. Avoiding that is not a hard engineering problem for an alternative.
A world where a record amount of code is pushed to GH but a large amount of paying customers leave would be "cooked"
Yeah I don't feel bad for Microsoft, but I don't feel bad for people who keep paying them for it either. This has been going on forever now. Nothing will happen until people start leaving. I don't honestly understand why so much shit is on GitHub when there are so many alternatives, anyways. It's fucking git! Can't people do or try anything different, ever?
Near the end of the 8.5 years that I worked at GitHub as an enterprise support engineer, I asked in an all hands if a "GitHub Classic" product had been considered. Much like World of Warcraft Classic, I imagined it would be a rewrite focused on matching the simpler feature set of the past.
I was basically given the same response that blizzard gave that question; "you think you want that but you don't".
Ahhh, I think the author mixed up two values here. That value seems to actually be the average over the past 3 months.
const incidents = e.detail.incidents;
// ...snip...
const now = new Date();
const threeMonthsAgo = new Date(now);
threeMonthsAgo.setMonth(threeMonthsAgo.getMonth() - 3);
// ...snip...
var currentFreq = recentIncidents.length / 3; // <- We out here, smoking these guns with our homeboy Claude
// ...snip...
var earliest = null;
for (var j = 0; j < incidents.length; j++) {
var dd = new Date(incidents[j].started_at); // <- eventually incidents[j].started_at is "2016-03-01T07:07:37.000Z"
if (!earliest || dd < earliest) earliest = dd;
}
// ...snip...
document.getElementById('n-since').textContent = earliest
? earliest.toLocaleDateString('en-US', { month: 'long', year: 'numeric' })
: '?';
document.getElementById('n-rate').textContent = Math.round(currentFreq * 10) / 10;
#n-since is going to be either march or feburary. It'll change depending on your timezone because JS's Date object always shifts the date around to match the same instant but in the system's timezone.
#n-rate has nothing to do with the #n-since month, it's just the last trailing 3 months. And even then, it's sort of underbaked? It's moving the date back by 3 calendar months not taking into account differing numbers of days, so it'll under-report short months.
I wouldn't trust the stats here.
Edit: whoops, author updated the template while I was writing this! It now says "Over the last 3 months", though that's still calendar months.
Not sure whether it has been updated since your comment, but the sentence now reads:
GitHub has had 1125 incidents since March 2016. Over the last 3 months, they've averaged 24 incidents per month
edit: although they also have 1.2 days of downtime (in a day) for their 'worst days' of downtime table, which suggests some auto number crunching is not working as expected.
Yes I tweaked it! The number and copy were mismatched and are no longer!
That worst day is likely an overlapping incidents accounting issue; I tried to account for overlapping incidents in another view but probably failed to port it over there.
I feel for the team having to deal with these problems at github today. Its also beyond me how those in charge for years wouldn't have seen this coming.
Github was bought by Microsoft, whether they wanted to acknowledge that internally or not. Microsoft went deep on LLMs, and specifically on LLMs for coding use cases. They must have recognized that LLM generated code and PRs would effectively DDoS GitHub.
I can only assume they simply didn't care, likely driven by greed.
It's not making headlines, but Google and Apple seem to be getting hammered by AI as well. Getting app updates approved to release on prod on Play Store / App store used to take days, now it's taking weeks.
People don't hate Excel. Well - maybe programmers who don't use it hate it.
In reality, Excel is the only reason Microsoft still exists.
Excel is how they lure you in. Excel really is the best spreadsheet, its the only one every doc you are sent works with, but also the one people know inside and out. And if you are into Pivot tables, PowerQuery, etc there is just no escaping it. You are utterly stuck.
But here is how they get you: its not much more to rent the whole office suite compared to just Excel. And as long as you are into it that far, Outlook and Teams are much cheaper than GSuite + Slack. And still...okish? No, not really. But we pretend. And it wouldn't happen without Excel.
Getting rid of Actions and Copilot and other secondary services almost halves Github's incident rate: https://i.imgur.com/XPcMIFr.png
I'm a big fan of Github Actions and I think people are often a little too harsh on it, but it's clear that it's sad that it's come at such a high cost to the platform's stability
GitHub Actions is really, really badly designed. The security model is fundamentally broken, the YAML hell is as bad as any, the log streaming lags like hell, they charge self-hosted runners for using their coordination plane, jobs queuing is really slow, and their software for actually running jobs is cursed and designed in a way that is practically hostile to self-hosting.
GitHub Actions is god-awful. Have you ever used any other CI tools?
It’s ironic that the SCM that 2026 software engineers landed on, is one with a complicated distributed usage model and is slow/down because of a centralised service
An important thing to consider is how much of their uptime without incidents is not the normal working hours. Their incident-free uptime on 9-5 EST, Mon-Fri, is probably like 60%.
6 years is a good run. For comparison, every 6 years there's a not insignificant chance you die. A restaurant that lasted 6 years would be an outstanding success. 6 years ago the very first LLMs were a novelty toy. 6 years before that was just after HTTPS became popular. 6 years before that was about when the iPhone came out.
Hey! isgithubcooked.com is my site; I made this site in February '26, I think
The contribution graph as an outage calendar idea is a commonly recurring one :). I definitely saw it somewhere else as a static asset first before I made this site.
I think we need to have a little more sympathy for GitHub. You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figured out, but a little sympathy goes a long way. I personally wish them the best and hope their on-call people can go back to getting normal amounts of sleep soon.
Sympathy? It's a Microsoft company that is being ran with a consistency of a startup in early seed rounds. Their downtime is abhorrent and unacceptable as far as enterprise goes. Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.
This is uncharitable, rude, and pretty baseless. Unless you have a lot of direct, personal information about GitHub engineers, they’re dealing with a huge spike in traffic.
Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.
8 replies →
Sympathy negated by their 2026 Pwnie award: "Lamest Vendor"
https://this.weekinsecurity.com/microsoft-wins-lamest-vendor...
> Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.
I gather that you have intimate and deep knowledge on the teams and the problems they try to solve there.
6 replies →
If you haven't read this, you really have no idea what they are dealing with. https://cursor.com/blog/git-at-any-scale
Their engineers almost certainly are not the ones making the decision to let all the new traffic degrade their service for their existing paid customers.
That seems a bit crass but the underlying sentiment stands. Microsoft has more money than God. 4T valuation. When you say you’re worth that much, no excuses. Figure it out.
3 replies →
Typically when a company gets acquired, they get thrown a bunch of terrible initiatives like “change your cloud provider” that distract from their mission, executives saying “more AI!” and they definitely don’t hear anyone in management say “prioritize technical debt”.
and it has been so damn slow for a number of years now, it drives me insane.
> I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers.
I don't think this is commendable at all. I give GitHub a lot of money and I'm tired of it being wasted with downtime.
It's a shame that the GitHub org that we use at my job that we pay a lot of money for gets affected the same way my personal nonsense does.
I don't know the architecture or any of that, but I feel like there could be (and it's not like they would've really known this until the last year or two with the massive spike) separate infrastructure for paid users/orgs vs free the same way they make the distinction with enterprise.
I get the massive load changes that they are under over the last two years, but why does a bunch of vibe coded slop take down the same resources that my company pays for every single month and has for years? I imagine properly splitting that out would be an absolute headache and not worthwhile for them vs stabilizing the rest of the service, but damn it sucks when I get blocked at work because GH is down.
3 replies →
I think somewhere down the line you have an answer to problem itself. Its for a while that companies are not into the business of sympathies, we should probably not get into that side for the topic.
1. Github has enterprise users who paid for the service, their day job requires Github to be available and working
2. Github has generous free tier which is the one which is exploring a lot more with the AI generated code.
It is a complexity in itself but the traffic should have been separated, the free users should not be allowed to bring down Github for enterprise customers (Just to clarify, I am free user myself). And if they do not have capacity it would have been perfectly fine to push back or throttle new users/repositories.
I agree that they should reserve capacity for enterprises and paying users.
But I think they’re in a tough spot. GitHub has historically been a huge supporter, proponent, and provider for open source projects. Engineers are difficult customers, to say the least, and the community would likely freak tf out of the segmented traffic.
The logical, pragmatic, and justifiable answer doesn’t always align with your market.
1 reply →
Do we have a breakdown on how much traffic is free users vs paid users?
1 reply →
We have a business relationship so no we shouldn’t have any sympathy. They sell a service and they’re failing to provide it.
Why? If we have sympathy for Apple then we can have sympathy for Microsoft ...
2 replies →
They did that to themselves, and doubly so.
They decided they needed to capture the whole open source ecosystem by turning open source work into social networking... on a proprietary platform (because open source is great, especially when it's others' software). That was before they joined Microsoft.
And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
I would have liked if they had left the open source projects alone and didn't create that FOMO for not using them.
I have no sympathy.
> And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
Ding ding ding, we have a winner. I like AI. I work for an AI company. Still, Microsoft aggressively pushed GitHub users toward Copilot. They don't get to do that and complain about increased volume from AI-generated changes.
No Copilot + reasonable operation: the way things were
Copilot + reasonable operation: Well done!
No Copilot + being overwhelmed by AI commits: Sympathy.
Copilot + being overwhelmed by AI commits: "Where did that petard come from that's hoisting us?"
I don't typically have sympathy for businesses that fail to deliver a service as advertised.
I can have sympathy for the humans caught in the crossfire but only managing one nine of availability on a commercial service is not acceptable.
I have a lot of sympathy for the people working at GitHub trying to keep it going, but I do not have sympathy for GitHub. That's what they're paid for. I have things to get done, and it has no sympathy for me.
> they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
That is an active choice they are making, over and instead of any reliability for the rest of their users.
It is a problem they are embracing, and actively encouraging, for themselves.
"Leave the big trillion dollar corporation alone" is not the right idea.
Microsoft made $134B in net profit last year, and is currently valued at nearly $4T. They have the resources to solve this if they wanted to.
Both Microsoft and GH issue is capacity which is being solved with huge capex in DC investment, you can't create a new DC overnight and its even harder in the current climate where everyone assumes the DC is for AI and not just normal workload compute like what GH needs...
If you do the math, their numbers are something like 1000 commits per second. That's not pushes per second either, it's just the total number of commits summed up over a week, not all of which trigger much of anything to happen.
Yes, that is scale. And yes, that's not actual requests per second. But it's the sort of scale that big (and even mid-sized) tech has known how to deal with for decades. Microsoft doesn't have an excuse.
Their excuse is called Azure and if you’ve ever had to deal with it you’d know it’s an amazing(ly awful) excuse to have.
Maybe for a free account.
But we pay enterprise license and GitHub is a big dependency in our software flow.
If this continues to be a problem as an enterprise product they need to do something. Otherwise theyre are going to to start losing business
> have a little more sympathy for the billion dollar company
No.
Their leadership went all in on AI and in the last blog post essentially admitted some missing test coverage for a critical path. Time to learn lessons and fix your vibe coded shitslop and stop using "user graph go up" as some kind of excuse.
Yeah, all those 1100 commits per second. Like, is this the standard for 'scale of traffic' that we're impressed by these days? And it's not like this snuck up on them. The graph shows a very consistent trend that gave them months and months to respond.
idk man, we pay stupid amounts of money to microsoft, we are an enterprise customer, expecting better availablity compared to my laptop isnt really a high bar.
For that matter, I admire their relative transparency about their incidents. I can think of other big players who will cheerfully show green statuses across the board while everyone can see that their pants are down...
Constant building with AI is something that they (Microsoft) promote and are heavily invested in.
And then get penalized for in the market when they say we need to build more DCs and spend huge capex... which is what GH needs right now
I strongly suspect that quite a few HN members are busy creating agents that are furiously stuffing GH.
As I said before (and it was not a popular comment), it's easy to be an "armchair quarterback," with these services, as I think the brittleness was already baked in, and just waiting for the right time to crack. The only true way to have a robust platform, is to design something that will scale, from the start, and many startups don't do that, because they are feverishly trying to get out an MVP; even if it is a mess of bubblegum and baling wire.
They always say "We'll get it done right, once we get funding," but that never happens.
But my sympathy is limited by the fact that MS paid a lot of money for this, quite a while ago, and that is one company that knows all about issues of scale. They should have seen this coming.
> I think we need to have a little more sympathy for GitHub
We certainly do not.
Paying enterprise clients who can not push/pull code and collaborate on PRs (table-stakes) disagree with having a little more sympathy.
It’s commendable that they let people give them training data for free?
GitHub literally went all in on doing almost nothing but AI-related feature additions for Copilot. So they built their own prison.
I don't have sympathy for them. They have disrupted my work so much in the past month that it's ridiculous, and my company likely pays them millions of dollars. And you can see from this site's plotting that there is a trend towards more frequent and more critical outages. It has been extremely bad the past month.
Depends who you are aiming sympathy at. C-levels or Engineering?
No, we don’t.
Our company pays for GitHub. We're paying for a broken product that stops our work. I don't have sympathy for the companies whom I pay for a product and give me broken shit in return. This is entirely preventable and their own fault.
A restaurant makes pizzas. They suddenly get 100x more popular. They can't make 100x more pizzas. But they are still taking orders from 100x more people. Not only are they not getting enough pizzas delivered that they took orders for, but in their rush to make and deliver more pizzas, they set the kitchen on fire, which makes an even longer wait for pizzas.
When the pizza you ordered doesn't get delivered, do you have sympathy for the restaurant? Or do you tell them to stop taking orders they can't fill and try not to set the kitchen on fire?
Now consider the pizza restaurant has 21 billion dollars in cash, is taking your money, and not giving you pizza.
I’d have sympathy for the pre-Microsoft GitHub. For the thing they are now, backed by a top-5 company with trillions of dollars? Cry me a river.
"sympathy" for one of the worst corporations in history is some insane corp bootlicking logic.
This is a multi billion dollar corporation. History of robbing and stealing from others of their labor or IP. History of enshittifying once great services.
Give me a break. Sympathy? For microsoft? That might have flown when github was like seven people, but they have nearly unlimited resources to make it better. They're just choosing not to. Let's talk contracts and money before we pull the sympathy card.
I used to be on-call in a high-traffic environment where single customers pushed more bits than entire nations. I chose the role. I didn't want people's sympathy, if anything, I wanted them to complain to management.
If it gets too bad they can quit. Maybe that would be for the best, just wear the thing down until it outright fails and no one wants to touch it. One less bullshit service sucking all of the oxygen out.
Given that the outages are caused by record traffic, so far they are only cooked in the Yogi Berra sense of “Nobody goes there anymore, it’s too crowded”
Gets me wondering… if I get used to the fact that some location is busy, I stop going there. Other people might do the same. Could this create a detectable wavy signal over time? People stop going because it’s busy. It stops being busy. People go because it’s not busy. Repeat. Is this a known phenomenon?
Closest thing would be the “El Farol Bar problem”
https://en.wikipedia.org/wiki/El_Farol_Bar_problem
Many people think we're on the path to having separate webs for AIs and humans. This may be the beginning of that.
As much as I would like that to happen, I don't think it is at all possible until we solve the "is a human" problem.
Random AI slop code pushers don't pay the bills. Traffic isn't revenue.
I've started suggesting to my CTO that we move to something else because we do pay the Github bills and yet we are also affected by other people's activity. Avoiding that is not a hard engineering problem for an alternative.
A world where a record amount of code is pushed to GH but a large amount of paying customers leave would be "cooked"
Yeah I don't feel bad for Microsoft, but I don't feel bad for people who keep paying them for it either. This has been going on forever now. Nothing will happen until people start leaving. I don't honestly understand why so much shit is on GitHub when there are so many alternatives, anyways. It's fucking git! Can't people do or try anything different, ever?
1 reply →
Near the end of the 8.5 years that I worked at GitHub as an enterprise support engineer, I asked in an all hands if a "GitHub Classic" product had been considered. Much like World of Warcraft Classic, I imagined it would be a rewrite focused on matching the simpler feature set of the past.
I was basically given the same response that blizzard gave that question; "you think you want that but you don't".
> "GitHub has had 1125 incidents since February 2016, implying a monthly incident rate of 24"
1125 incidents / 126 months ≈ 8.9 incidents per month, not 24
still terrible, but why such an obvious error in the first sentence...
Ahhh, I think the author mixed up two values here. That value seems to actually be the average over the past 3 months.
#n-since is going to be either march or feburary. It'll change depending on your timezone because JS's Date object always shifts the date around to match the same instant but in the system's timezone.
#n-rate has nothing to do with the #n-since month, it's just the last trailing 3 months. And even then, it's sort of underbaked? It's moving the date back by 3 calendar months not taking into account differing numbers of days, so it'll under-report short months.
I wouldn't trust the stats here.
Edit: whoops, author updated the template while I was writing this! It now says "Over the last 3 months", though that's still calendar months.
Not sure whether it has been updated since your comment, but the sentence now reads:
edit: although they also have 1.2 days of downtime (in a day) for their 'worst days' of downtime table, which suggests some auto number crunching is not working as expected.
Yes I tweaked it! The number and copy were mismatched and are no longer!
That worst day is likely an overlapping incidents accounting issue; I tried to account for overlapping incidents in another view but probably failed to port it over there.
Should be fixed soon!
> GitHub has had 1125 incidents since March 2016. Over the last 3 months, they've averaged 24 incidents per month (↓ 5% vs prev 3mo).
Looks like they fixed it already
Prob used Copilot to write the excel formula...
I feel for the team having to deal with these problems at github today. Its also beyond me how those in charge for years wouldn't have seen this coming.
Github was bought by Microsoft, whether they wanted to acknowledge that internally or not. Microsoft went deep on LLMs, and specifically on LLMs for coding use cases. They must have recognized that LLM generated code and PRs would effectively DDoS GitHub.
I can only assume they simply didn't care, likely driven by greed.
could have been a page with a static 'Yes' and a significant portion of time it'd be accurate.
Relevant xkcd: https://xkcd.com/2236/
It's not making headlines, but Google and Apple seem to be getting hammered by AI as well. Getting app updates approved to release on prod on Play Store / App store used to take days, now it's taking weeks.
If backend GitHub services are anything like GHES then I’m surprised it even managed to scale this much.
Is GitHub Cooked? Decidedly yes.
Migrating to something else has been raised as a concern during every engagement with clients and prospective clients in the past year.
"Do you use GitHub?"
"Yes though we'd like to move to something else, we just don't know what yet."
As soon as the next big thing shows up, they're done. And they know it.
I wonder if this is a Microsoft Excel situation where everyone
1. Hates the status quo
2. Only uses a handful of the 10,000 features the status quo has
3. Uses a different handful than everyone else
Because that model kept Excel as the center of buisnesses around the world for 30 years and counting
People don't hate Excel. Well - maybe programmers who don't use it hate it.
In reality, Excel is the only reason Microsoft still exists.
Excel is how they lure you in. Excel really is the best spreadsheet, its the only one every doc you are sent works with, but also the one people know inside and out. And if you are into Pivot tables, PowerQuery, etc there is just no escaping it. You are utterly stuck.
But here is how they get you: its not much more to rent the whole office suite compared to just Excel. And as long as you are into it that far, Outlook and Teams are much cheaper than GSuite + Slack. And still...okish? No, not really. But we pretend. And it wouldn't happen without Excel.
Getting rid of Actions and Copilot and other secondary services almost halves Github's incident rate: https://i.imgur.com/XPcMIFr.png
I'm a big fan of Github Actions and I think people are often a little too harsh on it, but it's clear that it's sad that it's come at such a high cost to the platform's stability
Actions aren't secondary to most paid GH users; and if they are down it usually means no deployments and no tests, which can often halt work.
GitHub Actions is really, really badly designed. The security model is fundamentally broken, the YAML hell is as bad as any, the log streaming lags like hell, they charge self-hosted runners for using their coordination plane, jobs queuing is really slow, and their software for actually running jobs is cursed and designed in a way that is practically hostile to self-hosting.
GitHub Actions is god-awful. Have you ever used any other CI tools?
Didn't GitHub CTO Vladimir Fedorov just posted this a couple days ago?
https://github.blog/news-insights/company-news/the-august-17...
My first thought today seeing GH actions were degraded was "some good those 3 million cores did"
It’s ironic that the SCM that 2026 software engineers landed on, is one with a complicated distributed usage model and is slow/down because of a centralised service
I’ve reached the point where I just feel a mourning kind of sadness for GitHub
https://mrshu.github.io/github-statuses/
Yes, but I did not need to look at the tracker to know that.....
I literally chuckled out loud when I saw the headline :D
An important thing to consider is how much of their uptime without incidents is not the normal working hours. Their incident-free uptime on 9-5 EST, Mon-Fri, is probably like 60%.
As a daily GitHub user in Australia I still haven’t figured out why everyone’s complaining about uptime. :)
> Their incident-free uptime on 9-5 EST, Mon-Fri, is probably like 60%.
And it may be even worse in EDT, which is currently in effect!
what's normal working hours? very US centric comment IMO. Europe, India, China, Latam etc. don't fall into your 9-5 EST normal working hour bucket.
A service like GH will still show a daily usage pattern, often with a peak somewhere around 16:00 UTC when most of the US and Europe are at work.
Im fairly certain that SWEs only exist on the US west coast
Now add an rss/atom feed link and you have my support
Oh, I thought it said "Github Outrage Tracker".
I was ready to click...
The GitHub Outrage Tracker is at this link: https://news.ycombinator.com/from?site=githubstatus.com
That one would be like iscaliforniaonfire.com
They were cooked the moment they got acquired by Microsoft.
This is why I foresaw that centralizing everything to GitHub was just generally a bad idea 6 years ago. [0]
Now that there is no CEO of GitHub, there is no point to GitHub improving.
[0] https://news.ycombinator.com/item?id=22867803
6 years is a good run. For comparison, every 6 years there's a not insignificant chance you die. A restaurant that lasted 6 years would be an outstanding success. 6 years ago the very first LLMs were a novelty toy. 6 years before that was just after HTTPS became popular. 6 years before that was about when the iPhone came out.
Possibly inspired by:
https://red-squares.cian.lol/
Hey! isgithubcooked.com is my site; I made this site in February '26, I think
The contribution graph as an outage calendar idea is a commonly recurring one :). I definitely saw it somewhere else as a static asset first before I made this site.
https://news.ycombinator.com/item?id=48034587