Perspective from the trenches: I teach at a university that uses Canvas. We are in our final exams period right now.
We got our first email (from Academic Affairs) notifying us that it was down at 5:17pm EDT this afternoon, with little info; followup emails were sent at 6:24 and 6:57 with more info, but mostly about how we would be compensating for it and not about what actually was going on (other than, "nationwide shutdown" and "cybersecurity attacks", no further detail). I don't get a sense that they know much more than that, not that I would expect them to.
A perhaps telling detail: they're instructing us to have students email us directly with any work that had been submitted via Canvas. That suggests that they have no particular confidence that it will come back up soon.
I personally am only slightly affected; as a CS professor a lot of my students' work is done on department machines, and submitted that way, and I do the actual exams on paper. More importantly, I've never liked or trusted Canvas's gradebook, and so although I do upload grades to Canvas so students can see them, my primary gradebook is always a spreadsheet I maintain locally.
But I have a lot of colleagues for whom this is catastrophic at a level of "the whole building burnt down with all my exams and gradebooks in it"---even many of those that teach 100% in person have shifted much or all of their assessment into Canvas (using the Canvas "quiz" feature for everything up to and including final exams), and use the Canvas gradebook as their source-of-truth record. We've been encouraged to do so by our administration ("it makes submitting grades easier"). For faculty in that situation, they have few or zero artifacts that the students have produced, the students themselves don't have the artifacts to resubmit via email because they were done in Canvas in the first place, and they have no record of student grades or even attendance (because they managed that all inside Canvas). I guess they have access to the advisory midterm grades from March, if they submitted them (most do, some don't), but that might be it.
My gut feeling on this is that this is either resolved in hours (they have airgapped backups and can be working as soon as they can spin up new servers), or weeks (they don't). Very little in-between. And if that's true and we wake up tomorrow with this unresolved, I really have no idea what a lot of professors at my university and across the country are going to do to submit grades that are fair and reasonable. In the extreme case, they may have to revert to something we did in the pandemic semester (and before that, at my school, in the semester that two major academic buildings actually did burn to the ground a week before finals): let classes that normally count for a grade just submit grades as pass-fail. Because what else can you do?
(Well, one thing you can do is not put your eggs all in one basket, and not trust "the cloud" quite so much, but that ship's already sailed. I do wonder if in the longer term, anybody learns any lessons from this....)
UPDATE: As of 11:45pm EDT, my university's canvas instance is up and running! Here's hoping it stays (but I'll be downloading some stuff just in case...)
> the students themselves don't have the artifacts to resubmit via email because they were done in Canvas
It’s so simple to send an e-mail to the student with relevant records on completion of a quiz or whatnot. They don’t do it, because they want to control the data. (And universities don’t insist on it for who knows what reason.)
I've never used Canvas before, but all the LMSes that I've used allow students to enable emails whenever anything is updated, including when grades are posted. This is off by default because it's often 10+ emails a day, because many teachers post notes once a day, and with 5 classes, that adds up pretty quick. I personally have it enabled because it's pretty manageable with some custom Outlook rules, but setting this up is well beyond the capabilities of most students.
Students having records of what their score was doesn't prove to the professor / university what score they received. "FWD: Exam 1 Results" is not especially auditable.
Makes me glad I've always avoided doing my work on web platforms. When we used to have to make presentations in Google Slides I used to do them in Org-mode, then export to Sheets. I still have all those assignments sitting on my disk. Sure, there's versions of them on Google Drive, but I always make sure that the canonical version is the one on my disk.
Just to add one more data point, we also use Canvas at my university. The deadline for submitting who are eligible (i.e. passed compulsory assignments and labs) to take the exam was yesterday, and I couldn’t meet that deadline because Canvas went down. I usually do corrections offline so I have backups of my own evaluations, but these are courses with many teachers and many TAs, so Canvas is the way we sync our assessments.
I work in the Education sector as IT. We don't know much else either.
Everything we know has come from reddit threads / hackernews threads. There has been 0 official communication today indicating this was an attack, yet the login page was defaced by ShinyHunters.
I don't understand what's the panic and doomerism about. Any competent IT team has backups and will be up and running as they go back to a state before the breach. This is HN. I'm disappointed that everyone is talking about losing grades and going back to pen and paper. I don't see how that could happen in 2026.
And from the hacker's message itself, it's clear they want money in exchange for not releasing private info, not for the data itself.
Do we live in a fear based culture? Why the panic? Even if everything was hosted on Instructure's infrastructure, it's all AWS. I'd be VERY surprised if there aren't multiple way to go back to a previous state.
Most of the work and delay is to make sure they figure out where the breach occurred.
Here in the Netherlands a data center's power source (not even the machines) burnt down, data center is offline and University of Utrecht, one of the biggest universities here, is closed. Access passes don't work, work from home environment doesn't work, student information system is down, system for grading doesn't work. No failover for any of them (or maybe it was in the same DC?)
Backups can be sabotaged (turned off or schedules manipulated) or compromised (say, by lateral movement).
> Even if everything was hosted on Instructure's infrastructure, it's all AWS.
AWS Backup isn't foolproof. Get your hands on administrator credentials as an attacker and suddenly the only thing between everything being gone for good and unrecoverable even for AWS is remembering to have put a permanent deletion protection on all resources in AWS Backup.
To my European ears this just sounds like a disaster like this waiting to happen. God bless the annoying privacy OSS advocates and bureaucrats, I guess.
Reminds me of the incident last year when a South Korean government's server room caught fire, which contained the government equivalent of Google Drive, and the only backup was in the same room, and they all burnt down together.
Some data was permanently lost, and then officers told reporters that multi-regional backup was not yet built because it was too hard at such a massive scale... of 858 TB.
Backups are definitely helpful in ransomwares, but before systems can be restored and brought back online, victim organizations still need to assess the scope of the breach, find the initial access vector, identify compromised accounts, and evict the threat actor. That can take time.
I’m not certain, but it appears you’re giving Instructure a pass here, as if this is the first time they were hacked. But, it’s the second, by the same group.
As a parent of kids who are impacted by this, I’m not super concerned about the data being held for ransom, but I sure as fuck am concerned about how much it’s going to cost the district to move to another provider.
> let classes that normally count for a grade just submit grades as pass-fail. Because what else can you do?
Schedule a single exam and that's your grade for that subject? That's how it should work anyway, credits for work during semester (or worse attendance) are not needed to evaluate if someone learned the material, give them an exam and done.
That's just bad outdated practice. It leads to cramming and less remembering than of the demand is for students to do work and show learning and effort throughout the year.
I'm surprised how few comments there are on this thread. This is probably affecting millions of students at the most stressful time of the year.
Incidentally I've always hated Canvas and probably every other LMS provider, but what is particularly amusing about this current outage is that it is occurring at exactly the time when universities are demanding that all professors put all of their materials on Canvas, without exception, due to ADA compliance regulations. It is explicitly forbidden for professors to, e.g., refer to pdfs posted on a personal website.
Other commentators here seem not to understand that many faculty also do not enjoy being forced to use Canvas.
They have not succeeded in forcing me, yet. But it's sad how many computing faculty apparently can't operate the basic online infrastructure needed to support their courses. Not that universities make it easy for us.
And of course the other serious concern I have with Canvas is that they are likely using all the materials faculty upload to train their AI replacements. Many of my colleagues engage in dark humor about this but I haven't noticed much action.
Live streaming of class through Canvas is very popular. Quite a few people just watch from their dorms. So maybe people will have to come back to class, that will be entertaining. The class rooms are almost standing room only (sometimes they are) on the first day of class and then gradually thin out. Sometimes 10 or so people show up out of a class of 100. If Canvas is not back up soon I think it could actually be disruptive for that reason also.
This is awful to hear. The idea that students are just half assedly streaming the lectures is really just ruining things in the long run. This is a bit old manny, but showing up to lectures is good. You go to class, you get face time with professors, you can ask impromptu questions, you rub elbows with classmates, you talk on the walk between classes, you maybe run into a cute girl. Friction like walking to class and finding a nook in that annoying hour gap you have, are the things that make life enjoyable.
Not much overlap between students and HN these days, though? I’m an extremely rare outlier afaik :)
The administration has so far opened with one “Canvas said” and then an hour later one “Canvas is down indefinitely” email noting that they’re aware it’s serious.
(Canvas is a glorified wiki for teaching students, with quizzes and such, for those unaware.)
What? What makes Canvas accessible in a way that HTML and PDF files are not? It's true that PDF readers aren't the best for screenreaders, but surely you can just upload a .html copy as well.
Canvas has an easy way of checking if a pdf or other course material is accessible, so many universities are forcing faculty to put all their materials on Canvas. That way if a pdf or powerpoint is not compliant it is immediately flagged. The goal is to reach a "100% accessible" metric.
Note that little of this really helps the students that it is supposed to help, because as you wisely point out, raw HTML is almost by definition extremely accessible. I work in a field that uses Latex and the source code of Latex should also be considered more accessible than the compiled pdf. But for university administrators the only important thing is that the accessibility metric that appears (or used to appear, before today!) on Canvas shows 100% accessible.
Putting aside the "So you hate waffles?" non-sequitur, surely the entire topic of this thread should be a bit of a hint that this misguided policy has not, in fact, "[made sure] courses are fully accessible".
Accessibility regulations, implemented with feedback from faculty and with the support of university resources, are certainly a good thing. But that is not what is happening in my experience.
1. It should be illegal for any company to pay ransomware attacks. Period. No pay out ever.
2. The penalty for being the attacker should be linked to the system they violated. If you do this to a hospital and someone dies you are life in prison / chair. The minimum sentence should be so painful that it deters the attack.
No this will not stop this and companies need to be held accountable for their lack of security investment. Every attack should be investigate if the company met an agreed industry standards best practices and staffing, etc. The penalties for not meeting the requirements should be punitive.
It should be illegal to host insecure services, especially when you're dealing with PII. Breaches keep happening and nobody gives a fuck, because the worst that'll happen is you might lose a handful of customers and buy some "credit monitoring".
Incidents like this should be followed by an audit and charges being laid. Send corp officers to jail for negligent security failures. If you can go to jail for accounting fraud, you should be able to go to jail for cybersecurity-promises-fraud.
They claim to be compliant with a number of security standards [1]. I would love to see a postmortem audit of how much of this they actually implemented.
I don't think that criminal negligence is the most helpful legal tool for incentivizing improved security. It's too hard to prove negligence.
Instead, there should be standard civil penalties for leaking various degrees of PII paid as restitution to the affected individual. Importantly, this must be applied REGARDLESS of "certification" or whether any security practices were "incorrect" or "insufficient". Even if there's a zero-day exploit and you did everything right, you pay. That's the cost of storing people's secrets.
This would make operating services whose whole "thing" is storing a bunch of information about individuals (like Canvas) much more expensive. Good! It's far to cheap to stockpile a ticking time bomb of private info and then walk away paying no damages just because you complied with some out-of-date list of rules or got the stamp of approval from a certification org that's incentivized to give out stamps of approval.
People who haven’t been hacked just haven’t been looked at. If someone wants to hack you, they will hack you. It’s really unfortunate that people have this level of confidence in their ability.
If Boeing claimed a plane was airworthy, but it crashed because basic engineering controls were skipped, we have collectively put our faith in the NTSB to preserve evidence, run an independent technical investigation, etc. There is no such authority for software - most security auditors (SOC2, HITRUST, etc) are just looking at self-reported data.
Just take a look at the recent Epic vs. Health Gorilla lawsuit to see how nonexistent the protection is around exchanging your medical records, one of the most sensitive types of PII.
Shouldn’t we be focusing on making it harder to pay overseas criminals in the first place? /ahem/ crypto platforms facilitating transfers to bad actors /ahem/
interestingly, having actually done the law enforcement side of these investigations, 50% of them are local. And I understand that this is not 100% solution, but neither is any form of law enforcement, but that doesn't mean we should fail to attempt it.
Kids from the local uni having a lark, stalkers, vindictive ex employees, local gangs, criminals who understand their victims because they hail from the same community. These are your local hackers. Sift them from the nation states and international crime groups, then deal with the International as a matter of diplomacy. Because we do this so poorly locally, we have little ammunition to when it comes to diplomacy. "reduce attacks by your crime groups and we buy your natural gas, seel you wheat etc"
Want more motivation?- 75% of the local attacks by volume send funds back to terrorist or separatist organizations.
It is not an in-soluble problem. Sentences are a fraction of the answer, effective and receptive reporting processes are more important, then government backing for investigation and enforcement, then policy around home-team activities (ie don't do the bad things yourselves Mr Gov). Deterrence comes after all that.
Yeah, they identified themselves as ShinyHunters, and the IP they've put on the demonstration page is geocoded to Russia. Notice this is the same group responsible for the Infinite Campus hack last year.
Really, though, if you want someone to blame, Instructure is not a particularly compelling target. Let's review:
1. Iran is intentionally targeting infrastructure due to a war started by the current administration.
2. China is actively seeking corporate secrets to steal and commercialize for themselves, spurred by extreme protectionism and retaliatory tariffs.
3. North Korea is doing anything they can -- including just taking a remote job by proxy -- in order to extract any money.
4. And Russia is working with and aiding all of them, after everything else going on has forced the embargo to break.
5. All of this while completely alienating every single one of the United States' allies.
6. Meanwhile, the American DHS is currently shut down.
7. And this is after Trump cut funding and personnel for CISA severely enough they've had to end the contract with MS-ISAC, meaning all state and local entities can only remain in the organization if they foot the bill for it directly and CISA and other agencies responsible for cybersecurity are more thinly staffed than they have been in decades.
In short, the current administration systematically disassembled all the protections we have built over the last 100 years, and then placed infrastructure -- schools, in this case, but also power companies, water treatment facilities, communications companies, local governments, hospitals, food producers -- directly on the front lines of the modern geopolitical conflict.
That vast ocean that has kept us safe historically is a poor moat in the modern era.
Complete internet blockage of nations allowing the attacks. If foreign governments are you can always execute them. We are living in a different world where this is no longer a zero probability occurrence.
When will countries start treating cyberattacks as an act of war? If the North Korean military came to America and robbed fort Knox of $200M in gold there would be retribution. But hack an American company for the same amount and the feds do nothing.
Ok, so we treat it as an act of war. Now what? Attack North Korea? Great, the entire city of Seoul gets shelled within five minutes of your attack and hundreds of thousands of innocent people die.
It's very easy to play with lives that aren't yours.
How do you know which country to blame? It is standard practice for foreign actors (or just hackers in general) to use proxies around the world to misdirect and insert false clues as to their origin. It could be an American teenager proxying through North Korea, and it could be a North Korean proxying through another American teenager's residential connection, there's no way to know.
They already do. This is what asymmetric warfare looks like, your weakest links will break in a time of crisis. Focusing on retribution for the Dunder Mifflin cyberattack is pointless, the adversarial motivation is purely to disrupt and extort.
The best response to a cyberattack on critical systems is to take security seriously. Document the offense, avoid the same mistakes and invest in penetration testing. Of course, nobody is incentivized to do that until they're attacked, so the cycle perpetuates itself.
If someone robs a bank and someone inside dies of a heart attack, thats felony murder. I would be happy if the same applied to ransom attacks or other blackmail/leaking of info. If someone commits suicide because of it, its murder.
felony murder is pretty widely regarded as a leading factor in incredibly unjust prosecutions and sentencing decisions. perhaps not the best concept to build your ideas on top of.
> If you do this to a hospital and someone dies you are life in prison / chair.
If you're going to get the chair you might as well murder some witnesses or destroy some systems to hide the fact you got hacked. "Hack? What hack? Our servers all burned down in an arson attack".
We could also throw the CEOs of companies who don't properly secure their infrastructure and pay their security engineers enough in jail. A little justice on both ends.
Uh, who determines that the infrastructure wasn't properly secured? Who is willing to risk prison because some intern accidentally committed an API key or made a dumb mistake. Conversely, what's the chances that no one actually gets prosecuted regardless of how sloppy their security practices are?
Failure to protect computer system from forseen failure should result passing corporate veil and resulting all stock holders and managers/leadership of funds to be jailed for same period as perpetrator. It is only way to ensure that these things are taken seriously and enough pressure is put on leadership of companies.
> No this will not stop this and companies need to be held accountable for their lack of security investment.
I think in principle, its sound. Im also just baffled hearing anecdotes from friends that are in big corp world and hearing the type of incidents they have, and how they respond to it.. It makes me wonder if there is enough capable talent to go around for the "boring corp" crowd.
Hint: I don't think there is nearly enough talent to go round, but for these companies, its either that they think they have solid experts (and didn't), OR its not a real priority until you get hit.
My kids are in the middle of their finals week. What a mess. Universities know nothing, Canvas claims to be in a "scheduled maintenance", and one Prof claims to "not have any copies of material offline" which seems pretty negligent. Sounds like one section of a popular class will be doing paper exams while other sections had Canvas-based "half points for 2nd attempt"-type exams earlier today. How soon before names & grades appear in data dumps?
This would be like TurboTax "scheduling maintenance" on April 14th in the US.
The "Scheduled Maintenance" is just total B.S. and just honestly makes them look worse. Apparently according to their status pages this is what 99.996% uptime looks like. Pay attention lol.
It has been over 5 hours now and there has not been any communication about this being an attack, despite many of us seeing the ShinyHunters message on the login page.
There is a lot of people who likely are unaware the latest outage is because they were compromised again.
Them marking the incident as 'Under Maintenance' means the status page isn't reporting this as an outage and adding to downtime%.
I was going to make a joke that they should have just taken a page from the military and said “Rapid Unscheduled Maintenance”, but I guess that’s actually the phrase for it.
Once again, an example of why corporations should not have free speech. Corporate statements that are transparent lies should be criminally actionable.
A friend who teaches at MIT said they were hit by this. I found it ironic and a little sad that a place like MIT doesn't have an IT staff that can maintain their own on-prem solutions for things like this.
But it turns out that MIT used to have their own homegrown system, and recently switched to Canvas. Bet they're regretting that now.
The build vs. buy decision seems to have swung very hard toward buy in the last decade, and I think that's a shame. Yes, orgs need to focus on their core competency, and sometimes that means outsourcing things that aren't core competencies to third parties. But there are always downsides.
Homegrown systems are expensive to maintain and usually still fail to match up to the commercial options available at this point. LMS's are also just really complicated pieces of software. I worked on my university's own version as an undergrad.
My highschool, for a while, had a website, which was eventually replaces by a large corporate CMS. Was the website as complicated or complex as the CMS? No, you would have needed to know HTML to publish to it. The CMS was no doubt "more user friendly", I suppose.
But … the original site had a soul. It was unique to the school. There was a student directory! All lost, because the CMS meant utter standardization between all the schools using it (their pages were all identical, except for each got like a different picture of the school as the banner at the top) and the CMS did not do directory anything.
Of course, the directory largely didn't matter in the end. (This was when you needed people's landlines! Quite laughable nowadays…) But it was still sad to see it lost, and several of us students worked on it, which provided us with some early real-world experience.
A large number of my college professors published their own sites, too, where they'd put their lecture notes, homework, etc. I loved those far more than I loved "Canvas" or whatever the ugly LMS we used was.
I started my tech career in EDU. I’m not at all surprised.
IT staff who are ambitious and talented don’t last long in education. The pay is very low compared to industry. Where I worked, you could retire with a comfortable pension after a number of service years, so the IT staff outsourced as much as possible so they needed to take zero risks to their nest egg. Blame all the problems on the consultants and do as little as possible.
It’s literally where dreams go to die.
MIT is known for the brilliant professors and students but at the end of the day, running a university is pretty standard stuff. They don’t need a genius rockstar to admin the courseware servers.
MIT has an incredible IT staff and they do some cool stuff. Every time I interact with any other organizations IT stuff I find it inferior. They just aren’t super big from what I gathered and probably don’t want to do the incredibly boring work of an LMS.
The one they had before Canvas was very very inadequate.
edit: also some of the more popular cs classes have custom websites and don’t really use canvas, but that isn’t the centralized IT department’s doing.
I'm a student at Stanford — this is hitting the whole school hard. Unlike a lot of schools on the east coast that are affected (Brown, Harvard, MIT) we are on the quarter system so we're just ending Midterms right now. We're also lucky enough to have our CS department entirely independent from Canvas, but most of my humanities classes are not so lucky. One art history class is having us submit our midterm papers by uploading to a google drive folder—another is pausing weekly quizzes. The main thing this has revealed is just how dependent students and teachers are on Canvas... I hope that this re-prompts discussions about moving off of a platform that was already (from a student perspective) not very good.
Canvas is handling this terrible. No communication, no status updates, etc. Also looks pretty bad their whole platform was compromised and not a single real report for the breach that already had happened. Wonder how long it will take for SLA violations and lawsuits to manifest, especially with most U.S. schooling having finals right now.
Yeah like their page says "Scheduled Maintenance" which is total B.S. Talking to people at my university's IT side of things Canvas has said nothing to any clients.
So many universities used to run homegrown or on-prem student systems. This is the downside of consolidating in the cloud. If the infrastructure is compromised, it affects everyone, not just isolated or single installations. I wonder how they are feeling about that decision now? I guess they can say "not our fault" so they might be feeling better than if it was a vulnerability in their own system.
If an exploit is found in the software, hackers will often be able to attack hundreds of separate institutional installations in an automated way just as easily. And depending on the exploit, potentially more easily if on-prem admins fail to take all recommended security steps.
I'm actually much more interested if there is any financial liability for Instructure here? It's interesting that it's the universities being ransomed, while the technical failure was Instructure's. We're used to uptime SLA's -- what about security breach SLA's?
> It's interesting that it's the universities being ransomed, while the technical failure was Instructure's.
My guess would be they get likelihood of getting paid when blackmailing 9,000 schools (at least a few would pay up) than blackmailing Canvas/Instructure.
I don't think any SLA/terms would change who gets to feel the pain.
My guess is that they believe by maximizing their attack coverage, the odds are greatest that some of the institutions will pay up. And otherwise, they can still make a bit of money by selling the data.
Yeah, if they had spent the time and money to roll their own that got hacked, they'd be responsible. Now, they can just clap their hands and show them palms up to you like a black jack dealer and walk away from the table with no responsibility. Probably one of the biggest benefits of using a product instead of building your own.
You’d think this is how it works but universities and schools will still end up holding the bag at the end of the day, irrespective of who is responsible.
It's annoying that this is how internal politics usually works. Decision-makers at an org should be considered just as responsible when a third-party choice goes bad as when an internal tool goes bad.
I remember when I was in high school (2016? 2017?), I found a super simple XSS in the assignment submission form and told the programming teacher. Canvas then proceeded to lock my account and got me my first (only?) detention. Good times.
Somewhat similar vein, the school's blocking software would block YouTube and embeds unless they came from Canvas. They were smart enough to disable the HTML editor for posting discussion comments, but forgot that since it was a rich text editor, you could just copy-paste in an embed by putting the code in data:text/html, then copying the element as formatted html.
I also ran the entire DOMPurify sample XSS and managed to find one way to download custom content onto someone's computer.
Goddammit. Anyone in the know, know if Parchment was also impacted by this potentially? They were acquired by Instructure a few years ago, and deal with a LOT of transcripts.
Edit: https://status.parchment.com/ says "While Canvas, Canvas Beta and Canvas test are currently unavailable, we are simultaneously monitoring all of our other product environments, including Parchment. We continue to see no reason to believe any Parchment resources have been impacted."
I remember circa 2010 a friend of mine at college was like “blackboard sucks, let’s build something new”. At the time I poo pood the idea and lo and behold canvas came out a year later. Outside looking in, they been crushing it.
One of my mentors created Blackboard. It used to be very very good, but he sold it to private equity, and they immediately fired all of the customer support and developers, 3xd prices overnight leading to the 'blackboard sucks' problem. This gave the opening for Canvas to eventually come on to the scene and dominate.
My wife and I each have to use it as we're both following an online master's at the same university... it's definitely gone downhill (compared to the days where I originally used it ~20 yrs ago in college; tracker-riddled, slow); surprisingly, a recent change made it so that you can only attend online lessons in Chrome (haven't had time to see if this is just a user-agent thing).
I worked in a college IT department around that time and the common belief was that all LMSes suck. There are just too many different ways that too many different people want to do things that it's just bound to be hated. Kind of like Jira / Asana for software dev project management.
I used both and could not tell you the major differences. I feel like they are equivalent in the bread and butter features. Most people don't use 99% of the functions they bake into these. Just use it to hold the syllabus, maybe hold the slides, submit assignments, and spreadsheet for grades. All stuff you can do with email + spreadsheet already. Maybe throw in a shared drive for larger files, which every university in the country already pays for.
As someone who has used both as a student and a TA I find blackboard miles better, much easier to find what i'm looking for and my professors seem to have better luck laying out their course on blackboard than canvas.
I actually disagree, based on my time using Blackboard as an admin, student, and teacher. Although my experience is a few years out of date, I found the interface cumbersome and the performance slow.
They are definitely crushing it on sales. The actual product is a radioactive dumpster fire that is simultaneously hostile to students, teachers, and parents.
The Canvas instance at the nearby university is now down (May 7, 4 PM Eastern), but was briefly displaying the message in this screenshot (1). The ransom message implies that today's problem is the second wave in an attack on Instructure after ignoring their first breach in recent days.
We received communication that Canvas is down for "Under Maintenance" although it seems ShineyHunters have compromised Canvas again with that message you posted.
My wife is in grad school at a major university and is dealing with this right now the week of midterms for spring quarter.
I totally understand why a university wouldn’t want to bake their own learning portals but just feels like such a single point of risk to use third party solutions for something like this.
Back in my day… all we had was a school email via on-premise services. I guess we registered for classes in a web portal but that’s about it. The idea of online class was entirely foreign at the time. Ain’t nobody hacking a blue book.
It’s wild to me that people in this comment section are suggesting that schools should improve their security by rolling their own platform, which is bound to be filled with security holes, instead of using a popular, maintained, open source option.
To be fair to the idea, though, while this would make individual instances less secure, it would drastically decrease the leverage for the work bad actors put in.
There is a saying in the software security industry that (I'm paraphrasing from rusty memories) a system is secure if the cost of hacking it is higher than the value it protects.
Each system being completely distinct from another means that the cost of hacking the average student goes up by 9000 (from the article, Canvas is used by 9000 schools).
Still not saying that rolling out your own is the preferred solution, but the idea is not as ludicrous as it would seem, and should definitely be entertained and discussed, at least.
Maybe. I still remember the Drupal community sneering at the New York Times when they unveiled their homegrown online news platform bitd. After 15 years of recursively scraping ad-hoc porn sites off of server hard drives when clients dragged their feet on migrating to latest versions I 'm less certain the assumption that homegrown == less secure is as valid as it sounds.
Universities used to do this sort of stuff themselves. Then it became a business handled by purchasing rather than needs met by the department themselves.
Because faculty didn’t want to do it anymore. They want it handled by others but also they want oversight and veto power but also they don’t want to be bothered. But it better always work, and if they make a mistake the software is broken because don’t tell them it’s a user error they used to write Fortran.
As a faculty member at a large university…I have a deep respect for the impossible job of university IT departments.
We originally rolled our on LMS decades ago. When we switched to canvas we kept the home brew running for five years past its expiration date because faculty refused to remove their files. Finally each one was manually moved by IT for the recalcitrant old faculty.
I totally understand why a university wouldn’t want to bake their own learning portals
They used to, in the pre-cloud/SaaS era; and they were much simpler and better UX than the slop that they're renting today, because the actual users were not far from the developers.
Counterpoint: I was a PhD student in 2004 and on the universities board* which oversaw the roll-out of the campus management system. It cost > 10m EUR to implement a shitty system with the worst UX and years of stabilizing to make it somewhat work.
The amount of corner cases and performance requirements during rush times (semester start) made it really infeasible for a university to roll their own.
* German universities have this funny system where 51% of such boards are controlled by the professors and the rest is made up of other employees/staff and students. They call it academic participation.
Ongoing. It is not "down" but purposefully offline for "maintenance." Main status does show the LMS (all the course stuff) down, and my instance shows "up" but that's because (I assume) you can reach it and the maintenance page. But that's not useful, if technically not "down."
Canvas LMS is the core service that universities rely on. I assume they're trying to develop a fix and that's why the service is labeled "Under Maintenance". I'm a Berkeley student and can confirm that our instance (bcourses.berkeley.edu) is still down.
The boy is a biochem PhD student at UIUC and reports that all their finals are now cancelled. "Is this good news?" I ask. "Yes. Everything coming up Milhouse."
I'm shocked universities don't host their own LMS? At least large universities have the IT departments to do this. They host compute clusters, so they can certainly host an LMS.
The same reason hospitals don't have their own Patient Information System but all use Epic. The amount of customization you need and continuous churn due to changing curricula and regulatory requirements makes it hard to keep up without scale.
Hugs going out to the teams at Instructure working to fix this. I've been through a similar Ransomware attack (national news stories, lots of customers dead in the water, etc.), and it's about as bad a situation you can wind up in.
Back when I worked for Instructure ~10 years ago, Canvas was effectively a single, giant, monolithic multitenant app with one instance backed by several thousand app servers and ~100 separate Postgres database clusters that any app server could talk to.
Schools were grouped onto pools of app severs and Postgres database clusters more or less according to locality and cluster availability. I want to say a handful of the largest schools got their own clusters, but I'm not certain, and at any rate their clusters could certainly all talk to each other.
It was actually kind of neat from a technical perspective: any Rails model across the entire Canvas world could have a "foreign key" pointing to any other Rails model anywhere else. Among other things, this allowed for users who could administer multiple Canvas organizations, even if those organizations resided on different Postgres clusters. https://github.com/instructure/switchman is their gem that made that all work. (I put "foreign key" in quotes because the whole thing was implemented in software, not with actual database FKs, for obvious reasons.)
---
Of course, the massive downside to that sort of thing is that if you manage to pop one Canvas app server, you have the keys to the kingdom. I wonder if they'll sharpen the edges between clusters in response to this...
---
(Disclaimer: I left Instructure back in 2017; much could have changed since then, and my memory could be faulty about the specifics. Caveat emptor.)
It's possible that Instructure's servers got compromised:
dig canvas.ucdavis.edu
[...]
;; ANSWER SECTION:
canvas.ucdavis.edu. 1974 IN CNAME ucdavis-vanity.instructure.com.
ucdavis-vanity.instructure.com. 60 IN A 18.173.121.125
ucdavis-vanity.instructure.com. 60 IN A 18.173.121.103
ucdavis-vanity.instructure.com. 60 IN A 18.173.121.15
ucdavis-vanity.instructure.com. 60 IN A 18.173.121.18
dig canvas.duke.edu
;; ANSWER SECTION:
canvas.duke.edu. 300 IN CNAME duke-vanity.instructure.com.
duke-vanity.instructure.com. 60 IN A 18.173.121.125
duke-vanity.instructure.com. 60 IN A 18.173.121.18
duke-vanity.instructure.com. 60 IN A 18.173.121.103
duke-vanity.instructure.com. 60 IN A 18.173.121.15
I used canvas for some Harvard extension classes 10 to 5ish years ago. It worked Ok. Work distributed, grades posted. I didn't realized so many schools used it, or that it was all schools on one instance, which seems kind of nuts.
I lost access when I left as it was tied to my work email. I downloaded a lot, but there was still some useful stuff on the boards.
I wonder what the havkers found out about me. Perhaps the class notes will be lifted to train AI, higher quality than a lot thats on the internet anyway.
I discovered one of my old school assignments ended up on some homework help website. I had never posted this document publicly and had only uploaded it to the schools work submission page. Presumably at that point it was shared with multiple third parties for plagiarism checking and such. And then was exposed to a data breach years later and ended up on the public internet.
One thing I remember from my days in the LMS world is that obfuscated copies of prod tenants were used for testing. Almost every dev had at least one tenant from prod on their local computer. So with some de-obfuscation at least some of the data is plausibly retrievable. Whether that data is also public depends on how the negotiations go.
Our whole testing center is down. This is inconvenient, but mainly it's amusing. I swear strangers are talking to each other more. I'm noticing people just sitting in the sun and relaxing. Nature is healing.
(Of course, plenty of people have also just finished their exams, so it's hard to know the cause.)
Any idea what data Instructure-and-also-now-ShinyHunters even purport to have beyond names, profile photos, pronouns, homework assignments, school communications, phone numbers, and email addresses?
i.e. What makes this threat so different from what any old data brokers have already scraped?
What leverage besides aura farming do the ShinyHunters really have?
All I can think of that's really valuable is passwords. And private communications in Canvas DMs. But if you're being at all intimate over your school email, that's kinda on you.
Anyway surely Instructure only stores user public keys or something?
Alternate history question: If they just sold the data, never revealed the hack, and didn't make a scene, from a customer perspective, how different would this be from business as usual?
I use Canvas for some postgraduate studies, and my teenage daughter uses it at her high school.
We already bond over how awful the Canvas UX is (and she has a bunch of Chrome extensions to improve it.) Now we’ve got something else to gripe over together.
I vibecoded a pretty extensive CLI for Canvas and using it is very pleasant. Joyful, even, when combined with an LLM. Especially when compared to the developer hostile Blackboard Ultra.
I wonder when the public is going to start calling for corporate liability for malpractice in software development and corporate liability for malpractice in IT deployments. Even if the tech industry fights it, it probably won't be that much longer.
I'll never understand this point of view. If someone would please explain how to create perfectly secure software, I will gladly start writing perfectly secure software. Only after, if it's clear I ignored obviously correct advice, should there be malpractice penalties.
Consider surgery instead of software development. There are general best practices, but the difference between a good surgeon and a poor one is a small number of deaths. Malpractice insurance is high. Litigation is constant. And patients still die on the operating table. It's unclear what all the malpractice tort law actually gets you in the end.
> "Consider surgery instead of software development."
Is that really the analogy you want to use the bolster your argument? Licensing was forced on the medical profession because of rampant quackery causing a large number of deaths. Some of the horrors that went on before enforced medical licensing are well-nigh unbelievable, e.g. https://en.wikipedia.org/wiki/John_R._Brinkley
> Only after, if it's clear I ignored obviously correct advice, should there be malpractice penalties.
In most of these cases, the companies involved did NOT follow standard security practices.
I am pretty sure that is what people mean when they say "held responsible", they mean "held responsible for failing to follow standard security practices", not for the actual act of getting hacked.
I agree that even if companies do everything right, they can still get popped. But most companies do not do everything right, and they should be legally responsible for those things.
But even if they do everything right, is it really fair to let the companies just shrug their shoulders and say "it happens"? While their users are the ones who really get hurt.
> Consider surgery instead of software development. There are general best practices, but the difference between a good surgeon and a poor one is a small number of deaths.
I like this analogy, but deaths shouldn't be the leading indicator just an indicator. Family member had a surgery with well known procedures, say removing a gall bladder. Unfortunately, this surgeon skipped a step in lieu of setting a record for fastest procedure. Because steps were skipped, the gall bladder was not scooped into a net to avoid spilled gall stones which resulted stones spilling into the abdominal cavity requiring numerous follow up surgeries to remove the spilled stones as they made themselves known. So clearly not following accepted procedures should be a clear win in a malpractice case, yeah? Wrong. No doctor would testify against the surgeon and the case was dismissed. I feel like this is exactly how it would work in software security incidents as well.
I do wonder if that won't just end up INCREASING ransom-type attacks, though?
If we increase the penalties for a company being hacked, you create even MORE incentive for hackers to try to break in, because if they succeed, they have a pretty big stick to threaten companies with when demanding a random payment - not only will the company have the negative effect of the data being leaked and the PR that accompanies it, they now know that if they don't pay and the attack becomes public knowledge, they face a big fine or other punishment.
A company is much more likely to pay a big ransom if they know they are just going to end up paying that much or more in fines if they refuse the ransom and report the hack instead.
If you take this route, and increase punishment for being hacked, you are making a pretty big bet that the main reason companies are hacked is because of poor security practices. I am not sure if that is true or not.
Tbh this is extremely annoying for high school/college students too. High schools are in the middle of AP tests, and many universities have yet to finalize grades, so overall this is a terrible time for this to happen. After the first issue a few weeks ago Canvas should have upped their security and prepared for another attack. They also should provide better communication. If Canvas is down for more than a few days, many schools and universities will have a lot of trouble when it comes time to publish course grades.
I bet it depends on the institution and the IT team behind said institution, but at least for my university we apparently don't delete old course shells or anything.
I'm friends with a professor who complained to me a couple times about how sometimes he will need to scroll through pages and pages of courses he taught in the past. He also mentioned that profs aren't able to delete their own course shells either.
It wouldn't surprise me if most of it is still around. The amounts of data are probably fairly small, and thus unless intentionally deleted, it's probably still there (maybe unis in Europe are more likely to bother to click the relevant buttons as to comply with the GDPR?). I can't imagine storage becoming an issue unless you've got a huge uni or classes that deal with video (and even then, those probably end up on Youtube as private videos, or only as really small clips).
Grades, records, etc I would assume. Someone else pointed out that they recently acquired https://www.parchment.com/ so they may have also been able to scoop up those records too
It is how classes (even in person ones) are organized. Assignments, quizzes, links to online textbooks, discussion boards, student/teacher messaging, student group messaging, etc. From the teacher side, I'm not sure if there is a backup copy for things like grades outside of Canvas. It's that pervasive.
Everything from middle school up to grad school.
It's a particularly interesting time to have this happen too -- many finals going on now.
It’s a “learning management system.” It replaces a course website in most instances. It’s also used for course grades and you can submit assignments or take quizzes.
Great. More data gone astray. Given Canvas' handling of the situation, I doubt they're going to learn much.
The timing probably isn't a coincidence. Great time to stress out students and staff alike. Hopefully it doesn't affect them too much in the end, but I imagine it will.
i work tech at a university that's impacted by this. while it doesn't impact me directly, many many other staff and instructors i know are heavily affected by this outage. the students are absolutely outraged, mostly because the university hasn't been providing updates as quickly as they'd like, but since the staff/admin are waiting on word from instructure -- and there hasn't been a lot from them, it just generally sucks for all of us.
this is really, really, REALLY bad. it's not great that names/emails/etc will potentially be leaked, but also private messages between students and instructors. and since many of the campus systems rely on canvas integration, things have pretty much ground to a halt a week before finals.
after they were breached on the 1st of this month, instructure had an announcement yesterday that "everything is great! we're good! hackers are gone! we've rotated our keys!".
My wife’s grades are due tomorrow. She was in the middle of finishing exams when it happened. She can’t even access the exams to grade by hand. Total mess.
Nothing to see here folks. Just another predicable data breach from allowing companies to do whatever the hell they want with sensitive personal information.
This will keep happening, more and more, and never stop, until we create a software building code and legally require it for all online businesses.
Universities, Parents: ya'll actually have the political and economic power to get a software building code passed. This incident isn't the last.
It is absolute chaos at my institution. This is the last day of finals and grades are due Monday morning. Most faculty are spending today, tomorrow, and through the weekend finalizing grades.
What we don't have access to includes:
* Already graded work
* Ungraded work
* overall adn assignment grades
* lists of students and student emails from the course
* messages from students that are often sent through gradescope
At the same time,
Aussie tech giant pauses work, devotes entire week to AI
Design software giant Canva has halted normal operations across its 5300-strong global workforce for five days of nothing but AI learning and hackathons, bucking the global wave of technology giants that have slashed jobs, citing the technology.
https://www.smh.com.au/technology/aussie-tech-giant-pauses-w...
Terrible that this affects children and that their information may be ultimately leaked. They need to be greater consequences in the law for security breaches.
I saw this happen to my Canvas account today. At first I thought it was a prank from the school or Instructure. The message was sent to students which makes no sense. Second, the message that was sent basically implies that ShinyHunter is actively getting patched out, and no one is ever going to give into their demands. They're basically saying that they're done and desperate. It's a strange message for ShinyHunter to send, but I think they were trying to pull off a psyop / FUD.
Looking into the payload they sent me this is how they hijacked the screen. Everything in the payload is unchanged except for one line of code:
body::after {
content:
"\A\A"
"S H I N Y H U N T E R S"
"\A"
"rooting your systems since '19 ;)"
"\A\A\A"
"ShinyHunters has breached Instructure (again)."
"\A"
"Instead of contacting us to resolve it they"
"\A"
"ignored us and did some \201Csecurity patches\201D."
"\A\A"
"\26A0 W A R N I N G"
"\A\A"
"If any of the schools in the affected list are"
"\A"
"interested in preventing the release of their"
"\A"
"data, please consult with a cyber advisory firm"
"\A"
"and contact us privately at TOX to negotiate a"
"\A"
"settlement. You have till the end of the day by"
"\A"
"12 May 2026 before everything is leaked."
"\A\A"
"Instructure still has until EOD 12 May 2026"
"\A"
"to contact us."
"\A\A"
" \25BC DOWNLOAD AFFECTED_SCHOOLS.TXT \25BC"
"\A"
"91.215.85.103/pay_or_leak/"
"\A"
"instructure_affected_schools_list.txt"
"\A\A"
"visit us: shnyhntww34phqoa6dcgnvps2yu7dlwzmy5"
"\A"
"lkvejwjdo6z7bmgshzayd.onion" !important;
Do you remember how Canvas was a gigantic improvement over Blackboard?
And GitHub doesn't provide a way to record grades that remain private per student last I checked, much less sync them to the university, or 99% of other things Canvas does.
I don't love Canvas, but it's far, far preferable to a world without it.
I've written a bunch of LMS integrations so I've had the opportunity to use all of the major LMSs. Basically, all LMS systems are rather user unfriendly and complicated with a ton of customization options hidden under layers of sub-menus/configuration settings. At their core they provide a grade book, student management tools, and some basic CMS type functionality for posting class messages/coursework/etc. They've all adopted a standard for interacting with external tools (LTI).
Canvas generally is the 'easiest' to use, and the 'cleanest' looking one although D2L Brightspace is pretty good too. Moodle out of the box is pretty confusing and ugly, but I've seen some heavily customized instances that look a lot better. Blackboard is the worst of the bunch IMO.
Perspective from the trenches: I teach at a university that uses Canvas. We are in our final exams period right now.
We got our first email (from Academic Affairs) notifying us that it was down at 5:17pm EDT this afternoon, with little info; followup emails were sent at 6:24 and 6:57 with more info, but mostly about how we would be compensating for it and not about what actually was going on (other than, "nationwide shutdown" and "cybersecurity attacks", no further detail). I don't get a sense that they know much more than that, not that I would expect them to.
A perhaps telling detail: they're instructing us to have students email us directly with any work that had been submitted via Canvas. That suggests that they have no particular confidence that it will come back up soon.
I personally am only slightly affected; as a CS professor a lot of my students' work is done on department machines, and submitted that way, and I do the actual exams on paper. More importantly, I've never liked or trusted Canvas's gradebook, and so although I do upload grades to Canvas so students can see them, my primary gradebook is always a spreadsheet I maintain locally.
But I have a lot of colleagues for whom this is catastrophic at a level of "the whole building burnt down with all my exams and gradebooks in it"---even many of those that teach 100% in person have shifted much or all of their assessment into Canvas (using the Canvas "quiz" feature for everything up to and including final exams), and use the Canvas gradebook as their source-of-truth record. We've been encouraged to do so by our administration ("it makes submitting grades easier"). For faculty in that situation, they have few or zero artifacts that the students have produced, the students themselves don't have the artifacts to resubmit via email because they were done in Canvas in the first place, and they have no record of student grades or even attendance (because they managed that all inside Canvas). I guess they have access to the advisory midterm grades from March, if they submitted them (most do, some don't), but that might be it.
My gut feeling on this is that this is either resolved in hours (they have airgapped backups and can be working as soon as they can spin up new servers), or weeks (they don't). Very little in-between. And if that's true and we wake up tomorrow with this unresolved, I really have no idea what a lot of professors at my university and across the country are going to do to submit grades that are fair and reasonable. In the extreme case, they may have to revert to something we did in the pandemic semester (and before that, at my school, in the semester that two major academic buildings actually did burn to the ground a week before finals): let classes that normally count for a grade just submit grades as pass-fail. Because what else can you do?
(Well, one thing you can do is not put your eggs all in one basket, and not trust "the cloud" quite so much, but that ship's already sailed. I do wonder if in the longer term, anybody learns any lessons from this....)
UPDATE: As of 11:45pm EDT, my university's canvas instance is up and running! Here's hoping it stays (but I'll be downloading some stuff just in case...)
> the students themselves don't have the artifacts to resubmit via email because they were done in Canvas
It’s so simple to send an e-mail to the student with relevant records on completion of a quiz or whatnot. They don’t do it, because they want to control the data. (And universities don’t insist on it for who knows what reason.)
I've never used Canvas before, but all the LMSes that I've used allow students to enable emails whenever anything is updated, including when grades are posted. This is off by default because it's often 10+ emails a day, because many teachers post notes once a day, and with 5 classes, that adds up pretty quick. I personally have it enabled because it's pretty manageable with some custom Outlook rules, but setting this up is well beyond the capabilities of most students.
13 replies →
Students having records of what their score was doesn't prove to the professor / university what score they received. "FWD: Exam 1 Results" is not especially auditable.
21 replies →
Makes me glad I've always avoided doing my work on web platforms. When we used to have to make presentations in Google Slides I used to do them in Org-mode, then export to Sheets. I still have all those assignments sitting on my disk. Sure, there's versions of them on Google Drive, but I always make sure that the canonical version is the one on my disk.
>It’s so simple to send an e-mail to the student ...
What seems easy on hobby projects gets way more difficult at scale. Source: experience.
Just to add one more data point, we also use Canvas at my university. The deadline for submitting who are eligible (i.e. passed compulsory assignments and labs) to take the exam was yesterday, and I couldn’t meet that deadline because Canvas went down. I usually do corrections offline so I have backups of my own evaluations, but these are courses with many teachers and many TAs, so Canvas is the way we sync our assessments.
I work in the Education sector as IT. We don't know much else either.
Everything we know has come from reddit threads / hackernews threads. There has been 0 official communication today indicating this was an attack, yet the login page was defaced by ShinyHunters.
I don't understand what's the panic and doomerism about. Any competent IT team has backups and will be up and running as they go back to a state before the breach. This is HN. I'm disappointed that everyone is talking about losing grades and going back to pen and paper. I don't see how that could happen in 2026.
And from the hacker's message itself, it's clear they want money in exchange for not releasing private info, not for the data itself.
Do we live in a fear based culture? Why the panic? Even if everything was hosted on Instructure's infrastructure, it's all AWS. I'd be VERY surprised if there aren't multiple way to go back to a previous state.
Most of the work and delay is to make sure they figure out where the breach occurred.
Schools don't have competent IT teams.
Here in the Netherlands a data center's power source (not even the machines) burnt down, data center is offline and University of Utrecht, one of the biggest universities here, is closed. Access passes don't work, work from home environment doesn't work, student information system is down, system for grading doesn't work. No failover for any of them (or maybe it was in the same DC?)
https://nos.nl/artikel/2613485-storingen-in-hele-land-door-b...
> Any competent IT team has backups
Backups can be sabotaged (turned off or schedules manipulated) or compromised (say, by lateral movement).
> Even if everything was hosted on Instructure's infrastructure, it's all AWS.
AWS Backup isn't foolproof. Get your hands on administrator credentials as an attacker and suddenly the only thing between everything being gone for good and unrecoverable even for AWS is remembering to have put a permanent deletion protection on all resources in AWS Backup.
Maybe a hybrid approach. Scramble to create a final exam/project and give them the option to do pass/fail or a real grade, their choice.
And then wish for the death of saas and a day where you can deploy your own software you can control and modify as you need.
> day where you can deploy your own software you can control and modify as you need.
Canvas is mostly FOSS
https://github.com/instructure/canvas-lms
What is the strategic response then? Assuming I'm a student and my grades are gone, and I want to graduate, shouldn't I pick pass/fail?
Does a future employer look at pass/fail vs the grade? do they care? Are there even jobs that matter enough to care out there for them?
This seems like, solving the problem but without actually seeing the broader goal or trajectory education is supposed to follow.
1 reply →
To my European ears this just sounds like a disaster like this waiting to happen. God bless the annoying privacy OSS advocates and bureaucrats, I guess.
> they have airgapped backups and can be working as soon as they can spin up new servers
... and assuming they have a documented, tested, and trusted restore process.
Reminds me of the incident last year when a South Korean government's server room caught fire, which contained the government equivalent of Google Drive, and the only backup was in the same room, and they all burnt down together.
Some data was permanently lost, and then officers told reporters that multi-regional backup was not yet built because it was too hard at such a massive scale... of 858 TB.
2 replies →
Ah yes the “recovery” part of the continuity plan. We tested that right? Right?
Backups are definitely helpful in ransomwares, but before systems can be restored and brought back online, victim organizations still need to assess the scope of the breach, find the initial access vector, identify compromised accounts, and evict the threat actor. That can take time.
I’m not certain, but it appears you’re giving Instructure a pass here, as if this is the first time they were hacked. But, it’s the second, by the same group.
As a parent of kids who are impacted by this, I’m not super concerned about the data being held for ransom, but I sure as fuck am concerned about how much it’s going to cost the district to move to another provider.
2 replies →
> let classes that normally count for a grade just submit grades as pass-fail. Because what else can you do?
Schedule a single exam and that's your grade for that subject? That's how it should work anyway, credits for work during semester (or worse attendance) are not needed to evaluate if someone learned the material, give them an exam and done.
That's just bad outdated practice. It leads to cramming and less remembering than of the demand is for students to do work and show learning and effort throughout the year.
1 reply →
That's maybe something a school can do if exams are next week, or after.
At my school, tomorrow is the last day of exams. Most of the students have left campus. There's no time or mechanism to schedule an(other) exam.
Then you're testing how good someone is at exams as much as anything
Exams have performance variance. Otherwise you're only getting a pass/fall signal in any case.
1 reply →
[dead]
I'm surprised how few comments there are on this thread. This is probably affecting millions of students at the most stressful time of the year.
Incidentally I've always hated Canvas and probably every other LMS provider, but what is particularly amusing about this current outage is that it is occurring at exactly the time when universities are demanding that all professors put all of their materials on Canvas, without exception, due to ADA compliance regulations. It is explicitly forbidden for professors to, e.g., refer to pdfs posted on a personal website.
Other commentators here seem not to understand that many faculty also do not enjoy being forced to use Canvas.
They have not succeeded in forcing me, yet. But it's sad how many computing faculty apparently can't operate the basic online infrastructure needed to support their courses. Not that universities make it easy for us.
And of course the other serious concern I have with Canvas is that they are likely using all the materials faculty upload to train their AI replacements. Many of my colleagues engage in dark humor about this but I haven't noticed much action.
> they are likely using all the materials faculty upload to train their AI replacements
Instructure (Canvas's developer) partnered with OpenAI last year [1], about a year after KKR and Dragoneer (PE firms) acquired it [2].
[1] https://www.forbes.com/sites/rayravaglia/2025/07/23/instruct...
[2] https://www.pehub.com/kkr-and-dragoneer-complete-4-8bn-take-...
I'm sure the engineers at instructure are not capable of building systems that can do that. You give them too much credit.
1 reply →
instructure/canvas-lms is open-source -- is there anything preventing universities from hosting it themselves?
1 reply →
Live streaming of class through Canvas is very popular. Quite a few people just watch from their dorms. So maybe people will have to come back to class, that will be entertaining. The class rooms are almost standing room only (sometimes they are) on the first day of class and then gradually thin out. Sometimes 10 or so people show up out of a class of 100. If Canvas is not back up soon I think it could actually be disruptive for that reason also.
This is awful to hear. The idea that students are just half assedly streaming the lectures is really just ruining things in the long run. This is a bit old manny, but showing up to lectures is good. You go to class, you get face time with professors, you can ask impromptu questions, you rub elbows with classmates, you talk on the walk between classes, you maybe run into a cute girl. Friction like walking to class and finding a nook in that annoying hour gap you have, are the things that make life enjoyable.
(Comments were split across multiple threads and we've since merged them.)
Definitely not a criticism of your (hard) work here. Thank you!
1 reply →
We all appreciate the work you do! Thank you!
Not much overlap between students and HN these days, though? I’m an extremely rare outlier afaik :)
The administration has so far opened with one “Canvas said” and then an hour later one “Canvas is down indefinitely” email noting that they’re aware it’s serious.
(Canvas is a glorified wiki for teaching students, with quizzes and such, for those unaware.)
> Not much overlap between students and HN these days, though?
That's my biggest fear.
12 replies →
What? What makes Canvas accessible in a way that HTML and PDF files are not? It's true that PDF readers aren't the best for screenreaders, but surely you can just upload a .html copy as well.
Canvas has an easy way of checking if a pdf or other course material is accessible, so many universities are forcing faculty to put all their materials on Canvas. That way if a pdf or powerpoint is not compliant it is immediately flagged. The goal is to reach a "100% accessible" metric.
Note that little of this really helps the students that it is supposed to help, because as you wisely point out, raw HTML is almost by definition extremely accessible. I work in a field that uses Latex and the source code of Latex should also be considered more accessible than the compiled pdf. But for university administrators the only important thing is that the accessibility metric that appears (or used to appear, before today!) on Canvas shows 100% accessible.
2 replies →
Are you saying that making sure your courses are fully accessible to your students by following disability regulations is a bad thing?
Putting aside the "So you hate waffles?" non-sequitur, surely the entire topic of this thread should be a bit of a hint that this misguided policy has not, in fact, "[made sure] courses are fully accessible".
1 reply →
Not GP, Incompetent policy makers are the bad thing.
Accessibility regulations, implemented with feedback from faculty and with the support of university resources, are certainly a good thing. But that is not what is happening in my experience.
1. It should be illegal for any company to pay ransomware attacks. Period. No pay out ever. 2. The penalty for being the attacker should be linked to the system they violated. If you do this to a hospital and someone dies you are life in prison / chair. The minimum sentence should be so painful that it deters the attack.
No this will not stop this and companies need to be held accountable for their lack of security investment. Every attack should be investigate if the company met an agreed industry standards best practices and staffing, etc. The penalties for not meeting the requirements should be punitive.
> It should be illegal
It should be illegal to host insecure services, especially when you're dealing with PII. Breaches keep happening and nobody gives a fuck, because the worst that'll happen is you might lose a handful of customers and buy some "credit monitoring".
Incidents like this should be followed by an audit and charges being laid. Send corp officers to jail for negligent security failures. If you can go to jail for accounting fraud, you should be able to go to jail for cybersecurity-promises-fraud.
They claim to be compliant with a number of security standards [1]. I would love to see a postmortem audit of how much of this they actually implemented.
[1] https://www.instructure.com/en-au/trust-center/compliance
I don't think that criminal negligence is the most helpful legal tool for incentivizing improved security. It's too hard to prove negligence.
Instead, there should be standard civil penalties for leaking various degrees of PII paid as restitution to the affected individual. Importantly, this must be applied REGARDLESS of "certification" or whether any security practices were "incorrect" or "insufficient". Even if there's a zero-day exploit and you did everything right, you pay. That's the cost of storing people's secrets.
This would make operating services whose whole "thing" is storing a bunch of information about individuals (like Canvas) much more expensive. Good! It's far to cheap to stockpile a ticking time bomb of private info and then walk away paying no damages just because you complied with some out-of-date list of rules or got the stamp of approval from a certification org that's incentivized to give out stamps of approval.
4 replies →
How could you possibly make it illegal to host insecure services? Is any service 100% secure? And if it were how would we know?
I do agree with the audit and punishments for clear failure to adhere to established standards.
11 replies →
People who haven’t been hacked just haven’t been looked at. If someone wants to hack you, they will hack you. It’s really unfortunate that people have this level of confidence in their ability.
Here’s an example. https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin...
If Boeing claimed a plane was airworthy, but it crashed because basic engineering controls were skipped, we have collectively put our faith in the NTSB to preserve evidence, run an independent technical investigation, etc. There is no such authority for software - most security auditors (SOC2, HITRUST, etc) are just looking at self-reported data.
Just take a look at the recent Epic vs. Health Gorilla lawsuit to see how nonexistent the protection is around exchanging your medical records, one of the most sensitive types of PII.
1 reply →
Has a corporate officer ever gone to jail or been meaningfully fined for a data breach?
> Incidents like this should be followed by an audit and charges being laid
What? Why? Who died? This whole thing is perfectly dealt with through civil process.
Shouldn’t we be focusing on making it harder to pay overseas criminals in the first place? /ahem/ crypto platforms facilitating transfers to bad actors /ahem/
Criminals should focus on proven methods, like Steam Gift cards.
But, then, how would Trump’s family and cronies get paid?
[dead]
Your "minimum sentence so painful" will certainly dissuade foreign nationals, even foreign governments.
interestingly, having actually done the law enforcement side of these investigations, 50% of them are local. And I understand that this is not 100% solution, but neither is any form of law enforcement, but that doesn't mean we should fail to attempt it.
Kids from the local uni having a lark, stalkers, vindictive ex employees, local gangs, criminals who understand their victims because they hail from the same community. These are your local hackers. Sift them from the nation states and international crime groups, then deal with the International as a matter of diplomacy. Because we do this so poorly locally, we have little ammunition to when it comes to diplomacy. "reduce attacks by your crime groups and we buy your natural gas, seel you wheat etc"
Want more motivation?- 75% of the local attacks by volume send funds back to terrorist or separatist organizations.
It is not an in-soluble problem. Sentences are a fraction of the answer, effective and receptive reporting processes are more important, then government backing for investigation and enforcement, then policy around home-team activities (ie don't do the bad things yourselves Mr Gov). Deterrence comes after all that.
3 replies →
Yeah, they identified themselves as ShinyHunters, and the IP they've put on the demonstration page is geocoded to Russia. Notice this is the same group responsible for the Infinite Campus hack last year.
Really, though, if you want someone to blame, Instructure is not a particularly compelling target. Let's review:
1. Iran is intentionally targeting infrastructure due to a war started by the current administration.
2. China is actively seeking corporate secrets to steal and commercialize for themselves, spurred by extreme protectionism and retaliatory tariffs.
3. North Korea is doing anything they can -- including just taking a remote job by proxy -- in order to extract any money.
4. And Russia is working with and aiding all of them, after everything else going on has forced the embargo to break.
5. All of this while completely alienating every single one of the United States' allies.
6. Meanwhile, the American DHS is currently shut down.
7. And this is after Trump cut funding and personnel for CISA severely enough they've had to end the contract with MS-ISAC, meaning all state and local entities can only remain in the organization if they foot the bill for it directly and CISA and other agencies responsible for cybersecurity are more thinly staffed than they have been in decades.
In short, the current administration systematically disassembled all the protections we have built over the last 100 years, and then placed infrastructure -- schools, in this case, but also power companies, water treatment facilities, communications companies, local governments, hospitals, food producers -- directly on the front lines of the modern geopolitical conflict.
That vast ocean that has kept us safe historically is a poor moat in the modern era.
1 reply →
Complete internet blockage of nations allowing the attacks. If foreign governments are you can always execute them. We are living in a different world where this is no longer a zero probability occurrence.
1 reply →
When will countries start treating cyberattacks as an act of war? If the North Korean military came to America and robbed fort Knox of $200M in gold there would be retribution. But hack an American company for the same amount and the feds do nothing.
Ok, so we treat it as an act of war. Now what? Attack North Korea? Great, the entire city of Seoul gets shelled within five minutes of your attack and hundreds of thousands of innocent people die.
It's very easy to play with lives that aren't yours.
3 replies →
How do you know which country to blame? It is standard practice for foreign actors (or just hackers in general) to use proxies around the world to misdirect and insert false clues as to their origin. It could be an American teenager proxying through North Korea, and it could be a North Korean proxying through another American teenager's residential connection, there's no way to know.
> When will countries start treating cyberattacks as an act of war?
When appropriate. I.e. never.
They already do. This is what asymmetric warfare looks like, your weakest links will break in a time of crisis. Focusing on retribution for the Dunder Mifflin cyberattack is pointless, the adversarial motivation is purely to disrupt and extort.
The best response to a cyberattack on critical systems is to take security seriously. Document the offense, avoid the same mistakes and invest in penetration testing. Of course, nobody is incentivized to do that until they're attacked, so the cycle perpetuates itself.
If someone robs a bank and someone inside dies of a heart attack, thats felony murder. I would be happy if the same applied to ransom attacks or other blackmail/leaking of info. If someone commits suicide because of it, its murder.
felony murder is pretty widely regarded as a leading factor in incredibly unjust prosecutions and sentencing decisions. perhaps not the best concept to build your ideas on top of.
> If you do this to a hospital and someone dies you are life in prison / chair.
If you're going to get the chair you might as well murder some witnesses or destroy some systems to hide the fact you got hacked. "Hack? What hack? Our servers all burned down in an arson attack".
We could also throw the CEOs of companies who don't properly secure their infrastructure and pay their security engineers enough in jail. A little justice on both ends.
Uh, who determines that the infrastructure wasn't properly secured? Who is willing to risk prison because some intern accidentally committed an API key or made a dumb mistake. Conversely, what's the chances that no one actually gets prosecuted regardless of how sloppy their security practices are?
8 replies →
Failure to protect computer system from forseen failure should result passing corporate veil and resulting all stock holders and managers/leadership of funds to be jailed for same period as perpetrator. It is only way to ensure that these things are taken seriously and enough pressure is put on leadership of companies.
> No this will not stop this and companies need to be held accountable for their lack of security investment.
I think in principle, its sound. Im also just baffled hearing anecdotes from friends that are in big corp world and hearing the type of incidents they have, and how they respond to it.. It makes me wonder if there is enough capable talent to go around for the "boring corp" crowd.
Hint: I don't think there is nearly enough talent to go round, but for these companies, its either that they think they have solid experts (and didn't), OR its not a real priority until you get hit.
1. It should be illegal to run insecure services. Massive Fines.
2. The payout to the hackers should form part, but not all of the penalties. Pay those guys for their great service to humanity they earned it.
My kids are in the middle of their finals week. What a mess. Universities know nothing, Canvas claims to be in a "scheduled maintenance", and one Prof claims to "not have any copies of material offline" which seems pretty negligent. Sounds like one section of a popular class will be doing paper exams while other sections had Canvas-based "half points for 2nd attempt"-type exams earlier today. How soon before names & grades appear in data dumps?
This would be like TurboTax "scheduling maintenance" on April 14th in the US.
The "Scheduled Maintenance" is just total B.S. and just honestly makes them look worse. Apparently according to their status pages this is what 99.996% uptime looks like. Pay attention lol.
It has been over 5 hours now and there has not been any communication about this being an attack, despite many of us seeing the ShinyHunters message on the login page.
There is a lot of people who likely are unaware the latest outage is because they were compromised again.
Them marking the incident as 'Under Maintenance' means the status page isn't reporting this as an outage and adding to downtime%.
3 replies →
I was going to make a joke that they should have just taken a page from the military and said “Rapid Unscheduled Maintenance”, but I guess that’s actually the phrase for it.
Once again, an example of why corporations should not have free speech. Corporate statements that are transparent lies should be criminally actionable.
Crazy that kids data are getting leaked before they even had a chance to properly understand the consequences and consent to it being used
A friend who teaches at MIT said they were hit by this. I found it ironic and a little sad that a place like MIT doesn't have an IT staff that can maintain their own on-prem solutions for things like this.
But it turns out that MIT used to have their own homegrown system, and recently switched to Canvas. Bet they're regretting that now.
The build vs. buy decision seems to have swung very hard toward buy in the last decade, and I think that's a shame. Yes, orgs need to focus on their core competency, and sometimes that means outsourcing things that aren't core competencies to third parties. But there are always downsides.
Homegrown systems are expensive to maintain and usually still fail to match up to the commercial options available at this point. LMS's are also just really complicated pieces of software. I worked on my university's own version as an undergrad.
> LMS's are also just really complicated pieces of software
it's MIT.
2 replies →
… so?
My highschool, for a while, had a website, which was eventually replaces by a large corporate CMS. Was the website as complicated or complex as the CMS? No, you would have needed to know HTML to publish to it. The CMS was no doubt "more user friendly", I suppose.
But … the original site had a soul. It was unique to the school. There was a student directory! All lost, because the CMS meant utter standardization between all the schools using it (their pages were all identical, except for each got like a different picture of the school as the banner at the top) and the CMS did not do directory anything.
Of course, the directory largely didn't matter in the end. (This was when you needed people's landlines! Quite laughable nowadays…) But it was still sad to see it lost, and several of us students worked on it, which provided us with some early real-world experience.
A large number of my college professors published their own sites, too, where they'd put their lecture notes, homework, etc. I loved those far more than I loved "Canvas" or whatever the ugly LMS we used was.
I started my tech career in EDU. I’m not at all surprised.
IT staff who are ambitious and talented don’t last long in education. The pay is very low compared to industry. Where I worked, you could retire with a comfortable pension after a number of service years, so the IT staff outsourced as much as possible so they needed to take zero risks to their nest egg. Blame all the problems on the consultants and do as little as possible.
It’s literally where dreams go to die.
MIT is known for the brilliant professors and students but at the end of the day, running a university is pretty standard stuff. They don’t need a genius rockstar to admin the courseware servers.
MIT has an incredible IT staff and they do some cool stuff. Every time I interact with any other organizations IT stuff I find it inferior. They just aren’t super big from what I gathered and probably don’t want to do the incredibly boring work of an LMS.
The one they had before Canvas was very very inadequate.
edit: also some of the more popular cs classes have custom websites and don’t really use canvas, but that isn’t the centralized IT department’s doing.
CYA is a powerful drug for the C Suite
I'm a student at Stanford — this is hitting the whole school hard. Unlike a lot of schools on the east coast that are affected (Brown, Harvard, MIT) we are on the quarter system so we're just ending Midterms right now. We're also lucky enough to have our CS department entirely independent from Canvas, but most of my humanities classes are not so lucky. One art history class is having us submit our midterm papers by uploading to a google drive folder—another is pausing weekly quizzes. The main thing this has revealed is just how dependent students and teachers are on Canvas... I hope that this re-prompts discussions about moving off of a platform that was already (from a student perspective) not very good.
I really feel like SH fucked up by sinking this low hitting students and Americas young minds like this....
One thing to target coroporations but leave the students alone....
It's not so bad, I'd say the Christmas PS3 hack was worse
1 reply →
And what's your opinion on the em dash?
Canvas is handling this terrible. No communication, no status updates, etc. Also looks pretty bad their whole platform was compromised and not a single real report for the breach that already had happened. Wonder how long it will take for SLA violations and lawsuits to manifest, especially with most U.S. schooling having finals right now.
Lot of experience dealing with Canvas/Instructure. Tech is o-k. Culture seems to be full of themselves due to market position.
Yeah like their page says "Scheduled Maintenance" which is total B.S. Talking to people at my university's IT side of things Canvas has said nothing to any clients.
1 reply →
So many universities used to run homegrown or on-prem student systems. This is the downside of consolidating in the cloud. If the infrastructure is compromised, it affects everyone, not just isolated or single installations. I wonder how they are feeling about that decision now? I guess they can say "not our fault" so they might be feeling better than if it was a vulnerability in their own system.
If an exploit is found in the software, hackers will often be able to attack hundreds of separate institutional installations in an automated way just as easily. And depending on the exploit, potentially more easily if on-prem admins fail to take all recommended security steps.
I'm actually much more interested if there is any financial liability for Instructure here? It's interesting that it's the universities being ransomed, while the technical failure was Instructure's. We're used to uptime SLA's -- what about security breach SLA's?
> It's interesting that it's the universities being ransomed, while the technical failure was Instructure's.
My guess would be they get likelihood of getting paid when blackmailing 9,000 schools (at least a few would pay up) than blackmailing Canvas/Instructure.
I don't think any SLA/terms would change who gets to feel the pain.
My guess is that they believe by maximizing their attack coverage, the odds are greatest that some of the institutions will pay up. And otherwise, they can still make a bit of money by selling the data.
Don't ransom all your eggs in one basket
Yeah, if they had spent the time and money to roll their own that got hacked, they'd be responsible. Now, they can just clap their hands and show them palms up to you like a black jack dealer and walk away from the table with no responsibility. Probably one of the biggest benefits of using a product instead of building your own.
You’d think this is how it works but universities and schools will still end up holding the bag at the end of the day, irrespective of who is responsible.
It's annoying that this is how internal politics usually works. Decision-makers at an org should be considered just as responsible when a third-party choice goes bad as when an internal tool goes bad.
It's still more secure this way, especially with AI hacking making it harder to rely on obscurity.
Also yeah there is value in being able to blame another party, and also being down when everyone else is down.
Is there a good self-hostable FOSS version of Canvas/Blackboard?
Canvas is open-source and can be self-hosted.
I remember when I was in high school (2016? 2017?), I found a super simple XSS in the assignment submission form and told the programming teacher. Canvas then proceeded to lock my account and got me my first (only?) detention. Good times.
Somewhat similar vein, the school's blocking software would block YouTube and embeds unless they came from Canvas. They were smart enough to disable the HTML editor for posting discussion comments, but forgot that since it was a rich text editor, you could just copy-paste in an embed by putting the code in data:text/html, then copying the element as formatted html.
I also ran the entire DOMPurify sample XSS and managed to find one way to download custom content onto someone's computer.
Uh, did you tell the teacher by exploiting the vuln?
Goddammit. Anyone in the know, know if Parchment was also impacted by this potentially? They were acquired by Instructure a few years ago, and deal with a LOT of transcripts.
Edit: https://status.parchment.com/ says "While Canvas, Canvas Beta and Canvas test are currently unavailable, we are simultaneously monitoring all of our other product environments, including Parchment. We continue to see no reason to believe any Parchment resources have been impacted."
I remember circa 2010 a friend of mine at college was like “blackboard sucks, let’s build something new”. At the time I poo pood the idea and lo and behold canvas came out a year later. Outside looking in, they been crushing it.
One of my mentors created Blackboard. It used to be very very good, but he sold it to private equity, and they immediately fired all of the customer support and developers, 3xd prices overnight leading to the 'blackboard sucks' problem. This gave the opening for Canvas to eventually come on to the scene and dominate.
I believe Canvas was also sold to private equity pretty recently too. https://www.instructure.com/press-release/instructure-to-be-...
1 reply →
My wife and I each have to use it as we're both following an online master's at the same university... it's definitely gone downhill (compared to the days where I originally used it ~20 yrs ago in college; tracker-riddled, slow); surprisingly, a recent change made it so that you can only attend online lessons in Chrome (haven't had time to see if this is just a user-agent thing).
..and be acquired by PE so the cycle can continue.. https://www.instructure.com/press-release/instructure-to-be-... sigh. Barbarians at the gate probably didn't double down on security
I worked in a college IT department around that time and the common belief was that all LMSes suck. There are just too many different ways that too many different people want to do things that it's just bound to be hated. Kind of like Jira / Asana for software dev project management.
LMS’s are a lot like programming languages. There’s the ones people complain about and the ones no one uses.
I used both and could not tell you the major differences. I feel like they are equivalent in the bread and butter features. Most people don't use 99% of the functions they bake into these. Just use it to hold the syllabus, maybe hold the slides, submit assignments, and spreadsheet for grades. All stuff you can do with email + spreadsheet already. Maybe throw in a shared drive for larger files, which every university in the country already pays for.
"Equivocal describes something ambiguous, uncertain, or open to multiple interpretations, often used to intentionally mislead or evade."
do you mean equivalent ?.
1 reply →
Blackboard got a lot better in response to the flood of customers heading to canvas.
As someone who has used both as a student and a TA I find blackboard miles better, much easier to find what i'm looking for and my professors seem to have better luck laying out their course on blackboard than canvas.
I actually disagree, based on my time using Blackboard as an admin, student, and teacher. Although my experience is a few years out of date, I found the interface cumbersome and the performance slow.
1 reply →
How does canvas compare to Brightspace?
> circa 2010
Instructure, "the developer and publisher of Canvas," was founded in 2008 [1].
[1] https://en.wikipedia.org/wiki/Instructure
That sounds like “circa 2010” to me. And Canvas was launched in 2011, according to the article you linked.
Blackboard, the Canvas predecessor, was so unstable that we called it BlackOutBoard
Maybe schools should be self-hosting something like Sakai instead.
They are definitely crushing it on sales. The actual product is a radioactive dumpster fire that is simultaneously hostile to students, teachers, and parents.
Yeah but the customer is the administrators who never have to make contact with the real world
The Canvas instance at the nearby university is now down (May 7, 4 PM Eastern), but was briefly displaying the message in this screenshot (1). The ransom message implies that today's problem is the second wave in an attack on Instructure after ignoring their first breach in recent days.
1: https://ibb.co/r29RjdnH
Yeah, this is ongoing.
We received communication that Canvas is down for "Under Maintenance" although it seems ShineyHunters have compromised Canvas again with that message you posted.
We do not see that message anymore, although all instrucuture.com URLs are down. The list of schools in the ShinyHunters publication can be found here: https://web.archive.org/web/20260507042014/http://91.215.85....
https://web.archive.org/web/20260507042014fw_/http://91.215....
Original now shows 404.
Seems like Canvas instances of schools not listed are also down (at least my alma mater is)
4 replies →
My wife is in grad school at a major university and is dealing with this right now the week of midterms for spring quarter.
I totally understand why a university wouldn’t want to bake their own learning portals but just feels like such a single point of risk to use third party solutions for something like this.
Back in my day… all we had was a school email via on-premise services. I guess we registered for classes in a web portal but that’s about it. The idea of online class was entirely foreign at the time. Ain’t nobody hacking a blue book.
It’s wild to me that people in this comment section are suggesting that schools should improve their security by rolling their own platform, which is bound to be filled with security holes, instead of using a popular, maintained, open source option.
To be fair to the idea, though, while this would make individual instances less secure, it would drastically decrease the leverage for the work bad actors put in.
There is a saying in the software security industry that (I'm paraphrasing from rusty memories) a system is secure if the cost of hacking it is higher than the value it protects.
Each system being completely distinct from another means that the cost of hacking the average student goes up by 9000 (from the article, Canvas is used by 9000 schools).
Still not saying that rolling out your own is the preferred solution, but the idea is not as ludicrous as it would seem, and should definitely be entertained and discussed, at least.
Maybe. I still remember the Drupal community sneering at the New York Times when they unveiled their homegrown online news platform bitd. After 15 years of recursively scraping ad-hoc porn sites off of server hard drives when clients dragged their feet on migrating to latest versions I 'm less certain the assumption that homegrown == less secure is as valid as it sounds.
1 reply →
Universities used to do this sort of stuff themselves. Then it became a business handled by purchasing rather than needs met by the department themselves.
In fairness in the era where universities did it themselves the tech requirements and expectations were dramatically lower.
3 replies →
Because faculty didn’t want to do it anymore. They want it handled by others but also they want oversight and veto power but also they don’t want to be bothered. But it better always work, and if they make a mistake the software is broken because don’t tell them it’s a user error they used to write Fortran.
As a faculty member at a large university…I have a deep respect for the impossible job of university IT departments.
We originally rolled our on LMS decades ago. When we switched to canvas we kept the home brew running for five years past its expiration date because faculty refused to remove their files. Finally each one was manually moved by IT for the recalcitrant old faculty.
2 replies →
> Ain’t nobody hacking a blue book.
Well not with that attitude
A university doesn't need to bake its own learning portal, Moodle exists and is used by a lot of large schools.
Moodle is an open-source LMS that can be self-hosted.
https://moodle.org/
Another open-source LMS that can be self-hosted is... Canvas.
2 replies →
I totally understand why a university wouldn’t want to bake their own learning portals
They used to, in the pre-cloud/SaaS era; and they were much simpler and better UX than the slop that they're renting today, because the actual users were not far from the developers.
Counterpoint: I was a PhD student in 2004 and on the universities board* which oversaw the roll-out of the campus management system. It cost > 10m EUR to implement a shitty system with the worst UX and years of stabilizing to make it somewhat work.
The amount of corner cases and performance requirements during rush times (semester start) made it really infeasible for a university to roll their own.
* German universities have this funny system where 51% of such boards are controlled by the professors and the rest is made up of other employees/staff and students. They call it academic participation.
Also here: https://news.ycombinator.com/item?id=48054386
https://status.instructure.com/ implies Canvas became available again about thirty minutes ago from the time of this post.
Is this accurate? Or is this still an ongoing issue?
Ongoing. It is not "down" but purposefully offline for "maintenance." Main status does show the LMS (all the course stuff) down, and my instance shows "up" but that's because (I assume) you can reach it and the maintenance page. But that's not useful, if technically not "down."
Thanks
Canvas LMS is the core service that universities rely on. I assume they're trying to develop a fix and that's why the service is labeled "Under Maintenance". I'm a Berkeley student and can confirm that our instance (bcourses.berkeley.edu) is still down.
Federated logins appear to now be broken for the campus I’m affiliated with. So more action is needed.
It looks like Instructure has been removed from the ShinyHunters website. Both the entry and the list of schools has been removed.
Look for large BTC moves recently?
Ransom paid?
The boy is a biochem PhD student at UIUC and reports that all their finals are now cancelled. "Is this good news?" I ask. "Yes. Everything coming up Milhouse."
I'm shocked universities don't host their own LMS? At least large universities have the IT departments to do this. They host compute clusters, so they can certainly host an LMS.
The same reason hospitals don't have their own Patient Information System but all use Epic. The amount of customization you need and continuous churn due to changing curricula and regulatory requirements makes it hard to keep up without scale.
> Canvas is currently undergoing scheduled maintenance
doesn't seem that scheduled to me
ex-Instructure employee here (though it's been about 10 years since I worked for them).
That's just the quickest page/status update to throw up; it was a one-liner to push it live back when I was on the deploy rotation.
I'd hazard a guess they have more important things to worry about right now than exact status page messaging ;)
I thought the same. The "scheduled" part of the message is gone now, at least on the instance I use.
Well, scheduled by whom? :)
Whoever it is, is likely defended by Cloudflare. They seem to like the booters.
https://news.ycombinator.com/item?id=48025001
Hugs going out to the teams at Instructure working to fix this. I've been through a similar Ransomware attack (national news stories, lots of customers dead in the water, etc.), and it's about as bad a situation you can wind up in.
A post on the official Canvas forum: https://community.instructure.com/en/discussion/666027/ranso...
Somehow I have less distaste for ShinyHunters than I do for the companies who don't secure user data
When you picture the attacker, don't picture a bored nerdy teanager. Picture a selfish, $$ motivated psychopath.
Let's not side with the parasites.
yep, i work for a major university and our canvas instance is down. this is really, really bad.
edit: here's the list of impacted universities (unsure if they all have their canvas instances offline, but i'd be surprised if not): http://91.215.85.103/pay_or_leak/instructure_affected_school...
The source txtfile has since either been dos'd or deleted (at least it was when I tried to access)
Someone dumped the content into a google doc on reddit[1] if anyone's interested.
[1]: https://docs.google.com/document/d/1MTktVSwTUM5I_w7bKNGj94sT...
> The source txtfile has since either been dos'd or deleted (at least it was when I tried to access)
> Someone dumped the content into a google doc on reddit[1] if anyone's interested.
> [1]: https://docs.google.com/document/d/1MTktVSwTUM5I_w7bKNGj94sT...
Thanks for linking this. Ended up finding my kids school district on the list unfortunately.
https://web.archive.org/web/20260507042014fw_/http://91.215....
tbh this has me wondering if canvas "instances" are actually as isolated and segregated from each other as they're supposed to be.
Define "as they're supposed to be".
Back when I worked for Instructure ~10 years ago, Canvas was effectively a single, giant, monolithic multitenant app with one instance backed by several thousand app servers and ~100 separate Postgres database clusters that any app server could talk to.
Schools were grouped onto pools of app severs and Postgres database clusters more or less according to locality and cluster availability. I want to say a handful of the largest schools got their own clusters, but I'm not certain, and at any rate their clusters could certainly all talk to each other.
It was actually kind of neat from a technical perspective: any Rails model across the entire Canvas world could have a "foreign key" pointing to any other Rails model anywhere else. Among other things, this allowed for users who could administer multiple Canvas organizations, even if those organizations resided on different Postgres clusters. https://github.com/instructure/switchman is their gem that made that all work. (I put "foreign key" in quotes because the whole thing was implemented in software, not with actual database FKs, for obvious reasons.)
---
Of course, the massive downside to that sort of thing is that if you manage to pop one Canvas app server, you have the keys to the kingdom. I wonder if they'll sharpen the edges between clusters in response to this...
---
(Disclaimer: I left Instructure back in 2017; much could have changed since then, and my memory could be faulty about the specifics. Caveat emptor.)
It's possible that Instructure's servers got compromised:
dig canvas.ucdavis.edu
dig canvas.duke.edu
1 reply →
It depends on what you pay for. If you need FedRamp or IL4+ compliance you are likely on dedicated infrastructure. Everyone else uses multi tenancy.
Here's an archive https://archive.is/eB2hE
[dead]
My son was in the middle of an exam and then his screen went black and it showed the message from ShinyHunters. Hasn’t been able to get back in since.
https://archive.is/5v693
What are we even coming to when even internet blogs are paywalled. Verge? Next thing Gizmodo is gonna be paywalled.
Just learned the defacement page was hosted from instructure's own aws bucket so seems pretty bad.
I used canvas for some Harvard extension classes 10 to 5ish years ago. It worked Ok. Work distributed, grades posted. I didn't realized so many schools used it, or that it was all schools on one instance, which seems kind of nuts.
I lost access when I left as it was tied to my work email. I downloaded a lot, but there was still some useful stuff on the boards.
I wonder what the havkers found out about me. Perhaps the class notes will be lifted to train AI, higher quality than a lot thats on the internet anyway.
I discovered one of my old school assignments ended up on some homework help website. I had never posted this document publicly and had only uploaded it to the schools work submission page. Presumably at that point it was shared with multiple third parties for plagiarism checking and such. And then was exposed to a data breach years later and ended up on the public internet.
One thing I remember from my days in the LMS world is that obfuscated copies of prod tenants were used for testing. Almost every dev had at least one tenant from prod on their local computer. So with some de-obfuscation at least some of the data is plausibly retrievable. Whether that data is also public depends on how the negotiations go.
A college student I know just sent me a screenshot, he can't access canvas for his school at all
Same, my daughter just sent a screenshot, she was trying to study for finals.
Student at an impacted university here.
Our whole testing center is down. This is inconvenient, but mainly it's amusing. I swear strangers are talking to each other more. I'm noticing people just sitting in the sun and relaxing. Nature is healing.
(Of course, plenty of people have also just finished their exams, so it's hard to know the cause.)
Any idea what data Instructure-and-also-now-ShinyHunters even purport to have beyond names, profile photos, pronouns, homework assignments, school communications, phone numbers, and email addresses?
i.e. What makes this threat so different from what any old data brokers have already scraped?
What leverage besides aura farming do the ShinyHunters really have?
All I can think of that's really valuable is passwords. And private communications in Canvas DMs. But if you're being at all intimate over your school email, that's kinda on you.
Anyway surely Instructure only stores user public keys or something?
Alternate history question: If they just sold the data, never revealed the hack, and didn't make a scene, from a customer perspective, how different would this be from business as usual?
Qld, Australia was also affected: https://www.itnews.com.au/news/qld-gov-says-students-staff-c...
QLD Government vendor selection is always terrible.
I use Canvas for some postgraduate studies, and my teenage daughter uses it at her high school.
We already bond over how awful the Canvas UX is (and she has a bunch of Chrome extensions to improve it.) Now we’ve got something else to gripe over together.
It is open source, so you could send pull requests with improvements: https://github.com/instructure/canvas-lms
https://github.com/instructure/canvas-lms/pulls?q=is%3Apr+is...
haha i went to go check and they haven't merged a PR since 2017
1 reply →
I vibecoded a pretty extensive CLI for Canvas and using it is very pleasant. Joyful, even, when combined with an LLM. Especially when compared to the developer hostile Blackboard Ultra.
Canvas seems like it’s not that great. But if you then use Blackboard Ultra it makes canvas look amazing.
I wonder when the public is going to start calling for corporate liability for malpractice in software development and corporate liability for malpractice in IT deployments. Even if the tech industry fights it, it probably won't be that much longer.
I'll never understand this point of view. If someone would please explain how to create perfectly secure software, I will gladly start writing perfectly secure software. Only after, if it's clear I ignored obviously correct advice, should there be malpractice penalties.
Consider surgery instead of software development. There are general best practices, but the difference between a good surgeon and a poor one is a small number of deaths. Malpractice insurance is high. Litigation is constant. And patients still die on the operating table. It's unclear what all the malpractice tort law actually gets you in the end.
> "Consider surgery instead of software development."
Is that really the analogy you want to use the bolster your argument? Licensing was forced on the medical profession because of rampant quackery causing a large number of deaths. Some of the horrors that went on before enforced medical licensing are well-nigh unbelievable, e.g. https://en.wikipedia.org/wiki/John_R._Brinkley
> Only after, if it's clear I ignored obviously correct advice, should there be malpractice penalties.
In most of these cases, the companies involved did NOT follow standard security practices.
I am pretty sure that is what people mean when they say "held responsible", they mean "held responsible for failing to follow standard security practices", not for the actual act of getting hacked.
I agree that even if companies do everything right, they can still get popped. But most companies do not do everything right, and they should be legally responsible for those things.
But even if they do everything right, is it really fair to let the companies just shrug their shoulders and say "it happens"? While their users are the ones who really get hurt.
> Consider surgery instead of software development. There are general best practices, but the difference between a good surgeon and a poor one is a small number of deaths.
I like this analogy, but deaths shouldn't be the leading indicator just an indicator. Family member had a surgery with well known procedures, say removing a gall bladder. Unfortunately, this surgeon skipped a step in lieu of setting a record for fastest procedure. Because steps were skipped, the gall bladder was not scooped into a net to avoid spilled gall stones which resulted stones spilling into the abdominal cavity requiring numerous follow up surgeries to remove the spilled stones as they made themselves known. So clearly not following accepted procedures should be a clear win in a malpractice case, yeah? Wrong. No doctor would testify against the surgeon and the case was dismissed. I feel like this is exactly how it would work in software security incidents as well.
1 reply →
Well, you don't know how many more would have died if doctors and hospital didn't care about their insurance going higher???
I do wonder if that won't just end up INCREASING ransom-type attacks, though?
If we increase the penalties for a company being hacked, you create even MORE incentive for hackers to try to break in, because if they succeed, they have a pretty big stick to threaten companies with when demanding a random payment - not only will the company have the negative effect of the data being leaked and the PR that accompanies it, they now know that if they don't pay and the attack becomes public knowledge, they face a big fine or other punishment.
A company is much more likely to pay a big ransom if they know they are just going to end up paying that much or more in fines if they refuse the ransom and report the hack instead.
If you take this route, and increase punishment for being hacked, you are making a pretty big bet that the main reason companies are hacked is because of poor security practices. I am not sure if that is true or not.
There's precedent for simply making it illegal to pay the ransom, e.g. https://www.reuters.com/world/uk/uk-plans-ban-public-sector-...
That is already happening in the EU [1][2]. Most of the world will catch up soon I suspect, with some notable exceptions.
[1] https://digital-strategy.ec.europa.eu/en/policies/cyber-resi... [2] https://ec.europa.eu/commission/presscorner/detail/en/ip_22_...
Tbh this is extremely annoying for high school/college students too. High schools are in the middle of AP tests, and many universities have yet to finalize grades, so overall this is a terrible time for this to happen. After the first issue a few weeks ago Canvas should have upped their security and prepared for another attack. They also should provide better communication. If Canvas is down for more than a few days, many schools and universities will have a lot of trouble when it comes time to publish course grades.
I wonder how much old data Canvas keeps around? Are students who graduated in 2016 going to be at risk of having their academic data leaked?
I bet it depends on the institution and the IT team behind said institution, but at least for my university we apparently don't delete old course shells or anything.
I'm friends with a professor who complained to me a couple times about how sometimes he will need to scroll through pages and pages of courses he taught in the past. He also mentioned that profs aren't able to delete their own course shells either.
It wouldn't surprise me if most of it is still around. The amounts of data are probably fairly small, and thus unless intentionally deleted, it's probably still there (maybe unis in Europe are more likely to bother to click the relevant buttons as to comply with the GDPR?). I can't imagine storage becoming an issue unless you've got a huge uni or classes that deal with video (and even then, those probably end up on Youtube as private videos, or only as really small clips).
Our public school system here in Maryland got hit, ransom screen.
Down for all students at my University… it’s going to be a headache for all professors to deal with extending due assignments.
My daughter says that Northeastern is also affected. Is it more widespread? Did they infect all SaaS Canvas universities?
Yes, all 8000+ institutions that use Canvas.
What's in the files they've already released? Some of them are > 800GB.
Where are you getting that information from?
I'm under the impression files are getting released 12th May. I don't see any reporting on 800GB?
Grades, records, etc I would assume. Someone else pointed out that they recently acquired https://www.parchment.com/ so they may have also been able to scoop up those records too
Also discussions between students and teaching staff.
I'm guessing loads of student work? If so, it'll be great for anyone who wants to research AI usage in papers.
I remember this group did something else a while back too.
I’m not surprised. Canvas kind of sucks. And their development is slow. And they are poor at communicating during mundane events.
They're also apparently poor at communication during highly interesting events as well
Pretty cruel to do this right around finals.
That's exactly the point, I'm sure.
Even more incentive to pay up. I wonder if the timing was intentional or just coincidental.
That is the point. Get an extra million or two $ in btc from Instructure.
It's been a long time since I was in school. What does this software do?
It is how classes (even in person ones) are organized. Assignments, quizzes, links to online textbooks, discussion boards, student/teacher messaging, student group messaging, etc. From the teacher side, I'm not sure if there is a backup copy for things like grades outside of Canvas. It's that pervasive.
Everything from middle school up to grad school.
It's a particularly interesting time to have this happen too -- many finals going on now.
It’s a “learning management system.” It replaces a course website in most instances. It’s also used for course grades and you can submit assignments or take quizzes.
Grades, lessons, quizzes, exams, homework submission, rosters, messaging platform. Lots of things.
If you’re a student or teacher: nearly everything that matters. Homework, materials, lectures, grades. It’s all on canvas.
Great. More data gone astray. Given Canvas' handling of the situation, I doubt they're going to learn much.
The timing probably isn't a coincidence. Great time to stress out students and staff alike. Hopefully it doesn't affect them too much in the end, but I imagine it will.
Some instances seem to be recovering. I wonder if a ransom was paid.
It looks like Instructure has been removed from the ShinyHunters website. Both the entry and the list of schools has been removed.
It looks like every CSU System is on the list (California State University). Surprised this hasn't hit the front page yet.
Possibly because they haven't released the data yet?
I'm honestly surprised more people aren't talking about this.
i work tech at a university that's impacted by this. while it doesn't impact me directly, many many other staff and instructors i know are heavily affected by this outage. the students are absolutely outraged, mostly because the university hasn't been providing updates as quickly as they'd like, but since the staff/admin are waiting on word from instructure -- and there hasn't been a lot from them, it just generally sucks for all of us.
this is really, really, REALLY bad. it's not great that names/emails/etc will potentially be leaked, but also private messages between students and instructors. and since many of the campus systems rely on canvas integration, things have pretty much ground to a halt a week before finals.
after they were breached on the 1st of this month, instructure had an announcement yesterday that "everything is great! we're good! hackers are gone! we've rotated our keys!".
no. nothing is great. we are not good.
And grades are due in the next week or so for many of these (usually a quick deadline at the end of the semester due to graduation happening)...
My wife’s grades are due tomorrow. She was in the middle of finishing exams when it happened. She can’t even access the exams to grade by hand. Total mess.
Graduation is just a ceremony. The actual credential award depends on whether you finished all your coursework and is not time-boxed by that event.
Of course if you can't complete your exams because of this, that's more of an issue!
Nothing to see here folks. Just another predicable data breach from allowing companies to do whatever the hell they want with sensitive personal information.
This will keep happening, more and more, and never stop, until we create a software building code and legally require it for all online businesses.
Universities, Parents: ya'll actually have the political and economic power to get a software building code passed. This incident isn't the last.
It is absolute chaos at my institution. This is the last day of finals and grades are due Monday morning. Most faculty are spending today, tomorrow, and through the weekend finalizing grades.
What we don't have access to includes:
* Already graded work
* Ungraded work
* overall adn assignment grades
* lists of students and student emails from the course
* messages from students that are often sent through gradescope
Just...complete implosion.
What happens if the system isn't back up in time for grades to be submitted? Just a delay?
going after systems that affect students is beyond bad taste
Eek I bet there are a few people at Instructure who won't be getting much sleep tonight!
At the same time, Aussie tech giant pauses work, devotes entire week to AI Design software giant Canva has halted normal operations across its 5300-strong global workforce for five days of nothing but AI learning and hackathons, bucking the global wave of technology giants that have slashed jobs, citing the technology. https://www.smh.com.au/technology/aussie-tech-giant-pauses-w...
Canvas is not Canva. Is this a bot reply
You learn all the technical details only to harm people like that instead of making a modest and honest living.
Shame on your existence basically.
Damn, all schools in our district in Washington moved to Instructure last year.
They moved away from Teams because it objectively sucked, but I haven't heard of widespread compromises like this in Microsoft's systems so...
Well instructure is slightly better than the somehow legal torture of having to use the "product" Microsoft Teams
Canvas shouldn't exist in its current form, and neither should have Blackboard.
It's always been as stupid as requiring that your chalkboard, chalk, chairs, bluebooks, pens, paper, gradebook etc etc all come from the same company.
I, for one, am very much looking forward to my IT Gov council meeting tomorrow.
Terrible that this affects children and that their information may be ultimately leaked. They need to be greater consequences in the law for security breaches.
I saw this happen to my Canvas account today. At first I thought it was a prank from the school or Instructure. The message was sent to students which makes no sense. Second, the message that was sent basically implies that ShinyHunter is actively getting patched out, and no one is ever going to give into their demands. They're basically saying that they're done and desperate. It's a strange message for ShinyHunter to send, but I think they were trying to pull off a psyop / FUD.
Looking into the payload they sent me this is how they hijacked the screen. Everything in the payload is unchanged except for one line of code:
<link rel="stylesheet" href="https://instructure-uploads.s3.amazonaws.com/account_9363000..." media="all"/>
This links to the following styling sheet:
@import url('https://fonts.googleapis.com/css2?family=Orbitron:wght@500;7...');
html, body { height: 100% !important; overflow: hidden !important; margin: 0 !important; padding: 0 !important; }
body > * { display: none !important; }
body { display: flex !important; align-items: center !important; justify-content: center !important; background: #07080c !important; }
body::before { content: "" !important; position: fixed !important; inset: 0 !important; z-index: 999998 !important; background: radial-gradient(ellipse at 50% 20%, rgba(255,59,59,.06), transparent 55%), radial-gradient(ellipse at 50% 85%, rgba(125,70,152,.04), transparent 45%), repeating-linear-gradient(0deg, rgba(255,255,255,.035), rgba(255,255,255,.035) 1px, transparent 1px, transparent 3px), #07080c !important; pointer-events: none !important; }
body::after { content: "\A\A" "S H I N Y H U N T E R S" "\A" "rooting your systems since '19 ;)" "\A\A\A" "ShinyHunters has breached Instructure (again)." "\A" "Instead of contacting us to resolve it they" "\A" "ignored us and did some \201Csecurity patches\201D." "\A\A" "\26A0 W A R N I N G" "\A\A" "If any of the schools in the affected list are" "\A" "interested in preventing the release of their" "\A" "data, please consult with a cyber advisory firm" "\A" "and contact us privately at TOX to negotiate a" "\A" "settlement. You have till the end of the day by" "\A" "12 May 2026 before everything is leaked." "\A\A" "Instructure still has until EOD 12 May 2026" "\A" "to contact us." "\A\A" " \25BC DOWNLOAD AFFECTED_SCHOOLS.TXT \25BC" "\A" "91.215.85.103/pay_or_leak/" "\A" "instructure_affected_schools_list.txt" "\A\A" "visit us: shnyhntww34phqoa6dcgnvps2yu7dlwzmy5" "\A" "lkvejwjdo6z7bmgshzayd.onion" !important;
}
@keyframes pulseWarn { 0% { box-shadow: 0 0 20px rgba(255,59,59,.15), 0 40px 90px rgba(0,0,0,.65), inset 0 0 0 1px rgba(255,255,255,.06); } 50% { box-shadow: 0 0 55px rgba(255,59,59,.4), 0 40px 90px rgba(0,0,0,.65), inset 0 0 0 1px rgba(255,255,255,.06); } 100% { box-shadow: 0 0 20px rgba(255,59,59,.15), 0 40px 90px rgba(0,0,0,.65), inset 0 0 0 1px rgba(255,255,255,.06); } }
The hack is crude, and it seems unlikely that they have any access to Instructure's developer tools.
[dead]
[dead]
[dead]
[dead]
[flagged]
[flagged]
Where did you find information on the nature of the attack?
This is an AI bot
I hate Canvas. I would rather run a course on GitHub. But our university forces it on us. And now this.
Do you remember how Canvas was a gigantic improvement over Blackboard?
And GitHub doesn't provide a way to record grades that remain private per student last I checked, much less sync them to the university, or 99% of other things Canvas does.
I don't love Canvas, but it's far, far preferable to a world without it.
It is really convenient and stays out of the way. As much as I'm enjoying the mess, I am forced to appreciate its value.
> remain private per student last I checked
last I checked it appears grades remain private per planet or so ...
How does Canvas compare to things like Moodle?
Or is it an entirely different class of beast?
I've written a bunch of LMS integrations so I've had the opportunity to use all of the major LMSs. Basically, all LMS systems are rather user unfriendly and complicated with a ton of customization options hidden under layers of sub-menus/configuration settings. At their core they provide a grade book, student management tools, and some basic CMS type functionality for posting class messages/coursework/etc. They've all adopted a standard for interacting with external tools (LTI).
Canvas generally is the 'easiest' to use, and the 'cleanest' looking one although D2L Brightspace is pretty good too. Moodle out of the box is pretty confusing and ugly, but I've seen some heavily customized instances that look a lot better. Blackboard is the worst of the bunch IMO.
Wow, I last used Moodle in 7th grade, 2008. It seemed like a similar thing.