Comment by Nevermark
19 days ago
All the fiascos in education raise a simple question. Why are big changes not arrived at by first gaining experience with them in in some reduced scale, then spreading the improvements incrementally as they continue to be validated.
And why isn't this experimentation being done all the time, not randomly but competitively/cooperatively between school districts and individual schools? Each making small changes toward getting better results and sharing what they have learned. With most cross adoption happening naturally.
Creating and managing the context for the latter is what people with power should be doing. Not making top-down decisions devoid of the bottom-up wisdom and visible exemplars that big changes need to succeed.
> Why are big changes not arrived at by first gaining experience with them in in some reduced scale
Because the results that come back are always politically inexpedient to agendas--generally for all sides.
Examples:
1) Charter schools: As soon as you force charter schools into an actual lottery which normalizes their student body relative to the public schools, their performance relative to the public schools craters. Quelle surprise: Expensive students are expensive and take up a disproportionate amount of your smaller budget. Quelle surprise deuxième: motivated parents means better student performance.
2) Low end performance: low performers actually make up some of the gap during the school year. This creates the obvious suggestion of year round school which runs into the fact that would require an immediate 25% pay raise to every teacher.
3) Raising the average/median: Even the Gates foundation documented the solution but stopped short of suggesting it--focus most of your resource on the lowest performers as they are the easiest to improve. I don't even have to suggest the firestorm that causes.
4) Proper student:teacher ratios: Again even the Gates Foundation (whom I loathe) documented it correctly--1 classroom with 2 credentialed teachers (randos aren't enough) per 15 students (middle and elementary was the focus--high school is a bit different). Every program that followed that formula had solid documentable success. Every single program that followed that formula got closed for being "too expensive".
I can go on and on. The problem is that the US education system is at a solid local minimum and getting out of it requires significant amounts of focused resource. And when you finally ask folks to start writing checks for your education system, you suddenly find out exactly how much folks want to improve education (aka nothing for teachers or students, but they'll happily fund that new stadium).
And, I would like to point out that it was school spending that went up by 45% more than inflation (which was 35% over the same period). In addition, teacher salaries didn't go up 45% relative to inflation. So, might I suggest that perhaps the problem is what we are spending the money on?
> focus most of your resource on the lowest performers as they are the easiest to improve
That depends heavily on the pedagogical approach. There are approaches that are quite effective in bringing low performers up to near-par (so-called "direct instruction", in a broad sense) but teacher actively hate them because they're viewed as "demeaning" the profession, and ed schools don't teach them. Special Ed teachers actually get extensive instruction in these approaches, but obviously we cannot and should not treat every low performer as Special Ed.
> Proper student:teacher ratios
What's "proper"? Teacher-centered and direct approaches cope quite well with greater class sizes, but again they're unpopular among teachers.
> Teacher-centered and direct approaches cope quite well with greater class sizes, but again they're unpopular among teachers.
Cite. All research I have seen completely contradicts this.
The limitation with larger class sizes is not "knowledge transfer"; it's "classroom management" aka dealing with a student causing an interruption for some reason (bathroom, injury, sickness, etc.).
I use the Gates Foundation as my primary citations because they are easily findable on the web and simply match all of the other findings. You max out at about 15 students per 2 teachers because one of the teachers can handle the inevitable disruption while the other teacher can continue teaching. The more students you add on top of the roughly 15, the more likely you wind up with 2 interruptions which stops the class cold irrespective of teaching technique.
And, as I have stated, most of the research focuses on elementary to middle levels. High school requires teacher specialization which confounds a lot of the data.
2 replies →
If you make a change in only one school, you end up with selection effects where interested parents move their children into (or away from) catchment areas based on vibes.
Then you can't really measure outcomes, because the strongest predictor of student performance is parents interest and resources.
You also run into issues with teaching skills and standards, you need a high level of planning and adherence to the supplied plan in order to measure outcomes; otherwise it's just vibes based on individual teachers.
Big studies have an important role. Especially for dramatically different approaches, such as the different approaches to teaching reading. The differences are so acute, that careful A/B, or A/B/Control studies are the best approach.
But most improvements in any complex system happen iteratively, and benefit from clusters of subtle changes found to work well together. At some point enough experience is gained to characterize the change, and give others a chance to consider it.
I suggested incremental adoption, and organic adoption, of successful changes, precisely because of this need for significant bottom up testing before spreading something widely. Success at one scale, and location, doesn't always translate directly to another context, or might not work in another context at all.
> You also run into issues with teaching skills and standards, you need a high level of planning and adherence to the supplied plan in order to measure outcomes; otherwise it's just vibes based on individual teachers.
You point out a very important concern. Good measurement doesn't make bottom-up improvement impossible, unnecessary, or any less important. The point of measurement is to make improvements easier to see, and not get in there way.
This is what I was referring to when I said at the top, the job is to create a context where improvements can happen.
One of the simplest ways to balance top-level and bottom-up concerns, is to communicate the actual top-level needs (not just current practices) clearly, then let front line educators propose changes to measurement practices, where they feel the current practice is holding them back. That gets both scales working together to enable improvements to happen, and to be seen. There is no (competent) conflict here, the opposite.
Yeah, maybe we should have given a control group of kids infinite doomscrolling before we gave it to all them.
What you want is done all the time.
What happens a lot:
1) Someone (a researcher, usually) comes in and tries some radical new program in some school.
2) (sometimes) It works! It works great, in fact.
3) This new system or approach or framework gets publicized. This may include dissemination through academic channels, but also (and especially if it's really going to take off) through a kind of reform-grifting network that turns the whole thing into a bunch of stuff that can be sold, for actual money (training, materials, consultants). Turns out being an education researcher pays dick-all, but selling a "system" pays real cash dollars—for many researchers, admin, and curriculum-design folks, getting a windfall from being part of one of these is their most promising path to "making it" before they're old.
4) Some districts adopt the new thing, often with initial pilot programs. Some spend a lot of money doing it.
5) Few of them spend much time considering whether there are material differences between their schools and the one(s) where the system was proven (the experimental program was proven in a troubled inner city school? Surely our middling suburban school can expect similar improvements!). Expertise of and authority granted to the person or persons implementing the system also isn't considered as a factor (one or both are usually lacking, compared with the case or cases on which the promise of the system is judged).
(My personal "here's what to do if you want to fix schools" is "fix our justice/corrections system, worker protections, healthcare, and our social safety net". I think the biggest improvements to our schools would be found there. It's all stuff outside schools. That's why we keep struggling to make headway by monkeying around with schools themselves. It's why more money for schools doesn't help much. That the US finds it basically impossible to do anything constructive about any of those problems is... a sign we can expect not to see any huge across-the-board positive changes in US public school performance any time soon, I reckon)
Meanwhile, within and among districts, individual schools do pilot new programs, et c. All this stuff happens. Does it always happen with everything that turns into a broader reform? No, not always. Is this kind of activity constant, and common, in schools? Absolutely. Frankly it happens way too much (because people are desperate and flailing around to find a path to improvement through school reform, but see above about why I think they are doomed to remain desperate). There's an absolute shitload of process and curriculum churn in schools.
Consider also that while all the above is going on, you have the usual incompetence and principal agent problems you see in any organization. Important tasks are handed to the person an assistant superintendent's having an affair with (god, so common) for whom they invented a paid position. Systems are picked apart and bits adopted piecemeal while ones admin find too uncomfortable or scary are dropped up-front without even trying them, while anyone used to analyzing systems like this can see that the parts their dropping support and are necessary for the success of the parts they're keeping, dooming the reform before it's even implemented. Empire-building happens. Things get hijacked for personal gain. Powerful folks' own inept efforts at breaking into the reform-grift industry get pushed on those under them, as they try to get their own success story to sell. Superintendents or principals fall for obvious bullshit at one of their drinking-and-driving retreats er I mean conferences, because frankly most of them are kinda dumb, and then a whole district gets to suffer for a couple years. Et cetera. Same crap you see in big corporations.
But! Despite all that, lots of people are out there running experiments and reform pilot programs just as you suggest, and for the right reasons, and sometimes even competently. It's just that as soon as it goes past that, it tends to get caught up in all the above. However, even the best-considered reforms that show promise in early experiments and trials are rarely broadly-applicable enough, and familiar enough, and simple enough, and easy enough, and effective enough, to survive that process of wider application without being destroyed. Plus (to repeat, and IMO) I just don't think there are many big wins to be had with educational reform on its own, without working on things outside schools that are resulting in lots of hard-to-educate-in-a-classroom kids.
Every now and then, though, you get a really solid improvement, like, "hey that Whole Language thing that sure seemed to a lot of us to be backwards-ass garbage that really looked like it was making kids worse readers, in-fact, whatever its proponents claimed? Yeah, turns out it is backwards-ass garbage, we can improve reading markedly by knocking that off". (see process and systemic pitfalls outlined above for how it ended up widely in-use in the first place)
>> incompetence and principal agent problems you see in any organization
Principal agent problem is a huge part of it. How do you keep school leadership accountable and effective? It's super hard in any organization.
The answer to that is relatively simple to identify, difficult to acheive.
School system leadership at the very top. Instead focusing on fixes or specific initiatives at the top, needs to understand their job as creating the culture, climate and support for improvement by lower levels of leadership, with strong emphasis on everyone working to enable and support teachers support students.
That is how accountability and responsibility, and initiative get distributed together to everyone who can contribute. And power doesn't "choose" what is expedient.
But given education at the State and Federal levels is highly politicized, this necessity of enabling and supporting the organization, as apposed to controlling it, is hard to accomplish directly.