In her 2010 polemic, The Death and Life of the Great American School System, Diane Ravitch has praise and criticism for KIPP charter schools. On the one hand, she recognizes the organization improves scores for students. On the other hand, she credits this success, in part, on the schools ability to kick out hard to educate students:
…KIPP schools often have a high attrition rate. Apparently many students and their parents are unable or unwilling to comply with KIPP’s stringent demands. A 2008 study of KIPP schools in San Francisco’s Bay Area found that 60 percent of the students who started in fifth grade were gone by the end of eigth grade. The students who quit tended to be lower-performing students. The exit of such a large propotion of low-performing students –for whatever reason- makes it difficult to analys the performance of KIPP students in higher grades. In addition, teacher turnover is high at KIPP schools as well as other charter schools, no doubt because of the unusually long hours. Thus, while the KIPP schools obtain impressive results for the students who remain enrolled for four years, the high levels of student attrition and teacher turnover raise questions about the applicability of the KIPP model to the regular public schools.
Note that teacher turnover in-and-of-itself is considered a problem. I find it baffling to consider success alongside high turnover as evidence of a limitation of the sucees rather than as evidence that turnover is not necessarily a problem.
The italicized portion of her quote is of particular interest, since it is directly contradicted by a study from Mathematica Policy Institute that showed that KIPP improves test scores for students that ever attend KIPP, including those that leave early. This is a direct contradiction of her claim.
As she does throughout the book, Ravitch drives the point home with a rhetorical flourish befitting of a speech at an NEA pep rally, lamenting the unfair advantages that charter schools have, and how easy that makes it for them compared to the underdog public schools:
Regular public schools must accept everyone who applies, including the students who leave KIPP schools. They can’t throw out the kids who do not work hard or the kids who have many absences or the kids who are disrespectful or the kids whose parents are absent or inattentive. They have to find ways to educate even those students who don’t want to be there. That’s the dilemma of public education.
Ravitch creates the image of KIPP schools taking better students from public schools, and simply kicking out bad students, sending them back into the public school system. This negative model of charter success is an important theme in the book. However, another recent study by Mathematica Policy Institute shows that her claims here are also false. They found that students leave KIPP schools at the same rate as they do for nearby public schools. In fact, for black and hispanic students, the attrition rates for KIPP were lower.
Ravitch also credit’s the lottery admissions for KIPP’s success. Her argument is that
“Like other successful charter schools, KIPP admits students by lottery; by definition, only the most motivated families apply for a slot. Charters with lotteries tend to attract the best students in poor neighborhoods, leaving the public schools in the same neighborhoods worse off because they have lost some of their top-performing students. They also tend to enroll fewer of the students with high needs – English-language learners and those needing special educaiton.”
This complaint puzzles me. Ravitch once was a supporter of charter schools. But if lotteries are “by definition” going to cream skim and advantage charter schools, how did she ever support them? Her argument here is definitional, and not a matter of data. When criticized for changing positions on education reform Ravitch likes to quote Keynes who, perhaps apocryphally, said ”When the facts change, I change my mind. What do you do, sir?”, but have the definitions changed as well?
Furthermore, the Mathematica study found that KIPP did not admit the “best students”. On average, KIPP entrants were “not more advantaged than other students in their communities, as measured by poverty and prior achievement levels”. For instance, 84% of students who attended the sample of KIPP schools qualified for a free lunch program, compared to 64% in KIPP host districts, and 72% in the elementary schools that send any children to KIPP. They enroll more minorities, and they enroll students with lower test scores than the district average and the same as the average for the public schools that KIPP students came from.
They do find that KIPP tends to enroll less ESL students and students with disabilities, but this is not the same as admitting the “best students”. Ravitch clearly agrees with this, as you can see in the quote above she includes the complaint that KIPP doesn’t admit enough ESL and disabled students as distinct from and in addition to the complaint about only letting in “the best students”.
These results are important because if Ravitch claims that KIPP cream skimming higher acheiving students makes public schools worse off, then the fact that they take lower performing students must make public schools worse off. This doesn’t just remove a complaint about charters then, it actually represents a benefit to the public school system. This positive impact of more charters and choice on public schools is reinforced by studies that have shown public schools improve in response to more competition, an entire vein of literature ignored in Ravitch’s book.
These are clear examples of the facts disproving Ravitch’s claims. Will future editions of her book correct this? Will she call attention to this fact and publicly reverse her opinion of KIPP?

19 comments
Comments feed for this article
Tuesday ~ April 19th, 2011 at 8:56 am
Roland
Thanks Adam good take down.
Tuesday ~ April 19th, 2011 at 9:40 am
Anup
Your takedown is not quite convincing. How can not admitting as many dsl and disabled students not boost performance. Such students affect performance of non esl and non disable by taking resources away ( money, teachers and aids). The also typically do not perform as well non esl and non disabled. Plus, charter schools admission and lottery based admission self-select student families who are committed to do better from the same socio-economic background.
Tuesday ~ April 19th, 2011 at 9:57 am
Alex
You would think that there would be some self-selection bias for charter schools. Only parents who care about there children and value education would go through the trouble of putting them in a charter school. These children already have advantages and are better off then other children of similar statistical conditions. I don’t think any of your arguments refute this point adequately.
Tuesday ~ April 19th, 2011 at 10:25 am
Adam Ozimek
Alex and Anup,
If performance difference was simply due to a positive selection bias then you would observe that randomized studies based on lottery applicants would find no difference. But they do find large differences, which means there is a positive impact of charters controlling for self-selecting.
Also, it’s hard to square the story of disproportionately motivated charter applicants with lower prior performance of charter applicants. It may be that a more motivated segment of lower performing students apply to charters, but this is still a subset of *low performing* students, which seems like it should trump motivation. In short: however motivated they are, it’s not motivated enough to overcome their poor performance.
Finally, it seems just as likely to me that a selection problem is working in the opposite direction: e.g., students who are willing to leave their current schools to attend a charter aren’t motivated in unmeasured ways, but rather struggling in unmeasured ways. This is an issue never considered by Ravitch.
Tuesday ~ April 19th, 2011 at 11:49 am
Gepap
Charter schools were first conceived (by teachers, including, GASP!, union members) as pedagogic testing grounds, where new and innovative teaching techniques could be tried out and if found to be more effective, transitioned to all public schools, for the benefit of all students, which justified the public expense.
What pedagogic innovations have been found at KIPP? Can every public school be a KIPP school and still meet the public commitment to providing a certain level of education to every child?
Tuesday ~ April 19th, 2011 at 12:28 pm
Adam Ozimek
I’m puzzled why charter school critics and public school defenders seem to revel in the fact that charters succeed in ways that may not be replicable by public schools. If it were true, it only weakens the argument for public schools and strengthens the case for replacing them entirely with charters.
Tuesday ~ April 19th, 2011 at 3:27 pm
Gepap
Except you CAN’T replace all public schools with charters and expect all charters to do well – that is the point of people not sold on charters. Charters forced to deal with every kid in a neighborhood would eventually be indestinguishable from the existing local public schools.
Do you see charter schools coming up as an option in the affluent suburbs? No, because those constituencies are happy with their public schools, which do quite well. You see charters pop up in poor urban neighborhoods beset by unemployment, underemployment, and relatively high crime, and it is in these neighborhoods that you come up with the issue of selection that Ravitch spoke of. Which comes back to your reply to an earlier comment on selection:
Children do not apply to charter lotteries – their parents do! No primary or middle school child picks their schools, that is something parents or guardians do.
At most a child will spend say 9-10 hours five to six days a week in a school, usually less than that. They spend the rest of their lives at home or their neighborhood. An interested parent makes a huge difference – a disinterested parent on the other hand won’t push their kids, ignore when they have problems, so forth. So the selection bias is one of the child having a home situation supportive of education vs. one not supportive.
Now, if we decide to get rid of laws mandating an education for every child and leave whether a kid gets an education at all or not to parents, then you could set up a universal charter system, but such a system would never face the same problems as the current public school system.
Wednesday ~ April 20th, 2011 at 7:41 am
Adam Ozimek
You just said KIPP wasn’t producing lessons that public schools could adopt, and yet now your saying that charters would become exactly like public schools. How is that possible? Also, you’re argument that charters aren’t really dealing with the hard students is the same as Ravitch’s, which I’ve rebuked above.
Parents in the suburbs surely are more happy with their schools than those in urban areas. And since you think selection isn’t a problem there, there’s no reason to oppose letting them choose between charters and public schools, so I’m gathering you’d support removing all barriers to charters in suburban areas. I agree.
Re: your point about motivated parents, it’s hard to believe the parents are more motivated given that the students are doing worse. This is especially true given that you say “parents make a huge difference”. You’re arguing with yourself here.
Wednesday ~ April 20th, 2011 at 12:44 am
govt_mule
Suppose you have a Little League team that takes every kid in the neighborhood, guarantees each kid equal playing time and never kicks anyone off the team. I have a team that only accepts kids whose parents apply a year in advance and who pledge to practice 5 hours a week with their kids, and I kick any kid who is disruptive or tardy or doesn’t perform well off the team.
Would you be surprised to find that my team scores 10% better than yours? Would you be offended if I claimed I was a coaching genius and/or insinuated that there must be something wrong with your coaching skills or abilities?
Should all Little League teams be like mine?
Wednesday ~ April 20th, 2011 at 7:43 am
Adam Ozimek
This would be an apt metaphor were Ravitch correct about attrition and selection and if all the facts I laid out above didn’t exist.
Thursday ~ April 21st, 2011 at 3:07 pm
govt_mule
It’s a perfectly valid metaphor – there is no mention of attrition or selection
Tuesday ~ April 19th, 2011 at 3:52 pm
Holly
I don’t have a problem with the most motivated parents and children being able to exit a failing public school for a charter. Perhaps it leaves the public school worse off, but that’s not really the concern for the individual parent. They since they’re on a sinking ship and see a life raft to put their child on. If every parent that gave a damn was able to access a charter school, yes, the public schools would be populated with children with uninformed and/or apathetic parents. Then, perhaps, we’d be able to stop blaming the teachers at those schools and start working with the parents that are the crux of the problem.
Wednesday ~ April 20th, 2011 at 10:08 am
Gepap
Adam:
My point is that charters skim by taking those kids with the best support system – the measures you claim show that KIPP takes the same kinds of students as the public schools does not address that point:
1. The fact that kids in KIPP are as likely to be in poverty as not is not a way to measure the involvement of the parents unless you want to claim that parents who care about their kids must by definition be richer, which is not something I think can be sucessfully argued.
2. Past testing results are likely the result of student’s inate skills and the quality of teaching they experienced – neither of those things measures the support system for the child directly.
As for attrition, if my argument is correct, then you should expect lower attrition – the home environment will encourage the kids to stay with it (which reinforces the success). There is a reason why the best single predictor of what level of education someone will get is the level of education their parents reached. Having parents that expect kids to earn a degree does not make all of them A students, but they do ensure most of the time that the kids stay at it, and practice, and over time if the kid has decent instruction, their achievement increases.
If you go to KIPP’s own website, and look at their five pillars, they lay out rules for the parent’s commitment as well. KIPP itself recognizes the importance of the support system at home. This is likely also why Catholic schools have better achievement than public schools in these neighborhoods given similar student metrics – because the parents are more involved and invested.
Regular public schools ask nothing of the parents beyond what is required of parents legally. So how do we create a system in which every child is given a chance, even in the face of lousy or uninterested parents? No one is out there campaigning to take kids away from non-abusive but still non-supportive parents. Can we find some pedagogy that can overcome the failure at home? As of yet, charters do not seem to have found it even if some claim they have.
Wednesday ~ April 20th, 2011 at 10:42 am
Adam Ozimek
“As for attrition, if my argument is correct, then you should expect lower attrition”
And yet you observe the same attrition rate. Which suggests, again, that your argument is not correct.
“At most a child will spend say 9-10 hours five to six days a week in a school, usually less than that. They spend the rest of their lives at home or their neighborhood. An interested parent makes a huge difference – a disinterested parent on the other hand won’t push their kids, ignore when they have problems, so forth. So the selection bias is one of the child having a home situation supportive of education vs. one not supportive.”
“Past testing results are likely the result of student’s inate skills and the quality of teaching they experienced – neither of those things measures the support system for the child directly.”
So past testing is mostly about teaching and innate ability, yet the home situation is ultimately determines their achievement? At what point does the home situation kick in and start displacing past teaching and innate ability as the determinant of achievement? The day before they apply for charter school, I presume?
Wednesday ~ April 20th, 2011 at 1:12 pm
Gepap
Parental involvement can’t overcome bad pedagogy or instruction when it comes to tests, but it can enhance behaviors meant to keep a kid interested and involved in the academic enterprise, which is what gets someone to graduate.
To speak from personal annecdote, I went to a public middle school in NYC, and that school tracked all students – the “good” students were placed in the “1″ class, which was generally taught material one grade ahead, while the kids in the lower classes were taught but not to the same level of course. The school was also very strict so violence was a non-issue. When I joined the school as an immigrant, I was placed in one of the lower classes. It was my mother noticed that I was acing all the material, she demanded that I be moved upwards in the classes, and eventually I ended up in the “1″ class. My mother’s deep involvement in my education had nothing to do with my own ability to score on a test, but had she not pushed, the school would have likely not moved me as it did, so the level of education received would have been much lower.
I eventually went to a highly selective and prestigious public high school and I was there with a lot of very bright students. At that school, sucess in tests was never the metric by which you could tell which students would come out on top – it was the amount of work and their dilligence, and this is certainly somthing that parental expectations influenced highly. The relatively higher emphasis on educational achievement as a metric of personal achievement was clearly a reason for why that high school was over 50% Asian when the population of NYC was at best 12% Asian.
Wednesday ~ April 20th, 2011 at 1:00 pm
govt_mule
Adam
I’m having a hard time understanding what exactly is being reported in the Mathematica studies. Can you clarify?
They don’t seem to be reporting actual test scores, rather some kind of statistical estimate of performance “trajectory”
Another thing that puzzles me is the apparent lack of cumulative improvement. The authors mention several times that if X annual gain in performance were continued for 4 years there would be a cumulative gain of Y. But overall, the data (Table III.3, Fig III.3, Fig III.5 of the first report) show little or no cumulative improvement. In fact there is a decline from year 3 to year 4 (if I understand the data correctly).
Thursday ~ June 2nd, 2011 at 8:10 am
Defending KIPP, again « Modeled Behavior
[...] ~ June 2nd, 2011 in Teaching | Tags: Diane Ravitch, KIPP | by Adam Ozimek I’ve defended KIPP before from accusations that their performance is due to selection bias and cream skimming. A new paper [...]
Friday ~ June 24th, 2011 at 6:45 pm
Diane Ravitch: Teachers’ Hero or Education Hypocrite? | LesBnB.com
[...] belief today that school choice by definition constitutes a damning cream-skimming, which I discuss here (what has changed to make this so?). Or her new fixation with neighborhood democratic institutions [...]
Sunday ~ June 26th, 2011 at 7:31 pm
Diane Ravitch: Teachers’ Hero or Education Hypocrite? | Brian Brown's Website
[...] belief today that school choice by definition constitutes a damning cream-skimming, which I discuss here (what has changed to make this so?). Or her new fixation with neighborhood democratic institutions [...]