It is often said by education reform critics that if you look at U.S. education by state, some states actually excel in international comparisons. The following charts are from a paper by Hanushek, Peterson, and Woessman on how U.S. States perform compared to other nations by looking the percent of students that scored advanced on 2006 PISA math exams. Massachusetts clearly outperforms all the other states, but is still outdone by 14 countries, 11 of which perform better by a statistically significant margin. The U.S. overall does much poorer:
I thought this chart served as a useful reminder that all states have room for improvement, and it’s not true that some of them can simply pat themselves on the back, say “great job!”, and turn their backs to education reform.
10 comments
Comments feed for this article
Wednesday ~ March 9th, 2011 at 12:08 pm
Nick Bradley
Control for demographics and you’ll get a completely different picture.
East Asians living in the US score like East Asians.
Norwegians living in Minnesota score like Norwegians.
Latinos score like Chileans, Portuguese, and Spaniards.
Black Americans score like Africans.
With the exception of Black America (the state of schools in black neighborhoods is borderline criminal), US ethnic groups score similarly to their cultural brethren abroad. But the longer their families are in the US, these difference wash away. 2nd-generation Latinos score much, much better than 1st-generation Latinos.
Please see the official government statistics on the matter below:
http://nationsreportcard.gov/science_2009/g8_state.asp?subtab_id=Tab_4&tab_id=tab1#tabsContainer
Wednesday ~ March 9th, 2011 at 2:21 pm
Jeff
I appreciate the use of data to bolster your point. Clearly, no state, not even Massachusetts, should pat itself on the back and say, “great job”, and turn away from education. It is also worth pointing out, that of the the 10 states that are clearly above the line, 8 states are more than 90% unionized, with unions that are active in educational policy. Only VA (6th) and NC (9th) do not fit that description, whereas the majority of those clearly below the line do not fit that description. While these data do show that there is plenty of work to be done, they do not support the contention that the biggest problem with education in America is teachers’ unions.
Wednesday ~ March 9th, 2011 at 2:25 pm
Adam Ozimek
Look at the other charts in the report. Texas does pretty well too.
Wednesday ~ March 9th, 2011 at 11:49 pm
Jeff
Texas does beat the average with respect to white kids. Even on that scale, however, the best states tend to be highly unionized and the worst (e.g., Alabama) tend to have weak or non-existent unions. Moreover, I’m not sure what you think pointing to one case proves. Using the data reported in the appendix, student achievement on these measures (percent of students classified as ‘advanced’) clearly tracks the strength of teachers’ unions.
Again, these data show that American achievement levels are unsatisfactory, and again, while unions’ could be part of the problem, these data contradict that hypothesis.
Thursday ~ March 10th, 2011 at 8:36 am
Adam Ozimek
Reliance on correlations here presumes a lot about “all else equal”, and I don’t think they are sufficient to test any meaningful hypothesis about unionization and outcomes.
Friday ~ March 11th, 2011 at 3:01 am
Jeff
Just to be clear: In your world, a single cherry-picked example of a state, Texas, with little unionization and decent scores on one metric proves that teachers’ unions are the biggest problem with education in America. Meanwhile, consistently strong, positive correlations between the degree of unionization in all states and the quality of educational outcomes on every metric assessed should be completely dismissed, because we cannot be sure of the assumptions about “all else being equal”.
I think you and I have a different approach to methodology. For what it’s worth, I have never claimed that these correlations are determinative (I have mentioned this above in each of my comments). What I do say is that conditional on these data, the hypothesis that unions are the problem and the hypothesis that unions are not the main problem are not equally likely. That is, from a Bayesian perspective, these data should shift your subjective beliefs; your posterior distribution should not equal your prior. In addition, you have suggested various papers from the economics literature that I have subsequently looked up. *None* of them ran true experiments; *all* of them used correlational data.
Friday ~ March 11th, 2011 at 12:17 pm
Adam Ozimek
I think you’re overrelying on correlations and quantifiable measures in understanding this issue. Don’t get me wrong, I really am a numbers person, but I think a mindless devotion to them can lead you astray, especially when you have such huge endogeneity problems. Here is my basic framework for how I think about this: It’s quite observable in the real world what kind of policies unions push for. Unions fight against charter schools, and once you have some they fight for caps on the number. Unions fight against merit based pay, and fight for LIFO. They’ve been giving on these issues somewhat, but still remain instransigent, furthermore these are just the policies we want to change today, not tomorrow. But overall it’s pretty clear the kinds of policies unions tend to fight. Now these policies are much more amenable to data analysis, and here there is a lot to talk about and look at. That I think is the proper frame for this debate: we know what kinds of things unions fight for and against, and on balance what do we know about them? Another difficult to quantify is the effect of the ability to experiment and be flexible. Unions make this harder since they negotiate lengthy contracts and require administrators to only seek experimentation that doesn’t affect the contract or that the union agrees on.
So yes, I think we have differences in methodology in understanding this. But I don’t think ignoring realities we can see in the world around us, e.g. unions fight against charter schools, and insisting on finding a correlation for every fact, is the best way to understand this issue.
Saturday ~ March 12th, 2011 at 2:31 am
Jeff
Well, I appreciate your response. I think this response is helpful, and allows us to inch towards a productive discussion.
I’m quite clear about the endogeity problems with these data, and I’ve consciously tried to make that manifest in just about every comment I’ve posted. I do think that cuts both ways, though. It *is* “quite observable in the real world” that teachers’ unions have fought against charter schools and merit pay, among other things. Lets assume for the moment that these policies will improve educational outcomes. (It’s worth noting that the literature on charters is that they don’t, and that the literature on merit pay is mixed. My personal take is that KIPP and HSZ do work, so some good may come from charters in the long run, and similarly, that the lag between merit pay policies and educational outcomes is going to be long and so the literature won’t be worth much for a while. But set all that aside for the moment and assume that these policies are an unalloyed good.) There can be many causal pathways from unions to educational outcomes. So one pathway can be stronger unions -> fewer charters -> worse education. But unions can have many effects, even many that we’ve never thought of; heck, there could be thousands. So there could be another pathway from stronger unions -> more satisfied teachers -> better teachers stay in the field -> better education. That might exist, or it might not. Many others that have not occurred to either of us could exist, and they might have varied effects. What matters in this situation is how the positives and negatives weigh out on balance. If we only look at the policies that fall out of some theory or our own prejudices, we will have a biased view of what is going on. (Although it may not be apparent, on a deep level this is related to the idea of endogeneity.) This means there is a role for looking at what actually happens in the world and taking that into account. Once again, it’s not determinative, it’s entirely possible that on net unions are a big negative. But if you want to claim that, you are obliged to provide a plausible account of how that’s true given the evidence to the contrary. It is possible to do that, but the more of a negative you want to claim, and the more of a positive they appear to be in the real world, the harder that is to do. And that is why, I suspect, it is more convenient to just ignore or dismiss the first-order data.
Saturday ~ March 12th, 2011 at 12:50 pm
Adam Ozimek
I think I’ve certainly made the case for the positive effects of unions elsewhere. The point of this (http://modeledbehavior.com/2011/03/04/the-market-for-teachers/) post is that counterbalancing monopsony power is a legitimate aim of unions, but that there are better ways to limit that monopsony power. Ironically, charters are one of the best ways available to limit competition. But as the study in that post suggests, in areas with very little competition, and even without unions, workers appear to be sharing in the economic rents. I’d argue that the positive aspects of unions can be had in ways that don’t come with the negatives.
Another point I’d like to make is to strongly disagree with this statement: “Lets assume for the moment that these policies will improve educational outcomes. (It’s worth noting that the literature on charters is that they don’t, ”
I think it’s fair to say that the literature on education reforms is mixed, and no gordian knot has been found, but this statement misleading. Most of the studies do find that charters improve outcomes for poor students and in urban areas. In some states charters do outperform public schools, and this suggests performance depends in part on state policies. So it’s inaccurate to look at charter performance as a whole and use that to gauge whether states can utilize charters to improve outcomes.
Wednesday ~ March 9th, 2011 at 2:41 pm
teageegeepea
What do you think of Heckman‘s claims that there’s little marginal gain from improvements by the time kids are old enough for school?