Chad Aldeman at the excellent Quick and the Ed explains how improvements in the NAEP reading scores are masked by¬†looking at overall scores because of Simpson’s Paradox. To put it simply, this means that the you can see progress when looking at group-by-group numbers, but then see no progress overall due to relative group sample sizes. In the context of the NAEP, what this means is that overall scores show no progress because of changing demographics, and that looking within demographic groups progress is obvious.

Here are the overall long-term NAEP scores which show very little progress:

And here are the same scores broken down by race/ethnicity, which paint quite a different picture:

Chad sums up these graphs:

Each group has actually made greater gains over time than the overall total. White students increase 11 points, one more than the national average. Black students scored 23 points higher, and Hispanic students were scoring 24 points higher in 2008 than they were in 1975 despite quadrupling in size. In other words, the white-black and white-Hispanic gaps are closing and every group is scoring higher, but the national score is showing more modest improvements because of demographic changes.

About these ads