Is our kids learning? Probably, but at least when it comes to reading they aren’t learning any better than in the past. NAEP reading scores for 2009 have just been released and they’re pretty uninspiring. Scores for 8th graders have been flat since 2002, and as the chart on the right shows, they’ve been flat at pretty much all achievement levels. Good readers are reading as well as they did in 2002 and poor readers are reading as poorly. Black-white and Hispanic-white gaps have narrowed ever so slightly, and the male-female gap has stayed about the same. Broken out by public, private, and Catholic schools, scores still remain flat across the board.
So is there any good news? Well, scores for the worst readers have improved at the 4th grade level, and it’s possible that these improvements will eventually filter up to the higher grades as well. That hasn’t happened in the past (in fact, gains in the lower grades tend to wash out at higher grades), but you never know. Maybe this time they will.
Of course, there is another bit of good news: American students may not be improving much in reading, but neither has there been a wholesale collapse, as news reports sometimes suggest. Reading scores are slightly up over the past two decades, and math scores are up considerably.
When it comes to education, though, it’s always worth peering into the demographics, since socioeconomic factors are so important to student achievement. It looks to me that black eight graders are reading somewhat better. And so are hispanic eighth graders. White eighth graders look to have plateaued since 2002, but improved a lot in 1992-2002. And scores for Asian American eighth graders are also up. Low income kids seem to be doing a bit better.
So I think what’s happening is that we’re actually seeing modest achievements that are being masked by a growing proportion of minority or poor students in the sample. Black kids, Latino kids, and poor kids have all long done worse than white kids or middle class kids. But the performance of the education system is best judged with some kind of demographic controls in place. One way or the other, however, any change taking place is pretty modest.
Chad Aldeman at The Quick and The Ed:
Stories from the NY Times, Mother Jones, and the Washington Post bemoaned the flat National Assessment of Education Progress (NAEP) reading scores released Wednesday. Jay Matthews called it the epitaph of the No Child Left Behind era. The results aren’t quite so simple.
See, NAEP is different than most standardized tests. It takes a sample of the current population in every state, so this year’s population of kids is compared to the last time the test was administered. There’s an automatic correction for changing demographics, so as America has gotten less white, so has NAEP. In statistical terms this creates something called Simpson’s Paradox, which makes trend lines seem worse than they really are because of a hidden variable, in this case, race (Matthew Yglesias touched on this point yesterday).
Just a quick followup to yesterday’s post about NAEP reading scores. Over at The Quick and the Ed, Chad Aldeman suggests that reading scores have improved more than I suggested — possibly due to a statistical anomaly called Simpson’s Paradox1 that can produce different results for an entire group than it does for all its subgroups. But that’s not what’s going on here. The main issue is that he’s looking at different test scores than I am: he uses the Long Term NAEP for 4th graders to look at progress since the mid-70s, while I used the Main NAEP for 8th graders to look at progress since 2002. The former shows significant progress for all ethnic groups, while the latter is virtually flat for every group except Asians.
Click the link for more. I don’t have any special axe to grind about the use of either version of NAEP or about which timeframe is appropriate to look at. It all depends on what you’re investigating. But it’s worth knowing that different data is out there.
Lindsey Burke at Heritage:
But, despite the bad news, there is an outlier among the states– Florida. The Sunshine State’s results are a bright ray:
Reading scores for all students. While 4th grade reading across the country remained flat, 4th grade reading in Florida rose two points and 8th grade reading role 4 points over 2007 figures.
Although these gains are good, the improvements made by sub-groups (special needs students, African American students, Hispanic students, English language learners, and low-income students) are the most impressive.
Special needs students. Special needs students in The Sunshine State made tremendous gains in reading. Gains in 8th grade reading were impressive, with special needs students scoring 11 points higher than they did in 2007. 8th grade special needs students in Florida are now a full 10 points ahead of the national average of special needs students. 4th grade special needs students also made impressive progress, scoring 9 points above 2007 levels in reading. Florida 4th graders with special needs are now 15 points ahead of the national average in reading for special needs students. But special needs students in Florida have an advantage over many other special needs students in most states – McKay Scholarships, which allow them to attend a private school of their choice.
African American students. African American students in Florida made significant gains in 4th grade reading, and now exceed or tie the statewide average for all students in eight states: Alaska, Arizona, California, Hawaii, Louisiana, Mississippi, Nevada and New Mexico.
Hispanic students. Hispanic students have made the most impressive progress in reading of all subgroups. Hispanic students in Florida now outpace or tie the statewide average of all students in 30 states. (As of 2007 Hispanic students outscored the statewide average of all students in 15 states).
Achievement in Florida started to trend upward in 2000 – two years after a set of sweeping education reforms were introduced by then governor Jeb Bush. Under Bush’s tenure – and prior to NCLB – Florida implemented the following reforms:
- Performance pay. Schools began receiving bonuses for student improvement, which go directly to principals and teachers and circumvent collective bargaining.
- Alternative teacher certification. Florida now allows reciprocity with other state teaching certificates, as well as on the job training and alternative routes to the classroom.
- Ended social promotion. Third grade students who fail to score at least 1 out of 5 on the Florida Comprehensive Assessment Test (FCAT) are not passed on to the 4th grade.
- State standards. Florida implemented rigorous state standards and corresponding assessments, testing students each year in grades 3-10.
- Grading schools. Florida moved to a more direct and comprehensible way of grading schools, transitioning from a I-V grading scale to an A-F grading scale. Not only does the A-F scale provide transparency for parents, it attracts tremendous media attention each year when grades are released.
- Options out of failing schools. Florida didn’t just demonstrate through the grading system whether a child was in a failing school, Florida gave parents options to act on that information. Florida parents have access to vouchers for special needs students, corporate tax credits, charter schools, and the largest online learning school in the country.
Unlike the administration’s plan for reauthorization of NCLB, Florida’s actions provide a much more reasonable blueprint for how to successfully raise student achievement. As reauthorization moves forward, conservatives need to push for policies that allow for the flexibility to innovate that Florida has made so successful.
Florida’s strong ‘09 NAEP scores continue an impressive trend—a trend that began well before No Child Left Behind was implemented, which should be a lesson about the limits of federal policy for lawmakers moving forward on its reauthorization this year.
Jennifer Cohen at New America Foundation:
We used Federal Education Budget Project data on state poverty levels and per pupil expenditure and merged it with 2009 NAEP data for the percent of 4th and 8th graders scoring proficient or above in reading and math. These data suggest that in some cases, high per pupil expenditures are connected to high student performance. For example, more 4th and 8th graders score proficient or above on both math and reading in Massachusetts than in any other state. Massachusetts had the 8th highest per pupil expenditure in the country in 2007 (the most recent year for which we have data) – $12,857. Similarly, New Jersey, the state with the highest per pupil expenditure, has high student achievement scores relative to other states. It ranks in the top five on each 4th and 8th grade test.
But not all high spending states demonstrate high achievement levels. The District of Columbia (though not technically a state) spends the third most per pupil in the country – $15,511 in 2007 – but has the worst student achievement. Seventeen percent or fewer of DC’s 4th and 8th graders score proficient or above in reading and math. Only Louisiana and Mississippi come close to such low achievement levels. It should be noted that these two states and DC have the highest student poverty rates in the country.
Some states that spend relatively less on a per-pupil basis do, however, demonstrate impressive NAEP results. Idaho, with the second lowest per pupil expenditure in the country at $6,648 (less than half of what New Jersey spends), still displays reasonable achievement levels, particularly in 8th grade. Thirty-eight percent of Idaho’s 8th graders scored proficient or above in math, the 16th highest in the nation. South Dakota’s students also perform well given the state’s low per pupil expenditure (42nd in the country at $8,064). Their 8th grade math performance levels rank 8th in the country and their 8th grade reading levels rank 9th.
Clearly, how much a state spends per pupil matters far less than how that state spends that money on education services. Unfortunately, these spending methods probably aren’t standardized either. Hence, good outcomes are far more expensive in New Jersey or Massachusetts than they are in Idaho or South Dakota. But one conclusion should stand out above all others – proficiency levels in even the most impressive states never break 60 percent. In fact, national averages on all four tests hover in the mid-30 percent range. That means that far more than half of the nation’s 4th and 8th graders are still scoring below proficient on both math and reading. Given that finding, it seems like every state needs to start rethinking how they spend their education dollars.
These data for all 50 states and the District of Columbia can be downloaded here.
Catherine Gewertz at Education Week:
Even as the scores of most student groups—girls and boys and those of various races and ethnicities—have risen over time, gaps since 2007 showed no improvement, and gaps since 1992 narrowed in only two areas: between black and white students in 4th grade and between boys and girls in 8th grade.
In fact, despite widespread concern about boys’ reading skills, the latest NAEP scores show boys making greater improvements than girls since 1992, Mr. Loveless pointed out.
The 2009 NAEP was the first based on a new reading framework, or testing blueprint. The framework places more emphasis on literary and informational texts, uses a new way of assessing students’ vocabulary knowledge, and includes poetry. A NAGB analysis concluded that results from tests based on the new framework can be accurately compared with results of tests based on the previous framework, which had been used since 1992.
Among states, Kentucky alone saw increases in reading scores at both grade levels since 2007. Rhode Island and the District of Columbia saw scores rise only at the 4th grade level, and seven states—Alabama, Connecticut, Florida, Hawaii, Missouri, Pennsylvania, and Utah—saw increases only in 8th grade.
Terry K. Holliday, Kentucky’s commissioner of education, attributed the gains to Reading First and to multiple state reading initiatives focusing on elementary and middle school. Reading coaches were dispatched to many schools to work with teachers, he said, and professional development in reading instruction was provided not just to English/language arts teachers, but to those in other subjects as well.