Rick Hess at Education Week:
The University of Arkansas School of Education, home to my good friends Patrick Wolf and Jay Greene, yesterday released new research showing that students in Milwaukee’s two-decade old voucher program (the Milwaukee Parental Choice Program) “scored at similar levels as their peers not participating in the school choice program.”
Wolf, who has led this effort as well as the federally-endorsed evaluation of the DC voucher program, summarized, “Voucher students are showing average rates of achievement gain similar to their public school peers.” Translation: when it comes to test scores, students with vouchers are performing no differently than other kids. (It is worth noting that MPCP students are being educated more cheaply than are district school students).
What to make of the results? First off, 20 years in, it’s hard to argue that the nation’s biggest and most established voucher experiment has “worked” if the measure is whether vouchers lead to higher reading and math scores. Happily, that’s never been my preferred metric for structural reforms–both because I think it’s the wrong way to study them (see “Science and Nonscience“) but, more importantly, because choice-based reform shouldn’t be understood as that kind of intervention. Rather, choice-based reform should be embraced as an opportunity for educators to create more focused and effective schools and for reformers to solve problems in smarter ways. Whether any of that pays off is much more a question of quality control, support, talent, investment, infrastructure, and the rest than it is of whether or not a choice program is in place.
Second, congrats to Wolf and his team for reporting this straight and for not trying to spin the results. There’s way too little of that, for my taste. And I was happy to see Wolf, who generally favors school vouchers (as do I), not engaging in tortured efforts to make the data tell a happy tale. Advocating for vouchers (or charters or merit pay) by struggling to extract some favorable coefficients from math and reading scores has always been a problematic way to argue the need to fundamentally rethink the design of schooling.
Kevin Carey at The Quick and the Ed:
I’m having a hard time understanding the distinction here. Vouchers aren’t a teaching method or curriculum or instructional intervention. They are, as Hess notes, purely structural, shifting control over resources toward parents and widening the range of institutions that can receive those resources. Since “more focused and effective schools” are properly defined as “schools where students learn more” i.e. “schools with higher reading and math scores,” if vouchers didn’t result in more such schools then vouchers failed. One might argue that vouchers created the opportunity for educators to create such schools and educators didn’t take advantage of it, but what’s the difference? The whole point of structural reform is to change incentives and conditions; if the change was insufficient to create desired behavior then ipso facto the reform failed. A purely structural metric for evaluating purely structural reforms misses the point altogether.
Hess then suggests that vouchers (or school choice programs more generally) are better understood as a necessary but not sufficient condition for improvement that also requires “quality control, support, talent, investment, infrastructure, and the rest.” Perhaps, although it’s worth remembering that those are things that can be effectively applied to the cause of school improvement without vouchers. All in all, I think this lends credence to my theory that you either try vouchers for real or you don’t. Half-measures, of which even the Milwaukee vouchers are an example, will never produce satisfactory evidence either way.
This strikes me as really wrongheaded. It takes time for more focused and effective schools to emerge. Moreover, if they do emerge, as they have in Sweden under that country’s choice regime, they’d likely scale up as national franchises. Milwaukee is a great American metropolis. It’s not going to spontaneously generate the dense marketplace that you’d need.
Am I just recapitulating the argument made by advocates of state socialism — it didn’t work because we didn’t really try it? I don’t believe so, in light of the socialist calculation problem and other vexing challenges facing central planners. I can see how one might feel otherwise. I do caution us, however, against approaching this problem too narrowly.
The truth is that Hess’s arguments are a little too sophisticated for a binary debate. He has made an active effort to make educational reformers, particularly on the right, to go beyond a narrow focus on school choice, and instead to focus on the secondary markets and institutions that can make the broader educational system better faster.
This, by the way, is a good way to think of lots of complicated regulated markets, including the health sector and the financial sector. Choice is important, but it’s certainly not enough. It’s just that in those sectors, we don’t even contemplate restricting choice as severely as we do in the educational space.
Giving parents choices about where to send their kids to school has certain kinds of virtues. It turns out, however, that parents of low-income kids don’t seem to particularly use this freedom to select schools that are good at improving kids’ academic performance. At least they’re not sufficiently invested in doing that so as to put a lot of pressure on schools to figure out ways to improve academic performance. The choice program does seem to lead to a lot of consumer satisfaction, but not actual improvements in performance. It’s sort of like when people switch to a “low fat” version of a product, find it’s surprisingly delicious, and don’t pay attention to the fact that it actually has just as many calories as the old variety.
At the end of the day, to improve academic performance you need policies that specifically focus on that goal. You can shut down charter schools that consistently deliver below-average demographically adjusted academic performance and allow operators of charter schools that consistently deliver above-average performance to open new franchises. You can pay teachers who consistently deliver above-average performance enough to persuade them to keep teaching. You can recognize that schools that teach a lot of poor kids are going to need more resources rather than fewer. You can try to research which pedagogical methods actually work instead of just guessing. But you can’t just throw some procedural switch and fix everything, especially if the process you put in place doesn’t even specifically focus on improved academic achievement.
Andrew Coulson at Cato:
The evidence of this literature is starkly one-sided. The vast preponderance of findings show private schools outperforming public schools after all the normal controls. What’s more, when we focus on the research comparing truly market-like systems to state-run school monopolies, the market advantage is found to be even more dramatic (see Figure 2 in the paper linked above). To draw policy opinions from a small, selective handful of those studies while ignoring the rest is policy malpractice, and it is dangerous to children.
Even the recent Milwaukee result described by Yglesias as a failure shows voucher students in private schools performing as well as public school students who receive roughly 50% more government funding. How is a program that produces similar academic results to the status quo at a much lower cost to taxpayers a failure? And what of the research suggesting that students in the Milwaukee voucher program graduate at higher rates than those in public schools?
More importantly from a long term policy perspective, how is a program limited to 20,000 or so children in a single city, being served almost entirely by non-profit entities, a test of market education? Would Apple have spent hundreds of millions developing the iPhone or the iPad if its market were limited to the same customer base? Of course not. The dynamism, diversity and innovation we have come to expect from competitive markets in other fields relies on the prospect of ultimately scaling up to serve mass audiences. Without the prospect of a large-scale return on investment, there is no incentive to invest in the first place.
Joseph Lawler at The American Spectator:
I think that Coulson has accidentally understated the argument that the vouchers are a lot cheaper because of what looks like a simple arithmetic error: Milwaukee spends $6,442 per voucher student and $14,011 per public school student. That means that they spend more than 100 percent more per public school student, not 50. (The government doesn’t pay all of the voucher students’ tuitions. The average cost per voucher student is $7,703, meaning that the schools must come up with over $1000 of private funding per student.)
The larger point that Hess and Yglesias are getting at is that these voucher students are still not getting a decent education, which truly is a failure. But given the available evidence that school choice works in general and the fact that the system is saving significant amounts (a very salient fact when you consider the condition of state and city finances), it’s probably wise not to overemphasize this one disappointing finding.
So the voucher program achieved the same learning objectives at a lower cost, or more bang for the buck. Since when is that regarded as failure? Let’s consider the following two possibilities:
1. Spending more money on education (at the margin) increases learning.
2. Spending more money on education (at the margin) doesn’t increase learning.
First assume case one is true. This would imply that if we adopted vouchers, and spent as much per student as the Milwaukee public schools spend per student, we would get higher test scores. That is called “success.”
Now assume case two is correct. This would imply that there is no point in spending more money on education. We should simply try to hold down costs. This means that the voucher program in Milwaukee succeeded in the only way schooling can succeed; it provided education at a lower cost than the public school system.
I’m sure that case two sounds very cynical to a progressive like Yglesias. I imagine that he thinks more spending can make a difference, perhaps if targeted to certain methods that have been shown to work. OK, then how about taking the tax saving from voucher schools, and giving those schools a government grant to improve education in whatever area progressives like Yglesias think that money can still help at the margin? Wouldn’t that be a win-win for everyone except unionized public school teachers? I wonder why such a policy has almost no chance of happening.
I suppose the progressive counter-argument is that the policy failed according to the criterion set by the voucher proponents. I have been a voucher proponent from the beginning, and certainly never thought success should be measured by test scores. I’ve always thought parental satisfaction was the proper criteria. Indeed, I would hope that all free market economists agreed on this point. There may be some conservatives who argued that test scores would improve, but why should we care what they think? Every day the progressive bloggers tell us that conservatives are morons. I’d rather judge the program on how well it actually did, using the standard economic criteria of costs and perceived customer benefits, not the single criterion used by central planners.
If a policy that leads to greater consumer satisfaction at lower cost, and produces no negative side-effects in test scores, is viewed as a “failure” by progressives, then I don’t think we need to worry very much when progressives criticize the free market. As Dylan once said: “There’s no success like failure, and failure’s no success at all.”
More Salam
Hess again:
The piece I penned last week on the new University of Arkansas findings on the Milwaukee voucher program has drawn a fair bit of reaction in the blogosphere. I observed that the unimpressive results from the Milwaukee voucher evaluation (touted as the most ambitious evaluation of a U.S. voucher program yet conducted) are not all that surprising and that the bleak results ought not be taken as evidence that vouchers don’t “work,” but as a reminder of how little attention choice proponents have devoted to creating the kinds of oxygenated ecosystems that can support dynamic markets. (For a lengthier discussion on this, check out my new book, Education Unbound).
As controversial as this stance has been with choice enthusiasts, and as inclined as choice skeptics are to regard it as an apologia or an attempt to move the goalposts, I have always thought it a rather unremarkable and commonsensical attitude. After all, it was Milton Friedman who once famously opined that the “market is not a cow to be milked” but is simply a powerful mechanism for channeling human ingenuity, energy, and talent. If markets are dysfunctional, corrupt, or inhospitable to law-abiding enterprises (think of post-Gorbachev Russia), they are more likely to lead to venality than socially productive work.
UPDATE: Charles Murray in NYT
Around The Sphere 
Pingback: What We’ve Built Today « Around The Sphere