Wisconsin Policy Research Institute School Choice Study: More

The following was left as a comment to an earlier post of ours, and we feel it deserves the prominence of a full-fledged post. It’s from George Mitchell, policy wonk and activist who has been a stalwart supporter of school choice.

George Mitchell said...
The following was sent to the study’s author:

October 26, 2007

Dave,

WPRI’s sponsorship of your recent study, combined with the Journal Sentinel’s misleading article, has generated wide interest in the findings. You can anticipate many questions regarding the methodology you employed and whether it was appropriate for the Milwaukee situation.

For reasons summarized below, I am interested in knowing if you or WPRI submitted your methodology to others for independent review. If so, can you share the responses you received? If the methodology was not reviewed by others, I think it would be advisable for you or WPRI to seek such a review.

You estimate “the extent and nature of public school choice and parental involvement” in Milwaukee by applying “a variety of demographic variables…correlated with…parental choice and parent involvement...” The variables, derived from a “national data set,” were:

• educational attainment of parents
• race and ethnicity of students
• household composition (single-parent vs. two-parent), and
• mother’s employment status.

The “national data set” consisted of answers to an NHES survey. These answers apparently show, for example, that high educational attainment is correlated positively with parental involvement and more engagement in school choice. Similarly, I assume that two-parent, non-minority families also were positively correlated with parent involvement and engagement in school choice. And so on.

You assume (or “extrapolate”) that MPS parents will behave in the same manner as those who answered the NHES survey. By constructing a demographic profile of MPS parents, you thus believe you can estimate the “extent and nature of public school choice and parental involvement within the Milwaukee Public Schools district.” In the specific case of school choice, you conclude that only 10 percent of MPS parents are discerning choosers.

George Lightbourn’s introduction to your report says, “The report you are reading did not yield the results we had hoped to find.” However, given your methodology, the results were completely predictable — even pre-ordained. Specifically, rather than gathering information about the actual behavior of MPS parents, the study simply assumes that they behave in a manner similar to respondents to a national survey. Given the demographic profile of MPS parents, you knew from the start that you would estimate relatively low levels of school choice and parent involvement. To quote one scholar who has read your report, “[WPRI] simply assumes that parents of a certain demographic profile behave the same way no matter what. In this way, the WPRI study assumes the very thing it claims to prove. The study finds that [parents] are not effective choosers based on data from environments where they have few choices.”

The key question, then, is whether the methodology is appropriate for Milwaukee’s unique situation.

Consider some actual numbers, beginning with your ten percent conclusion. Applied only to MPS enrollment, that yields a group of 9,000 discerning choosers. Applied to all Milwaukee students in tax-supported K-12 education, the total would be 12,000. So, you find that between 9,000 and 12,000 students have discerning parents. Yet:

• There are more than 17,000 students who attend schools in the Milwaukee Parental Choice Program.
• More than 5,000 other students attend independent public charters authorized by the City of Milwaukee and UW-Milwaukee. That’s more than 22,000 students.
• There are thousands — perhaps tens of thousands — of MPS students with parents who have chosen neighborhood schools. Indeed, a specific focus of MPS school selection in recent years has emphasized neighborhood schools. Your report appears to discount all those choices by labeling neighborhood schools as a “default” option.
• Roughly 10,000 other parents use the open enrollment process and the Chapter 220 program to make clear choices about where their students attend school.
• Thousands more parents enroll their children in so-called specialty MPS schools. These parents make clear choices. Many of those schools have waiting lists, yet another measure of choice.

In other words, by any reasonable estimate the number of actual choices made by Milwaukee parents substantially exceeds 9,000-12,000. Even allowing for a subjective definition of what a well-informed choice is, the number is simply much greater than your 10 percent estimate.

This raises the possibility that your methodology is highly inappropriate when it comes to the Milwaukee situation, one that is unique among American cities. Given that Milwaukee parents have so many options — a condition that has existed for more than a decade — it is questionable to assume that their behavior will not differ from other urban parents who don’t have such options.

While there are other assertions in your study that I find debatable, those are overshadowed by the methodology question. As part of the continuing discussion your report will engender, I reiterate that an independent assessment of that methodology is in order. This is all the more so given the existence of peer-reviewed scholarship (such as by Hoxby and Chakrabarti) that is at odds with your conclusions; it is unclear why your report does not acknowledge and discuss their work.

George