Gaming the College Rankings (It’s Not Just Marquette)

Three years ago we reported how Marquette had begun to manipulate the ACT and SAT scores of the entering freshman class, in order to improve our ranking with U.S. News & World Report.

We, for course, had no illusions that Marquette was the only university doing this. According to an essay on Minding the Campus.

The most obvious examples of gaming are efforts to increase colleges’ standing on the U.S. News rankings lists. Years ago the University of Miami created a scandal by omitting the scores of athletes and special admits when calculating average SAT scores of entering freshmen. The president of a major liberal arts college in New England told her admissions director to make SAT scores the major criterion for admissions decisions until the school made it into the top 25 schools in its category. Once that happened, she had the chutzpah to announce that they would no longer require SAT scores from applicants because they don’t really mean much.

Earlier this summer Catherine Watt, a former administrator at Clemson, gave a well-publicized presentation in which she explained how her institution approached the “reputation” part of the US News survey, which asks officials to assess the academic quality of peer undergraduate programs. This is a silly question in the first place - college administrators are the last people you would ever go to in order to get accurate information about a competing institution - but it accounts for a quarter of the rankings formula. Watt showed how Clemson simply sought an edge by rating competing schools as below average. Other gambits have included sending dollar bills to alumni with a request that they send them back to the Annual Fund so as to raise the proportion of alumni donors and inflating the number of fulltime faculty members.

And more:

Clemson has drawn heavy publicity for cutting corners on data supplied to U.S. News. But other colleges and universities are involved as well. Baylor recently asked students already accepted to the university to take the SAT again, hoping for higher scores, and gave the students $300 bookstore credits for complying. Albion College reported a $30 alumni donation as $6 a year for five years so the percentage of annual gifts from graduates would rise for several years. The University of Southern California reported that 34 of its professors were members of a prestigious engineering association, though an investigation showed that the number included people who had moved to industry or retired, while only 17 were currently on staff and teaching. In the mid-1990s Boston University raised its SAT scores by excluding the verbal scores of foreign students, while including their math scores, a practice believed to be fairly common. At about the same time, the University of South Florida raised the group SAT scores of its students by lopping off the bottom 6 percent of all scores and Monmouth University improved its statistics by simply adding 200 SAT points to its group score. More recently, in an exchange on the Inside Higher Ed Web site. a writer from an unnamed northeastern university said he knew for sure that a director of institutional research at his school had forced the resignation of the provost by telling the president that manipulated data had been sent to U.S.News. Middlebury College in Vermont has a director of institutional research and its provost was asked to resign in 2007.

In spite of all this, it’s still not the case that “everybody does it” (even if a lot of people do it) and it still leaves a bad odor.

Shouldn’t a Catholic university be teaching, by word and action, that “everybody does it” is not a sound form of moral reasoning?

Marquette Gets High National Rankings

From the Marquette PR people, an e-mail:

Washington Monthly and Princeton Review rank Marquette nationally

The Princeton Review and Washington Monthly magazines recently announced college rankings in which Marquette University ranked among the top universities nationally.

Marquette ranked 48th in the nation in Washington Monthly magazine’s annual College Guide, released this week. Unlike other college ranking systems, the Washington Monthly rates individual schools based on their tangible contributions to the public interest — the degree to which they recruit and graduate low-income students; produce PhDs and research; and encourage students to serve in ROTC, the Peace Corps and other service programs. In topic-specific areas, Marquette ranked seventh nationally in the percentage of students who participate in ROTC and 34th in the country for participation in the Peace Corps.

Marquette has also been named in The Princeton Review’s annual publication “Best 366 Colleges.” Only 15 percent of the four-year colleges in America and two Canadian colleges were chosen for the book. In its profile on Marquette, The Princeton Review described classes as challenging. Students in the profile said faculty are “truly top-notch and care about both teaching and research. Marquette is highly underrated for what we receive. Most professors personally care about the learning experiences of their students and make every effort to offer help or insight inside and outside of class.” The Princeton Review does not rank order the colleges but provides a two-page profile of each school.

Digging into the Princeton Review rankings (which requires a free registration) Marquette is classified on the sub-lists “Class Discussions Rare” (not so good!), “Best Midwest Colleges”, “Colleges with a Conscience.”

The data for all this is apparently a combination of information provided by the institution, and the results of a student survey. Since the student survey does not use any systematic sampling, and since a very small number of students participate, the validity is rather questionable.

For example, under “Academics” and “Students Say,” the favorable comments quoted above are found. But the blurb also comments that:

The few students with leftist political leanings warn that “the professors and the students are incredibly conservative, which can be difficult because the university requires a ton of theology and philosophy credits.”

In fact, Marquette professors are liberal and (sometimes) leftist in the main, and this kind of comment merely reflects the fact that some leftist students get entirely bent out of shape hearing any ideas that they find unorthodox.

The Washington Monthly rankings use an interesting set of criteria to rank schools.

Schools get points for having more students getting Pell Grants (a kind of financial aid reserved for students from low income families). You get points for getting a higher graduation rate than your freshman class SAT/ACT scores would predict -- which could mean the program lacks rigor, or that the school does a good job of engaging students intellectually and helping students who have problems. You get points if the college gets a lot of research grants and graduates a lot of Ph.D. students. You get points if a lot of your students go into the Peace Corps, but to provide a nice ideological balance, you also get points if a lot of your students are in ROTC.

Are any of these ranking valid? Not particularly. Having a lot of Ph.D. programs is associated with having faculty who are top scholars in their field, but also frequently means that teaching undergraduates takes a back seat to research.

Having a lot of students eligible for Pell Grants means that the college is performing a splendid social function of providing upward mobility for low-income kids, but is also likely to mean that a lot of students will need remedial instruction. It may also mean that bright kids will not be challenged the way they would in student bodies made up of kids who have had all the middle-class advantages.

Still . . . Marquette really is one of the better universities in the country. We would rather sit high in the rankings and explain why they don’t mean so much than to be ranked poorly and have what we say dismissed as sour grapes.

We are also not happy about the joy these rankings will bring to top Marquette administrators. Marquette is the fine school it is due to the faculty, the students, the traditions and the alumni, and not due to the current crop of administrators. The place is as good as it is in spite of them, and not because of them.