Academic

Rankings Talk

By Gordon Smith, Christine, Hurt, Vic Fleischer, Fred Tung, Lisa Fairfax, David Zaring

As noted below (and on every other law blog, it seems), this year's law school rankings by U.S. News have leaked. Ranking methodologies are not my forte, but I am responding to Dan Solove's invitation to discuss possible improvements. In the interests of full disclosure, one of the sub-items on my agenda is to speak to the alumni and students of BYU's law school about our numbers.

As to the overall ranking, Brian Leiter makes a convincing arguments that it "combines too many factors, in an inexplicable formula, and much of the underlying data isn't reliable, and some of it (e.g., expenditures on secretarial salaries and electriciy) isn't even relevant." Nevertheless, that overall ranking seems to be as important as it has ever been, judging by traffic on discussion boards and local conversations. And it's not clear to me that a proliferation of other rankings (like this new one from Vault) or the distribution of other information is going to change that.

One improvement would be to create clusters of schools (as is done with the lower-ranked schools), rather than the numbered ranking that prevails in the Top 100. Every school that experiences a major shift from one year to the next has a story about a single input or perhaps two inputs that caused the shift. The schools are so tightly clustered that small changes produce huge effects. Last year, for example, BYU dropped 10 spots, a result attributable largely to a small decline in our entering class GPA. You might think that entering class GPA is an important factor in law school quality, but I doubt that BYU is a materially different place as a result of the GPA of one class.

This year, the GPA and LSAT are both up, placing our student numbers comfortably among the Top 20-25 schools on that measure, but our overall ranking declined. We believe that this effect was the result of U.S. News' decision to change the placement calculation. In prior years, students who were not seeking employment were not counted in the denominator. This year, they were counted against our placement numbers. I am told that we have a number of former students who passed the bar, but decided to dedicate themselves to their families rather than to seek employment. I suspect that BYU has an inordinate number of such students, and given the small size of each class (approximately 150 students), this change in calculation dropped our placement number from over 97% to just over 91%. I understand that the category of "students who are not looking" is subject to abuse, but I hope U.S. News revisits their decision to include such students in the denominator.

The bigger issue here is that U.S. News is constantly changing the formula to produce changes in the rankings. This is good marketing, and it curbs to some extent the impulse to play to the rankings. By substituting clusters for numerical ranks, U.S. News would moderate some of the effects of these changes, though that is precisely why it has no incentive to embrace clusters. The interest among readers in small changes from one year to the next is what keeps this machine running. Quite brilliant on their part.

Full post as published by Conglomerate on March 26, 2008 (boomark / email).