Who's likely to benefit from the new USNWR law school rankings formula?
Melissa Korn at the Wall Street Journal dropped the news today that USNWR plans on changing its formula for the law school rankings:
In a letter sent Monday to deans of the 188 law schools it currently ranks, U.S. News said it would give less weight in its next release to reputational surveys completed by deans, faculty, lawyers and judges and won’t take into account per-student expenditures that favor the wealthiest schools. The new ranking also will count graduates with school-funded public-interest legal fellowships or who go on to additional graduate programs the same as they would other employed graduates.
This is a remarkable sea change. As I recently pointed out, I did not anticipate much of a match between the boycotting law school’s tactics and complaints, and the ultimate USNWR response. I pointed out earlier that many of the complaints were about information that was already publicly available. And sure enough, the per-student expenditures, which were not the subject of complaints but the thing USNWR could not independently collect without voluntary participation from schools, are the first on the chopping block.
So which schools might it benefit or adversely affect the most? Let’s take a look at these three categories, and one other variable I’ll mention at the end.
1. Reduced weight to reputational survey data. Reputational surveys get 40% of the overall weight in the rankings—a lot. That’s 25% for the “peer” survey (i.e., other law school survey respondents), and 15% from the “lawyer/judge” survey.
Let’s start with the “peer” survey. Among the current overall-rated “top 50” schools, the schools that would benefit the most from this change (i.e., their peer score has lagged their overall score). (And this is just a raw comparison of rank v. rank; there are more nuanced issues dealing with the weighted Z-scores, scaling, and the like for another day…. And, of course, differences can be exaggerated the farther down the rankings one goes, a reason I also confined this to the “top 50” for now.)
POTENTIAL WINNERS, PEER CHANGE
George Mason (65 v. 30)
BYU (52 v. 23)
Alabama (36 v. 25)
Florida (31 v. 21)
Utah (47 v. 37)
Wake Forest (47 v. 37)
Texas A&M (56 v. 46)
Georgia (36 v. 29)
How about the other side? That is, the schools that would be adversely affected the most from the change? Again, focusing just on the existing USNWR “top 50” for now:
POTENTIAL LOSERS, PEER CHANGE:
UC-Irvine (20 v. 37)
Washington (36 v. 49)
Colorado (36 v. 49)
UC-Davis (24 v. 37)
Wisconsin (31 v. 43)
Boston College (27 v. 37)
Emory (20 v. 30)
There are a few others in the middle of some interest, because near the top smaller variations matter more. NYU (3 v. 7) and Georgetown (11 v. 14) are harmed the most.
Now, the degree to which this benefits or harms a school entirely depends, of course, on how much USNWR chooses to reduce the weight of the category.
For schools presently outside the “top 50,” schools that stand to gain the most include Wayne State, Baylor, Penn State-Dickinson, Tennessee, and Penn State-University Park. Schools that stand to be harmed the most include Santa Clara, Howard, Brooklyn, Rutgers, Denver, Georgia State, American, and Hastings.
Now, over to the “lawyer/judge” survey. It’s a smaller percentage, and, again, it depends on how much the change in weight goes. For those who stand to gain the most:
POTENTIAL WINNERS, LAWYER/JUDGE CHANGE:
Texas A&M (90 v. 46)
Arizona (77 v. 45)
George Mason (54 v. 30)
BYU (43 v. 23)
Arizona State (49 v. 30)
Alabama (43 v. 25)
Utah (54 v. 37)
Maryland (64 v. 47)
Boston University (28 v. 17)
And who are likely to be adversely affected the most:
POTENTIAL LOSERS, LAWYER/JUDGE CHANGE:
Boston College (24 v. 37)
Washington & Lee (24 v. 35)
Washington (39 v. 49)
Wisconsin (33 v. 43)
William & Mary (20 v. 30)
Emory (20 v. 30)
It should be noted, schools that appear on both lists in the best/worst categories may have much more to be happy/unhappy about.
Outside the top 50, schools most likely to benefit include UNLV, Wayne State, Florida International, Hawaii, Georgia State, Penn State-University Park, and Arkansas. Schools most likelty to be harmed include Howard, Oklahoma, Miami, Michigan State, Hastings, Lewis & Clark, Pittsburgh, and Case Western.
2. Eliminating per-student expenditures. Again, this is not publicly-available data, but it doesn’t take much effort to realize that the wealthiest schools, typically (but not always!) private schools, tend to be harmed the most by this change. Public schools (but not all!) are likely to benefit most. It was by far the biggest differentiator among many schools, giving schools like Yale and Harvard nearly insurmountable leads. Of course, it was also notoriously opaque and subject to manipulation.
If you want to consider the schools most adversely affected, it would be also useful to look at private law schools that have risen sharply in the rankings in recent years, or law schools that have had recent naming gifts or building/capital expenditures that allow an influx of spending.
By the way, it’s worth considering a new and different incentive for law schools situated within universities right now. Law schools could presently make the case to central administration that high spending on resources, including on law professor salaries, was essential to keeping one’s place in the rankings. No longer. It’s worth considering what financial incentive this may have on university budgets in the years ahead, and the allocation of resources.
3. Counting graduates with school-funded public-interest legal fellowships or who go on to additional graduate programs the same as they would other employed graduates. A couple of caveats. First, schools do not have to self-report if their full-time, long-term, bar passage-required or JD-advantage jobs that they subsidize are “public interest” or not, although one could suspect they mostly are public interest. These categories are also very small overall. Among the 35,713 graduates in 2022, 427 were pursuing graduate degrees (1.1%), 276 had school-funded bar passage-required jobs (0.7%), and 56 (0.2%) had school-funded JD-advantage jobs. The change will take pursuing a graduate degree from lesser to full weight, and school-funded positions from a lesser weight to full weight. (This is the only change really directly responsive to some of the elite schools’ earliest complaints.)
But those positions are not all equitably distributed. Who has the most in these three categories, which I’m lumping together for present purposes:
Yale 17%
Stanford 8.7%
District of Columbia 8.3%
San Diego 8.3%
South Dakota 7.7%
BYU 7.6%
Berkeley 7.0%
UC-Irvine 6.6%
Harvard 6.2%
Penn 6.2%
It’s worth considering whether the totals in these positions will climb in future years as the incentives have changed.
4. What fills the gap. USNWR has just announced it would eliminate expenditure data (per-student is 9%, “indirect” expenditures at 1%, if that’s also on the chopping block; as to library resources, it remains unclear for now). It also announced it would give reduced weight to the 40% categories of reputational surveys.
How USNWR backfills its formula to get to 100% will matter tremendously to schools. If you are doing well in all other areas of the rankings, maybe not so much. But if you are a school with relatively poor admissions medians and strong employment outcomes (a great value for students, to be frank!), you benefit much more if the new formula gives more weight to the employment outcomes. And if you are a school with really strong admissions medians but relatively poor employment outcomes, you benefit much more if the new formula gives more weight to the admissions statistics. Time will tell. UPDATE: USNWR has disclosed that it intends to give “increased weight on outcome measures.”
In short, there’s much uncertainty about how it will affect many law schools. I feel fairly confident that a handful of the schools identified above as winners in several categories, including Alabama, BYU, Georgia, and Texas A&M, will benefit significantly in the end, but one never knows for sure. It also has the potential to disrupt some of the more “entrenched” schools from their positions, as the more “legacy”-oriented factors, including spending and the echo chamber of reputational surveys, will receive less value. Law schools must increasingly face the value proposition for students (e.g., lower debt, better employment outcomes), with some other potential factors in the mix, in the years ahead.
UPDATE: It’s worth adding that USNWR has indicated, “We will rank law schools in the upcoming rankings using publicly available data that law schools annually make available as required by the American Bar Association whether or not schools respond to our annual survey.” That suggests other data, including the “employed at graduation” statistics and the “indebtedness at graduation” statistics, at the very least, would also disappear.