On the heels of the new USNWR law school rankings, with its many pre-release errors, we also have a new metric: student debt. It’s actually two components: 3% weight to average law school debt incurred upon graduation among those incurring debt; and 2% weight to the percentage of students who took out loans.

The only problem? It artificially boosts schools with relatively high debt loads and relatively high percentages of students who do not take out loans. Call it the “independently wealthy law student enrollment” bonus. (Of course, it could also be disproportionate full tuition plus living expense scholarships, but that seems less likely.)

When I started tracking law school affordability years ago, I considered students who took out no debt as $0, weighted in the average of law school debt. I included the caveats as I’ve continued to do. But it doesn’t make much sense to treat these two cohorts differently. Some take out loans; others don’t. The average is the average. Include some caveats and weigh all the students together.

USNWR thinks otherwise.

It separately ranks average indebtedness and students who take out loans. This results in a skewing of results against schools with relatively low debt but a relatively high percentage of students who take out loans; and in favor of schools with relatively high debt but a relatively low percentage of students who take out loans. The latter is the independently wealthy law student enrollment bonus.

This is exacerbated by the tremendous compression for law schools around the percentage of students incurring debt. The difference between 74% and 75%—really, a rounding error, is 0.044 “scaled and weighted” points. That’s the same as the difference an extra $2300 in average tuition indebtedness, which converts to about 0.044 “scaled and weighted” points. Even though the average is 3% and the percentage incurring debt is 2%, the disparity is stark.

UPDATE: I should add, if one were to weight them at 2% and 3%, the figures would be much smaller, more like 0.003. I was doing a 60%-40% allocation using the larger figures to compare these figures to one another, not as components of the overall rankings. But the relative figures still hold, whichever “scaled and weighted” points one wants to use.

How stark? Assume a graduating class with 150 students. Let’s assume, for the moment, they have the average debt of $108,000 (about the median) and 74% indebtedness (about the median).

To move from 75% indebted to 74% indebted is 2 students. Pick off the two students with the least debt and tell them they qualify for some student loan forgiveness program—really, the two students least in need of it. That could bring you down 0.044 points.

In contrast, among the remaining 111 students indebted, they bear $11,988,000 in cumulative debt—nearly $12 million even. To bring the average debt loan down $2300—an equivalent amount of scaled and weighted points—you’d need to trim a whopping $255,300 from the cumulative debt load to bring the average down. (Admittedly, a school can only “game” this so much before the average indebtedness begins to rise substantially among the remaining students.)

It’s a poor metric, in my judgment, when the tradeoffs are placed against each other like that. I ran some figures to examine how the methodology would play out if we had a 5% weight for the single metric of average indebtedness, factoring in students who did not incur any debt among the overall debt figures. I then compared that metric to a 3%-2% weight separating the schools. (The raw figures are here.)

Here’s my rough calculation of the ten schools that benefited most, and the ten schools that benefited least, from the decision to separate the columns rather than to incorporate together. Remember, these aren’t sorted by lowest-to-highest debt loads. Instead, they’re sorted by those who benefited the most from the decision to separate the two metrics rather than put them together.

USNWR debt metric winners (over metric that averages all students together)

Southwestern Law School $190,184 - 82% ($155,950)

California Western School of Law $164,918 - 88% ($145,127)

St. Thomas University $161,701 - 86% ($139,062)

Nova Southeastern University (Broad) $155,193 - 87% ($135,017)

Columbia University $190,141 - 66% ($125,493)

Golden Gate University $151,854 - 83% ($126,038)

American University (Washington) $159,723 - 76% ($121,389)

Harvard University $170,866 - 70% ($119,606)

University of San Francisco $156,460 - 77% ($120,474)

Florida Coastal School of Law $145,245 - 86% ($124,911)

While some schools have percentage incurring debt above the median of 74%, even 85% indebtedness, when weighted, only presses schools so high because of compression. When factoring into the overall debt figures, however, it keeps these schools will above the overall median.

USNWR debt metric losers (over metric that averages all students together)

University of South Dakota $53,253 - 80% ($42,602)

Cleveland State University (Cleveland-Marshall) $69,727 - 90% ($62,754)

Florida A&M University $61,500 - 81% ($49,815)

Ohio Northern University (Pettit) $71,134 - 88% ($62,598)

University of Nebraska--Lincoln $63,027 - 78% ($49,161)

Rutgers University $62,210 - 75% ($46,658)

University of North Dakota $67,281 - 78% ($52,479)

University of Arkansas--Fayetteville $68,877 - 79% ($54,413)

Texas Tech University $56,898 - 72% ($40,967)

University of Utah (Quinney) $76,344 - 85% ($64,892)

These schools—unsurprisingly, public schools with low tuition and low indebtedness—suffered in the rankings because a 90% indebtedness looks absolutely terrible compared to the 74% median; in contrast, it would seem that these students are, on the whole, dramatically better off than some of the ones that benefited from a separate ranking.

One notable outlier is the University of Tulsa, with around $90,000 indebtedness, well below the median, but reporting 100% indebtedness, putting it at the very top and giving it an extremely poor score on this metric. But when balanced against the actual median of about $80,000, it performs only slightly above the median and would benefit significantly.

It’s worth noting that some schools that most benefited had above-median (against, about 74%) percentages, and some schools that benefited least had below-median percentages. But the size of the debt was spread far more significantly from top to bottom, which skewed the overall results.

In short, if schools want to manipulate this poor metric from USNWR, the solution is to buy off the least debt-ridden students before graduation to reduce their loans to $0—or admit far more independently wealthy students, or provide more full-tuition scholarships at the expense of partial tuition scholarships. Those, in my judgment, are the wrong incentives.