The hollowness of law school rankings

“We’re a top-ranked law school.”

Those words, in their various forms, are found everywhere in legal education marketing materials. They are hollow words. In my reflection, they grow more hollow each year.

It’s hard for me to think of where to begin a post like this one. Maybe I’ll start with what we think make a great law school. It’s great people, in a great community, doing great things. And others might have different definitions. But let’s start here.

*

Each of these requires a preexisting definition of “great.” Great at what?

The first is great people. It draws faculty who excel at writing and speaking, teaching and mentoring, reading and listening. It’s people who as passionate about these aspects of legal scholarship and legal education, who aspire to give their students meaningful guidance as they begin their careers. It draws students who are engaged and active in the classroom, inquisitive and active, eager for journals and service to the community.

The second is a great community. It’s one thing to have great people working in silos, or great students studying and going their own ways. But to have a great community builds upon those assets, people who can support one another to ensure that articles are even sharper in their clarity and argument, that classroom experiences are even more meaningful to students by learning from one another, that employment opportunities for students are supported across the faculty, staff, and students to build a culture commitment to student success.

The third is doing great things. This requires some look at the outputs—the quality of the articles and books from the faculty, the influence of law journals and centers at the law school, the success (more than just “elite placement”) of students in legal careers in the short-term and the long-term. It can take a lot of forms, traditional legal scholarship and engagement with the legislature, bar, and bench; placement in elite law firms and public interest work; advancing interests in the local community and in the nation as a whole.

The broader the pool in each class—more great people, stronger community engagement, higher output of great achievements—the better the institution.

*

From a prospective students view, assessing these things is difficult. It can be a challenge for a prospective law student to know exactly what “great” looks like. A student may want to do X or Y kind of law, but not really know what that means if an institution discusses its programs there or its alumni in that field, or how to weigh that against other competing concerns—or if it’s all just hype that doesn’t translate into the results one may want. Or a prospective student may not know exactly what she wants to do (particularly true of first-generation law students), and be at a loss of how to compare these things.

There is a temptation, then, to seek out advice. Undoubtedly, those with attorneys in the family or those in upper-class social strata or education circles get advice of varying types. But many also look for external validation, because it can be difficult to make assessments based on the representations of schools alone.

*

External validation can be rankings. I’ve been highly critical (admittedly, an easy position for a law professor to take!) of most law school rankings—at least, those rankings that purport to be comprehensive, to distill everything about a school into a single measure. But I acknowledge there’s a reason they're out there: prospective students in particular look for help evaluating schools.

I confess, I was particularly attracted to rankings early in my blogging career, even ranking the rankings. (Links, mercifully, herein omitted.) Over time, I realize that was largely a symptom of my desire to generate traffic by ranking something, anything, for someone’s feedback. That’s not to say comparing law schools is unimportant, particularly for prospective students. But it’s to turn rankings into, well, clickbait. And perhaps the most clickbait-y of all are singular rankings that aggregate a series of factors for one, “true” ranking. Rankings can't do that.

*

It’s the convergence of a few things, then, that may give rise to this hollowness of rankings. One is the tyranny of metrics, the obsession of measuring everything and evaluating everything on the basis of those measurements. I’m all for the "data-driven” or empirical evaluation of what we do. The tyranny part comes when those measures are used at the expense of all others, or used without proper acknowledgement of their limitations.

The bulk of rankings methodologies are much older than the available “analytics” we have and may desire to use today. Consider, again, USNWR, which includes a significant amount of inputs in its rankings, and which are not, in my judgment, useful. For instance, law students should worry much less about incoming metrics—essentially, self-congratulatory admissions-oriented metrics—and instead look at student outcomes.

I’ve tried to look more at student outcomes, from institutions’ commitments to reducing debt loads, to debt-to-income ratios of graduates, to employment outcomes at graduation, to federal judicial clerkship outcomes. Others have built on employment outcomes, too, in ways that are more helpful and more lucid than the USNWR figures (the published figures, for what it’s worth, are not the figures it uses in its actual ranking).

But unquestionably, the most alluring rankings are, really, any rankings, good or bad, that put a school in a good light (and may validate a prospective law student’s desire). Free pre-law magazines make them up. Blogs make them up. Clickfarms make them up.

*

Those rankings are everywhere. And it allows schools, with ease, to cite them. But the line, “We’re a top-ranked law school,” reflects two great weaknesses of so many law schools: lack of confidence and a lack of vision.

Lack of confidence arises from the inability to articulate to others—prospective students, current students, alumni, donors, faculty, staff, and the larger university—of what the school is accomplishing. It might be that too many overstatements of a school’s achievements now fall on deaf ears. Or that there’s simply distrust in self-promotional presentations of a school’s accomplishments. And it’s recognizing that these “others” won’t necessarily heed the list of accomplishments without some reference to some ranking—as weak or as hollow as the ranking may be—to shore up the chronicles of success about the institution.

Lack of vision arises from an inability to articulate success. Rather than define success to a public audience, they rely on others’ definitions of success as validated through a ranking, and they promote that ranking as the end, as the definition of success.

I admit, it might simply be that these others prefer to have some external validation of the school’s quality, rather than something internal. But schools could readily identify the things I pointed out at the beginning: what makes the school great? It can be data-driven, or it can be a qualitative narrative. Ideally, it’s a combination. Schools should have confidence in their own vision as they’ve articulated and measured it, and they should be able to persuade relevant outsiders about why the law school is succeeding on these terms, not on someone else’s terms.

Maybe that’s all too idealistic. It’s impossible to unring the bell of rankings. But I think schools should be spending much more effort thinking about how to define success and how to communicate that.

*

Here we sit on the eve of yet another USNWR ranking, one that gives weight to inputs, to dated measures like how much money a school spends on its electric bills—to an overall ranking that moderately correlates with some ways that we can think of “good” schools. But it’s time for schools to think about how hollow these rankings are, and to think about how to move beyond them in ways to persuade prospective students, the greater academic community, and the public about the institution’s value.

I’ve had a version of this post drafted in my blog queue for several years. I’ve been tweaking it now and then, and just never got around to posting it. These are my initial thoughts, that of course merit much deeper evaluation in the future!