Is the ABA any good at evaluating judicial nominees?
The American Bar Association ("ABA") has long been evaluating federal judicial nominees, and it has received some scrutiny for how it goes about doing so. There have been empirical studies to show that Republican-nominated judicial candidates tend to receive lower scores than Democratic-nominated candidates, studies which admittedly offer their own limitations.
...as an aside, I've also found it interesting to dig through the ratings of those who appeared on President Donald Trump's "list" of prospective Supreme Court nominees:
Brett Kavanaugh: revised rating Q (sm), WQ (min) (backstory on downgraded rating here)
Thomas Hardiman: Q (sm), NQ (min) [on nomination to Third Circuit, WQ (1 abstention)]
Raymond Kethledge: WQ (sm), Q (min)
Amul Thapar: WQ (1 abstention)
Diane Sykes: WQ (sm), Q (min)
Steven Colloton: Q (sm), WQ (min), NQ (min)
Neil Gorsuch: WQ
Timothy Tymkovich: Q (sm), NQ (min)
Bill Pryor: Q (sm), NQ (min)
To be fair, there are different traits that might make one a good district court judge, appellate judge, and Supreme Court justice. But it's worth noting, I think, that the very candidates a Republican presidential administration considers as most worthy of a Supreme Court nomination received, on the whole, fairly middling grades from the ABA.
Back to the topic at hand. I want to set those aside for a moment these debates, and look instead at something else. Is the ABA any good at doing what it purports to do?
As the Standing Committee on the Federal Judiciary reports, "the Committee focuses strictly on professional qualifications: integrity, professional competence and judicial temperament." The goal is to "ensure that the most qualified persons serve on the federal judiciary."
Is the ABA any good at that?
In part, that's because the ABA is typically looking backward at a candidate's record, then trying to project it forward to how the ABA believes that person will behave as a judge. It might be the case that past performance is an indicator of future success, but it also might be the case that the ABA is relying on weak measures of "qualifications."
One problem is the "rating" system itself, which lacks any nuance and instead offers the kind of thumbs-up/thumbs-down (and thumbs-sideways) of a movie review. Yes, there are probably several minutes of thoughtful film commentary that could precede that final rating, but, here, the ABA actually leaves all that commentary on the cutting room floor. All we have are opaque inputs and a single output.
One of the criteria that the ABA uses is "experience," and it includes some hard-and-fast proxies for experience: "The Committee believes that a prospective nominee to the federal bench ordinarily should have at least twelve years’ experience in the practice of law." This isn't a terribly thoughtful criterion, even if it has the advantage of being a fairly clear rule. That said, one would be hard-pressed to think a rule like this does very much to fill out the term "qualified" or "not qualified." After all, Roger Ebert might well have said, "If a movie comes in under an hour twenty, I give a thumbs down." But if we have no rush to get younger judges, then perhaps it's a fairly harmless criterion.
Additionally, the committee makes other kinds of ex ante determinations about what makes a good judge, like "substantial courtroom and trial experience as a lawyer or trial judge is important." These tend to skew the judiciary toward those with more practical experience, true; it also skews toward litigators and trial lawyers. For appellate judicial nominees, the ABA places "somewhat less emphasis on the importance of trial experience as a qualification for the appellate courts." It prizes certain types of experience: "While the Committee recognizes that civic activities and public service are valuable experiences for a prospective nominee, they are not a substitute for significant experience in the practice of law in either the private or public sector."
For those presidents who pre-screened their lists of applicants with the ABA, the results can be frustrating. President Barack Obama saw the ABA reject 14 of his prospective judicial nominees as "not qualified" his first three years in office. As Obama administration officials complained, "In particular, they have questioned whether the panelists — many of whom are litigators — place too much value on courtroom experience at the expense of lawyers who pursued career paths less likely to involve trials, like government lawyers and law professors."
Now, perhaps these ABA litigators are right, and perhaps their criteria are superior. Could that be measured? That would be a new and valuable place for future ABA studies. But that is also difficult to quantify. Allow me to offer a few thoughts.
First, we have a handful of notoriously bad-behaving judges we can examine.
Thomas Porteous was rating unanimously "qualified" (not "well qualified"), but he was impeached and removed for committing perjury by signing false financial disclosure forms and abusing his judicial office.
Samuel Kent was unanimously rated "well qualified," but he was impeached and later resigned from office for lying about sexual misconduct involving female employees.
Mark Fuller, in contrast, received a "qualified" rating with a minority "not qualified," before resigning after an investigation involving allegations about spousal abuse. (I should add, maybe this is a hard thing to measure ex ante....)
Second, I looked to a couple of the examples cited recently of more controversial nominees, and then I examined what litigants had to say about those judges in the Federal Judicial Almanac.
Roger Benitez received a substantial majority "not qualified," with a minority "qualified." Here's highlights from the Federal Judicial Almanac on him:
Alison Nathan received a majority "qualified," with a minority "not qualified." From the Almanac:
As a law professor who has to read (often inconsistent) student evaluations of my own performance each semester, I'm well aware of the limitations evaluations like these might present. But, in my view, they reflect, on the whole, that both judges are, with some possible weaknesses, capable and competent (even "qualified") judges. And, of course, perhaps someone will counter that using the Almanac has an entirely different set of flaws to evaluate judges.
Now, I have no idea how many Type I & Type II errors come out of the ABA judicial nominating process, at least to the regard that it's trying to anticipate who is "qualified" for the bench. I just cherry-picked a few examples, and I made no effort to dig deeper.
Additionally, these judges are hardly a random sample. They are selected, at times, to comport with the ABA criteria, and, with a couple of recent Republican administration exceptions, exclude candidates unanimously rated "not qualified." Other political reasons sink some nominees. Some items arise so far in the future that perhaps the ABA could not reasonably have been thought to be capable of evaluating that candidate's qualifications.
That said, I do think there would be tremendous value in examining the Federal Judicial Almanac entries of recent nominees and trying to compare them with ABA ratings. I assume, but perhaps I am wrong in the assumption, that the correlation between "qualified" ratings and the feedback from litigants is uneven. Additionally, I wonder if, over the course of a judge's career, the judge's capabilities (particularly instincts regarding exercises of discretion) improve to a degree that lessen the significance of any shortcomings. (Admittedly, learning on the job may be cold comfort to some early litigants!)
All this is to say, I wonder, setting aside the political critiques of the ABA at the moment, whether its evaluation process is even doing what it's designed to do in the first place.