The American Bar Association ("ABA") has long been evaluating federal judicial nominees, and it has received some scrutiny for how it goes about doing so. There have been empirical studies to show that Republican-nominated judicial candidates tend to receive lower scores than Democratic-nominated candidates, studies which admittedly offer their own limitations.
...as an aside, I've also found it interesting to dig through the ratings of those who appeared on President Donald Trump's "list" of prospective Supreme Court nominees:
Brett Kavanaugh: revised rating Q (sm), WQ (min) (backstory on downgraded rating here)
Thomas Hardiman: Q (sm), NQ (min) [on nomination to Third Circuit, WQ (1 abstention)]
Raymond Kethledge: WQ (sm), Q (min)
Amul Thapar: WQ (1 abstention)
Diane Sykes: WQ (sm), Q (min)
Steven Colloton: Q (sm), WQ (min), NQ (min)
Raymond Gruender: Q
Neil Gorsuch: WQ
Timothy Tymkovich: Q (sm), NQ (min)
Bill Pryor: Q (sm), NQ (min)
Federico Moreno: Q
To be fair, there are different traits that might make one a good district court judge, appellate judge, and Supreme Court justice. But it's worth noting, I think, that the very candidates a Republican presidential administration considers as most worthy of a Supreme Court nomination received, on the whole, fairly middling grades from the ABA.
Back to the topic at hand. I want to set those aside for a moment these debates, and look instead at something else. Is the ABA any good at doing what it purports to do?
As the Standing Committee on the Federal Judiciary reports, "the Committee focuses strictly on professional qualifications: integrity, professional competence and judicial temperament." The goal is to "ensure that the most qualified persons serve on the federal judiciary."
Is the ABA any good at that?
In part, that's because the ABA is typically looking backward at a candidate's record, then trying to project it forward to how the ABA believes that person will behave as a judge. It might be the case that past performance is an indicator of future success, but it also might be the case that the ABA is relying on weak measures of "qualifications."
One problem is the "rating" system itself, which lacks any nuance and instead offers the kind of thumbs-up/thumbs-down (and thumbs-sideways) of a movie review. Yes, there are probably several minutes of thoughtful film commentary that could precede that final rating, but, here, the ABA actually leaves all that commentary on the cutting room floor. All we have are opaque inputs and a single output.
One of the criteria that the ABA uses is "experience," and it includes some hard-and-fast proxies for experience: "The Committee believes that a prospective nominee to the federal bench ordinarily should have at least twelve years’ experience in the practice of law." This isn't a terribly thoughtful criterion, even if it has the advantage of being a fairly clear rule. That said, one would be hard-pressed to think a rule like this does very much to fill out the term "qualified" or "not qualified." After all, Roger Ebert might well have said, "If a movie comes in under an hour twenty, I give a thumbs down." But if we have no rush to get younger judges, then perhaps it's a fairly harmless criterion.
Additionally, the committee makes other kinds of ex ante determinations about what makes a good judge, like "substantial courtroom and trial experience as a lawyer or trial judge is important." These tend to skew the judiciary toward those with more practical experience, true; it also skews toward litigators and trial lawyers. For appellate judicial nominees, the ABA places "somewhat less emphasis on the importance of trial experience as a qualification for the appellate courts." It prizes certain types of experience: "While the Committee recognizes that civic activities and public service are valuable experiences for a prospective nominee, they are not a substitute for significant experience in the practice of law in either the private or public sector."
For those presidents who pre-screened their lists of applicants with the ABA, the results can be frustrating. President Barack Obama saw the ABA reject 14 of his prospective judicial nominees as "not qualified" his first three years in office. As Obama administration officials complained, "In particular, they have questioned whether the panelists — many of whom are litigators — place too much value on courtroom experience at the expense of lawyers who pursued career paths less likely to involve trials, like government lawyers and law professors."
Now, perhaps these ABA litigators are right, and perhaps their criteria are superior. Could that be measured? That would be a new and valuable place for future ABA studies. But that is also difficult to quantify. Allow me to offer a few thoughts.
First, we have a handful of notoriously bad-behaving judges we can examine.
Thomas Porteous was rating unanimously "qualified" (not "well qualified"), but he was impeached and removed for committing perjury by signing false financial disclosure forms and abusing his judicial office.
Samuel Kent was unanimously rated "well qualified," but he was impeached and later resigned from office for lying about sexual misconduct involving female employees.
Mark Fuller, in contrast, received a "qualified" rating with a minority "not qualified," before resigning after an investigation involving allegations about spousal abuse. (I should add, maybe this is a hard thing to measure ex ante....)
Second, I looked to a couple of the examples cited recently of more controversial nominees, and then I examined what litigants had to say about those judges in the Federal Judicial Almanac.
Roger Benitez received a substantial majority "not qualified," with a minority "qualified." Here's highlights from the Federal Judicial Almanac on him: