Politifact fact-check: the Ninth Circuit is, in fact, the most reversed federal court of appeals
Recently, cable news personality Sean Hannity commented that the Ninth Circuit is the "most overturned court in the country." Politifact rated that claim as "false." But Politifact's analysis is seriously flawed and suffers from selective analysis of the evidence, and misrepresentation of the evidence in other respects.
I recently had the opportunity to appear on NPR's AirTalk to support a proposal from Arizona Senator Jeff Flake's office to split up the United States court of Appeals for the Ninth Circuit. The "Judicial Administration and Improvement Act of 2017" would keep California, Oregon, Hawaii, and some U.S. territories in the Ninth Circuit. It would create a new Twelfth Circuit out of Arizona, Nevada, Montana, Idaho, Washington, and Alaska.
One reason for splitting the circuit is systemic dysfunction in the Ninth Circuit. It has 29 active judges, (but four vacancies at the moment) nearly the size of the 30-member Arizona Senate (to speak on terms for those from the Grand Canyon State). That's nearly twice as large as the 17-member Fifth Circuit. It's little wonder that decisions take longer (often much longer) to issue from the Ninth than anywhere else. Splitting the circuit would help create a new "Mountain Circuit" that would function fairly effectively, and the new Ninth Circuit would remain the largest circuit in the country.
More judges might help the Ninth speed along cases, but it would not help its high reversal rate. That's because the court lacks the ability to self-correct with true en banc procedures; the entire 29-member court can't really assemble, and it's left instead to a lottery of some subset of these judges to correct errors from three-judge panels. And in a lottery of three judges among 29, some combinations are sure to be greater outliers than others. Splitting the circuit would allow it to have true en banc procedures and minimize reversal rates.
The Ninth Circuit's legacy is cemented by instances like October Term 1996, when it went 1-for-28, the stuff of legend. The Ninth Circuit is reversed more often perhaps because its size accounts for poorer outcomes.
So, to Mr. Hannity's claim that the Ninth Circuit is the "most reversed." While there are arguments raised in the Politifact piece that "most reversed" may have limited normative significance, that's a separate argument. (But, Politifact can't help itself to weigh in on the normative claim, concluding, "More broadly, experts say this statistic is a poor way of comparing courts.") Instead, as a matter of pure math, is the Ninth Circuit the "most reversed"?
Politifact says no. The answer is resoundingly yes.
Politifact chooses a cohort of data from 2010 to 2015 to conclude that the Ninth Circuit is the third-most reversed, behind the Sixth and Eleventh Circuits. But why pick this window of time?
I used the same analysis with slightly different data--Politifact uses the SCOTUSBlog Stat Pack, which is slightly simpler and less comprehensive in time than the Harvard Law Review statistics, which I opted to use (and may result in some slight variations of the numbers). Because of changes to the Harvard Law Review statistics system, I also didn't have ready access to Eleventh Circuit data between OT1994 and OT1996.
The chart below displays the cumulative reversal rate, which includes opinions from the Supreme Court that reverse a court, vacate an opinion from a court, or reverse in part or vacate in part. It is the cumulative reversal rate based on the term listed: so, for instance, OT2014 data is the cumulative reversal rate for OT2014 & OT2015; OT2013 is the cumulative reversal rate for OT2013, OT2014, and OT2015. I ran the figures for the Sixth, Ninth, and Eleventh Circuits.
It would be hard for Politifact to manufacture a window that more perfectly enables it to refuse Mr. Hannity's claim. If you begin to include any earlier terms, the Eleventh Circuit quickly falls behind the Ninth in reversal rates; by including a 12-year window back to OT2004, the Ninth Circuit is the most reversed, and holds that trend back through cumulative data to at least OT1994.
Now, there are, of course, many ways to slice such data. Is the Ninth Circuit relatively better than it has been? (Answer: yes, but not by much--its cumulative reversal rate since OT2012 was around 80%, not much better than the rate of around 82% since OT1994). Is the Ninth Circuit not as bad as other circuits in recent years? (Answer: yes, but only with a fairly limited window of time.)
But for Politifact to so brazenly rate Mr. Hannity's claim as "false" displays its choice to evaluate his claim through a single and precise window--perhaps to achieve a result that the fact-checker desired to reach.
Below are charts for the 9th Circuit and 6th Circuit reversal rates dating back to OT1994.
| 9th Circuit | rev/vacate | total | rev rate |
| OT1994 | 14 | 17 | 82% |
| OT1995 | 11 | 13 | 85% |
| OT1996 | 27 | 28 | 96% |
| OT1997 | 14 | 17 | 82% |
| OT1998 | 14 | 18 | 78% |
| OT1999 | 9 | 10 | 90% |
| OT2000 | 13 | 17 | 76% |
| OT2001 | 14 | 18 | 78% |
| OT2002 | 18 | 23 | 78% |
| OT2003 | 19 | 25 | 76% |
| OT2004 | 16 | 19 | 84% |
| OT2005 | 15 | 18 | 83% |
| OT2006 | 19 | 21 | 90% |
| OT2007 | 8 | 10 | 80% |
| OT2008 | 15 | 16 | 94% |
| OT2009 | 11 | 15 | 73% |
| OT2010 | 19 | 24 | 79% |
| OT2011 | 18 | 24 | 75% |
| OT2012 | 12 | 14 | 86% |
| OT2013 | 10 | 11 | 91% |
| OT2014 | 10 | 15 | 67% |
| OT2015 | 8 | 10 | 80% |
| Total | 314 | 383 | 82% |
| 6th Circuit | rev/vacate | total | rev rate |
| OT1994 | 4 | 7 | 57% |
| OT1995 | 2 | 4 | 50% |
| OT1996 | 2 | 3 | 67% |
| OT1997 | 3 | 3 | 100% |
| OT1998 | 2 | 4 | 50% |
| OT1999 | 3 | 4 | 75% |
| OT2000 | 5 | 7 | 71% |
| OT2001 | 9 | 10 | 90% |
| OT2002 | 5 | 7 | 71% |
| OT2003 | 6 | 8 | 75% |
| OT2004 | 7 | 11 | 64% |
| OT2005 | 6 | 8 | 75% |
| OT2006 | 4 | 7 | 57% |
| OT2007 | 2 | 3 | 67% |
| OT2008 | 5 | 5 | 100% |
| OT2009 | 7 | 7 | 100% |
| OT2010 | 5 | 6 | 83% |
| OT2011 | 5 | 5 | 100% |
| OT2012 | 2 | 2 | 100% |
| OT2013 | 9 | 11 | 82% |
| OT2014 | 4 | 5 | 80% |
| OT2015 | 3 | 4 | 75% |
| Total | 100 | 131 | 76% |
Additionally, and inexplicably, the Politifact analysis includes this absurd claim:
We also found that the 9th Circuit never had the highest reversal rate in any individual term between 2004-15. (That’s the farthest back we could go.)
Just as it would be wrong to look at the total number of reversals--the Ninth Circuit is the largest circuit, and we would expect it to have the most raw reversals (and the most raw affirmed opinions) over a period of years, such as 314 reversals in a little over 20 years, which dwarfs all others--it would be just foolish to look at a single year's data for which circuit had the highest reversal rate. While it sounds impressive that the Ninth Circuit was "never" the "highest" in a single year for 12 years, Politifact's own reporting in this same piece explains why a single year's data is a silly metric:
In 2014, for instance, the 2nd Circuit had a reversal rate of 100 percent, which sounds pretty bad until you find out that the Supreme Court only heard one case from the 2nd Circuit that entire season.
There are plenty of years where a single circuit's record is 0-1, or 0-2, giving it a 100% reversal rate; meanwhile, the Ninth Circuit, with an appeals load of at least 10, and often more than 15 cases, it almost assuredly guaranteed at least one decision affirming what the court did.
Politifact's fact-check, then, is false.
It's worth emphasizing that what bearing this particular claim has on the merits of a decision to split the Ninth Circuit into smaller courts is a different matter. There are many good reasons for dividing the Ninth Circuit up, which I discussed in my AirTalk interview, and which Mr. Flake's offices will surely raise to his constituents.
On the precise point raised by Mr. Hannity, however, he is, at least in some measure, quite right--the Ninth Circuit is, in recent history, the most reversed federal court of appeals.
Please notify me of any errors in the data!