Early signs point to dropping July 2021 bar exam results

Back in 2014, I noted early warning signs of a precipitous drop in bar exam pass rates around the country. The MBE scores were much lower. Part of that may have been attributable to an ExamSoft error, which I ultimately concluded was unlikely, and, at first tentatively blaming the NCBE’s administration of the exam, later backed off that claim as well, attributing the bulk of the decline to a decline in overall student quality as a major part of the answer.

What’s old is new again. July 2021 saw a major ExamSoft error, and now we’re beginning to see a downtick in bar passage rates. From five jurisdictions, and reporting overall results (not first-time results) so far:

Iowa, -12 points: July 2020, 83%; July 2021, 71%

Nebraska, -17 points: July 2020, 89%; July 2021, 72% [Nebraska offered a much smaller second exam in 2020, too]

New Mexico, -18 points: September 2020, 89%; July 2021, 71%

North Dakota, -11 points: July 2020, 76%; July 2021, 65%

South Dakota, +3 points: July 2020, 70%; July 2021, 73%

West Virginia, -19 points: July 2020, 77%; July 2021, 58%

Wyoming, -13 points: July 2020, 85%; July 2021, 72%

It becomes increasingly difficult to compare year-over-year exam results as the format and timing change, and as new variables enter the mix. Some possible variable to consider….

ExamSoft. Here we open with problems attributable to a remotely-administered exam as administered by ExamSoft. Not all of these jurisdictions were remote (e.g., Nebraska was in person). But certainly same-day exam problems are a significant issue. Stress and sleeplessness from the first day could trickle over to a second day, albeit with a more indirect effect. North Carolina, for instance, has already announced it would lower its cut score by 2 points specifically because of software issues. This is not unprecedented—California, for instance, made modest adjustments to the cut score for some test-takers after an earthquake hit testing sites in 2008.

Online learning. Many law students taking the bar exam had nearly half of their law school educational experience shifted largely, if not exclusively, online. It’s not clear what pedagogical effect that had on students in the long term. It would not surprise me that graduating 3Ls “lost” some amount of learning in the pandemic, which later translated to lower bar exam performance.

Pandemic fatigue. Relatedly, one could easily multiple the concerns from the July 2020 administration of the bar exam to prolonged difficulties arising from the coronavirus pandemic. Early summer 2021 may have been some of the better moments for most test-takers, so it’s difficult to know how it may have affected test-takers later in the summer.

Credentials/academic dismissal. These tend to be more individualized assessments at institutions. The incoming class of 2017 was not materially better than the incoming class of 2018—they were, from all I can see, largely comparable. Whether individualized decisions at schools, including academic dismissal rates, affected scores remains to be seen.

In short, I’m cautious about assigning any, all, or none of these responsibility for the decline in scores so far. We may see scores at other states increase, negating the premise of this post. But I’d say these are more likely to be canaries in the coal mine. I’d anticipate lower scores in many states, and a lot of questions about the cause. I wouldn’t rule out these possibilities so far, and I hope to see clear-eyed analysis of the possible sources for declining rates.

This post has been updated as results come in or are corrected.