Despite protests from the National Council of Bar Examiners to the contrary (PDF), it increasingly appears that the NCBE had some role in the decline of Multistate Bar Exam scores and, accordingly, the decline in bar passage rates around the country.

Causation is hard to establish from my end--I only see the data out there and can make guesses. But with California's bar results released, we now have 34 of 51 jurisdictions (excluding American territories but including the District of Columbia) that have released their overall bar pass rates. Comparing them to 2013, we see that 20 of them experienced at least a 5-point drop in scores. Louisiana is the only state that does not use the MBE, and it's quite the outlier this time around.

A single state, of course, cannot establish that the MBE is to blame. But it's a data point of note.

Some have blamed ExamSoft. On that, I remain skeptical. First, it would assume that the exam-takers on Tuesday were "stressed out" and sleepless as a result of the upload fiasco, which caused them to perform poorly on Wednesday's MBE. Perhaps I'm too callous to think it's very much of an excuse--it might be for some, but I would have doubts that it would have a dramatic effect on so many. One problem is that reporting of the actual problems of ExamSoft has been spotty--there have been no journalists who did the legwork of investigating which states had the problems, or to what extent.

But we have a couple of data points we can now use. First, jurisdictions that do not use ExamSoft, but use some other exam software like Exam4 or require handwriting. Second, the jurisdictions whose essay components occurred on Thursday, not Tuesday--meaning there was no ExamSoft debacle the night before the MBE.

Again, there does not appear to be a significant trend in any of these jurisdictions--they appear to be randomly distributed among the varying scores. While it might be a cause for some, I am not convinced it's a meaningful cause.

Finally, the NCBE has alleged that the class of 2014 was "less able." That's true, as I've pointed out, but only to a point--the decline in scores should not have been as sharp as it was. One small way of trying to compare this point is to examine repeater test-taker data.

A problem with measuring repeater data right now is that few jurisdictions have disclosed it. Further, most bar exams are quite easy, and repeaters are few. Finally, repeaters should fail the bar at extremely high rates, as it would prove the validity of the test--and which skews the results figures. But it might be useful to extract the data and compare first-time from repeater pass rates this cycle, at least in jurisdictions that had a significant number of repeaters. If the Class of 2014 was "less able," then we might expect the first-time takers' pass rates to decline at a higher rate than the repeat takers' pass rates.

Places like California saw identical declines. Others, like Texas and Pennsylvania, actually saw a slightly increased rate of failure from repeaters than from first-time takers. Ohio is on the other side, with a decline pass rate for first-time takers but a decent increase in the rate for repeat takers.

In short, I haven't been able to find an explanation that would identify the cause of the sharp decline in rates. Some, I think, is explained by a slightly lower-quality incoming class--one I've noted will lead to still sharper declines in the years ahead.

But after looking at all this information, I'm increasingly convinced that some decision in the NCBE's scoring of the MBE had some role in the decline of the scores, and of the pass rates around the country. That's speculation at this point--but it's a point, I think, worth investigating further, assuming additional data would be made available.

Previous posts on this subject

A more difficult bar exam, or a sign of declining student quality? (October 2, 2014)

Bar exam scores dip to their lowest level in 10 years (October 14, 2014)

Bar exam posts single-largest drop in scores in history (October 27, 2014)

Did ExamSoft cause the bar passage rate decline? (October 27, 2014)

National Conference of Bar Examiners: Class of 2014 "was less able" than Class of 2013 (October 28, 2014)

Class of 2014 LSAT scores did not portend sharp drop in MBE scores (November 11, 2014)

The bleak short-term future for law school bar passage rates (November 17, 2014)