Class of 2014 LSAT scores did not portend sharp drop in MBE scores
UPDATE: Jerry Organ (University of St. Thomas) has posted an even more thorough and thoughtful analysis of the LSAT scores and projected bar passage rates at the Legal Whiteboard. He, too, finds the NCBE's conclusion difficult.
I've blogged (here and here and here and here and here) about the sharp drop in bar passage rates around the country from the July 2014 administration of the bar exam, largely due to the unprecedented drop in MBE scores. A recent Wall Street Journal blog post about the reaction of the dean of Brooklyn Law School shows the sides in the fight. Did the NCBE screw up its exam, yielding a sharp drop in scores? Or did law schools admit a disproportionately unqualified class?
Here's an attempt to measure the quality of the class and correlate it with MBE scores. (Maybe it's just awful math.)
The LSAT is fairly highly correlated with MBE scores. Consider this NCBE report (PDF). I extrapolated those figures for the LSAT and the average MBE scores. I then weighted them against the number of matriculants in law school: LSAC reports the number of matriculants with scores of 175+, 170-174, and so on. I took a rough estimate of the expected MBE score for each range; I then averaged it out for the entire class.
When I first charged it, the projected MBE scores were much higher than the actual MBE scores that arose three years later. (I used the 2009-2010 LSAT matriculant data, for instance, and mapped it on the MBE results three years later, in 2013.) I attributed this to several possibilities, the most significant of which is that repeaters probably significantly drag down the MBE score. But subtracting five points from the projected MBE score lead to an almost perfect match with the actual MBE score, with one exception.*
Note that the LSAT score reporting changed beginning in the 2009-2010 cycle (i.e., the Class of 2013): schools could report the highest LSAT scores, rather than the average LSAT scores, of matriculants. That meant that the LSAT scores were probably overstated in the last two graduating classes.
But in the charge, we see a fairly significant correlation between my extremely rough approximation of a projected MBE score based on the LSAT scores of the matriculating classes, and the actual MBE scores, with one exception: this cycle.
My math is rough--and maybe it's just bad. But as this comports with every other analysis I've done, and as I've not been able to find any other factors that would contribute to an across-the-board decline in scores, I'm increasingly convinced that a problem occurred on the NCBE's end--and not that the Class of 2014 was somehow disproportionately and dramatically worse than other classes.
That said, we should expect to see declining MBE scores (and bar passage rates) of some kind in the next few years, as academic quality of entering classes continues to decline; and, we should expect bar passage-required employment outcomes to see some (likely negative) effect due to a sharp drop-off in bar passage rates.
*I should add that I could have simply plotted the projected results so that you could observe the similarity (or differences) in the rise and fall; or, in the alternative, I could have plotted them on two different Y axes. Subtracting five points, however, seemed like the easiest way to make the visualization more obvious.