I've visualized a lot about the decline in bar pass scores and bar passage rates in the last few years, including a post on the February 2017 decline here. For some reason, this post in particular drew criticism as being particularly deceptive. It caused me to think a little more about how to best visualize--and explain--what the decline in multistate bar exam ("MBE") scores might mean. (I'll channel my inner Tufte and see what I can do....)

In the February 2017 chart, I didn't start the Y-axis at zero. And why should I? No one scores a zero. The very lowest scores are something in the 50s to 90s. And the score is on a 200-point scale, but no one gets a 200. So I suppose I could visualize it on the low to high ends--say, 90 to 190.

When you put it that way, it looks completely unremarkable. MBE scores have dipped a bit, but they've hardly moved at all. And it looks like my last post was simply clickbait. (It's worth noting I generate no revenue from this site!)

But that surely can't be right, either. After all, bar passage rates have been declining fairly sharply in the last few years even if this mean score has only moved relatively nominally. (For extensive discussion, see the "Bar exam" category on this blog.)

That's because what really matters is the passing score or the "cut score" in each jurisdiction.

Suppose the cut score in a jurisdiction is 100. A decline from a mean score of 135 to 134 should have essentially no effect if the results are distributed among a typical bell curve (and they usually are). That's because virtually everyone would still pass even if scores dropped a bit. In contrast, if the cut score were 180, a decline from a mean score of 135 to 134 should also have essentially no effect--virtually everyone would still fail.

But the reason for the perilous drop in bar pass rates is because this is exactly the spot where the mean scores have begun to hit the cut scores in many jurisdictions. Here's a visualization of what looks like, with a couple of changes--a larger y-axis, historical data for the February bar back to 1976, and gridlines identifying the cut scores in several jurisdictions. (It's worth noting that this is the national MBE mean, not individualized state means; July scores are somewhat higher; and it is a mean, not a median.)

You can see that the drop in the means plunges scores past what have been cut scores in many jurisdictions.

One more way of explaining why a drop at this point of the bell curve is particularly significant. The NCBE has not yet released the distributions of scores, but the bell curve linked above should be instructive, and the change from 2011 to 2016 is useful to consider.

In February 2011, just 39.6% of all test-takers had a score of 135.4 or lower. 13.7% had a score in the range of 135.5 to 140.4, and 46.6% had a score of 140.5 or higher. (Consider the chart above for references as to what those scores might mean.) In February 2016, however, 51.1% of all test-takers had a score of 135.4 or lower, a 11.5-point jump. 13.7% had a score in the range of 135.5 to 140.4, and just 35.1% had a score of 140.5 or higher.

That's because this particular drop in the score is at a very perilous spot on the curve. Bar takers are performing just a little worse in a relative sense. But when the distribution of performance is put up against the cut score, this is precisely the point that would have the most dramatic national impact.

I hope these explanations help illustrate what's happening on the bar exam front--and, of course, I welcome corrections or feedback to improve these visualizations in the future!