Blockchain and the bar exam

Over the last few years, the word “blockchain” has been sprinkled around everything as one of the hottest buzzwords in technology. I confess, I use the word tongue in cheek. I think most references to “blockchain” are hype, and many are duped by believing that this word makes the product to which it’s attached is somehow more valuable, more efficient, or more likely to succeed. It’s a Theranos or WeWork level of hype.

So you can imagine my skepticism when I saw the Massachusetts and California announcements that their bar exams would be administered “online” September 9 and 10.

I’m still skeptical.

The NCBE, in something of an understatement, said there are “significant issues,” including security, in providing an online exam.

But I want to put this in a bit of perspective. These bar exams are scheduled to take place in four and a half months. These state bar licensing authorities believe they can create a secure remotely administered test by then. Because, recall, no one does this now.

Let’s put aside the security issue for a moment and simply focus on reliability of software. Six years ago, ExamSoft had an issue during the July 2014 bar exam where thousands test-takers were unable to upload their answers for hours. Some (I think, wrongly) even blamed this debacle on a decline in bar passage rates that cycle. Exam software is not sufficiently reliable even in the best of times. Add to that the remote (and secure) delivery of materials that have previously been printed, and the collection of those materials after the exam.

In-room security is a huge problem, too. Bar exams are notorious for picayune requirements, like a small clear plastic bag containing limited personal effects, sign-in sheets to use the restroom during the exam, and so on. Remote proctoring software purports to watch the eye movement of exam test-takers during the exam, to scan the room before and after to make sure no one else is present, and other rather theatrical promises. Let’s face it—those probably work in much lower stakes tests.

Now, as a small pushback, perhaps cheating on the bar exam doesn’t yield much. The MBE is difficult to cheat on given short periods of time and its intensive fact-application component, which makes cheating difficult (unless, I suppose, someone else is literally taking the test for you). MPTs turn on a closed universe of facts, so, again, unless someone else is writing the exam for you, looking at an outline or something won’t help much. But the ability to outline dump rightly-stated black letter law is probably a huge temptation for the essay components, and probably the very easiest thing for cheating.

And, I think, those most inclined to cheat on the bar exam—and be advantaged by cheating—are probably the ones most at risk of failing and most likely to commit malpractice later in their careers. Maybe we’re not really worried because most who take the bar exam pass anyway, and these are, after all, extraordinary times. But when I consider the repeaters—those who’ve failed before—and wonder about the pressures (and incentives) to cheat, it gives one some pause.

Really, the ideas strike me as the kinds of things pitched to bar licensing authorities with some hype: “Oh, we totally can do this online!” Perhaps a string of buzzwords about security—AI, blockchain security, and so on—were persuasive. But to build something like this out in four months—not just build, but test, fix, and feel comfortable using—strikes me as unrealistic. Even a years-long preparation of the digital LSAT led to some small problems in the first widespread use (even if it was mostly seamless).

A proposal from Dean Jennifer Mnookin at UCLA and Dean Erwin Chemerinsky at Berkeley was far better. They proposed canceling the July 2020 test to reduce uncertainty about later scheduling and postponing, and allowing recent graduates “to practice law for a defined—and relatively limited—period, such as until the July 2022 bar exam releases its results.” Granted, this would only help some cohorts, and it would exclude, say, repeaters from the proposal, but in trying times there are going to be tradeoffs in all decisions.

Finally, I’m not a Luddite! I think if we can develop a secure remotely-administer bar exam—perhaps one that looks different than the one we have today—we should go for it. Remotely, of course, is the great challenge. Moving to a digital, or a year-round test that one can self-schedule in a secure location, seem more promising. But this is a years-long project, and one that probably must start with volunteers on a small scale before ramping up.

I doubt these bar licensing authorities will actually move forward with a remotely administer bar exam this September. These licensing authorities can change their minds later, of course. Time will tell.

UPDATE: Professor Josh Blackman has more here, with comparisons to the Iowa caucuses and healthcare.gov.

UPDATE: It should be noted that Massachusetts intends to administer an alternative examination in the event the Uniform Bar Exam cannot be administered. It would be interesting to see how much notice it gives prospective test-takers about its form and contents; whether that meets the other security and practical concerns I raised; and whether it would be as reliable an exam. Of course, we’ll see plenty of experimentation, as Utah is doing!