Eight questions I have about the summer/fall 2020 bar exam
In the midst of the many changes to the bar exam this summer or fall, here are eight things I’m watching with some interest and some open questions.
1. Online security and reliability. Some states have announced the move to an online test. I’m pretty skeptical that such plans can be built in such a short time (so is Professor Josh Blackman, and the authors of a white paper on the bar exam describe an online exam as “very risky”). The botched AP test administration this spring is just the latest confirmation of my skepticism. I hope states will be able to build something secure and reliable, but we’ll have to wait and see.
2. A new format for test-takers to learn. I think it’s fair to say that bar exam test preparation providers have crafted study programs designed to assist learning in a particular format: in virtually all states, essays and multiple choice covering particular topics. Now, some state bar exams are literally creating a new exam they’ve never administered before out of thin air, whole cloth, or whatever metaphor comes to mind—short essay, short answer, no multiple choice, etc. Indiana, for instance, now will create “short answer questions” on MBE topics. It remains to be seen how students will study for this new test, what materials will be provided to them ahead of time to understand for the format of the test, and whether test-prep companies can accommodate in a timely and effective fashion.
3. Scaling, equating, and pass rates. In most states that administer the Multistate Bar Exam, licensing authorities scale essay exam answers to the equated MBE results. To greatly simplify, it ensures that the test is measuring the same things year after year. But some states are creating their own tests this year. It will be impossible to do that, and I haven’t seen any good statements about what states plan to do except generic claims that they’ll ensure comparable scoring. Michigan, for instance, announced, “Experts will work with the BLE to determine an appropriate passing score based on results from previous July exams.” Nevada “will take all reasonable measures to address the reliability of an essay only exam.” It might be something as crude as making sure the pass rate this year looks similar to past years. Or it might result in a dramatic fluctuation in pass rates. It’s hard to say without seeing more.
4. Out-of-state bias. New York and Massachusetts have announced rules prioritizing in-state ABA law school graduates over others. (Utah accommodated a number of out-of-state law schools, but not all, in its “diploma privilege-plus” model.) It’s not clear whether litigation will follow, whether alternative accommodations will be provided, or whether other states will follow suit.
5. The limitations of the UBE and questions about reciprocity. As UBE states move away from administering a UBE exam this term, students who have increasingly relied on the UBE as a way of improving portability of their license or ensuring licensure in multiple jurisdictions will face a setback. It might be that some states will soften reciprocity requirements (which, I think, seems unlikely)—but, more likely, a cohort of graduates will simply be out of luck.
6. What will “supervised practice” really look like. Utah has the most at stake with supervised practiced as it transitions to a type of “diploma privilege-plus” model, where recent graduates at a number of law schools. But many other jurisdictions, like Ohio and New York, are also allowing graduates to engage in supervised practices. Experience in Canada and elsewhere suggests that such proposals tend to disadvantage first-generation attorneys (i.e., those without a family support structure to help supervise practice) and those with limited socioeconomic means. It remains to be seen how this plays out in practice. It’s also not clear how many graduates take advantage of them, or simply extend their bar exam study periods longer as the bar exam is pushed later.
7. Discipline rates. We know that attorneys tend to be disciplined later in their careers. We also have some evidence that lower bar exam scores correlate with higher ultimate discipline rates. To the extent that alternative exams or admissions practices are adopted, we wouldn’t expect to see much for a decade at least in terms of attorney discipline rates. So we can revisit this in, maybe, 2035 (!) to see if the standards this year changed anything.
8. USNWR fallout. While some have worried about whether law schools will meet their ultimate 75% pass rate for graduates within two years, I think that’s less of a concern—failing to meet the 75% ultimate pass rate means schools must justify to the ABA why they’ve fallen below and what steps they’re taking, and, if in 2022 some schools fall below and can point to the circumstances of the summer/fall 2020 bar exam, I think it would be a small problem for schools to explain non-compliance.
A bigger problem, I think, is the USNWR rankings. USNWR includes as one of its twelve components how a school did in its pass rate compared to the overall pass rate for the modal jurisdiction of its graduates taking the bar exam. It remains unclear how Utah’s new proposal, how novel scoring systems, how pass-fail policies in the last semester, or how increased disruption that may particularly affect law school graduates in, say, New York City will affect pass rates.
To be fair, bar passage rates are a very small portion of the overall formula. But another and larger component is employment statistics. As bar exam grading gets pushed back several weeks (in California, the goal will be to complete it by December 31!), it remains to be seen if a softer employment market coupled with delayed bar exam grading leads to weaker job figures for some schools as of March 15, 2021, the 10-month employment figure date.