Puerto Rico lowers its bar exam cut score in response to threats that its law schools may lose accreditation
Back in 2019, I assessed the potential effect of the American Bar Association’s revised Standard 316, which requires an “ultimate” bar passage rate of 75% within two years for a graduating class. There, I noted:
Let’s start with the schools likely in the most dire shape: 7 of them. While the proposal undoubtedly may impact far more, I decided to look at schools that failed to meet the standard in both 2015 and 2016; and I pulled out schools that were already closing, schools in Puerto Rico (we could see Puerto Rico move from 3 schools to 1 school, or perhaps 0 schools, in short order), and schools that appeared on a list due to data reporting errors.
…
Will state bars lower their cut scores in response?
It’s possible. Several state bars (like South Dakota as mentioned above) have lowered their cut scores in recent years when bar passage rates dropped. If states like California and Florida look at the risk of losing accredited law schools under the new proposal, they may lower their cut scores, as I suggested back in 2016. If the state bar views it as important to protect their in-state law schools, they may choose the tradeoff of lowering cut scores (or they may add it to their calculus about what the score should be).
The ABA Journal recently reported the plight of two of Puerto Rico’s law schools that have failed to meet that standard in several years. Indeed, for Pontifical, their pass rates have worsened fairly dramatically in recent years: 71% for 2017, 52% for 2018, and 46% for 2019.
That article tipped me off to changes in Puerto Rico’s bar exam cut score. Puerto Rico does not use the UBE or a standardized bar exam score, so their passing score of “596 out of 1000 points” doesn’t offer a whole lot of information. But the Supreme Court of Puerto Rico did choose to lower the cut score to 569.
A 2021 report offers some reasons to be skeptical of this change, after studying predictors and exam performance:
For both set of analyses completed, the results did support the hypothesis that the applicants in the more recent years were not as well prepared than the applicants in previous years. Average P-values for a common set of items declined over time, and when comparing specific test administration pairs, the pattern consistently saw applicants from earlier test administrations performing better.
. . .
The hypothesis that the steady decline in overall pass rate on the Puerto Rico Bar Examination is a result of applicants being less prepared for the examination is supported by the decline in performance on the 14 anchor items administered on every test administration.
The Supreme Court of Puerto Rico expressly considered the effect of the new ABA Standard 316 on Puerto Rico’s law schools as an impetus for change.
Ante la necesidad de determinar si, además de las medidas ya concretadas por el Poder Judicial para atender los efectos de la aplicación del Estándar de Acreditación 316 de la ABA en nuestra jurisdicción, era necesario disminuir o modificar la nota de pase de los exámenes de admisión al ejercicio de la profesión legal, en el 2020 la Oficina de Administración de los Tribunales (OAT) comisionó a la compañía ACS Ventures un análisis sobre este particular.
A standard-setting study for the cut score had two rounds of standard-setting. One recommended a score of 584 (with a range of 574 to 594), and the other 575 (with a range of 569 to 581). The Supreme Court took the lowest of these ranges, 569. That said, the pass rate would still be at 46.4% even with that score, better than the rate of closer to 33% under the present standard:
We recommend that the program consider a final passing score for the Bar Examination somewhere in the range of the recommended passing score (575) and a score that is two standard errors of the mean below this score (569). The rationale for this recommendation is that the reference point for the panelists during the study was the Minimally Competent Candidate and panelists made judgments to predict how these candidates would perform on the multiple-choice questions and essay questions for the examination. This means that the distribution of reference candidates was all intended to be minimally competent. In creating that distribution, the lower bound would likely best represent the threshold of minimum competency suggested by the panelists. Setting the passing score at 569 would mean that approximately 46.4% of candidates would pass the examination while setting the passing score at 575 would mean that approximately 41.5% of candidates would pass. This range is consistent with the recommendations of the panelists as characterizing the performance of the minimally competent candidate.
The ABA has given Puerto Rican law schools an extra three years to try to comply. The lower cut score will make it easier to do so, although it remains unclear that even with this cut score all schools will be able to meet the standard.
But it also shows the rarity of the ABA of actually enforcing this standard, except for continuing to give schools more time to demonstrate compliance. We’ll see what happens in the next three years.