Increasingly appears NCBE may have had role in declining MBE scores and bar pass rates

Despite protests from the National Council of Bar Examiners to the contrary (PDF), it increasingly appears that the NCBE had some role in the decline of Multistate Bar Exam scores and, accordingly, the decline in bar passage rates around the country.

Causation is hard to establish from my end--I only see the data out there and can make guesses. But with California's bar results released, we now have 34 of 51 jurisdictions (excluding American territories but including the District of Columbia) that have released their overall bar pass rates. Comparing them to 2013, we see that 20 of them experienced at least a 5-point drop in scores. Louisiana is the only state that does not use the MBE, and it's quite the outlier this time around.

A single state, of course, cannot establish that the MBE is to blame. But it's a data point of note.

Some have blamed ExamSoft. On that, I remain skeptical. First, it would assume that the exam-takers on Tuesday were "stressed out" and sleepless as a result of the upload fiasco, which caused them to perform poorly on Wednesday's MBE. Perhaps I'm too callous to think it's very much of an excuse--it might be for some, but I would have doubts that it would have a dramatic effect on so many. One problem is that reporting of the actual problems of ExamSoft has been spotty--there have been no journalists who did the legwork of investigating which states had the problems, or to what extent.

But we have a couple of data points we can now use. First, jurisdictions that do not use ExamSoft, but use some other exam software like Exam4 or require handwriting. Second, the jurisdictions whose essay components occurred on Thursday, not Tuesday--meaning there was no ExamSoft debacle the night before the MBE.

Again, there does not appear to be a significant trend in any of these jurisdictions--they appear to be randomly distributed among the varying scores. While it might be a cause for some, I am not convinced it's a meaningful cause.

Finally, the NCBE has alleged that the class of 2014 was "less able." That's true, as I've pointed out, but only to a point--the decline in scores should not have been as sharp as it was. One small way of trying to compare this point is to examine repeater test-taker data.

A problem with measuring repeater data right now is that few jurisdictions have disclosed it. Further, most bar exams are quite easy, and repeaters are few. Finally, repeaters should fail the bar at extremely high rates, as it would prove the validity of the test--and which skews the results figures. But it might be useful to extract the data and compare first-time from repeater pass rates this cycle, at least in jurisdictions that had a significant number of repeaters. If the Class of 2014 was "less able," then we might expect the first-time takers' pass rates to decline at a higher rate than the repeat takers' pass rates.

Places like California saw identical declines. Others, like Texas and Pennsylvania, actually saw a slightly increased rate of failure from repeaters than from first-time takers. Ohio is on the other side, with a decline pass rate for first-time takers but a decent increase in the rate for repeat takers.

In short, I haven't been able to find an explanation that would identify the cause of the sharp decline in rates. Some, I think, is explained by a slightly lower-quality incoming class--one I've noted will lead to still sharper declines in the years ahead.

But after looking at all this information, I'm increasingly convinced that some decision in the NCBE's scoring of the MBE had some role in the decline of the scores, and of the pass rates around the country. That's speculation at this point--but it's a point, I think, worth investigating further, assuming additional data would be made available.

Previous posts on this subject

A more difficult bar exam, or a sign of declining student quality? (October 2, 2014)

Bar exam scores dip to their lowest level in 10 years (October 14, 2014)

Bar exam posts single-largest drop in scores in history (October 27, 2014)

Did ExamSoft cause the bar passage rate decline? (October 27, 2014)

National Conference of Bar Examiners: Class of 2014 "was less able" than Class of 2013 (October 28, 2014)

Class of 2014 LSAT scores did not portend sharp drop in MBE scores (November 11, 2014)

The bleak short-term future for law school bar passage rates (November 17, 2014)

The bleak short-term future for law school bar passage rates

This is the last (for now!) about the bar exam. And it's not about what caused the MBE and bar passage rate declines--it's what it means for law schools going forward. The news is grim.

There's no question there was a decline in the law school applicant profile from the Class of 2013 to the Class of 2014. The dispute that Jerry Organ and I (and others) have had is whether the decline in bar passage rates should have been as stark. But going forward, the Class of 2015, and 2016, and likely 2017, and probably 2018, will each be incrementally worse profiles still.

And it's not simply at the median LSAT and GPA. It's at the below-median profiles, particular LSAT, that should concern schools.

Those with LSAT scores below 150, and even 155, are at a substantially higher risk of failing the bar in most jurisdictions. For the Class of 2016, about 2/3 of schools have a 25th percentile LSAT of 155 or lower--that is, 25% of their incoming classes have an LSAT at or below 155. And over 80 schools have a 25th percentile LSAT at 150 or lower.

Furthermore, about half of schools have a 50th percentile LSAT of 155 or lower, and a full 30 schools have a 50th percentile of 150 or lower.

The increasing willingness of schools to accept these low-LSAT performers is a function of a combination of decisions made years ago. U.S. News & World Report evaluates only LSAT medians. This decision distorts evaluation of law student quality. To ensure that their medians remained as strong as possible, schools increasingly admitted more "imbalanced" students--students with a median or better LSAT and substantially below-median GPA, or a median or better GPA and a substantially below-median LSAT. That meant the 25th percentile LSAT began to sag at more schools--the bottom of the class became worse at a higher rate than the middle of the class. (There are other, less-measurable decisions at the moment, such as factoring the highest LSAT score instead of the average LSAT score of applicants, which probably distorts student quality; possible decisions to academically dismiss fewer students; educational programming decisions that may channel more students toward the kinds of courses that may not sharpen legal analysis for the bar exam, to the extent it affects bar passage; transfer-related decisions at schools; and much more.)

As bar passage rates decline--perhaps sharply--we should see still-falling rates, particularly from institutions that made the admissions decision years ago to prop up the median but sacrifice the quality of the bottom of the class.

For schools that made this decision years ago, the results will become increasingly sharp in the years ahead. If a school did not sufficiently reduce its class size, or worried about LSAT medians, it favored short-term interests; those short-term interests are becoming long-term as those classes graduate; are likely face the more significant debt (as the below-median students are less likely to have obtained merit-based aid); pass the bar at lower rates; in that cohort, are likely find employment at lower rates (if they are unable to pass the bar); and trickle back out to an already-reluctant applicant pool.

I've said before that I'm not a "doomsday" predictor. But these bar results portend a significantly worsening portrait for law school bar passage rates in the years ahead, if schools made short-term decisions years ago and are now facing the long-term results. For the long-term schools with visionary deans and faculties anticipating the long-term future of the institution, the results may not be quite so grim. (But we shall see how many of those there are.)

Why Kentucky may bar Rand Paul from running for president and the Senate simultaneously

This is the fourth in a series of posts about my forthcoming article, Scrutinizing Federal Electoral Qualifications, 90 Indiana Law Journal (forthcoming), available on SSRN. Comments, critiques, and feedback are welcome.

Senator Rand Paul may want to run for President of the United States in 2016. But he is also very likely interested in running for the United States Senate in 2016, too--just in case the president thing doesn't work out for him. But his home state of Kentucky has a law that prohibits him from running for both offices at the same time--a law of some difficulty for the potential Paul campaign. Is it constitutional?

Drawing upon my earlier analyses, the answer is yes--because it is a condition on running for president, and not a condition on running for Senate.

Kentucky cannot add to the qualifications to candidates for office in the Senate. That much is obvious from the Supreme Court's decision in U.S. Term Limits v. Thornton. It could not condition running for Senate on refusing to run for president.

One might argue that this is merely a ballot access rule. After all, Mr. Paul could not run for the Senate unless he filed paperwork, or obtained a certain number of signatures to get on the ballot or won a qualifying primary election, or something like that. The line occasionally blurs. "Sore loser" laws--laws that prohibit candidates who lose a primary election from running for federal office in the general election--have been held as permissible regulations, not additional qualifications. "Resign to run" statutes--laws that require a state official to resign office before running for federal office--have been upheld as restrictions on conditions on state offices, not as additional qualifications for federal office. Perhaps it's the case that the state has a more permissive regulatory interest in making sure someone is not running for two offices when he can only fill one. But that's a nuanced claim, and one that, I think, would take time before I felt comfortable settling on one side.

But the other side is different--a condition on running for President might be that one cannot also be on the ballot for the Senate.

Under the Constitution, the state legislature may direct the "manner" of selecting presidential electors. It may select electors itself, or it may permit the people to vote. But, perhaps, it may cabin the discretion of the people in how they vote--perhaps by excluding certain candidates it does not wish the people to vote for. Akhil Amar, for instance, in his book America's Unwritten Constitution, posits that prior to the Nineteenth Amendment, which guaranteed women the right to vote, states could have excluded women from running for the office of president--but that the "unwritten" Nineteenth Amendment now would prohibit.

It might be, then, that U.S. Term Limits does not extend to presidential candidates. Indeed, the state is only selecting the manner of appointing presidential electors; how those presidential electors act is another level of review. It might simply prohibit a candidate's name from being listed on the ballot, but, in the case of presidential elections, the final decision is left with the presidential electors. And Kentucky has no law (that I found) that would require electors to vote for the named candidates. In a sense, then, the candidates names on the ballot are mere proxies for a slate of presidential electors and, perhaps, the state legislatures possess the latitude to decide how to condition the appearance of candidates of the ballot.

The best possible defenses of the law, then, are that it is a reasonable ballot access provision, not a "qualification"; and, further, even in the event it is something closer to a "qualification," it is within the discretion of the legislature to decide the manner of the selection of presidential electors, and that may include cabining the discretion of the voters with a requirement that a candidate for President cannot be listed for another office.

But, it's a heavily gray area without obvious answers. We'll see if the Kentucky legislature arises at a novel solution to avoid a judicial intervention into this political thicket.

A novel (and weak) attack on a Tennessee ballot initiative

The people of Tennessee approved Amendment 1 this election cycle, which removed the matter of abortion from the Tennessee Constitution, effectively leaving the only legal protection for abortion in the state in the jurisprudence of the United States Constitution. (It's called an "amendment" as something proposed by the Tennessee legislature but ratified by the people--slightly different than a typical initiative process, which is often voter-initiative by petition.)

Shortly after the Amendment was passed, a federal lawsuit challenged the Amendment. The claim is somewhat complex. A provision of the Tennessee Constitution provides: "And if the people shall approve and ratify such amendment or amendments by a majority of all the citizens of the state voting for governor, voting in their favor, such amendment or amendments shall become a part of this Constitution."

That's usually construed to mean that the denominator for a ballot initiative to pass is the total ballots cast for governor, not for the initiative; and that the "yes" votes on the initiative must be a majority as compared to the total votes cast for governor. That's because down-ballot races often yield fewer votes cast, and it's a way of ensuring that the initiative is enacted only with a slightly stronger majority.

The total "yes" votes cast, at the time I write this, is 728,751 votes. But while 1,385,178 votes were cast for the Amendment; just 1,352,608 votes were cast for the office of governor. That means the Amendment passed with nearly 54% of the vote.

But plaintiffs in this case are claiming that that isn't the right way to read the statute. Instead, it should be read to require that only those who voted for governor should have their votes on Amendment 1 counted. That, they argue, is what the literal reading of the Tennessee Constitution demands.

Basic math illustrates the silliness of this claim. There are, at this point, at least 32,570 fewer votes cast for governor. The current margin of victory for Amendment 1 is 104,894 votes. Even if every single person who abstained from voting for governor voted "yes" on Amendment 1, it would not come close to changing the result. (Indeed, the complaint only demands the right to an accurate count, claiming a type of dilution. There are nuanced situations in which the total might be different based on some more complicated contingencies, but that's extremely unlikely.)

But I want to set aside the sheer outcome-ineffectiveness of this claim. And, further, I want to set aside the post hoc nature of this claim--it might, for instance, have altered behavior of the voters if they had know ex ante how to vote, rather than invalidating the votes of thousands of voters on Amendment 1 after the fact.

Instead, I want to look at the claim itself--that, as a condition for voting on Amendment 1, one must vote for governor, too. That almost assuredly fails.

In an analogous context, California and Colorado had conditioned the right to vote for a replacement in a recall election on voting "yes" or "no" on the recall. (I've blogged about these issues here and here.) A California federal court and the Colorado Supreme Court struck them down.

The basic approach has been to follow Burdick v. Takushi. A court examines the nature of the voting right being asserted, the burden on that, and, in the case of a "severe" burden, the regulation must be subject to strict scrutiny.

Plaintiffs in this case claim that their votes are diluted by having non-gubernatorial voters' votes counted in the Amendment 1 tally. But that's really asserting a "right" under the text of the Tennessee Constitution, and not a burden per se on the fundamental right to vote. The real "right," I think, is actually the reverse: the "right" of voters to vote for Amendment 1 without being compelled to vote on the governor's race. That's how the recall cases have articulated the right--without a concern about the "dilution" to voters who voted on both under state law.

The best justification for the two-step recall system is articulated by the dissent in Colorado's case, explaining that in the old days, the recall consisted of a list of all the candidates, including the sitting politician, and the winner would simply win the recall (or, the sitting politician would win and never be recalled). States broke this into two steps--yes or no, and then a vote on the recall, which empowered voters who support the incumbent but would still want a "say" in the recall down ballot.

Regardless, that narrow possible justification for recalls does not, I think, tie in any way to the proposed novel interpretation of Tennessee's law. It strikes me (at least) as fairly onerous to compel voters to vote for one office in order to have the privilege of having their votes counted for initiatives--at least, not without some even plausible justification along the lines articulated in the recall dissent (which is, I think, certainly a matter for meaningful debate). There doesn't, I think, appear to be such a justification articulated anywhere in the complaint.

My initial view, then, is that this attack, somewhat novel and, based on this analysis, weak, would not last long in federal court.

A version of this discussion was originally emailed to the Election Law Listserv.

Quick thoughts from oral argument in today's Alabama redistricting case

Today, the Supreme Court heard oral argument in Alabama Legislative Black Caucus v. Alabama and Alabama Democratic Conference v. Alabama. (The SCOTUSBlog page is here.)

There are many moving parts in this litigation. (Rick Hasen has a nice summary here.) Redistricting involves a number of decisions that legislators must make, and they are constrained by a variety of provisions of the Constitution, federal law, and state law. Very, very briefly: there must be an equal--or, at least, roughly equal--number of individuals residing in each district. The districts cannot be drawn with the primary intent to draw them among racial lines. They must, however, consider the impact that drawing the lines will have on racial groups if there is polarized voting. The drawing of lines could not, when Section 5 was in effect, be retrogressive on minority groups. And while partisanship is a permissible basis for redistricting, there may be circumstances in which a judiciary would find invalid redistricting for being excessively partisan.

Some of these are slightly vague standards; other are in tension with each other. It's not like these are new--this blog is dedicated, in part, to Elbridge Gerry, and "Gerry's salamander" (i.e., gerrymander) is the graphic at the top of the page.

I had the privilege of attending my first Supreme Court oral argument this morning. Here's what I observed. (Quotations are not direct; they are based on my scrawled notes!)

Bottom line: there seemed to be more sympathy for Alabama's position over that of the petitioners, but the Court was quite scattered at oral argument, and the opinion could be quite narrow.

Chief Justice Roberts opened asking petitioners about the "sweet spot," whether a state could really be expected to navigate having too many minorities in a legislative district and having too few. Justice Scalia chimed in to assert that the "only way to be sure" that Alabama was complying with Section 5 of the Voting Rights Act was to ensure that each legislative district had the same number of minorities under the old and new systems.

Justice Kennedy then tried to wrap his mind around redistricting for partisan gerrymandering reasons, but using race as a proxy for party. He found it hard to conclude that one party could do it, but the other could not because the effect would be a racial gerrymandering claim--a "one-way ratchet" concern. (Justice Breyer would later challenge this conclusion and assert it was actually a two-way ratchet."

In one (gotcha?) moment, Justice Alito asked, "You're just as interested in quotas as Alabama, only lower quotas." To which counsel replied, "I meant to use 'targets'" instead of "quotas."

Justice Ginsburg then wondered about the nature of this claim as a statewide claim instead of a district-by-district claim. This issue was picked up by Justice Alito in further questioning of petitioners' counsel, wondering whether the complaint adequately alleged it or whether the district court misunderstood the claim. Instead, petitioners pressed back, the effect is the same--challenging all 36 districts or challenging each of the 36 districts.

Chief Justice Roberts returning to this concern of the state navigating too few minorities in a district and too many--"If Alabama had reduced the numbers in any significant way, the Attorney General would come in like a ton of bricks."

Justice Sotomayor expressed a more basic problem--the injury. She was worried that the Shaw injury alleged by petitioners was an "ephemeral injury"--"Explain to me why you don't have to be harmed by a specific application of this policy." She would later confess that she was "still having a psychological problem" about whether petitioners had been injured at all. (She never asked a question to Alabama's counsel during argument.)

Justice Breyer expressed pragmatic concerns about remanding the case--would it be incumbent on the petitioners to point to specific districts where the motivation was rational? And then it would be left to Alabama to articulate a reasonable attempt--or a good-faith reasonable attempt--to comply with Section 5?

Solicitor General Verrilli then went on to make the claim that Shaw demanded district-by-district examination. Justice Kagan pushed back--the policy in place was to avoid retrogression, defined in a particular way, and that definition was applied to every single district. Instead, General Verilli countered, the only way the criteria would rise to a Shaw claim is if the district were drawn in derogation of a traditional policy, and that that had not been shown in every district. The policy itself, he said, was not enough.

Justice Alito returned to the question about whether the district court properly understood the claims, to which General Verrilli conceded the issue was "murky." And Chief Justice Roberts pressed the concern that, if remanded and redistricting occurred, Section 5 would no longer control and the Department of Justice would not have a part in the process.

When Alabama's counsel took to the lectern, counsel claimed that the goal was to "preserve the status quo." Chief Justice Roberts answered that the problem is that perhaps the status quo is not the right standard.

Justice Kagan came out with a full-throated defense of petitioners' claims. "You were determined, come what may," to keep the same percentage of minorities in each district? That it was "just a coincidence" that the same number of minorities were in each district?

She noted that this is perhaps a sui generis Shaw claim. Usually, she explained, Shaw claims are circumstantial. But here, we have a policy statement, from the state, that nonretrogression is the second-most important goal (behind Reynolds), and then a clear statement that nonretrogression demanded the same percentage of minority voters in each district. Alabama responded that it was a plausible way of interpreting Justice Souter's opinion in Georgia v. Ashcroft and a basis for Congress's amendments to the Voting Rights Act in 2006.

Justice Kennedy was concerned that the state did not say it was gerrymandering--it said it was trying to comply with Section 5. (In his eyes, the partisan reasons, I think, should have been more obvious.)

Justice Breyer questioned the practicalities of remanding the case--how would Alabama justify its system after Section 5 is out the window?

Finally, Justice Kagan even pressed the state's interest in complying with one person, one vote with a 2% deviation standard in population--she noted that as the Court has upheld deviations of up to 10%, perhaps 2% is not really a state interest.

During rebuttal, Justice Sotomayor again pressed the evidentiary point, asking whether the injury had been adequately established below.

My gut reaction, then, might not comport at all with reality--this case went in so many directions, it's hard to identify how the Court might proceed.

Class of 2014 LSAT scores did not portend sharp drop in MBE scores

UPDATE: Jerry Organ (University of St. Thomas) has posted an even more thorough and thoughtful analysis of the LSAT scores and projected bar passage rates at the Legal Whiteboard. He, too, finds the NCBE's conclusion difficult.

I've blogged (here and here and here and here and here) about the sharp drop in bar passage rates around the country from the July 2014 administration of the bar exam, largely due to the unprecedented drop in MBE scores. A recent Wall Street Journal blog post about the reaction of the dean of Brooklyn Law School shows the sides in the fight. Did the NCBE screw up its exam, yielding a sharp drop in scores? Or did law schools admit a disproportionately unqualified class?

Here's an attempt to measure the quality of the class and correlate it with MBE scores. (Maybe it's just awful math.)

The LSAT is fairly highly correlated with MBE scores. Consider this NCBE report (PDF). I extrapolated those figures for the LSAT and the average MBE scores. I then weighted them against the number of matriculants in law school: LSAC reports the number of matriculants with scores of 175+, 170-174, and so on. I took a rough estimate of the expected MBE score for each range; I then averaged it out for the entire class.

When I first charged it, the projected MBE scores were much higher than the actual MBE scores that arose three years later. (I used the 2009-2010 LSAT matriculant data, for instance, and mapped it on the MBE results three years later, in 2013.) I attributed this to several possibilities, the most significant of which is that repeaters probably significantly drag down the MBE score. But subtracting five points from the projected MBE score lead to an almost perfect match with the actual MBE score, with one exception.*

Note that the LSAT score reporting changed beginning in the 2009-2010 cycle (i.e., the Class of 2013): schools could report the highest LSAT scores, rather than the average LSAT scores, of matriculants. That meant that the LSAT scores were probably overstated in the last two graduating classes.

But in the charge, we see a fairly significant correlation between my extremely rough approximation of a projected MBE score based on the LSAT scores of the matriculating classes, and the actual MBE scores, with one exception: this cycle.

My math is rough--and maybe it's just bad. But as this comports with every other analysis I've done, and as I've not been able to find any other factors that would contribute to an across-the-board decline in scores, I'm increasingly convinced that a problem occurred on the NCBE's end--and not that the Class of 2014 was somehow disproportionately and dramatically worse than other classes.

That said, we should expect to see declining MBE scores (and bar passage rates) of some kind in the next few years, as academic quality of entering classes continues to decline; and, we should expect bar passage-required employment outcomes to see some (likely negative) effect due to a sharp drop-off in bar passage rates.

*I should add that I could have simply plotted the projected results so that you could observe the similarity (or differences) in the rise and fall; or, in the alternative, I could have plotted them on two different Y axes. Subtracting five points, however, seemed like the easiest way to make the visualization more obvious.

DOJ, pundits fear North Carolina elections have returned to the barbaric year 2000

I read the Department of Justice's lawsuit regarding North Carolina's new election law, HB 589, with interest. I then read the judicial opinion denying a preliminary injunction. And I watched the case unfold over the past year, leading up to the 2014 midterm election.

I want to bracket the discussion about its voter identification law, over which, I think, there is a very vigorous and good-faith debate among many as to its necessity, its impact on voters, and so on.

I want to focus on the fact that the DOJ--and, more generally, pundits and critics of North Carolina's law--didn't challenging just the voter identification part of the law. It also challenged three other provisions: a change in early voting days from 17 to 10, the elimination of same-day voter registration during early voting, and a rule that provisional ballots cast in the wrong precinct will not be counted.

These three changes hearken back to a barbaric year in America's election history: 2000. 

In 2000, early voting simply did not exist. It was created by the North Carolina legislature in 2001.

And same-day voter registration during the early voting period could not exist, because early voting did not exist--but the same-day registration procedures were instituted in 2007.

Additionally, 2000 was before the Help America Vote Act of 2002, which required states to institute procedures for provisional balloting. When North Carolina did institute provisional balloting, there was a controversy in 2004 as to whether out-of-precinct votes should be counted. In James v. Bartlett (PDF) in 2005, the North Carolina Supreme Court unanimously construed the provisional ballot statute to preclude the counting of out-of-precinct provisional ballots, as the State Board of Elections, it concluded, made a "unilateral decision" to count such votes for the first time in this election. The North Carolina legislature then amended the statute in 2005 to allow for the counting of out-of-precinct ballots.

For perspective, then, early voting did not exist before 2001; same-day voter registration during early voting did not exist before 2007; and out-of-precinct provisional ballot counting did not exist before 2005.

It may well be that a court one day agrees with the Department of Justice's complaint: that this law was enacted for a discriminatory purpose; that despite the small sample size of data, it appears that the law will have a discriminatory impact on minority voters; that minority voters taking advantage of recent novations in election law means that such novations are essentially unalterable by future legislatures; and so on.

But there is a risk, I think, in the DOJ overplaying its hand. (And a touch of sarcasm in this post title doesn't hurt to make that point!) All litigation, of course, is fraught with risks, and post-Shelby County Voting Rights Act litigation is no exception.

Indeed, the worst fearmongering appears never to have materialized. In North Carolina this cycle, voter turnout increased; African-American voter turnout apparently increased; early voting increased.

There is much more nuanced statistical analysis to be done, of course. And these statistics can never capture whether, on an individualized level, potential voters were unable to vote as a direct result of these changes. But, it does, I think, suggest that one could be too quick to engage in rhetoric, and too slow to examine the actual, material impact of voting laws.

Sample ballot with racial designations from Louisiana, 1961

In Anderson v. Martin (1964), the Supreme Court concluded that Louisiana's practice of listing the race of each candidate on the ballot was unconstitutional. The opinion is here.

Here's a sample ballot from a Louisiana election from that era, printed in the Lake Charles American-Press, April 7, 1961. Enlarge the image to see "Caucasian" or "Negro" listed beside each candidate.