NCBE has data to prove Class of 2014 was worst in a decade, and it's likely going to get worse

I have blogged extensively about the decline in bar pass rates around the country after the July 2014 test. My original take was more inquisitive, and I later discounted the impact that ExamSoft may have had. After examining the incoming LSAT scores for the Class of 2014, I concluded that it was increasingly likely that the NCBE had some role, positing elsewhere that perhaps there was a flaw in equating the test with previous administrations.

The NCBE has come back with rather forceful data to show that it wasn't the MBE (and that my most recent speculation was, probably, incorrect)--it was, in all likelihood, the graduates who took the test.

In a December publication (PDF), the NCBE described several quality-control measures that confirmed it was the test-takers, and not the test. First, on re-takers v. first-time test-takers:

Among the things I learned was that whereas the scores of those we know to be retaking the MBE dropped by 1.7 points, the score drop for those we believe to be first-time takers dropped by 2.7 points. (19% of July 2014 test takers were repeaters, and 65% were believed to be first-time takers. The remaining 16% could not be tracked because they tested in jurisdictions that collect inadequate data on the MBE answer sheets.) The decline for retakers was not atypical; however, the decline for first-time takers was without precedent during the previous 10 years.

I had suggested from earlier data from a few states that re-takers and first-time test-takers performed similarly; but, disclosing data from a much broader dataset and using the more precise issue of MBE performance, first-time test-taker performance was much worse.

Second, on equating the test:

Also telling is the fact that performance by all July 2014 takers on the equating items drawn from previous July test administrations was 1.63 percentage points lower than performance associated with the previous use of those items, as against a 0.57 percentage point increase in July 2013.

As equating the test is probably the biggest possible flaw on the NCBE's end, it's extremely telling that the equating of specific items on previous administrations yielded such a significant decline, and such a sharp contrast with the July 2013 test.

Third, and, in my view, one of the most telling elements, the MPRE presaged this outcome:

The decline in MPRE performance supports what we saw in the July 2014 MBE numbers. In 2012, 66,499 candidates generated a mean score of 97.57 (on a 50–150 scale). In 2013, 62,674 candidates generated a mean score of 95.65. In 2014, a total of 60,546 candidates generated a mean score of 93.57. Because many MPRE test takers are still enrolled in law school when they test, these scores can be seen as presaging MBE performance in 2014 and 2015.

A steady decline in MPRE scores, then, foretold this problem. This further undermines any notion that ExamSoft or other test-specific factors impacted the outcome; the writing was on the wall years ago. But as few schools carefully track MPRE performance, it might not have been an obvious sign until after the fact.

The NCBE bulletin then points out additional factors that distort student quality: a decrease in quality at the 25th percentile of admitted students at many institutions (i.e., those at the highest risk of failing the bar), the impact of highest-LSAT score reporting rather than average-LSAT score reporting for matriculants (a change embraced by both the ABA and LSAC despite evidence that taking the highest score overstates student quality), and an increase in transfer students to higher-ranked institutions (which distorts the incoming student quality metrics at many institutions). Earlier, I blogged that a decline in LSAT scores likely could not explain all of the decline--it could explain part, but there are, perhaps, other factors at play.

The NCBE goes on to identify other possible factors, ones that may merit further investigation in the legal academy:

  • An increase in "experiential learning," including an increase in pass-fail course offerings, and which often means students take fewer graded, more rigorous, "black-letter" courses;
  • A decline in credit hours required for graduation and a decline required (i.e., often more rigorous) courses;
  • An increase in bar-prep companies over semester-long coursework to prepare for the bar;
  • A lack of academic support for at-risk students as the 25th percentile LSAT scores of matriculants worsens at many institutions.

So, after I waffled, and blamed some decrease in student quality, and then started to increasingly consider the NCBE as a culprit, this data moves me back to putting essentially all of the focus on student quality and law school decisionmaking. Law schools--through admissions decisions, curriculum decisions, academic support decisions, transfer decisions, as a reaction to non-empirical calls from the ABA or other advocacy groups, or some combination of these factors--are primarily in control of the students' bar pass rates, not some remarkable decision of the NCBE. How schools respond will be another matter.

Further, the NCBE report goes on to chart the decline in the 25th percentile LSAT scores at many institutions. The declines in many places are steep. They portend some dramatic results--the decline in bar pass rates this year is only the beginning of probably still-steep declines in the next couple of years, absent aggressive decisions within the present control of law school. (The admissions decisions, after all, are baked in for the current three classes.)

Coupled with the decline of prospective law students, law schools are now getting squeezed at both ends--their prospective student quality is increasingly limited, and their graduates are going to find it still harder to pass the bar. And we'll see how they respond to this piece of news from the NCBE--I, for one, find the data quite persuasive.

Visualizing the continuing decline of the law school student body, 2014

One of the posts that has had the most staying power on this site was a post and a chart last year, "For legal education, the worst may be yet to come." We can now confirm that the decline continues, and the Class of 2017 is much smaller than previous classes--and that the bottom still has not been reached, given LSAT and applicant trends.

An ABA Journal piece discloses that the total incoming 1L class in 2014 was 37,675, the smallest since 1974, and down from the peak of 52,488 in 2010. Coupled with the declining LSAT data from LSAC, it paints a grim picture for legal education through at least 2017, and likely through 2018.

Top 25 law schools ranked by law student transfer preferences

How about another law school ranking--this time, one that measures tangible law student preference for one school over another?

The ABA has released the Standard 509 disclosures from law schools for 2014. The Standard 509 includes new data this year. Schools formerly listed only the number of transfers in and out. Now, if a school accepts more than five transfer students, it must list the schools from which the transfers came. Additionally, enough transfer students requires the school to the median GPA, and with an even larger number of transfers the 75th and 25th percentile GPAs of transfer.

The data is all concealed in those PDFs. But with a little magic (text recognition, data scraping, and a little time-consuming manual cleanup), we can aggregate the transfer data. Schools logged 2221 transfers in for the Fall of 2014. Because of disclosure requirements, we know the migration patterns of 1968 of them. So we know 1968 decisions of law students to leave one institution and instead attend another.

Students applying to law schools often don't have a good idea about what schools have to offer. But once they are in law school, they have some additional information about their own institution and have a better perspective about law schools themselves.

My colleague Rob Anderson (WITNESSETH) thought that using the Bradley-Terry model would be the best way of comparing schools. (This method is used in, among other things, Peter Wolfe's college football rankings, once a component of the BCS formula.)

Using that method, here are the top 25 schools. (The full list will be revealed at WITNESSETH later this week.) Comments below.

1. Yale University

2. Stanford University

3. Harvard University

4. New York University

5. University of California-Berkeley

6. Columbia University

7. University of Chicago

8. University of Pennsylvania

9. Northwestern University

10. University of Texas at Austin

11. Duke University

12. University of Washington

13. University of California-Los Angeles

14. Vanderbilt University

15. University of Michigan

16. University of Virginia

17. Cornell University

18. George Washington University

19. Brigham Young University

20. Georgetown University

21. University of Minnesota

22. University of Southern California

23. Southern Methodist University

24. Washington University

25. University of Notre Dame

As with any ranking system, there are obviously imperfections. Zero students transferred out of Yale, Harvard, or Stanford, so they are compared only indirectly; so too with others that had fewer transfers. Many schools had five or fewer transfers, which they did not disclose. Many students transfer for personal reasons, which may not reflect an evaluation of law school quality. Schools generally benefited if they had no (or few) transfers out; schools generally suffered if they were not required to disclose their transfer data, or if they accepted few transfers (when, of course, the most stable schools may accept the least transfers!).

Glancing at the top 25, one may wonder about some of the rankings. But consider SMU: they accepted transfers from Baylor, Hastings, and Fordham (among other institutions), but sent students only to Texas. A Bradley-Terry model would rank SMU quite high for precisely the results of this head-to-head matchup.

The data set includes all schools with at least one disclosed transfer, including the three schools in Puerto Rico, recently-accredited schools like the University of California-Irvine, and schools seeking accreditation like Concordia University.

Stay tuned: the full list of schools is forthcoming.

(If you are at a law school that did not disclose the schools from which students transfers, email us the information and we'll post an update. It will usually, but not always, help your school's ranking.)

UPDATE: This post has been modified in light of a correction in data.

Ninth Circuit affirms Measure B condom requirement

More than nine months after oral argument, the Ninth Circuit has issued its decision in Vivid Entertainment v. Fielding, a challenge to Los Angeles's Measure B (PDF), which, among other things, requires performers in pornographic films to wear condoms.

I wondered earlier if Prop 8 litigation may have undermined governmental defense of certain ballot initiatives, given that Los Angeles County urged intervenors to participate in the litigation so that it would not have to defend the measure. After challengers to the measure largely lost, they appealed, arguing, among other things, that the defendant-intervenors could not serve as appellees, because defendant Los Angeles County refused to continue to defend the measure.

Here's what the Ninth Circuit held regarding jurisdiction:

Citing Perry, Plaintiffs argue that we lack jurisdiction over this appeal, because Intervenors lack Article III standing. We disagree with their reading of Perry and with their contention that Intervenors must have standing for this appeal to proceed.
The Supreme Court has held that a party must have Article III standing both to initiate an action and to seek review on appeal. Arizonans for Official English v. Arizona, 520 U.S. 43, 64 (1997). But an intervenor who performs neither of those functions and no other function that invokes the power of the federal courts need not meet Article III standing requirements. Yniguez v. Arizona, 939 F.2d 727, 731 (9th Cir. 1991), vacated by Arizonans for Official English, 520 U.S. at 80, as recognized in League of United Latin Am. Citizens v. Wilson, 131 F.3d 1297, 1305 n.5 (9th Cir. 1997); see also Perry, 133 S. Ct. at 2661 (citing Art. III, § 2) (holding that “any person invoking the power of a federal court must demonstrate standing to do so” (emphasis added)). Nothing in Perry, which concerned the question whether an intervenor who sought to appeal had Article III standing, affects that conclusion. Plaintiffs have standing, and it is they alone who have invoked the federal courts’ jurisdiction. For that reason, we need not and do not decide whether Intervenors satisfy the requirements of Article III standing.

Jurisdiction, then, exists--but not obligation, apparently, exists for the government to defend a publicly-enacted ballot measure that the government officials may not agree with, or simply deem too sticky to provide a public defense. The Ninth Circuit went on to largely affirm the district court and, accordingly, upheld enforcement of much of Measure B.

New MP3: Eminem - '97 Bonnie and Clyde ft. Chief Justice John Roberts

I grew up on 12 Mile Road in the 313.* It's with some affection, then, that I hold the music of Detroit, from Motown to techno, in high esteem.

Eminem supporters storm the State of the Union address and beg onlooking Supreme Court justices to "Protect the 1st Amendment." (Screenshots from Eminem, "The Mosh Continues" (2004).)

Eminem is probably one of the most gifted lyricists in rap--he moves far beyond rhyme to embrace assonance, consonance, alliteration, and many more poetic forms in his lyrics. (This 90-second video clip of his interview with Anderson Cooper on the word "orange" teaches more about poetry more than most students learn in all of high school.) His music, however, is not for the faint of heart, given the obscenity and... shall we say, adult content of those lyrics.

An early album in 1997 featured "Just the Two of Us," a song about domestic violence in which Eminem and his daughter conspire to kill the girl's mother. (The song samples heavily from the 1981 Grover Washington Jr. and Bill Whithers song.) The song was re-recorded, extended, and released under the title "'97 Bonnie and Clyde" on another album in 1999.

Last week, the United States Supreme Court heard oral argument in Elonis v. United States, in which Defendant was convicted of threatening another in a Facebook post. Asking about the scope of the First Amendment when it comes to threatening speech, Chief Justice John Roberts quoted from "'97 Bonnie and Clyde." From the transcript (PDF):

CHIEF JUSTICE ROBERTS: What about the language [in] the Petitioner's brief? You know, "Da-da make a nice bed for mommy at the bottom of the lake," "tie a rope around a rock," this is during the context of a domestic dispute between a husband and wife. "There goes mama splashing in the water, no more fighting with dad," you know, all that stuff.
Now, under your test, could that be prosecuted.
MR. DREEBEN: No. Because if you look at the context of these statements--
CHIEF JUSTICE ROBERTS: Because Eminem said it instead of somebody else?
MR. DREEBEN: Because Eminem said it at a concert where people are going to be entertained.

With that, I thought a mashup was in order. With the enormous help and creative talent of an old friend and talented musician, Nate Wazoo, here's "Roberts and Clyde." an MP3 of a 52-second sample of the song.

You can purchase a copy of the original "'97 Bonnie and Clyde" at sites like Amazon.

*Strictly speaking, that was Royal Oak, not Detroit--the 313 area code was subdivided in 1993.

Copyright notice: the use of any copyrighted material is fair use for non-commercial, satirical, and educational purposes.

The law student applicant pool still hasn't bottomed out

On the heels of 8- to 9-point drops in LSAT test-takers year over year, LSAC is now reporting an 8.5% decline in applicants year-over-year. I explained last year that the worst may be yet to come for law schools, and noted that the shrinking 2013-2014 applicant pool would linger with law schools until 2017. This cycle promises to have still fewer applicants, which means even more deep impacts to law school budgets through at least 2018. Added with declining bar passage rates, law schools are facing the most dire circumstances yet. Some may not truly feel the financial effects for another year or two, but even more difficult decisions will be made this applicant cycle that may impact which schools survive.

(While I usually accompany a post like this with a chart displaying a precarious decline, I'm holding off for more concrete numbers from the ABA for the Class of 2017....)

Fictional Attorney of the Month: Barry Zuckerkorn

Barry Zuckerkorn is the Bluth family attorney in Arrested Development. Unlike rival attorney Bob Loblaw, he's the original family attorney.

To put it mildly, Mr. Zuckerkorn is deeply incompetent. He never actually passed the California bar, as he hired a lookalike to take it in his place. After a judge lists off pending charges (conspiracy, racketeering, evidence tampering, fraud, theft, grand theft, petty theft), he responds, "Wow, I... I did not get that page." Under the pressures of typical litigation, he lashes out with scare quotes, "That's why I want to settle. I'm not 'super prepared.'" His misunderstanding of law, ranging from spousal privilege (advising his clients that a husband and wife as coconspirators cannot both be charged) to basic ethical obligations (advising his clients to lie to police).

Another exchange illuminates how quickly he's impressed at the legal perception of non-lawyers around him:

Barry: And you don’t want to go in front of that judge. I caught him in a drag club.
G.O.B.: What were you doing there?
Barry: Wow... you should be the lawyer.

There's much more to read about Henry Winkler's portrayal of Barry Zuckerkorn at devoted fansites, but for now, that's all on this month's Fictional Attorney of the Month.

Missouri court issues TRO against Common Core on Compact Clause grounds

I blogged earlier about a challenge to Missouri's participation in "Common Core," an education system developed in part by encouragement from the United States Department of Education. The challenge alleged that the standards violated the Compact Clause, because it effectively undermined federal power and the power of non-compacting sister states, and it had not been approved by Congress. (I've written about the need for congressional consent of interstate compacts before.)

Yesterday, a state court issued a temporary restraining order prohibiting Missouri from making payments to the education consortium administering Common Core on the ground that plaintiffs made a preliminary showing of likelihood of success on the merits: "For the reasons stated in the Plaintiffs’ Memorandum in Support of their Motion for Preliminary Injunction, Plaintiffs have made a preliminary showing of likelihood of success on the merits on their claim that the Consortium is an unconstitutional interstate compact to which Congress has never consented, in violation of the Compact Clause of the U.S. Constitution. If Plaintiffs prevail on this claim, Missouri’s membership in the Consortium is unconstitutional." A copy of the TRO is here.