Excess of Democracy

View Original

Non-LSAT standardized test scores in admissions remain concentrated at a handful of schools

The ABA requires law schools to disclose when they have 10 or more enrolled students in an incoming class who were admitted using a standardized test other than the LSAT; and the 75th, 50th, and 25th percentile scores of that cohort. Sixteen schools reported at least 10 students admitted under such programs: 12 using the GRE, 3 using the ACT, and 1 using the GMAT.

Northwestern had 11 students admitted under the GMAT, 4.7% of the class.

BYU (19 students, 14.3%), Northern Illinois (10 students, 8.5%), and Georgia (10 students, 5.2%) were the three schools with ACT admissions.

And the 12 schools using the GRE:

Hawaii (24, 24.5%)

Arizona (20, 16.4%)

Penn State-Dickinson (10, 10.8%)

Harvard (36, 9.7%)

New Hampshire (20, 9.1%)

Georgetown (44, 8.5%)

Cornell (10, 5.3%)

Cal Western (12, 5.0%)

Columbia (18, 4.2%)

NYU (16, 3.5%)

Boston College (12, 3.5%)

Penn (10, 3.5%)

That’s 232 students admitted at these 12 schools under the GRE. Other schools may have admitted GRE students, but in smaller numbers. UPDATE: The ABA has supplemented this data, which will be the subject of another post.

Another interesting question to consider—do admissions for these students look different than LSAT admissions?

This is a complicated question, and it actually reveals a difference between ETS and USNWR. The bottom line, before you read everything below, is that USNWR appears to treat the GRE materially worse than ETS recommends; and it appears that admissions are, on the whole, a bit easier for GRE students.

USNWR converts GRE scores to percentile equivalents and weighs them against LSAT percentile equivalents. (It apparently does not do so for GMAT or ACT scores.) Now, it’s worth emphasizing that it does not appear that this is how ETS’s own studies of the validity of the GRE worked compared to the LSAT, as detailed in the ABA-commissioned study to examine the ETS report. (More on that in a moment.)

Here’s USNWR’s methodology:

These are the combined median scores on the LSAT and GRE quantitative, verbal and analytical writing exams of all 2020 full- and part-time entrants to the J.D. program. Reported scores for each of the four exams, when applicable, were converted to 0-100 percentile scales. The LSAT and GRE percentile scales were weighted by the proportions of test-takers submitting each exam. For example, if 85% of exams submitted were LSATs and 15% submitted were GREs, the LSAT percentile would be multiplied by 0.85 and the average percentile of the three GRE exams by 0.15 before summing the two values. This means GRE scores were never converted to LSAT scores or vice versa. There were 60 law schools – 31% of the total ranked – that reported both the LSAT and GRE scores of their 2020 entering classes to U.S. News.

It’s not clear where percentile equivalents come from, but the latest LSAC percentile equivalent tables cover 2019-2020, whereas the latest ETS percentile equivalent tables cover 2017-2020.

The first thing we can do, then, is to look at the 50th percentile LSAT scores for the incoming classes at each of these law schools:

See this content in the original post

Next, sticking with the same order, let’s look at the 50th percentile of the GRE-V, GRE-Q, and GRE-AW scores. At the end, I aggregate them, weighing each 1/3 like USNWR does, to give an overall percentile total. (There is tremendous compression in the GRE-AW scores, as you can readily see.)

See this content in the original post

Now I’ll compare those percentile equivalents with the LSAT percentiles to see if the admitted students in the GRE cohort have a higher or lower composite percentile equivalent than the LSAT cohort. One more feature: I’ll reverse-engineer the GRE percentile equivalent to its LSAT score (rounding to the nearest LSAT score, or in one place identifying the two scores it falls between) for some idea of what it looks like. UPDATE: I’ve been told that USNWR does not weigh the GRE sections equally. But because it does not disclose its methodology, I am awaiting an answer on how it does weigh them.

See this content in the original post

One more: ETS also has a “comparison tool” for law schools. As mentioned, the ETS methodology for the relationship between the LSAT and the GRE is not the one USNWR is using. Instead, it is based on a different methodology that, as far as I know, has not been disclosed. But you can use an ETS calculator to determine a “predicted LSAT score,” within “5 points.” The calculator only uses the GRE-V and GRE-Q. What outputs do we get from this proprietary engine for each school?

See this content in the original post

Note that the ETS “predicted” LSAT is higher—in some cases, much higher—than the USNWR formula.

At many schools, it’s still lower than the median LSAT score, but often not as dramatically different; at a few others, it’s higher (and at more than just the two schools in the USNWR table above).

So if schools were going by the ETS data and admitting comparable students, they will be penalized by USNWR, which has adopted a methodology that ETS does not use:

Why not compare the reported percentiles for the GRE and LSAT?

Percentiles represent how a test taker performed relative to other test takers who recently took the same test. The current test taker populations for the GRE General Test and LSAT exam are likely different in terms of background and ability, so the percentiles calculated for each test based on those different populations are not directly comparable. Additionally, reported percentiles for any test vary over time as the test taking population changes. Both GRE and LSAT scores are meant to be consistent across time and changes in test taking populations, so the most consistent and accurate comparisons are based on the statistical relationship between the reported scores, as provided by the Comparison Tool.

It appears that at many schools, the GRE cohort has lower incoming student credentials than the LSAT cohort. How much that affects USNWR rankings depends on the size, of course.

Now, at most of these schools, it’s a relatively small part of the incoming class. It might be that these are disproportionately joint degree students, for example. At institutions with n=10, it tells us very little about the overall class. We also don’t see all the other schools with n<10 and what they’re doing. But it’s worth noting that GRE admissions do not appear to be taking place in lockstep with LSAT admissions at most institutions.

This post has been updated and modified based on feedback received. Thanks for helpful suggestions.