The scores from the June 2017 LSAT have just been released, and the scoring scale is hot off the presses. We’ll have some section-specific discussions in the days ahead, but in the meantime I wanted to take a moment to analyze the scoring scale and what it tells us about the logical difficulty of this exam. As my colleague Dave Killoran has written elsewhere, each LSAT scoring scale is adjusted to fit the difficulty of that particular test:
“Although the number of questions per test has remained relatively constant over the years, the logical difficulty of each test has varied. This is not surprising since the test is made by humans, and there is no precise way to completely predetermine logical difficulty. To account for these variances in test “toughness,” the test makers adjust the Scoring Conversion Chart for each LSAT in order to make similar LSAT scores from different tests mean the same thing.”
So, the looseness or tightness of the scale will reflect the logical difficulty of the exam (loose scale with a harder exam, tight scale with an easier exam), and has a huge impact on final scores. With that in mind, how did this one turn out?
Here it is in full:
By itself, it is hard to determine what the scale tells you about the general nature of the exam: you can clearly note individual outcomes—you can miss only 1 question to get a 180, 9 questions to get a 170, 26 questions to get a 160, and so on—but what does that say for the test’s overall difficulty? To answer that we need to consider it relative to other exams.
When we compare this scale to the scoring scales from prior LSATs, what we find is that this scale is fairly normal, if not even a bit on the looser end (fewer questions needed to reach a target score) in the 140-155 scoring range, but becomes exceptionally tight for scores in the higher ranges of 160+. What that means is that this LSAT was easier than average, and particularly favored high-level scorers (while scorers in the mid-to-low range struggled about the same as they normally do).
Let me elaborate on a few of the more notable aspects, largely by comparing this test to the last released LSAT (December 2016) and to another test (December 2015) notable for an inverse trend: a tighter middle and more forgiving high-end.
- Scores of 145, 150, and 155 required minimum raw numbers of 46, 55, and 65, respectively. This is nearly identical to the last released test, the December 2016 LSAT, where those scores required minimums of 46, 55, and 64 questions correct. But when we compare both to the December 2015 LSAT, where those same three scores required at least 48, 57, and 65 questions correct, respectively, you can see right away that the June 2017 and December 2016 exams had a less strict scale. Fewer questions needed to attain a scaled score means that test takers at that level performed slightly worse (i.e. were challenged slightly more) than test takers at that same level on a different exam. So, again, in the mid-range the June 2017 LSAT was normal-to-marginally-generous.
- Scores of 160, 165, and 170 required a minimum of 75, 85, and 92 questions correct, respectively. Let’s again compare those raw results to both the December 2016 and the December 2015 tests. On December 2016, 74, 83, and 90 correct answers were needed for 160, 165, and 170 scores, and on December 2015 74, 82, and 89 correct answers were needed to attain those same final results. You should see right away that at 160 this June test is already getting tighter than either December exam (75 right answers for June, 74 for the two Decembers), and then the tables really turn: June 2017 required 85 correct answers for a 165. December 2016? 83, two fewer. December 2015? Only 82, so three additional mistakes for the same final score! And 170 repeats this discrepancy: test takers this June needed 92 correct answers, while in December 2016 90 correct was a 170, and December 2015 test takers hit that number with just 89 right. That means that June 2017 test takers far outperformed similarly-scoring students on those two past tests, and indeed on nearly every test released in the past decade! Not since September 2008 has a test allowed only 9 mistakes to stay in the 170s. Was this test easier, or were these people significantly better prepared? Hard to say, and probably a bit from both columns, but the numbers clearly show that the test makers tightened things up severely in the upper-160s and above.
- Look at those 180 numbers. 100 correct? Yikes. Only two tests in the last six years have been similarly tight: December 2016 and June 2016 (their two scales are incredibly similar, in fact). And the June 2017 scale is harsher than both of those tests for virtually every other score at 155 and above.
- Lastly, note that two scores are impossible in the 170s, a 175 and 179. People performed so well at 170+ that there simply weren’t enough questions available to allow the usual 10 possible results from 171-180, forcing the test makers to only allow 8 scores above 170 (while still giving a single-point buffer to 180). A single score removed in that range is fairly normal. Two is far less so, and again speaks to the vice-like grip at the highest reaches.
So all in all the June 2017 LSAT’s scale was forgiving for mid-range testers, but punishingly less so for students in the top 10-20%. Why? The test makers are attempting to close gaps between average performers and strong performers and maintain a smooth bell curve. But as a select subset of takers becomes better and better prepared for the LSAT—typically by wisely investing in courses and tutoring—this separation between average and elite continues to grow more pronounced.
Make sure you do all you can to reach those upper levels so as not to get left behind.
Have any questions or comments? Please post them below!