Any LSAT student reading the title above probably just felt their heart stop for a moment. I apologize for that! Fortunately, the LSAT won’t be getting harder immediately, but will likely be getting harder in the years to come. Let’s talk about why.
Big Data is Everywhere
The rise of Big Data has been widely discussed, usually in reference to social media and advertising. But with the dawn of the Digital LSAT and the launch of LSAC’s own online testing site, the makers of the test now have a unique ability to see exactly what causes the greatest problems for students. In the old days of the paper test, the only data the test makers had access to was very fuzzy and imperfect: the exact selection percentage of each answer choice, questions completed, and tests cancelled. While that may sound like a lot, it’s not really much at all—testing outcomes were visible, but student behavior and performance trends were not.
Now, the test makers can see everything about your performance as you practice in their system (known as LawHub), including how long you spend on each question, the questions you perceived as problematic (via flagging), what you thought was important in a passage (via highlighting), when and where you changed answers, and even how fast you guessed at the end of a section.
But Wait, There’s More
The above data is collected when students take individual LSATs via Lawhub, but LSAC also has the ability to see how often students study, the question and reasoning types that they miss over time, how much or how little they study prior to an official test day, and even the LSATs that are most frequently taken. This massive trove of data provides them with the clearest view they’ve ever had of how people are preparing for and taking this test. And that should worry you because sooner or later they will use this information against you.
There’s Reason To Make It Harder
One thing we’ve seen with the pandemic is that students have been studying more, and that has lead to a bubble of high scores on the LSAT-Flex exams (you can track daily updates to the Applicants’ LSAT Score here). With this bubble, there will be a desire in the coming year to return the percentiles back to their prior state. This suggests the exam will become harder in the short term. And, given that there been some percentile creep over the years, a possible desire to counter that as well.
If LSAC does make the test more difficult, they have all the tools to do so. They can identify what concepts and presentations students found the most troublesome, and then replicate those more frequently on each future LSAT.
What Does This Mean For you?
The popular adage, “You have to fight fire with fire” comes to mind. You need to use your practice tests and problems sets—and the resulting data they produce—to closely analyze where you are going wrong and what you find the most difficult, and then attack those areas. If you are in one of our courses, you receive tons of information via our digital platform. And, if you are self-studying, you can use our popular Testing and Analytics Package to receive all the stats and feedback you need.
wms says
Scores are up, but applications are up too. The 170-174 and 175-180 ranges have seen bumps disproportionate to the application increase, however. Maybe at least in part, the applicant pool has higher cognitive ability. But it’s mainly the fact that the LSAT-Flex is an easier test, because it’s taken at home and has only three sections. Remember, the actual LSAT is not simply four, but five, sections. That’s a major grind. Also, cheating on the LSAT-Flex may be a problem? What a shame. Not that LSAC cares or anything.
The change is clearly not a consequence of ‘more studying because of the pandemic’. What a prep company thing to say!
Dave Killoran says
Hi WMS,
I think you may have missed the point of what I was saying above 🙂 My main point is that LSAC now has access to testing data and pattern analysis that they never could access previously. They will, of course use that to their advantage! This is especially the case as there’s been a bit of score creep over the last decade or so; now they have the tools to offset that.
The point about a rise in top scores isn’t speculation, it’s a fact. This has happened, and we’ve been able to see daily data updates that prove it. More importantly, those high score bubbles increased faster than the rise in applications. So, it wasn’t simply tied to more people applying.
And last, ‘more studying because of the pandemic’ is also something we’ve seen (as have other companies), and it makes logical sense when people were stuck inside for months on end. It’s a point that we certainly believe contributed to higher scores (as did the shorter LSAT), but it wasn’t my main thrust here. Making a source argument doesn’t refute those points 🙂
As far as cheating being a problem on the LSAT-Flex, I invite you to show me examples of where that has occurred. As we’ve stated many times on our podcast, cheating on this version of the test is demonstrably harder than on any prior version of the LSAT. And I can most definitely confirm that LSAC cares quite a bit about cheating. It’s not only a central topic in the communications they give us but in the decisions they make (it is, for example, why the LSAT-Flex is only 3 sections and not more, and why LSAT Writing has had such delays).
Thanks!
wms says
A prep company has an incentive to encourage greater studying among test-takers, so a ‘source argument’ works just fine. By claiming test-takers are studying more, a prep company can increase its business. As should have been evident, however, I didn’t intend what I said in the final line of my post to be taken all that seriously.
I agree that the rise in scores is a fact, and I agree that it has outpaced applications…I said as much, if I am not mistaken.
I speculated that cheating could be occurring, not that it was occurring. If it’s not occurring, that’s great news.
Score Preview is another possible reason for score increases.
wms says
p1: a prep company can profit by claiming studying is increasing.
p2: therefore, a prep company has an incentive to claiming studying is increasing.
p3: because of this incentive, a prep company will claim studying is increasing regardless of whether it is decreasing, staying the same, or increasing.
c: therefore, skepticism with respect to a prep company’s claim that studying is increasing is warranted.
hmmmm says
Which of the following best describes a flaw in the argument’s reasoning?
C. it draws conclusions about the merit of a position and about the content of that position solely from evidence about the position’s source
Dave Killoran says
You nailed it! Where that’s thumbs up emoji when you need it?
wms says
Well, I didn’t say the claim was false or less likely, only that you can’t infer anything a/b its probability on the basis of the ‘prep company’s’ claim. But yeah, I think the arg is still guilty of ‘genetic fallacy’ reasoning – mea culpa. I don’t often miss flaw (much less source-flaw) questions on PT’s, either.
Dave Killoran says
I enjoyed this exchange, and your last point is the real key here: as long as you don’t miss these questions on the LSAT, that’s all that really matters to me!
Cheers!
Dave Killoran says
Hi WMS,
Thanks for the reply! I see we continue to agree and disagree on certain things 🙂
First, a source argument doesn’t work just because incentive is present–a source argument is a faulty form of reasoning that actually specifically recognizes that point as a cause of the error, and offers no reason other than the source of the claim as a problem. You still haven’t addressed our very factual claims here or the varied factors in support of our main point. But, that’s the LSAT teacher in me, and I’m happy to grant your point that your comment was more in jest than anything and let it go 🙂
Second, Score Preview will, I agree, lead to an average increase in scores (and this actually supports my main point that the LSAT will get harder since they will need to offset this), but in this case the bubble of scores appeared prior to the advent of Score Preview. So, while I expect it to be a factor going forward, it wasn’t in play for where we are now.
Thanks!