Below is Part III of our three-part discussion of the 15 hardest LSAT Logical Reasoning questions of all time. Here we discuss the last 5 on the list, which has been presented in chronological order. In Part I we discussed questions 1-5 on the list, and in Part II we presented questions 6-10. These selections are based on statistics from students’ practice tests, the number and variety of student questions we receive, and our own experience teaching and writing about logical reasoning questions. After presenting the list of the final five questions, we will discuss some of the interesting patterns and statistics associated with the questions on the list.
- October 2002, Section 2, #16: Confidence, Trust, and Abilities
Justify. In our discussion of question #10 on this list (in Part II), we mentioned an LSAT writer who tends to put abstract moral discussions into the stimulus. This question appears to have been produced by that same writer, and addresses the issues of confidence, trust, distrust, abilities, and difficult tasks. Almost invariably, the questions from this author are tough, and this one is clearly no exception.
- June 2003, Section 2, #17: Alex’s Capabilities
Parallel Reasoning. This question is probably another one from our friend who wrote questions #10 and #11 on this list, although this question’s stimulus is a bit easier to understand than the stimuli in those other two problems. However, it is paired with a Parallel question, which in this case increases the level of difficulty. Ultimately, only 17% of test takers were able to answer this question correctly.
- June 2004, Section 1, #15: Cigarette Advertisements
Resolve. In this question, the author argues that governments have the right to ban cigarette advertisements, but that they shouldn’t because other unhealthy practices are already legal to advertise. Resolve questions are generally fairly easy, but students correctly answered this one at a rate of only 20%.
- December 2004, Section 1, #12: Fish, Paper Mills, and Dioxin
Weaken. This infamous question wreaked havoc on test takers back in 2004. Only 21% of students answer this question correctly, and even LSAC’s discussion of the correct answer required two pages of explanation. It’s not in our character to hate LSAT questions, but this one would be near the very top of the list if we did.
- June 2005, Section 2, #24: The Existence of Money
Justify. This may be the easiest question on this list, which goes to show how hard the other questions on this list are. The author in this stimulus asserts that money doesn’t exist, namely because all that is needed to make money disappear is for all of us to stop believing in it. Uhh, ok.
Although we can no longer offer direct downloads (LSAC rules), this publication identifier link will tell you where to find these questions: https://www.powerscore.com/lsat/help/pub_ident.cfm.
Now that we’ve completed our list of the 15 hardest LSAT Logical Reasoning questions, let’s get to the most valuable and interesting part of this discussion: what we can learn from these questions. Here are a few fascinating observations drawn from an analysis of the 15 Hardest LSAT Logical Reasoning questions:
1. The spread of the answer choices is notably biased towards (A)
Questions within this set of 15 are overwhelmingly biased in favor of answer choice (A). Certainly this could perhaps be partially explained by the fact that a list such as this one is somewhat subjective, but consider that the correct answer for the next five questions we considered for the list but did not include was also (A).
Overall, the correct answer for 15 questions breaks down as follows:
(A) = 9, or 60%
(B) = 1, or 6.6%
(C) = 4, or 26.6%
(D) = 1, or 6.6%
(E) = 0, or 0%
Again, keep in mind that if we had added the next five most difficult questions, creating a list of 20 rather than 15, answer choice (A) would have been correct 14 out of 20 times, or 70%! Given that random chance would suggest that each correct answer appears at an average rate of just 20% and that overall on LSAT Logical Reasoning answer choice (A) is correct just 18.7% of the time, what is going on?
First, it’s not an accident. The statistics are too far out of kilter to be random, and they clearly show that this is the deliberate result of an intentional strategy used by the test makers. Second, it shouldn’t be that surprising, and in the LSAT Logical Reasoning Bible we discussed this exact phenomenon. Often in the case of a particularly hard stimulus or question stem, students can become so confused that when they hit answer choice (A) they really aren’t in tune with the problem yet, and they tend to move on without being certain of what (A) was really about. At that point, they often get caught up in the later answer choices, and usually select one of those.
So, the takeaway here is that if you are really stumped (meaning you know that a particular question is extremely difficult) and you have to guess on one of these tough problems, guess answer choice (A). And keep in mind that if you are reading a tough stimulus, you should not discount (A) too quickly—it may very well be the correct answer!
2. The spread of the answer choices is notably biased away from (E)
Another interesting point bears discussion as well: why was (E) not the correct answer for any of the questions on this list? Probably because of what we call the “reverberation effect.” Often, the very last thing a student reads “reverberates” in the memory, making one more prone to choose that last answer if no other answer has immediate appeal. With the questions on this list, which are specifically designed to be challenging, the test makers probably don’t want the reverberation factor leading students to the right answer, so they avoid making (E) the correct answer choice.
3. The questions on this list tend to show up late in the section.
The placement of these questions does not appear to be random. The range of question placement for the Top 15 Hardest questions is as follows:
Average Question Position: 20.33
Of the 15 questions, 0 appear in the single digits, 5 of them appear in positions #10-#19, and the remaining 10 appear from #23-#25. Note that last number: 2/3 of the Top 15 appear in the range of #23-25, the exact point that many great test takers are nearing the end of both time and energy. Thus, if you are shooting for a top score, at the very end of the section you must be careful of a subtle, sinister attack on your raw score: many of the test makers’ most brutal attacks will come at a point when most test takers are already worn down. And, if you are just reaching the end and hoping to optimize your score, be wary of anything that even feels tough from the outset.
4. Watch the question types
The question types of the Top 15 also tend to be non-random:
Must = 1
Flaw = 3
Parallel = 2
First Family = 6, or 40%
Strengthen = 1
Assumption = 2
Justify = 3
Resolve = 1
Second Family = 7, or 46.66%
Weaken = 2
Third Family = 2, or 13.33%
While this may appear to be a fairly random selection of question types, if you remove the pure fact based questions—such as Must and Resolve—the question types that involve more abstract reasoning account for 13 of the 15 questions, or 86.66% (These more abstract question types include Flaw, Parallel, Strengthen, Assumption, Justify, and Weaken questions). Thus, as would be expected, the problems that rank among the hardest of all time tend to be most commonly based on more abstract questions.
5. Science and philosophy rule
From a topical standpoint, 5 of the 15 questions were based on science topics, and 4 of the 15 were based on more abstract philosophical-style stimuli (as exemplified by question #10 on this list). This isn’t to say that every question on one of those two topics is sure to be difficult, but as you get deeper in the section, try to be a bit more deliberate when facing questions based on those two topics.
This ends our discussion of some of the most difficult Logical Reasoning questions of all time. We hope you find the list useful, but more importantly, we hope the analysis at the end provides you with some helpful insights into the LSAT. Again, keep in mind that this list is subjective, and you are welcome to add your personal least favorite LSAT questions below—we considered a many challenging questions for this discussion, and there are obviously a lot of extremely difficult questions that didn’t show up on this list.