I was recently asked about a specific question from the October 1999 LSAT (O99, LR1, #7) on our LSAT Discussion Forum. This question, known as Debbie's Magic Act, can be summarized as follows:
Debbie has a magic act where she identifies a card chosen randomly from a deck, without ever looking at the card beforehand. A skeptic examined the process, and conducted three separate tests. In the first test, he made a video of her selecting the correct card, and after examining it he determined she did not perform sleight-of-hand. In the second test he gave her his own deck of cards and she again succeeded, so the skeptic concluded it wasn't a stacked deck. In the third test, the skeptic selected the card on his own, and on that basis ruled out the possibility that a plant in the audience had been used. On the basis of these three tests, the skeptic concludes that none of the three methods—sleight-of-hand, stacked deck, or planted volunteer—had been used by Debbie to perform the magic trick. This stimulus is then followed by a Flaw in the Reasoning question.
The reason this question interests me is that a large number of students select the same incorrect answer, and are very confident they are correct even after reading the problem again. Even after looking at the right answer these students often do not necessarily understand where things went awry with the answer they chose. It's a tricky question! The trap used to create this confusion is one I sometimes refer to as the Natural Question Error, and the makers of the LSAT have preyed upon it in many Logical Reasoning questions. It's a type of Shell Game error, and if you study for the LSAT long enough, you will at some point make the very same mistake. So let's break this problem down and see how we can avoid this error.
The correct answer to the problem is (A), which states that the skeptic did not consider that Debbie might have used a different method for each test. If that was the case, she could have used a stacked deck or audience plant for the first video test, then an audience plant or sleight-of hand for the second test, and then a stacked deck or sleight-of-hand for the third test. In other words, each test rules out just one possibility for that test only; if Debbie was changing methods, then she could easily have used several of the methods over the course of the three tests, which would make the conclusion flawed. Note that answer choice (A) is a bit tricky and unexpected, and placing this answer first ensures that many students will somewhat gloss over it. It's supremely well-placed to get students to fly right past it.
The attractive incorrect answer I want to explore is answer choice (D), which states that the skeptic forgot to consider other methods besides the three mentioned in the stimulus. Around 27% of students select this answer, which means 1 out of every 4 test takers sees this answer as correct. Why is this answer so attractive? This is where the trap of the natural question comes in: when you read the stimulus, your first reaction is not to evaluate the various methods that didn't seem to work, but to instead wonder, "How did Debbie do it?" It's the most natural and common question to ask, and virtually everyone who does this problem thinks about it for at least a second. Answer choice (D) plays right into this by suggesting that there's another method that Debbie used that should have been considered. So why is (D) wrong? Because that question, although natural, is also irrelevant. The stimulus wasn't about other methods, but instead ruled out three specific methods. The skeptic wasn't obligated to research all the possible methods that Debbie used, just the three that the skeptic said weren't used. So, in this case, the natural question leads us right into a trap like answer choice (D).
How do we avoid this trap, then? By focusing very carefully on the wording of the conclusion. The conclusion here states that, "The skeptic concluded that Debbie uses neither sleight of hand, nor a trick deck, nor a planted "volunteer" to achieve her effect." The wording is very specifically limited to eliminating the three methods. Note that it doesn't say, "The skeptic concluded that she did not use magic to achieve her effect." If the conclusion said that, then (D) would be much more attractive. So, the way around the Natural Question Trap is to know the specifics of the conclusion, and not allow yourself to go beyond what was said.
If you'd like another example of this error from a source you probably have in your possession, take a look at the LSAT Logical Reasoning Bible, in the Weaken chapter. The second question in the text (page 224 in the 2016 version) is the Beverage Company Director question, and many students miss this problem because they ask the natural question of whether the new plastic rings will cause other issues, and end up picking answer choice (E). But, as explained in the book, that answer doesn't match the conclusion, and hence that answer is incorrect.
Identifying the conclusion in each Logical Reasoning problem is critical, and by looking at questions such as the two discussed above, you can see how the test makers set traps for the unsuspecting reader. Always make sure you know the details of the conclusion of each argument, and that you know exactly what it says. That will protect you from falling into one of the most insidious answer traps on the LSAT.
Questions or comments? Please post them below!
Photo credit: "Elige vestra religio sapienter" courtesy of Randy Heinitz.