Cognitive Biases in Test Reviews: When Smart People Can Be Stupid

    LSAT Prep

    LSAT biasOne of the most daunting tasks in any test preparation program, whether undertaken on your own or with an instructor,  is the practice test review. For many, it's an experience that is at once humbling and tedious, likely to repeat itself more than a dozen times over the course of their preparation. To make matters worse, the ideal test review requires more than simply going over the questions you got wrong. Ideally, you should review any question that you answered without confidence, whether you got it right or not. We've written extensively on this topic over the years, and you should probably have the following blog posts bookmarked: 

    The Best Way to Review LSAT Practice Tests

    The Ideal Way to Take an LSAT Practice Test

    How to Best Review LSAT Practice Questions

    In this blog, I'd like to highlight a few of the more prominent cognitive biases that I come across when reviewing practice tests with my students. What's a cognitive bias and why does it matter? Well, a cognitive bias refers to any type of perceptual distortion or irrational judgment, where the decision reached results from an unconscious process. Such distortions matter precisely because we aren't aware of them. We often trick ourselves into believing that, as long as we understand why we got a question wrong, our job is done. Alas, there is a bit more to it than that.

      • Hindsight Bias. The "I-knew-it-all-along" bias occurs when you believe that you knew the right answer to the question you got wrong, but just happened to choose the wrong answer. Just because you were able to narrow it down to two answer choices doesn't mean you almost got it right. You didn't. They still tricked you, and you fell for it. Unfortunately, now that you know what the correct answer choice is, you can easily buy into it, without any further introspection on your part. This is a classic example of a hindsight bias.

        • How to fix it? Don't review the questions you missed knowing what the correct answers are. Instead of explaining to yourself why a certain answer is correct, try to arrive at that answer on your own, avoiding the benefit of hindsight.

      • Choice Supportive Bias. A corollary to the hindsight bias occurs when you were stuck between two answer choices, but happened to choose the right one. When this happens, many students mistakenly believe that their choice was more righteously chosen than it actually was. The thing is, if you got the question right, but were not at all certain in your answer, you can easily miss a similar question in the future. Paradoxically, this is an even less desirable outcome than if you got the question wrong, because you probably won't even review the questions you did well on: you will either forget that you struggled on some of them, or you will pat yourself on the back for getting lucky. Consequently, you will forfeit the opportunity to analyze a potentially confusing situation, making you liable to missing a similar question in the future.

      • Overconfidence Bias. Some test-takers have an inappropriately high confidence in their answers. This is especially true when a novice test-taker comes across an attractive answer choice before closely examining the remaining choices. If you are overconfident that answer choice (A) is correct, for instance, you're unlikely to consider the remaining four answer choices on the same footing. You fell in love with an answer choice that has distorted your perception, as it were, and an objectively better answer choice may no longer receive the same level of attention or respect. 

        • How to fix it? Prephrase, but keep an open mind. Unless an answer choice matches your prephrase exceptionally well, do not assume that it is correct until you've given the remaining answer choices the respect that they deserve.

      • Self-Serving Bias. This bias occurs when you attribute a correctly answered question to your own brilliance (which is entirely possible, of course!), but blame the test-makers for your mistakes. I've seen this tendency time and time again, particularly among high-achieving students who think they can "outsmart" the test.

        • How to fix it? Don't ever argue with the test! Remember: this is a very well-assembled test, where every item is extensively vetted. If you screwed something up, the mistake is on you. 

      • Availability Heuristic. Tougher problems, such killer games or passages, convoluted Formal Logic questions, etc. tend to stand out as unusual or unexpected. This usually results in false assumptions or estimations about the overall difficulty of the test, and also makes students spend an inordinate amount of time focusing on esoteric concepts which are rarely tested. Just like there is no surge in births during a full moon (a classic availability heuristic that was actually tested on the October 2015 LSAT), there is no surge in particularly challenging questions on the tests you bomb. (That said, keep the LSAT Casino, as the emphasis on various logical concepts may change from test to test).

        • How to fix it? Don't focus your review on the toughest questions only! Review everything, and remember that most of your gains will come from answering low-to-medium difficulty questions quickly and correctly.

      • Planning Fallacy. This is the biggest one of all - the tendency to inaccurately predict the time necessary to reach your potential and excel at the test. Take my word for it: it always takes longer than you think.

        • How to fix it? Start preparing as early as possible. While 3-4 months is the minimum amount of time you should devote to test prep, some students can take a lot longer to see a marked improvement in their scores. Know thyself and plan accordingly!


    Photo courtesy of AJ Cann.