When the College Board, the makers of the SAT, published its test specifications for the redesigned SAT in 2015, the authors boldly stated that the SAT would be “the most transparent exam in the assessment field” and that “all would know what was on it and why.” But since the test debuted earlier this month, the only thing that’s transparent is that the new test is even more opaque than its predecessor and its competition, the ACT. Let’s look at three reasons why the new SAT is not the transparent assessment it claims to be:
1) The experimental section is shrouded in mystery.
Since the SAT’s inception in 1926, the College Board has asked students to complete “pretest” items, which are questions that do not count toward the students’ scores but do provide data to the organization about future questions. In recent years, there has been a single section of these pretest questions which was dubbed the experimental section. But with the announcement of the test redesign, there was no mention of the experimental section, leading most tutors and students to believe that it had been removed. I am less trusting of the College Board, so I called the test makers three times in 2015 to ask about the experimental section, and all three times I received different answers. Two out of the three representatives confirmed the survival of the experimental section, but both had different answers about where it would occur on the test and how long the timed section would be. To make matters worse, neither of these answers matched the answers given to the other doubting Thomas tutors when they called the College Board.
A month before the test, a College Board employee quietly announced to a group of Boston test center coordinators that there would be an experimental section for some students—but not all—who took the redesigned SAT. Astute students and tutors scrambled to learn more about it, but found the only mention of it in all of the College Board’s published materials was two sentences in a counselor’s guide: “To allow for pretesting, some students taking the SAT with no Essay will take a fifth, 20-minute section. Any section of the SAT may contain both operational and pretest items.” A counselor unfamiliar with psychometric terminology may not even recognize that the guide was referring to an experimental section.
The first test was administered nearly two weeks ago, but still the confusion lingers. There are conflicting reports about whether the experimental section is given before the test, in the middle of the test, or at the end. And if it is true that “any section of the SAT may contain both operational and pretest items,” questions within scored sections might actually be unscored. So despite the test makers’ claims that “The redesigned SAT is not mysterious,” many of us are still scratching our heads trying to put together the puzzle of the experimental section.
2) There is conflicting information about the order of the sections in the test.
In The Official SAT Study Guide, the four tests present the four sections in the exact same order: 1 Reading (65 minutes), 2 Writing and Language (35 minutes), 3 Math No Calculator (25 minutes), and 4 Math Calculator (55 minutes). Most test prep experts were then under the assumption that all tests would progress in this order, much like the ACT that follows a set subject order. But the SAT website warns this is not the case: “Don’t be surprised if your test experience isn’t exactly like the other students’. Your test book may have a different order of sections than those of the students sitting next to you. For example, your second section may have math questions, while your neighbors' books may have reading or writing and language questions.” If this is true, the mystery becomes how the proctors manage all of these timed sections with different time limits. If you are taking a 25 minute math section and your neighbor is taking a 65 minute reading section, how is the proctor calling time for you without interrupting your neighbor who still has 40 minutes left in her reading section? If the College Board created a test that was as “thoroughly transparent” as they claim, then the section order would clearly be explained as it was on the previous versions of test and as it is on the ACT.
3) The College Board postponed all adult test registrations six days before the March SAT.
Good SAT tutors will sign up for the test at least once a year to make sure that they are on top of any unannounced changes, and nearly all SAT tutors will take the first administration of a test after major changes are made. It’s the only way that we can get a true feel for the new test and to confirm that it follows the format documented by College Board. So I registered for the March SAT in the fall of 2015 to make sure I had a seat. I lined up childcare for my twins and I brushed up on the new content added to the test. But then, on February 29, just six days before I was due to take the test, I received an email stating that my test had been pushed to May “due to a test security measure.” I reached out to my colleagues across the country and learned that they, too, had received the same email. In fact, anyone over age 25 had their test postponed. Apparently, we responsible adults whose results have zero effect on our future were security threats in March, but we’re harmless in May. So let’s be transparent here, College Board: you really kicked us out because the first test administration stands the most chance of error and you did not want the scrutiny from experts like me who were the most likely test takers to discover those errors (much like the error I discovered in March of 2005, the last time the test changed. A hard-earned letter acknowledging my discovery hangs on the wall of PowerScore’s corporate offices). You also did not want us sharing information about this test with all future test takers until you had a better handle on student performance and test administration. The SAT may have been “designed for…transparency,” but by blocking adult test takers who are the watchdogs of the industry, you have failed in the execution of making it transparent.
So what does all of this mean? Mainly that you should avoid the SAT for the next six months or more while the College Board figures out how secretive it really wants to be. You have every right to know the format and content of an assessment test before it is administered, and since the test is playing hardball with informed students and test experts, you should be wary of what it will mean for you and the admissions officers who receive your results. I hope that someday in the future the SAT will become the “profoundly transparent” test it proclaims to be, but its current surreptitiousness (ooh! One of those SAT words they said you’d never need in the real world) should not be at your expense. Take the ACT instead.
Image: A lock, courtesy of Bilal Kamoon