What is an adaptive test? How should you change your study strategy to prepare for the GMAT Computer Adaptive Test (CAT)? Computer adaptive testing definitely puts pressure on your testing experience and forces you to adapt your test-taking strategy in small ways. It’s especially important to understand how the GMAT calculates your score and what this means for you as a test-taker.
Fact: The GMAT uses Computer Adaptive Testing (CAT)
This means, first of all, that each question you answer correctly or incorrectly determines what questions you will see later in the GMAT. It also means that any two people, even two people of nearly identical abilities and preparedness, will not see identical questions when they take their respective GMATs. BUT, because of the magic of psychometrics, two people who perform with comparable skill & strategy & focus will have comparable GMAT scores. The magic of how computer adaptive testing is different for everyone but the score is fair for everyone—if you don’t have a Ph.D. in Psychometrics or Statistics or something like that, then just take that magic as an article of faith.
It’s also very important to point out: only the GMAT Quantitative and Verbal sections are computer adaptive. The Integrated Reasoning is NOT computer adaptive, and of course, the AWA essay is not. Everything I say in this blog about the CAT concerns only the Q & V sections.
What is an adaptive test and how does the GMAT CAT work?
The GMAT is trying to figure out objectively your Quantitative Ability and your Verbal Abilities in a relatively short time. Think of computer adaptive testing as a big “twenty questions” game. Suppose your “opponent” picks a US city, and you are allowed to ask “horizontal yes/no questions” (e.g. “Is your city east/west of X?”) and “vertical yes/no questions” (e.g. “Is your city north/south of X?).
You might ask a bunch of horizontal questions. It is west of Albuquerque? No. Is it east of Atlanta? No. Is it east of Denver? No. Is it east of Santa Fe? No. OK, that narrows things down to a relatively thin vertical band.
Then a bunch of vertical questions. It is North of Wichita? No. Is it north of Birmingham, AL? No. It is north of New Orleans? Yes. Is it north of Tucson? No.
Among major cities, those answers are enough to home in on El Paso, TX. Much in the same way, the GMAT asks you two question types, Math and Verbal. By giving you easy and hard questions of each kind, the computer adaptive testing homes in on what is most appropriately your level.
How accurate is the Computer Adaptive Test?
The “twenty questions” analogy is helpful for understanding computer adaptive testing, but the latter is for more complex: after all, what the CAT is measuring is not cut-and-dry exact. If we want to know where a city like El Paso is, that’s totally objective, and the questions about whether such-and-such city is N/S or E/W of El Paso are also totally objective. That means, with very few yes/no questions, one could home in on an exact location.
Someone’s math & verbal ability is not so precise a thing. First of all, there are easy questions you definitely can answer, there are super-hard questions you definitely can’t answer, but for the questions in-between, it’s gray: there’s a difficulty level at which you usually get questions right, and another slightly higher level at which you usually get the questions wrong. For the sake of argument, let’s say that we have figured out questions that are exactly at your ability level if, on average, you get questions at that level right 50% of the time. Clearly, whether you answer any one question correctly or not is not enough information to tell whether it’s at your ability level or not; determining your level is going to be about an average over several questions, not simply the answer to one. Furthermore, there are frequent aberrations.
Super-brilliant people sometimes get an easy question wrong, and folks who are minimally prepared can still guess correctly on one of the toughest questions. With statistics, the computer can absorb such aberrations. What the computer is doing throughout your test is averaging over the difficulty ratings of all the previous questions, using the data about which you got right and which you got wrong to create a complex average that is the best estimate of your ability, and each new question it feeds you is the computer’s attempt to refine that best estimate. This process mostly involves questions around your level, but in its sweeps, the algorithm sometimes will give you a really hard or really easy question as a check.
Your score is a composite result that takes into account the difficulty of each question you got right and the difficulty of each question you got wrong. The exact details of the algorithm that the computer uses to do this are (a) probably incomprehensible if you don’t have a Ph.D. in Psychometrics, and (b) the secret proprietary information of GMAC. Legally, we can’t get access to that algorithm, and even if we could, we probably wouldn’t understand it anyway. Rest assured, though: it is accurate. 🙂
What does it imply about the accuracy of the computer adaptive testing if a person takes and retakes the GMAT multiple times, getting a different score each time? This empirically observable fact doesn’t compromise the accuracy, the statistical robustness, of the computer adaptive testing itself. Think about it this way.
Suppose, I walk into the GMAT on the first Tuesday in March. On that particular day, I have slept a certain amount over the nights of the preceding week. On that particular day, I have eaten whatever foods I have eaten over the past several days, and have had whatever alcohol or other toxins during the past week. On that particular day, I have whatever combination of life-stresses churning around in the back of my mind. On that particular day, I may feel healthy, but my immune system is in the process of fighting off whatever it is fighting off. On that particular day, my hormones may be at any particular level: this fluctuates somewhat more predictably in females and considerably less predictably in males. On that particular day, I will have whatever additional stress about the test itself, about my relative level of preparedness or lack thereof, my general level of self-belief and self-confidence, etc. All of these factors play into one’s GMAT readiness. The GMAT accurately records how ready I was for the GMAT that particular day.
Suppose I don’t do well on that sitting, then take a course, such as the Magoosh course. Then I schedule my second attempt, say, on the last Friday in May. The extra experience, the deeper understanding, and the familiarity with the test-taking procedure may boost my confidence considerably. Again, my performance will depend in part on all these other factors, but if I learned a great deal more, in all likelihood, I will be more ready for the GMAT, and the computer adaptive testing will accurately measure the GMAT readiness on this date.
Hold on! It sounds as if Mike is saying that all the GMAT measures is how well you can take the GMAT! Well, yes and no. It is not a pure intelligence test, insofar as one can take a course and improve: for example, Magoosh has a 50-point score guarantee. Also, eating well, getting adequate sleep, and practicing stress reduction all can help you put your best self forward on test day. Nevertheless, the folks at GMAC, the place that produces the GMAT, will tell us that, as a statistical tool, the GMAT CAT is not only robust but valid. To be statistically valid means that the test has predictive power: in particular, the GMAT is very good at predicting how one will handle the academic demands of business school. If by studying you make yourself more prepared for the GMAT, unwittingly you have also made yourself more prepared for your future business school classes.
Facts vs. Myths about Computer Adaptive Testing on the GMAT
Fact: If you get medium questions mostly right, the computer will start to feed you harder questions; if you get medium questions mostly wrong, the computer will start to feed you easier questions.
In general, this is true: this is the overall general pattern. The computer adaptive testing adjusts to your level throughout—much like the E/W and N/S question in the geography game above, it is constantly refining its picture of your ability, question by question.
Myth: If I suddenly get a ridiculously easy question, that means I got the last question wrong.
First of all, a question that seems easy to you may or may not actually be a truly “easy” question, that is, one that most people get right; it may just be that you saw a shortcut that most people don’t see.
Even if the question is, by objective standards, an easy question, no conclusion can be drawn about the previous question. The computer adaptive testing is running a complex algorithm, which sometimes involves giving you a very easy or a very hard question. Don’t take it personally: the adaptive testing is just running its algorithm.
Fact: You can get several questions wrong and still get a good score.
The computer adaptive testing has to give you several questions well above your ability, questions that you almost invariably will get wrong, in order for it to zero in on your actual ability. You are not penalized for that: that’s just what the computer adaptive testing must do as part of its algorithm.
In fact, here’s the irony: the person who gets a 460 and the person who gets a 780 may get approximately the same number of questions right & wrong. The number of questions correct, by itself, indicates nothing: all that matters is the difficulty of the questions. The person who got a 460 probably got several relatively easy questions wrong but even easier question right. The person who got a 780 got many super-hard questions correct but even harder ones wrong. The computer adaptive testing has an extremely sophisticated way of finding your level and customizing the analysis to someone at your level.
Myth: The first question or first five questions are super-important, because these determine the course of easy/hard questions from there.
Totally false. The computer adaptive testing is performing a complex process of estimation that can handle aberrations, even if one of the aberrations happens on question #1. Don’t worry: over the course of the whole test, the computer will give you the combination of questions it must in order to determine your abilities. Furthermore, the algorithm is such that order of the questions doesn’t affect your score at all. If you get a certain question right then whether it was the first question, a middle question, or the last question, doesn’t matter at all. What does matter for your score is the difficulty of the question, and whether you got it right or wrong, but not where it fell in the test.
This is a very common misconception, that for some value N, the first N questions are really important. First of all, objectively, this is patently false, but even disregarding that, think about it. What concrete strategy would this imply? Would it make sense to “try harder” on the first N questions? Wouldn’t that imply trying less hard on the rest, the majority of the test? By approaching a section with the thought that the first N questions are somehow “the most important,” typically all the student thereby accomplishes is to generate a great deal of stress right at the beginning of the section, a result that will almost invariably have counterproductive results.
You see, as far as effort is concerned, the only strategy that makes any sense at all is to bring your absolute best effort to each and every question that you see. That is the only strategy that makes sense, and as it happens, the computer adaptive testing is 100% designed to support such an approach. Excellence can be consistent when it is a habit: for more on this, see the discussion of the habits of excellence.
Fact: Not finishing all the questions in a section hurts your score.
That is quite true. It’s exceedingly important not only to learn content and strategy, but also to practice working efficiently so that you don’t run out of time. Ideally, you want to hone your time management skills so that you have abundant time on even the last questions on a section.
Myth: You can outsmart the CAT.
The algorithm is far too complex. There’s no sense stressing about “how did I do on those questions?” or “why is it asking this kind of question now?” Again just do your best on the question in front of you at any moment, submit it, and then forget about that question entirely.
Fact: Systematically reviewing math and verbal content, as well as strategies specific to each question type, can vastly enhance your GMAT score.
That is most certainly true, and that’s why Magoosh can give you such an advantage. With a couple hundred lesson videos discussing both content and strategy, and over 800 practice questions, each with its own video explanation, you will get top-notch preparation for the GMAT at only a fraction of what you would pay for a comparable course.
Editor’s Note: This post was originally published in February, 2012 and has been updated for freshness, accuracy, and comprehensiveness.