Charles Murray, champion for general intelligence, defender of gender and racial differences of IQ scores, surprisingly argues that the SAT should be abolished. His argument involves three premises:
- Achievement test scores predict first year college GPAs better than SAT scores.
- The rabid focus on SAT by students, parents, schools, and the coaching industry is harmful.
- And the cognitive stratification of the American society.
I’m going to focus on (1), and argue that Murray’s defense smacks of a shell-game.
Murray argues that the SAT is redundant if students also take the College Board’s achievement tests. Where the SAT-I (and the new version) tests verbal and mathematical ability, the SAT-II tests mastery of specific subjects. And the SAT-II predicts freshman GPAs better than the SAT-I according to a University of California study:
Achievement tests did slightly better than the SAT in predicting freshman grades. High school grade point average, SAT scores, and achievement test scores were entered into a statistical equation to predict the grade point that applicants achieved during their freshman year in college. The researchers found that achievement tests and high school grade point each had about the same independent role—that is, each factor was, by itself, an equally accurate predictor of how a student will do as a college freshman.
But the SAT’s independent role in predicting freshman grade point turned out to be so small that knowing the SAT score added next to nothing to an admissions officer’s ability to forecast how an applicant will do in college—the reason to give the test in the first place. In technical terms, adding the SAT to the other two elements added just one-tenth of a percentage point to the percentage of variance in freshman grades explained by high school grade point and the achievement tests.
What about students from families with low incomes? Children of parents with poor education? Here’s another stunner: after controlling for parental income and education, the independent role of the SAT in predicting freshman grade point disappeared altogether. The effectiveness of high school grade point and of achievement tests to predict freshman grade point was undiminished.
I don’t doubt the study’s claims or Murray’s conclusion, but you and I both know why the achievement tests predict so well: they are more g-loaded than the SAT. That is, they are better assessors of the students’ cognitive abilities. The post-1995 SAT-I was heavily massaged to reduce the gender and racial score disparities due to politically correct pressure in the 1980s and early 1990s. The pre-1995 SAT was much more g-loaded, evidence being its ability to discriminate very high IQs due to a higher testing ceiling. However, Murray anticipates this objection and points out:
Did the pre-1994 SAT measure something importantly different from what the post-1994 SAT had measured? Don’t bother asking the College Board. The data for answering that question would require the College Board to reveal just how well the original and revised SATs measure the general mental factor g, the stuff of intelligence/aptitude, and the College Board does not want to acknowledge that the SAT measures g at all or, for that matter, that g even exists.
Seen from an outsider’s perspective, the changes in 1993–1994 do not look particularly important. Twenty-five antonym items in the SAT Verbal were replaced with reading-comprehension items, on grounds that the antonym items could be compromised by students who memorized vocabulary lists. The math test saw some changes in the answer format. But samples of the new items appear to be plausible measures of g and not obviously inferior to the items they replaced.
Despite the College Board’s rhetoric about revamping the SAT to reflect curriculum, the changes in the test in 1993–1994 probably did not have much effect on the SAT’s power to measure g—in the jargon, its g-loading. (I would not make the same statement about today’s SAT, which has eliminated the highly g-loaded analogy items and added a writing component that carries with it a multitude of scoring problems.)
If I am wrong, and the pre-1994 SAT measured g much better than the SAT used for the UC study, then I hope some disaffected College Board psychometrician leaks that news immediately. I will thereupon join a crusade to restore the old SAT. But given the available information, I think it is probable that even analyses conducted prior to the revisions in the test would not have shown a major independent role for the SAT after taking high school transcript and achievement test scores into account. To put it another way, those of us who thought that the SAT was our salvation were probably wrong. Even coming from mediocre high schools, our scores on achievement tests would have conveyed about the same picture to college admissions committees as our scores on the SAT conveyed.
I’m sure the pre-1995 SAT is more g-loaded than Murray is willing to admit. And I’m also sure that his assessment is correct such that adding the pre-1995 SAT to a battery of achievement test scores and high-school GPAs will add little in better predicting freshman GPAs. But that’s because the College Board is performing a shell-game, and Murray knows it. The shell-game consists of moving the heavy g-loading from the pre-1995 SAT to the SAT-II tests while calling them “achievement tests.” The SAT-I is then massaged to placate the politically correct minded. We can focus less on the SAT-I, which wrongfully emphasizes “ability,” and focus more on achievement tests, which rightly emphasizes “mastery of specific subjects.” But this is a ruse; the achievement tests test cognitive ability better than the SAT-I.
If the status-quo continues, then Murray is correct—we should do away with the SAT-I while keeping the SAT-II achievement tests. However, premise (2) will continue to be true simply for the coaching industry will shift their focus from the SAT to the achievement tests. Students, parents, and universities will continue to stress, and the politically correct minded will have something new to complain about. What we truly need to abolish is current educational system. But that for another discussion.
Aunt Haley
29 July 2010
Having taught the SAT recently, I still think it is very much an intelligence test. No amount of coaching, tutoring, practice questions, and mock tests will enable a student of average intelligence to get significantly above-average scores. Mentally average students just don’t possess the quickness/cognitive ability required to excel on a test like the SAT within the time given. I think the SAT (and standardized testing in general) absolutely separates the wheat from the chaff in that respect.
What the SAT doesn’t measure is a student’s dedication, hard work, or good behavior. So Susie Plodder, who faithfully does her homework every night, does every extra credit assignment, knows the answer to every question in class, is on student council and is the president of three extra curricular clubs, could do just okay on the test, while Chip Slacker, who gets Cs and thinks school is boring, could slay the test. Some people think this is unfair because it doesn’t reflect the “real world” performance of either student. This claim has some merit – some people are just bad test takers, and there is a difference between having real world smarts and academic smarts – but university-level study requires a certain level of mental rigor, and being a responsible student is NOT an indication of such.
What is going on with the calls to dumb down or abolish the test is a bleeding heart desire to level the playing field across all economic and racial demographics. But the truth is that not all people’s intellects are created equal, and the SAT is the fairest and quickest way to rank students. Otherwise we are just left with a bunch of subjective calls, which is not a very efficient or even particularly accurate method of determining college admission.
Lover of Wisdom
30 July 2010
On your second point, schools use SAT scores and GPA to measure “ability” and “dedication.” Murray would agree.
Are you an SAT test prep teacher, Aunt Haley? How have you found the Reading test as far as difficulty (compared to the SAT-I) now that it doesn’t have the analogies and antonyms sections?
Aunt Haley
30 July 2010
The Reading test is still rigorous. Currently the test is composed of three sections: (1) grammar/usage, (2) short reading comprehension, (3) long reading comprehension. A student who does not have strong fundamentals in these will not do well. A six-week course can’t undo a lifetime of bad grammar, nor can you teach a person advanced reading skills in six weeks. It’s not unusual for a student of average intelligence to get half of the questions wrong in practice.
The now-defunct analogies and antonyms sections are roughly covered in the grammar/usage questions where students must “fill in the blank(s)” in a sentence based on the word choice. Without a large vocabulary, this section can be very difficult for students because they must be sensitive to sentence tone and be able to handle complex sentence structure.
Lover of Wisdom
31 July 2010
Well, it seems they were able to hide the g-loaded material in the new sections. Has the math section gotten harder or easier?
Aunt Haley
1 August 2010
I think the math section is pretty rigorous. There are tricks you can learn, but most students, especially those of average or lower intelligence, will not be able to have those tricks on hand and under pressure be able to think straight enough to put them into effect. In order to do well on the math portion under the time constraints, you really have to have the kind of brain that can quickly cut to the heart of the question and instantaneously assess the method required to get the answer.
I feel like there’s this insinuation that the test used to be hard, and now it’s not. This is not (imo) the case. It’s still hard, and being a classroom drone is not enough to excel.
Lover of Wisdom
2 August 2010
Well, it’s not so much an insinuation. The SAT before the pre-1995 recentering was harder. It had a higher ceiling than the post-1995 SAT-I. But this isn’t to say that the new SAT is easy. I’m sure it is still hard, just not as hard as it used to be.
Anonymous
9 June 2011
why doesn’t the US have standard national examinations in place of this school-by-school GPA bullshit?
Passingthrough
5 August 2011
Did the study control for the fact that different schools have vastly different expectations and grading schemes?
Most state universities are nowhere near as rigorous as private ones; they simply can’t afford to be. I wouldn’t be at all surprised if the results of the study could be explained by this fact alone. For example, Susie Plodder may have been near the top of her class in a rural high school, and had a 4.3 weighted GPA. But her SAT scores were just average, so she only secured a spot at Podunk State U. Students like Susie do very well at these schools, where the emphasis is on rote learning and professors tend to “teach the test.” So let’s say Susie has a 3.9 GPA at PSU, which is far higher than her SAT score predicted. If Susie were to transfer to Yale or Dartmouth, this effect might completely vanish: she in all likelihood would not able to keep up with the pace and depth of the coursework, and her GPA would drop to a 2.4 (she’s still plodding along, working hard!)
By the same token, Chip Slacker was not a valedictorian, but his phenomenal SAT scores secured him a spot at a top tier liberal arts school. Small class sizes, lots of competition. He’s still no Susie, but Chip finds his coursework interesting, and is intellectually engaged. He studies just enough to get a 3.3, but he aces the LSATs, and gets into Fordham Law. Susie ends up bombing the LSAT and going to a paralegal program.
SATs, in this scenario, obviously predicted the long-term “winner”, if you consider “winning” to be making more money. Anecdotally, I’d say I’ve seen this scenario play out dozens of times. The predictive power of the SAT is, if you control for school difficulty, very good. If it weren’t, one imagines schools would’ve abandoned them a long time ago and looked at GPA and class-rank alone. SAT is used because it measures the ceiling of a student’s intellectual potential, and not their ability to memorize facts and brown nose teachers.