October Surprise? The "Texas Miracle" & Its Critics

[See also, at WEJB/NSU:

“The General Patton of the Testing Wars.”]
 

A Three-Part Series

Part I: The Texas Testing Controversy
By Nicholas Stix
October 31, 2000

(Just before the 2000 presidential election, the socialist MSM, which was willing to do anything to help get Democrat candidate and then-Vice President Al Gore elected president, sprang two traps – “October Surprises” – on Texas Gov. George W. Bush, the Republican candidate. The SMSM first published and broadcast reports that Bush had been arrested for driving while intoxicated over twenty years earlier, and then, on October 24, promoted a new Rand Corporation “report” that insinuated that the “Texas Miracle” (Bush's phrase) in education was due to massive, institutionalized test fraud.

The grand irony of the story is that the “Texas Miracle” was the stuff of fraud, but not of test fraud.

It was not until 2003 that the Houston affiliate of CBS News, KHOU, would expose the real bum steer in the Lone Star State, and that was thanks to the courage of whistleblower Robert Kimball, who was then an assistant principal at Houston’s Sharpstown High School.

It seems that through a variety of dodges, the Houston Independent School District (HISD), the largest in Texas, had inflated 10th grade test scores. The tricks included phony bookkeeping that reduced dropout rates that were as high as 70 percent down to 1.5 percent, and holding back passing but borderline students in the ninth grade, and eventually leapfrogging them straight to the eleventh grade, so that they would not take the state tests given to 10th-graders.

Had Rand's Stephen Klein and his fellow anti-testing zealots in academia and the media done the real gumshoe work involved, instead of merely repeating leftwing teachers’ ed school test-bashing talking points, they could have gotten their man, Al Gore, elected president, and changed history -- though we’ll never know if a President Gore would have done any better than the man who beat him.

Note that Rod Paige, the superintendent of the HISD under Gov. George Bush, was rewarded by Pres. Bush for the “Texas Miracle,” by being appointed Secretary of Education.

The alleged success of the "Texas Miracle" was then used as the rationale for the "No Child Left Behind" Act (NCLB), which Pres. George W. Bush signed into law on January 8, 2002, and which was to be the legacy of Bush's education presidency. When CBS uncovered the real scandal, Rod Paige refused to comment on it.)


Was George W. Bush caught cheating in school?

That's what four researchers at the prestigious Rand Corporation, a non-profit, education research organization based in Santa Monica, California, are claiming. In a report released on October 24, "What Do Test Scores in Texas Tell Us?," Stephen P. Klein, Laura S. Hamilton, Daniel F. McCaffrey and Brian M. Stecher suggest that there's something rotten in the state of Texas, and that the "Texas miracle" of educational progress, is really so much Texas bull.

Based on 1990-98 testing figures, Texas education officials had reported gains in reading, writing, and math among Texas public school children that tripled those made in the same subjects by children across the nation. And unlike the rest of the U.S., in which the racial educational gap between whites on the one hand, and blacks and Hispanics on the other, has in recent years been growing, in Texas it was shrinking.

Texas Gov. George Bush dubbed the reported gains "the Texas miracle," and made them a pillar of his Presidential candidacy.

(The national figures I referred to above were those of the National Assessment of Educational Progress (NAEP), the closest thing we have to a uniform, national educational exam, which is given in 44 states. The Texas figures were from the Texas Assessment of Academic Skills (TAAS), which is unique to Texas.)

Immediately, one thinks: My, what a coincidence, just two weeks before Election Day. And only three months after a different team at the same Rand Corporation had published a major study, which wholeheartedly praised and endorsed Texas' educational progress.

Suspicious coincidences aside, the question is, Did Texas educators cheat, and fudge the results of the Texas Assessment of Academic Skills (TAAS)? My initial judgement was that beyond partisan politics, we might never know the truth. But it didn't look good.

But accompanying the report, before it could even be criticized, Rand President James Thomson issued a combative press release insisting, "We don't produce findings for political reasons, we don't distribute them for political reasons, and we don't sit on them for political reasons. This is a scrupulously nonpartisan institution. He did protest too much.

As I read the report, I searched for evidence of cheating, of fraud, of misconduct ... and found none. Not one shred. What I found instead were loads of speculation, innuendo, and language weasely enough to fertilize all the onion fields in southern California.

And then I examined the orchestration of the media attacks on George W. Bush's education record going back months before the report's release, led by, among others, the "lead researcher" of the October 24 report, Stephen Klein, and I realized that there was no "aside" beyond the "coincidences." George W. Bush was the recipient of a political hit, pure and simple.

The guilty parties are, first and foremost, Rand Corporation "researchers" Stephen P. Klein, Laura S. Hamilton, Daniel F. McCaffrey and Brian M. Stecher, and Rand President James Thomson.

But they had help. Help came from, among others, Walt Haney, a Boston College professor of education, researchers Linda McNeil of Rice University and Angela Valenzuela of the University of Texas, and Washington Post reporter John Mintz, who provided the aforementioned players with the platform from which they initially attacked the TAAS. (Mintz, however, is the most morally ambiguous figure here, because he is a hero, too. Information he provided was crucial to this story.)

Walt Haney is a leader of the cabal of anti-testing ideologues among the radical professors who control virtually all of the nation's major teacher education programs, and which is now making inroads in independent testing and research organizations. Linda McNeil and Angela Valenzuela are likewise members of the "test bashing" cabal, in the phrase coined by education critic Richard Phelps. Politically, these professors are what is euphemistically referred to as "liberal" in some circles, "multicultural" in others, and what I call "racial socialist," since they combine socialism and racialism.

As we shall see, A) the "research" claiming to disprove the Texas Miracle is laughably incompetent and biased; B) the new report is contradicted by a much more thoroughly researched Rand report that was published just three months earlier, and whose lead author, David W. Grissmer has sharply criticized Klein & Co.; C) Rand Corporation President James Thomson's statement in defense of Klein, et al., has all of the credibility of New York Yankees manager Joe Torre's defenses of his pitcher, Roger Clemens,' attempts to maim the New York Mets' Mike Piazza; D) the leaders of academia's anti-testing subculture, e.g., Walt Haney, in Richard Phelps' words, "never met a test they liked"; E) and finally, we shall see the context of this "research"; a campaign — itself characterized by contradictory and outrageous statements — by anti-testing zealots to discredit George W. Bush's Presidential candidacy, and win the election for Vice-President Al Gore.

And yet with all that said, Gov. George W. Bush is not going to come out of this debacle smelling like the yellow rose of Texas, either. As we shall see, the TAAS is not without thorns. So pay attention, because there will be a test — on November 7th!


Part II: “Dr. Spin” Flips the Rand Report
By Nicholas Stix
November 3, 2000

George W. Bush's “Texas Miracle” in education is a “myth.” We know that, because Dr. Stephen P. Klein said so. Dr. Klein is the lead researcher of the report, “What Do Test Scores in Texas Tell Us?,” released on October 24 by the prestigious, "nonpartisan" Rand Corporation.

Meanwhile, the Washington Post, Los Angeles Times, ABC News, and a host of other prestigious, "nonpartisan" news organizations, have repeated Klein's charges.

Stephen Klein's Rand report examines dramatic claims made by Texas education officials and Gov. George W. Bush. The Texans had reported that on the Texas Assessment of Academic Skills (TAAS), public school children showed gains in reading and math that were triple the gains made by their peers nationally. During the period in question, 1994-98, nationally the racial gap, whereby black and Hispanic children lagged far behind their white peers, widened. According to Texas officials, however, their state's gap was narrower to begin with, and black and Hispanic Texas children had come close to erasing it altogether.

In the October 25 Los Angeles Times, reporter Duke Helfand quoted Stephen Klein as charging that, “The soaring test scores in Texas do not reflect real improvement in students' ability to read and do math. Texas is doing better than the rest of the country in some areas, but nowhere near the miracle. It's a myth.”

Duke Helfand wrote further, that “The findings [from the National Assessment of Educational Progress] led the analysts to suggest that the widely ballyhooed gains on Texas' own tests may have been the result of extensive test preparation, a low standard for passing and some cheating prompted by pressure the system puts on teachers and administrators.”

Based on the press accounts and my own experience as an educator, I was tempted to believe Klein. But then I read and re-read his report, which made a disbeliever out of me.

Many education reporters don't seem to read the reports they write about. Apparently, they read only the press releases, and talk to the reports' authors and PR flacks. In other words, they consent to being used by spin campaigns. I've found that the actual reports often fail to prove, and at times even contradict the claims made by the press releases, “spokespersons,” and even the authors themselves. The Klein report is no exception.

Keep in mind, that while the Klein report never uses the phrase “test fraud,” the report has no point, except as an extended (if unfounded) allegation of massive test fraud in Texas.

For a point of reference, let's consider another recent case in which test fraud was alleged. Last December, New York City Board of Education Special Commissioner Edward Stancik published a report, “Cheating the Children: Educator Misconduct on Standardized Tests,” charging that conspiracies of teachers, administrators, and staffers in 32 public schools all over the city had been engaging in massive test fraud.

The tests in question were high-stakes, standardized exams, just like the Texas Assessment of Academic Skills (TAAS). The phrase “high-stakes” means that students' promotions and graduation, and educators' careers depend on the results.

The most popular method was for teachers to have children write their answers on scrap paper. The teachers would then correct the children, before having them write their "final" answers in their test booklets. Some teachers and administrators would “prompt” pupils, telling them, “That's wrong.” Others would “explain” questions, while some would simply tell the children the proper answers, or distribute typed “cheat sheets.” One teacher wrote the conclusion to a student's essay. An ambitious Manhattan teacher, Dennis Rej, made wholesale erasures on his students' examination “bubble sheets,” and entered the correct answers. Some enterprising educators managed to procure the actual exams in advance, which they then used to prep their students.

One teacher from Queens, Robin Smith, told investigators, “everyone does this and I never had a problem with it before.”

The investigation resulted in fifty-two New York City teachers and administrators being suspended for test fraud.

The 66-page Stancik Report was full of smoking guns. Enjoying the cooperation of outraged teachers in the affected schools, the investigators had gotten eyewitness testimony, and gathered (and published photographs of) physical evidence such as “bubble sheets” full of erasures, essays in which a child's handwriting gave way to an adult's, and “cheat sheets.”

I searched Stephen Klein's report for smoking guns, but found only smoke and mirrors. No cheat sheets, no sheets full of erasures, no advance copies of exams, no eyewitness testimony, no cooperating educators.

Dr. Stephen Klein turns out to be a spin-doctor.

What he engaged in, was a textbook, three-step spin-doctoring campaign. What he didn't engage in, was research.

Klein sought to create an atmosphere of doubt about the Texas Miracle; exploit that atmosphere to rationalize undertaking what was not a research report at all, but in the words of Bush campaign communications director Karen Hughes, a “14-page opinion paper … [that] directly contradicts every credible, nonpartisan scientific evaluation, including Rand's own official study”; and then, confident that few prominent reporters would read his “report,” go in for the kill, as if his claims had actually been proven.

Step one: In a major Washington Post story in April, Klein suggested that the TAAS was fraudulent. He claimed that this was based on his research, but according to Rand's current story, Klein had just begun his research on TAAS in the spring.

Since Washington Post reporter John Mintz was writing an anti-Bush story, he didn't demand any proof of Klein's insinuations. And in getting a lengthy story published in one of the most influential newspapers in America, Klein could rest assured that most major media outlets would assimilate his claims.

Step two: In Klein's “report,” he observes that, “For example, the media have reported concerns about excessive teaching to the test,” as justification for his study. He neglected to reveal to the reader that HE WAS THE SOURCE of the media reports. (Disguising one's own previous statements as independent corroboration is a basic form of scholarly and journalistic fraud.)

As John Mintz reported in the April 21 Washington Post, six months before the Klein report's release, “‘We knew something strange was going on,’ Klein said. He believes that, without meaning to, Texas officials design TAAS tests so they're vulnerable to Texas teachers' coaching.”

Step three: The moment the “report” was released, Klein again spoke to reporters at major news organizations. Playing off the momentum he had built up in steps one and two, he denounced the Texas Miracle as a “myth.”

We cannot discount the help that mainstream media organizations gave Stephen Klein.

In July, the Rand Corporation had published a major, 250-page study, by a team led by David W. Grissmer, Improving Student Achievement: What NAEP State Scores Tell Us, that supported Texas officials' claims.

As thorough and well-documented as the Grissmer study was, Rand released it four months before the election, but somehow it didn't get much coverage. Conversely, Rand released the Klein paper exactly two weeks before the election, and it got tremendous play.

As we'll see tomorrow, the structure of the Klein “report” is much like that of the Klein spin campaign. In lieu of evidence, Stephen Klein uses suggestion and innuendo, and then, acting as if he had proved that which he only insinuated, he piles on ever more dramatic suggestions and innuendoes. Another favored technique has him equivocate in his use of terms such as “coaching,” “cramming,” and “test preparation,” so that they mutate from innocent, even positive words into euphemisms for “cheating.”

Sandra Stotsky, a veteran researcher at the Harvard School of Education (but don't hold that against her!), is the Deputy Commissioner of Academic Affairs and Planning of the Massachusetts Department of Education. She is the author of, among other works, Losing Our Language: How Multicultural Classroom Instruction is Undermining Our Children's Ability to Read, Write, and Reason.

Arguably the foremost scholar of “K-12” reading curricula, Dr. Stotsky had earlier been commissioned to perform her own study of TAAS' reading component, and found it wanting. Stotsky said that since her study, the TAAS reading component had reportedly been reformed.

Of Klein's Rand study, Sandra Stotsky remarked, “There are various groups attacking Bush, because it's Bush. It's a political thing. … [The Klein report] destroyed their academic reputation. They shot their own person [David W. Grissmer]. It made Rand look terrible.”

Unfortunately, Stephen Klein is more interested in manipulating reality, than in discovering and describing it. That's why I call him, “Dr. Spin.”



Part III: Rand Researchers' Cheatin' Hearts
Nicholas Stix November 6, 2000

On October 24th, the Rand Corporation released its now notorious report questioning the gains that Texas public school children had made, as measured by the Texas Assessment of Academic Skills (TAAS). The report suggested that the "Texas Miracle," as Gov. George W. Bush had dubbed it, was closer to a hoax.

So far, I have examined the spin-doctoring campaign that the Rand report's lead researcher, Dr. Stephen Klein, whom I dubbed, "Dr. Spin," engaged in, with the help of mainstream news organizations.

While littering the 18-page "report" with suggestions and innuendoes, and playing word games, all to insinuate the existence of a regime of test fraud in Texas, Klein failed to offer one shred of evidence that Texas educators or officials were engaging in test fraud. In my last column, I showed what a real report on test fraud looks like — the 66-page, December, 1999 Stancik Report on massive test fraud in New York City's public schools.

The Klein Report claims to have two foundations: A study of students' performance on math tests that Klein and his research team had administered in 20 Texas schools in 1996, and a comparison of Texas and national scores on the National Assessment of Education Progress (NAEP) during the same period.

When Stephen Klein spoke to Washington Post reporter John Mintz in April — when his research team's study, according to Rand's official story, had just begun — he alluded to his team's study of Texas students' performance on math tests as the basis for his suspicions: "We knew something strange was going on."

But in the report itself, Klein denies the validity of his team's math test results: “Because our set of schools was small and not representative of the state, we decided to explore statewide patterns of achievement on TAAS and on NAEP.”

In simple English, the first study was a botch: “small and not representative.” So, how does a botched, 1996 study suddenly become proof that “something strange was going on?” When a Washington Post reporter is on the other end of the telephone line.

And how does a botched study become the basis for a second study trying to prove the same thing? When the White House is in danger of being lost.

In the case of the National Assessment of Educational Progress (NAEP), Klein isn't on quite such shaky ground. Texas eighth-graders did not blow away students across the country on the NAEP to the degree that the TAAS figures say they did, nor did Texas fourth-graders taking the NAEP reading test.

Those test groups of Texas students made some progress on the NAEP, but so did students everywhere else. And the racial gap that supposedly had been almost erased on the TAAS, persisted among Texas students on the NAEP.

However, on the NAEP, Texas fourth-graders of all races DID blow away their peers nationally on the math section, posting gains that were double those of children outside of Texas. And it was the math scores that had aroused Klein's suspicions. And yet, he turns this praiseworthy fact into an indictment: “The main finding is that over a four-year period, the average test score gains on the NAEP in Texas exceeded those of the nation in only one of the three comparisons, namely fourth grade math.”

The NAEP is so important, because it is the closest thing we have to a national exam; it is given voluntarily in 44 states, and as the Klein Report notes, is considered the “gold standard” of testing. (The NAEP's nickname is, “the nation's report card.”) Meanwhile, the various state examinations are all different, and thus not comparable with one another — or with the NAEP.

As influential, neo-conservative education scholar Chester E. Finn Jr. pointed out in an October 27 New York Times op-ed essay in defense of the TAAS, “Mr. Klein neglected to provide some vital context. It is normal for state tests to show better results than national ones. There are straightforward reasons for this: State tests are more narrowly designed; they test more basic skills; they intentionally align themselves to the state standards and curriculums (which national tests do not); and they provide more incentives, like grade promotion, for students to do well.”

Klein chooses to emphasize that gains were less for eighth-graders, but neglects to mention that in an intensive academic environment, it is normal for gains to be larger, the younger the students. The older students are, the harder it is for them to catch up.

Klein's method is one of suggestion, innuendo, and word games. He claims to smell smoke, and insists that the odor is proof that there is a fire, but in fact, he is the source of the smoke.

Klein writes, “For example, the media have reported concerns about excessive teaching to the test, and there is some empirical support for these criticisms.”

As I showed in my last column, Klein himself was the source of the media reports.

John Mintz of the Washington Post reported such “concerns” on April 21: “He believes that without meaning to, Texas officials design TAAS tests so they're vulnerable to Texas teachers' coaching. He also thinks that kids who 'prepped' for TAAS not only didn't get a deep understanding of the subject, but also weren't helped to pass non-TAAS tests.”

Klein's deeply problematic charge that the “TAAS tests ... are vulnerable to Texas teachers' coaching” implies that a legitimate test is impervious to coaching, and thus, that “coaching” is a form of cheating. Both implications are false, and thus misleading. Indeed, the Klein Report claims that the scores by the coached students are “inflated.”

Klein Report: “However, some educators and analysts have raised questions about the validity of these gains and the possible negative consequences of high-stakes accountability systems, particularly for low-income and minority students.”

“Raised questions”? “Possible negative consequences”? And why “particularly for low-income and minority students”?

The point of a narrow, scientific study is not to RAISE questions, but to ANSWER them, and to determine not whether a policy has POSSIBLE, but if it has REAL, negative consequences.

The heightened concern for “low-income and minority students” — a redundant phrase, since in the antiversity and in edworld, “low-income” is a euphemism for “minority” — is a cliche, like the phrase, “for the children.” (Actually, when it comes to education writing, “conservatives” and “liberals” alike profess exaggerated concern for black and Hispanic students. Apparently, nobody gives a hoot about white or Asian students.)

And the hits just keep on coming:

“There are also concerns that score trends may be biased by a variety of formal and informal policies and practices.”

“Concerns”? “May be”? Again, Klein provides no evidence of “bias,” which in the context of testing is a euphemism for “fraudulent,” “illegitimate,” “invalid.”
“Another concern is inappropriate test preparation practices, including outright cheating. There have been documented cases of cheating across the nation, including in Texas. If widespread, these behaviors could substantially distort inferences from test score gains.”

Klein is trying, through verbal sleight of hand, to go from “documented cases of cheating” outside of Texas, to the implication that — in the absence of any such documentation, widespread cheating in Texas has distorted TAAS test scores. “If ... could” is in the realm of possibility, not reality. IF I were seven feet tall, I COULD play in the NBA.

Although Klein mentions “documented cases” in Texas, he gives no specifics. That is because the cases were so limited, that they fail to bolster his case.

In New York, massive test fraud was uncovered, despite a lack of any pressure by anti-testing zealots to uncover the scandal.

Conversely, in spite of massive scrutiny by anti-testing zealots and the mainstream media, no major testing fraud scandal was ever uncovered in Texas. [N.S. 2007: I should have used “hostility” instead of “scrutiny,” since had the zealots and the media not been so lazy, they would have uncovered the real scandal.]

Stephen Klein then builds on his previous imaginary proof, upping the ante in the next paragraph to “the pressure to raise scores may be felt most intensely in the lowest-scoring schools, which typically have large populations of low-income and minority students.”

In other words, ‘I didn't prove my earlier point about widespread cheating, and now I'm going to build on that, by insinuating that the unproven or non-existent cheating's greatest prevalence is in black and Hispanic schools.’

Word games: “Evidence regarding the validity of score gains on the TAAS can be obtained by investigating the degree to which these gains are also present on other measures of these same general skills.”

Wrong again! Substitute for “evidence,” “suspicion.”

Seeing a discrepancy between the same group's scores on DIFFERENT exams is a reason for investigating IN SEARCH OF evidence of test fraud. Large discrepancies by the same group on the SAME test would arouse very strong suspicions, but even then, one would still have to prove one's case. The fact that the tests (TAAS and NAEP) are QUITE DIFFERENT, makes the case for test fraud harder, not easier to prove.

To my knowledge, in the early 1970s, multiculturalists began abusing the pseudo-scientific theory of “disparate impact,” which is central to affirmative action (aka “diversity”), to gut scientific standards of evidence in the social sciences and the already intellectually dubious field of education.

In the politically correct antiversity and in edworld, the mere existence of a statistical disparity between an officially defined “oppressor group” (white, heterosexual, able-bodied males) and an officially defined “oppressed group” (anyone else) has become accepted as “proof” of discrimination. Requesting evidence that the “disparate impact” in question is in fact due to white racism, as opposed to other factors (black racism, parental neglect, incompetent teachers, etc.), has long resulted in questioners being derided as “racists,” and fired from their jobs. That is why, one “goes along” in today's academic world.

More word games: In his report, Klein always uses “coaching,” “prepping,” and “intensive preparation” in a pejorative fashion, suggesting they are forms of cheating. Part of this web of suggestion is his reference to the use of older TAAS tests, and drilling for the TAAS using questions similar to those on the exam. And then there is the criticism, pervasive today in edworld (especially in the Anti-Testing Brigade) and the mainstream media, of “teaching to the test.”

Call it “coaching,” “prepping,” or “teaching to the test” — not only is this practice not a form of cheating which inflates test scores, but it is every teacher's duty, a duty I felt acutely during seven years as a college instructor teaching remedial and allegedly college-level courses.

And it is standard practice to use questions similar to those that will be on the exam, culled from past exams. A case of fraud or “bias” would only obtain, if a teacher used the questions from the actual test his students were about to take.

To read Stephen Klein, you'd never know that a multi-billion-dollar learning and test preparation industry existed, that middle-class and well-to-do students of all colors and ability levels routinely used, often in addition to private tutors, to increase their test scores.

Apparently, only poor “students of color” are to be denied such advantages. (Oops, there I go racially pandering! There's just no escaping it!)

And the commonsensical “tricks” students learn through their TAAS preparation — such as reading the test question before reading the passage it tests — are things all students should know. Besides, they are nothing, compared to the sorts of tricks that journalists and education scholars routinely depend on.

Finally, the report does not cite a single sample question from the TAAS. Many journalists did give examples of these questions, and used them to offer more scientifically-grounded explanations of the TAAS scores than Stephen Klein did.

At the same time, journalists Klein had spoken to gave — much like Klein himself — analyses of the Klein Report that went far beyond its actual contents. And the anti-Bush newspaper articles almost always contained quotes from the same anti-testing zealots — e.g., Angela Valenzuela of the University of Texas and Linda McNeil of Rice University — whom Klein cited approvingly in his report, even though in it, he adamantly denied being from the anti-testing camp.

Note too that the anti-Bush articles that immediately followed the release of the Klein Report combined analyses of the report's criticisms of TAAS (saying literally what the report only insinuated) with additional criticisms of TAAS by Valenzuela, McNeil, and other, unnamed sources, that bore no relation to the report, which often were irrelevant to it, and which in some cases flat-out contradicted Klein's criticisms.

However, a reader who had not studied the Klein Report, and who had a limited knowledge of the world of testing, would be hard-pressed to cut through the thicket of propaganda.

Originally Published in Toogood Reports.