Press "Enter" to skip to content

Standardized Tests Value Common Benchmarks over Classroom Excellence

An eager reader shares Florida scientist and science popularizer Rob Krampf's discussion of errors in the Science FCAT, a standardized test administered to all Florida fifth and eighth graders. The FCAT is high-stakes testing of the sort that South Dakota will impose on all K-12 students to carry out the mandates of Governor Daugaard's merit pay and math/science bonus plan, HB 1234.

While studying the fifth-grade test item specifications and sample questions to help students prepare for the test (right there we see on of the major problems with such testing, the adoption of test success over student engagement and learning as our goal), Krampf found a number of errors in the questions. He spotted incorrect definitions (one makes grass a "predator") and this flawed question:

This sample question offers the following observations, and asks which is scientifically testable.

  1. The petals of red roses are softer than the petals of yellow roses.
  2. The song of a mockingbird is prettier than the song of a cardinal.
  3. Orange blossoms give off a sweeter smell than gardenia flowers.
  4. Sunflowers with larger petals attract more bees than sunflowers with smaller petals.

The document indicates that 4 is the correct answer, but answers 1 and 3 are also scientifically testable [Rob Krampf, "Problems with Florida's Science FCAT Test?" The Happy Scientist, 2012.04.08].

Good citizen that he is, Krampf notified the state Department of Education of these errors. The state's test developers said the flawed definitions passed muster with their "Content Advisory Committee," so they must be o.k. As for the questions with multiple correct answers, the DOE said that the designated correct answer properly measures the benchmarks that are supposed to be taught by fifth grade. The benchmarks that would allow kids to know that the other two answers are correct aren't covered in the curriculum until later.

In other words, teachers who help their kids learn more than the benchmarks specify may end up causing their kids to get more wrong answers. Says Krampf:

I wonder how many students got "wrong" answers on the FCAT because their teachers taught them too much. How many "F" schools would have higher grades if those scientifically correct "wrong" answers were counted as correct answers. How many "B" schools would get the extra funding that "A" schools get, if those scientifically correct "wrong" answers were counted as correct answers? [Krampf, 2012.04.08]

Tie your standardized tests to teacher pay and funding, and you don't get better teacher performance and student outcomes. You get cookie-cutter lessons aimed strictly at test performance, even when the tests are wrong, and not at pushing kids to excel.

12 Comments

  1. David Newquist 2012.04.17

    In that sample question, one can see what the writer was trying to use as the distinguishing factor: personal taste, preference, and perception versus something that can be objectively tested. The test writers were not thorough and competent enough to critically examine the ambiguities and inconsistencies of the samples. In the cited case, it takes no degree of expertise to see the flaws.

    One can see the devastating result that the business model is having on education and the intellectual potential of the country. One is alerted by the dreadful jargon, i.e. "benchmark" and a content advisory committee of presumption witlings forging standards that fit the pseudo-science of MBAs, but can't stand up to even the feeblest muster of genuine cognition.

    At some point, true educators have to face the harsh fact that they cannot prevail over the posturing and bluffing that makes up much of the criticism of education. Rob Kempf puts his finger on the real problem with education, and it is not the teachers or the curriculum. It is rooted in that part of the political spectrum that thinks that anyone who actually knows valid information and can engage in actual critical thought is an elitist. It is another aspect of the class warfare that is trying to make serfs out of what once was a middle class.

  2. Bill Fleming 2012.04.17

    The correct answer is "all of the above." Every single one of those observations can be tested scientifically. Questions like that drive smart kids nuts. And the idea that teachers teach kids everything they know is ridiculous.

  3. David Newquist 2012.04.17

    Should read "presumptuous witlings."

  4. Steve Sibson 2012.04.17

    The standards are not about right and wrong, they are about indoctrination. This has been going on for decades. Want to pay teachers more, then cut the standards. which obsoletes the federal, state and University Departments of Education. How much money is that nation wide?

  5. David Newquist 2012.04.17

    Bill,

    You've added another great but devastating piece of evidence of how incompetently conceived testing exercises are demolishing education. There are multitudes of prompts devised over the years for eliciting writing from students. That teacher's letter describes another example of how some management model has been imposed on educators and how such incompetently devised exercises are the real issue in defining problems in education.

  6. tonyamert 2012.04.17

    The problem with using standardized testing of students to determine "teacher performance" is that it's like taking someone's temperature to determine if they small pox. Neither test directly indicates the conclusion being drawn. These are both symptoms but there are many other potential causes.

    A high temperature could also indicate the flu or any other number of diseases. Low student performance could indicate poor nutrition/lack of parental involvement, etc. The problem is that people are trying to create a one dimensional problem from a multivariate system.

  7. Troy Jones 2012.04.17

    I know Steve you see masonry and new agey stuff in much but really is there anything bad with regular standard testing to measure:

    Knowledge of Science, math, reading, writing?

    If so, what would you be the accountability measures or do you think there shouldn't be any.

    I don't think people are saying they are opposing standard testing (well Newquist appears to be but he might oppose any measure for teachers). They are just saying it shouldn't be the sole measure. I agree and so do most other people.

  8. David Newquist 2012.04.17

    What I'm saying, first and foremost, is that tests have to competent in design and purpose. Making a decision about anything on the basis of faulty tests, as the ones referred to here are, is fraudulent and destructive of anything that education can accomplish. If utter incompetence can be sentient enough to make it fraud.

    And proposing measures to test students and evaluate teachers which counter everything we know about the cognitive and pedagogical processes is, yes, something I am against. Most of the schemes currently proposed have nothing to do with education, but everything to do with "managing" teachers. And the examples alluded to above are so uninformed and incompetent that it is like the meat cutter's union writing and enforcing the standards and procedures for brain surgeons. More and more, it is apparent that the problems in education are not with the teachers. They are with the people who want to reduce them to a status of abject servitude and with the administrators who carry out the policies. Stupid schemes do not make smart children.

  9. Bill Fleming 2012.04.17

    In the examples given here, the important thing is that the people designing the tests have to be at least as human and intelligent as the people giving and taking it. i.e. a flawed test is going to yield flawed data. Garbage in, garbage out.

  10. Donald Pay 2012.04.17

    My daughter always did well on bubble tests. She told me that the key for many questions was not necessarily to fill in the correct answer, but to reason toward what the test maker would think to be the obvious answer. Standardized tests don't measure what you know only, but also how well you can "psych out" what the people who make up the tests want you to answer.

    The psychology of people in the standardized test business is pretty clear. Their goal is to find a battery of questions that will separate people along a normal distribution. To do that they need questions that they think are clear cut. As indicated by the responses here, many questions could have two or more correct answers, depending on how much you know about various subjects.

    That's not the test maker psychology, though. They generally write the questions with a correct answer in mind, then just throw in other possibilities, which they don't really bother to think deeply about.

  11. caheidelberger Post author | 2012.04.18

    I loved taking bubble tests. Being able to psych out a testmaker does indicate some strong intellectual skills. But those aren't the skills we're supposed to be teaching.

Comments are closed.