In an era of high-stakes and high-stress testing, how do we ensure that classroom instruction does not give way to inappropriate teaching?

Source: W. James Popham, Educational Leadership, pp 16-20, vol. 58, number 6, March 2001, ASCD, http://www.ascd.org/

In his article, Dr. Popham describes what teaching to the test means and discusses ways to tell whether a teacher is in fact doing that, or is teaching the curriculum. He suggests that teaching the curriculum, or curriculum-teaching, as he calls it, not only "will elevate students’ test scores," but will also "elevate students’ mastery of the knowledge or skills on which the test items are based."

Dr. Popham describes the differences between teaching to the test, or item-teaching, and teaching that is directed to the curriculum, or curriculum-teaching. Item-teaching is instruction that is focused on real test items or items similar to the actual test items, clone items (where the cognitive demand is unchanged, but only the names of things or insignificant factors and variables have been changed). In contrast, in curriculum-teaching, instruction is directed toward "a body of content knowledge or a specific set of cognitive skills represented by a given test."

Ideally, teachers, administrators, and parents can make accurate inferences from test scores about the levels of mastery, abilities, and knowledge base of students who take the test(s). "Curriculum-teaching, if it is effective, will elevate students’ scores on high-stakes tests and, more importantly, will elevate students’ mastery of the knowledge or skills on which the test items are based." However, if a teacher gets a copy of the test and resorts to item-teaching, then the validity of the scores and the inferences made from those scores are very questionable. As Dr. Popham states, "Because teaching either to test items or to clones of those items eviscerates the validity of score-based inferences- whether those inferences are made by teachers, parents, or policymakers – item-teaching is reprehensible."

Dr. Popham relates detecting inappropriate test preparation to screening for illegal use of drugs among athletes. He states that once the athletes have been warned that there will be random testing for drug abuse, inappropriate drug use drops significantly. Dr. Popham implies that perhaps the same effect would happen in schools regarding test preparation. However, he states that there is no practical procedure for detecting teachers who are engaged in inappropriate test preparation.

Dr. Popham describes a fictitious teacher who "is so frustrated by the overwhelming pressure to improve her students’ scores, she engages in some full-scale item teaching." As a result, her students make amazing and unbelievable improvement on the test. But, because of item-testing, these scores are really invalid indicators of her students’ actual mastery of the content. There is no actual way to prove that the scores changed significantly because of the item-teaching approach.

Dr. Popham describes several possible detection procedures: teacher self-reports, teacher-collected materials, pre-announced classroom visits, unannounced classroom visits, student self-reports, and score jumps. He points out that all have some merit, but none of them would be really successful for various reasons. The heavy pressure and stress on teachers for students to be successful and achieve high scores on tests weighs too heavily on many teachers.

Dr. Popham believes that "the vast majority of teachers, if they recognize the adverse effects of item-teaching, will abandon such teaching." He therefore advocates providing "a hefty dose of assessment literacy." Dr. Popham states that the majority of teachers he spoke with never considered the adverse effects or the appropriateness of their test preparation practices. Once confronted with the fact that their practices led toward "invalid inferences about their students," many teachers were "surprised and dismayed."

Just because teachers recognize the inappropriateness of their test preparation practices does not mean they will abandon those practices. For some, the pressures and the stress of high score achievement override their understanding of the implications for the students in the long run because the teachers feel they have no alternative. However, Dr. Popham does believe that the greater majority of teachers will make a change in their practices.

Dr. Popham suggests that teachers should be provided a clear understanding of the difference between item-teaching and curriculum-teaching. He also maintains that tests should provide teachers a clear understanding of the content knowledge and cognitive skills represented by the test items. He states that, "A teacher, looking over what curricular outcomes a high-stakes test represents, should understand those outcomes well enough to plan and deliver targeted lessons. Anything less descriptive drives teachers down a no-win instructional trail leading to item-teaching." He advocates encouraging policymakers to only adapt tests that have adequate item descriptions so teachers can focus on curriculum-teaching, and to view those tests that don’t adequately describe the content knowledge and cognitive skills measured as leading to probable invalid test scores due to item-teaching. He cites a positive example in Hawaii where not only were adequate descriptions provided, but "the number of content standards was reduced to a more intellectually manageable number of curricular targets."

In conclusion, Dr. Popham recommends that teachers and policymakers develop their assessment literacy.