To Comfort The Afflicted
And Afflict The Comfortable

To Comfort The Afflicted And Afflict The Comfortable

Wednesday, July 6, 2022

Observercast

Oh What A Tangled Web We Weave, When We Make Tests Punitive

on

BY JOHN THOMPSON

When I was in school in the 1960s, we occasionally took a standardized test, and we took ACT and SAT tests as a part of the college application system.

These were not the type of tests you could “teach to” or “study for.” Consequently, they provided a huge database of invaluable information about students and schools. These tests were a Consumer Reports on education, and they could have been an essential component of a data-informed – not data-driven – contemporary school reform movement.

I don’t want to get into the weeds of the social science, but the ACT and other tests were Norm Referenced Tests [NRTs] that assessed where students stood in comparison with others on the types of testable knowledge that students needed. They were not appropriate for holding individuals accountable. Back then, we showed a healthy respect for the proper use of test methodology, and ACTs and similar NRTs were not imposed for high-stakes purposes.

So, a new set of tests, Criterion Reference Tests [CRTs], were developed for measuring testable knowledge of Standards of Learning. Fifteen years ago, several outstanding experts in the Oklahoma City Public School System Planning Department explained, chapter and verse, how the seemingly arcane shift from NRTs to CRTs would corrupt tests and accountability systems.

With the passage of No Child Left Behind, the misuse of these new tests expanded further than we could have imagined. Then, Arne Duncan became the nominal secretary of education, and the test-loving Bill Gates became the real driver of school policies, and primitive bubble-in tests were misused in conjunction with not-ready-for-prime-time statistical models.

The entire testing system became hopelessly corrupted, undermining the quality of instruction in Oklahoma and the rest of the nation.

Now, Oklahoma is taking the lead in rescuing our schools from the test, sort, and punish mania known as school reform. A recurring suggestion, by good people who seek to reign in the testing frenzy, is the proposal that the ACT replace Oklahoma’s EOIs [see Rep. David Perryman’s HB 1497, for example].

While I don’t think that’s a good idea, I’d like to suggest a framework of discussing the potential value of using the ACT as a graduation examination.

The good news is that the ACT would help solve some of the problems that vex educators. The bad news is that the ACT could liberate adults, and many students, but dramatically increase the dropout rate.

Like most states, Oklahoma was coerced by the federal government into the use of “value-added” estimates of student test score growth in teachers’ evaluations. As an overwhelming majority of scholars explained, these models [known as VAMs] used inappropriate tests in an unreliable manner. They are biased against educators in high-poverty schools with large numbers of English Language Learners [ELLs] and special education students. If Oklahoma can’t find a way to abandon these invalid value-added evaluations, the exodus of our top teaching talent from poor schools will increase.

The courts have tended to conclude that value-added evaluations are improper and unfair, but that they cannot be struck down until they produce outcomes that are irrational. If systems tried to turn ACT test score into VAMs, that methodology would be so invalid that the entire evaluation system would likely collapse. That would be a great thing for educators and schools, and it would free Oklahomans from the single worst legacy of the education reform blame game.

Unfortunately, because of the way that testing has been misused over the last 12 years, the problem facing students is more complex. It took years to do so, but Oklahomans debated the type of End of Instruction [EOI] graduation tests that would be fair to all. We reached the reasonable conclusion that these exit exams should be minimum skills tests, and that the stakes attached to them should be phased in. Today, however, school reformers ignore the intent of the people and demand that EOIs be rigorous college readiness test. Moreover, corporate reformers insist that these high-stakes tests must be imposed immediately.

Today, the ACT is better than ever in providing diagnostic information. It would be far better for most students to be taught in schools where the ACT replaced the EOIs. It would be a wonderful test to teach with. But, it still has a huge problem as a high-stakes test to be taught to.

The biggest problem is setting an ACT cut score that is high enough to be politically viable, but low enough to prevent an explosion in the dropout rate.

In 2009, I attended an Oklahoma Senate interim committee on testing. I’m basing my analysis on data provided by the ACT to that committee. The ACT can provide updated data, but it would not be much different. Back then, 30% of eligible Oklahoma students were excluded because they didn’t take the ACT. A disproportionate number of them would be unable to pass such a test, then or now.

The ACT materials showed that 16% of Oklahomans failed to get a 14 on an ACT. If the cut score was set at 15 then 22% would fail. If it took a 16 to pass an exit exam, then 28% would fail. Add those numbers to those in the 30% who don’t take ACT [which includes disproportionately high numbers of poor, urban kids of color] and who wouldn’t pass it, and you can see the potential disaster.

Moreover, we should not go down the ACT path unless the votes are available to set a multi-year schedule for fully implementing the ACT. It would be unreasonable to expect students to shift over to the college readiness test without providing a several year grace period.

Of course, with an administration [like that of new Superintendent Joy Hofmeister] that has the trust of educators, some of these problems might be addressed, quietly, and a less dangerous set of rules might be possible. But such solutions might not be possible.

So, any legislation that adopted an ACT should include strong and explicit language. The State Education Department must be empowered to respect the will of the people and the Legislature. Regardless of what it takes, cut scores must be set so that the non-graduation rate will not increase.

Students, especially those with no intention of attending college, should not be denied a high school diploma because they fail to meet college readiness standards.

There could be an unintended bonus to such legislation. Today, students who fail the freshmen and sophomore EOIs often face a humiliating consequence. They basically forfeit the chance to have a holistic high school experience, and they lose their senior year to a focus on retaking EOIs. Too many students are sentenced to “EOI Boot Camps.”

We should think of what we would want for children who fail their EOIs if they were our own children. In poor schools, those children are subjected to repeated remediation and test prep. If they were our children, would we want a choice between an unflinching focus on passing the EOIs to a portfolio that builds on their strengths?

The law currently allows for portfolios as an alternative option, even though that path is largely spurned and stigmatized. The better path would be to make the portfolio/project option a normative and equally respected course for graduation.

I understand that money is short. Urban districts might be able to divert the resources that have been devoted to remediation into project-based efforts to meet graduation requirements. That would require leadership from the state, and I believe most districts would trust the Hofmeister Administration on this.

I also believe that many parents would choose high-quality portfolios if it was a common and respected path to graduation. All parents should be free to choose such a path without their children being stigmatized. I believe this could be a very popular, common sense policy that would enjoy the support of all stakeholders.

Dr. John Thompson, an education writer whose essays appear regularly at The Huffington Post, currently is working on a book about his experiences teaching for two decades in the inner city of OKC. He has a doctorate from Rutgers University and is the author of Closing the Frontier: Radical Responses in Oklahoma Politics.

Editor’s Note: This essay first appeared in the February 2015 print edition of The Oklahoma Observer.

Arnold Hamilton
Arnold Hamilton
Arnold Hamilton became editor of The Observer in September 2006. Previously, he served nearly two decades as the Dallas Morning News’ Oklahoma Bureau chief. He also covered government and politics for the San Jose Mercury News, the Dallas Times Herald, the Tulsa Tribune and the Oklahoma Journal.