WWNO continues its series “Behind the Test” with a look at test security. The paper booklets, and students’ answers inside, can determine things like teacher pay or the very existence of a school. It takes a lot of effort — and people — to keep the testing materials secure through delivery, administering the test, turning them in and then scoring.
The booklets and answer sheets for Louisiana’s LEAP tests come from a company called Data Recognition Corporation in Minnesota. When the Recovery School District's tests arrive they go straight to a warehouse.
“We have to have a pre-approved storage facility, and a pre-approved storage room," says Vera Triplett. The veteran educator and former charter school executive is now Deputy Superintendent for Achievement with RSD. "There are security guidelines around the room, the size of the room, who has access to the room and the way the room needs to be secured.”
She describes the scene when the thousands upon thousands of tests arrive.
“Essentially it looks like a team of people who are receiving these boxes, they are cataloging them, verifying what we’ve got in what we haven’t got in.”
Tracking all of it takes tactical precision. (A reporter on the scene is not welcome.) Around LEAP week the mantra seems to be: avoid anything that seems even a tiny bit like it might be a security risk.
Each school has a testing coordinator who ensures the pre-coded tests they pick up from RSD match their school’s order. The check the correct number of tests for each grade, and count any specially formatted tests for specific needs. Triplett says schools' needs often change between the time they order tests, and the materials arrive.
“With pre-coded tests you might have a test for a kid who’s no longer at your school, or you might have a kid who just started at your school weeks before testing, and so you did not account for them in your order,” says Triplett.
All special needs must be accommodated. That means Braille tests for blind students, or tests for students whose first language is not English. The list of special requests continues right up to test day.
“Maybe a kid broke his leg before testing, and now can’t test on the second floor, which is where the [approved] testing environment is,” Triplett explains. “We have to fill out a form to get another testing environment for that child.”
That one student’s broken leg might mean he needs a first floor testing room, but what if all approved testing classrooms on the first floor are full? Can he take his test in the cafeteria? Maybe. To do that properly, he he’ll need his own test proctor, and the space itself must be approved.
“There are guidelines around noise level, guidelines around movement, and so all of those things have to be controlled,” says Triplett. "Even if there is a single child testing, that child deserves to have the same testing environment that all the other children had.”
Triplett says after the several days of LEAP testing, students’ coded booklets are put back in the same coded box, and sent by UPS to Minnesota. Data Recognition Corporation, or DRC, also administers and scores tests for other states, including Nebraska, New York and Alabama. The company declined to be interviewed for this story. A firm specializing in detecting irregularity on tests, Caveon Test Services, shed some light on what happens once tests leave schools. David Foster, CEO of Caveon, says basic test scoring technology has mostly stayed the same, even as the stakes have grown higher.
“Scanning of answer sheets [first] occurred in the 1950s and that was a big change. Since then no huge changes,” Foster says.
What has changed, he says, is a rise in test fraud. The major concern today is not so much students cheating on tests as teachers and educators tampering with results to boost their students’ scores.
“Any time there’s a big breach, we get a call. A lot of that has been in state testing. So certainly our business has grown in that way,” says Foster.
States are on high alert after high-profile scandals. Last year educators in George were indicted for conspiring to alter or fabricate test answers. Caveon is conservative in its analysis, Foster says, looking for patterns that have less than one in a trillion chance of occurring. One of the most common scans his company uses is erasure analysis. Too many changes from wrong to right, and something might be up.
“When we see too much similarity, when we see too much aberrance, then we’ll report the findings," he says. “None of that means cheating, and we go to great lengths not to call it that. But the test score is suspect and should not be used.”
Soon erasure analysis won’t be necessary, he says, because new computer-based tests are becoming the norm. Louisiana is in the midst of that switch. Right now, Foster says, computer-based tests are mostly just digitized versions of the paper tests, but the technology will give rise to new forms of testing.
On a multiple choice question, for example, students may not get to see all the answers at once. They’ll see one answer at a time, with the answers randomized, and decide whether the one on their screen is the right answer before moving on to the next choice. That’s a more secure way to test, he says. But during the transition to computer tests, Foster notes the state will likely have to deal with both paper tests and computerized tests, possibly for several years. Computer tests have their own security issues, too.
“You just can’t test as many students at the same time, because there’s just not enough equipment,” Foster says. “A test that took three days is maybe going to take six, or nine. That longer testing window allows the harvesting of test questions early...to be shared with people taking the test later.”
The learning curve on new tests for students, teachers, school districts and testing companies could lead to less standardized – and therefore less reliable – test data.
No one is likely to mourn the paper tests and bubble sheets or those No. 2 pencils with their pink erasers, Foster says. Nor will they miss the scheduled deliveries, pickups, boxes and logistics that come with them. But school districts can’t let their guard down in making sure they’re doing everything right.
“A lot of thought goes into making sure the tests are fidelitous,” says Vera Triplett at the Recovery School District, “Because you are making decision about individual kids and schools, based on the results."
Test anxiety is yet another factor schools are trying to manage and mitigate. If the pressures of testing cause students to give wrongs answers on questions they actually know, then the test may not be doing its job: accurately measuring achievement.
Support for education reporting on WWNO comes from Baptist Community Ministries, Entergy and the John S. and James L. Knight Foundation.
