South Carolina public schools have required students to take at least 10 different standardized tests since the turn of the century, making it difficult for parents and policymakers to tell if their schools are improving over time.
In the 2017-18 school year, the state swapped out its mandatory 11th-grade workforce preparedness test, ACT WorkKeys, for a new test called Ready to Work. State education officials released results from the new test early this month but cautioned that they are not statistically comparable to WorkKeys, which had been in place for the previous three years.
South Carolina isn't alone among the states in churning through a large number of tests in a short period of time.
Changes in state standards, including many states' controversial adoption of Common Core, have prompted some shifts in testing, as has the change from the Bush-era No Child Left Behind program to the Obama-era Every Student Succeeds Act.
Still, educators and parents have talked less about their lack of ability to make apples-to-apples comparisons among students in different years than they have about too much standardized testing in general.
Similar, maybe too similar?
In the case of South Carolina's shifting from WorkKeys to Ready to Work, it's not immediately clear what prompted the change. The tests are remarkably similar — so similar, in fact, that ACT Inc. is currently suing the makers of Ready to Work in federal court claiming copyright infringement.
Both tests have three sections (Applied Mathematics, Locating Information and Reading for Information), and both tests allow students to earn one of four certification levels for career readiness (Bronze, Silver, Gold or Platinum).
But there is no apples-to-apples comparison between results on the two tests. "Gold" on WorkKeys is not equivalent to "Gold" on Ready to Work, according to state education officials.
"Our statisticians tell me that we cannot definitively claim that WorkKeys and Ready to Work are absolutely equivalent or even have uniformly roughly equivalent cuts," said S.C. Department of Education spokesman Ryan Brown, referring to the "cut scores" that are used to define bronze, silver, gold and platinum certificates on the two tests.
South Carolina changed career-testing vendors after a competitive bid process, awarding a five-year, $9 million contract to Tennessee-based Worldwide Interactive Network Inc., the maker of Ready to Work, starting in February 2018.
What parents need to know
The first few years of a new standardized test are often confusing for educators and parents alike, said Kevin Eakes, director of the College of Charleston's Office of Student Services and Credentialing.
"Any time you shift an assessment like that, there is always a time where you’ve got to learn the new assessment, learn how to administer it and interpret the scores," Eakes said.
Eakes, who previously worked in accountability and curriculum at the Charleston County School District, said teachers often find "formative assessments," such as school-administered MAP or STAR tests, to be more useful for pinpointing an individual student's needs.
Often administered to younger students two or three times in the course of a school year, these tests can show a student's exact reading level and identify specific skills they need to work on.
Still, when parents are researching a school online, it is much easier for them to get their hands on state standardized test data, like grade-level pass rates on the SC READY or SCPASS assessments — or, starting this year, the Ready to Work test for high schools.
One of the only standardized tests that has stayed largely the same since the 1990s is the National Assessment of Educational Progress, sometimes referred to as the "Nation's Report Card." On fourth-grade reading, an important benchmark for lifelong literacy, South Carolina slipped from 39th to 47th in the nation between 2015 and 2017.
While NAEP results are useful for comparing students in different states, the test has its own limitations: Because it is only administered to a small random sample of students in two grades every two years, it can't be used to track the progress of individual districts or schools.
Even tests that have remained in place for years are sometimes disrupted by technological change, as happened when the state began switching many of its tests from paper to computer-based versions. Technological hiccups and errors have rendered entire years' worth of data unreliable, according to some statisticians.
How SC students did
All 51,013 of South Carolina's 11th-graders took the Ready to Work test last spring, and 63.3 percent scored Silver or higher, a sign they have the skills needed for 65 percent of certain jobs. Most of those, 54 percent of all 11th-graders who took the test, scored Silver.
Fewer than 7 percent scored Gold, a sign they have skills for 90 percent of certain jobs. Only 2.5 percent reached the Platinum level, indicating they are ready for most any job.
In Charleston County, the percentages of students scoring Silver, Gold or Platinum were both above and below the state averages, at 52.1, 8.5 and 4.2, respectively.
In Berkeley County, the percentages of students scoring Silver, Gold or Platinum were 54.3, 6.8 and 1.6, respectively.
In Dorchester District 2, the percentage of students scoring Silver, Gold or Platinum were above the state average: 58.3, 7.7 and 3.8, respectively. In Dorchester District 4, those percentages were 48.3, 5.5 and 0.
Statewide, 16.6 percent of the students did not qualify for Bronze status, indicating they still cannot perform complex tasks that would qualify them for a broad range of jobs.