I Don't Know What to Think About America's Declining Test Scores and Neither Should You

The case for confusion

Rose Horowitch, in The Atlantic, says something is deeply wrong with America’s schools:

For the past several years, America has been using its young people as lab rats in a sweeping, if not exactly thought-out, education experiment. Schools across the country have been lowering standards and removing penalties for failure. The results are coming into focus.

Part of the problem, according to Horowitch, is we’re giving away good grades for poor performance in high school classes. But those same signs of academic decay—low standards, reduced penalties—infect all levels of American schooling. The result? Rapidly eroding skills:

The decline started about a decade ago and sharply accelerated during the coronavirus pandemic. The average eighth grader’s math skills, which rose steadily from 1990 to 2013, are now a full school year behind where they were in 2013, according to the National Assessment of Educational Progress, the gold standard for tracking academic achievement.

But there’s a problem with Horowitch’s argument. Her piece is anchored by a fascinating report from UC San Diego about their surging remedial math program. She thinks this surge reflects a decline in basic math skills. But there’s no decline on California’s state tests, which instead show increases up until the pandemic (and slow recovery since).

This isn’t to say that Horowitch is wrong—test scores have declined in California on NAEP. But when she points to a drop starting in 2013, that could only be in 8th Grade mathematics; 4th Grade math seems to have only dipped in California with the pandemic.

So—were standards only lowered for 8th Graders? And only for math? Because 8th Graders were looking pretty good at reading on NAEP in California until the pandemic.

My point isn’t to defend or focus on California. I’m also not saying that everything is fine. What I am saying is I find all of this very confusing. Some scores definitely started dropping around 2014, but not all of them. I don’t know how you can decide what’s going on in American schools based on these graphs. Whatever the cause of the decline is, it’s not obvious or simple—maybe it’s not even problematic.

It’s Mostly About the Weakest Students

Play around with NAEP scores and you’ll notice that the declines are concentrated in the weakest students. This seems to be the case also for TIMSS, an international assessment. On both the 4th and 8th grade exams, America’s lowest performing students peaked in 2011, while the strongest students continued improving through 2019.

TIMSS, 8th Grade Math

So are American students “getting dumber,” as Matt Yglesias says?

First, that’s rude. But second—no, they aren’t! I mean, yes, even high fliers were impacted by the pandemic. Everyone was. But strong students were looking fine up until then. That cuts out some possible explanations: this isn’t about detracking, gifted education, grading in AP classes, or anything else that would primarily impact the strongest 10% of students.

So…is it test-based accountability? Economist Joshua Goodman thinks so. He points out that 2015 is when the Obama administration started issuing waivers that defanged No Child Left Behind. Up until then, high-stakes testing pushed schools to focus on lifting weaker students up.

Listen—I don’t know! I’d quibble that since NCLB was introduced in 2001, it can’t explain the decade of improvements leading up to that law. And because NCLB was focused on math and reading it can’t explain declines in civics or US history.¹

But there’s another, more significant problem with this explanation…

It’s Not Just Kids

Chad Aldeman has been writing about declining test scores for years, and generally favors an explanation not unlike Goodman or Horowitch’s. But it was from him I first learned that American scores on PIAAC, a test of workplace skills for adults of ages 16 to 65, have also been on the decline, arguably also peaking around 2014.

There were declines in every cohort, in both literacy and numeracy, even in the 55-65 group that has been out of school for over forty years.

Isn’t that…deeply weird? So it’s not just kids that have lost progress, but adults. How could schools possibly be responsible for that? And not all countries experienced a decline. So something is going on in America that impacted both students and adults beginning in 2014. What could it be?

It’s Not Just Phones, and it’s Not Just Here

Adults have phones. Teens have phones. Maybe it’s phones?

Goodman thinks this is part of the story. Horowitch mentions this too. But as Aldeman points out, phones are everywhere and declines in TIMSS scores aren’t universal:

Smartphones and social media are global phenomena, and yet scores in Australia, England, Italy, Japan and Sweden have all risen over the last decade. A couple of other countries have seen some small declines (like Finland and Denmark), but no one has else seen declines like we’ve had here in the States.

(I’d also point out that 4th Graders typically don’t have phones, so they couldn’t explain declines in 4th Grade NAEP reading.)

But on the subject of international declines, and whether America is an outlier, there is another major international exam besides TIMSS—that’s PISA. And PISA scores have been down across the board since around 2014.

Weirdly, looking at this graph, America looks like it’s beating these trends in reading and science. How do you explain that? Why would countries participating in PISA have overall declines starting midway through the decade when TIMSS did not? Why would the US partly buck the trend on PISA, the test we traditionally do worse at, and then underperform the international trend on TIMSS, the exam we do much better on?

Now, there are differences between TIMSS and PISA. TIMSS is generally more of an achievement test with questions that closely resemble what students learn in school. PISA is designed to go a step further, requiring interpretation and application. Maybe that explains the different international trends…

But at this point, I want to throw my hands up—I just don’t know what’s going on!

So, what is it?

Here’s the situation: Americans are getting dumber...well, mostly not. But our lowest performing students seem to be losing ground. And, simultaneously, our adults. Some international tests show a similar decline happening in other countries. On other exams, America is on its own. What gives?

For a moment I had a theory that I liked—that it’s about the shift from paper to digital assessments. NAEP went digital in 2017. PISA went digital in 2015. TIMSS transitioned in 2019. Kids do a lot of lazy button pushing when they take these digital exams.

Maybe the issue is giving tests with the same devices we use to scroll YouTube?

This is definitely something to worry about (it’s a “mode effect”) but it seems as if the test-makers are on top of this. I guess I trust them not to mess up? Also, you’d figure that if there was a big problematic mode effect it would be more of a one-time hit. It probably wouldn’t cause a steady decline…right?

OK, another theory: is it just a coincidence that this tumult coincides with a huge influx of immigrants to the United States? Wouldn’t that change the adult workforce along with the school population? Demographic shift is the most accurate answer to so many educational questions—could it help us here?

I don’t know! This is all so speculative. I feel like I’m just making things up. And none of the explanations are completely satisfying.

Meanwhile the amount of confidence on display from our political writers is remarkable. They seem quite sure that American schools are in crisis, that students aren’t being taught, that there is some grand trick being played on the American public.

But the facts are confusing. So why the confidence?

“America has been using its young people as lab rats in a sweeping, if not exactly thought-out, education experiment,” Horowitch writes, and would it be crazy to say that she sounds a little bit jealous?

The American school system is stubborn. It evolves slowly and only in complicated ways in response to the demands of outsiders. It’s hard to reach into classrooms. Maybe that’s the point here. Writers, advocates, policymakers—many of them love the idea that schools have been steered in the wrong direction. After all, that means they can be steered, and maybe you can even take back the wheel. Climb aboard, everyone! We’re headed back to the 2000s.

1. TIMSS also measures science achievement and those declines were less severe for the US. NAEP declines are also less steep for science. Under NCLB, schools allocated more time and resources for math/reading at the expense of subjects like science and history. Maybe those less intense declines support the NCLB theory.