My quick thoughts on NAEP

By now you’ve heard the headlines–NAEP scores are down almost across the board, with the exception of 4th grade reading. Many states saw drops, some of them large. Here are my thoughts.

  1. These results are quite disappointing and shouldn’t be sugar-coated. Especially in mathematics, where we’ve seen literally two decades of uninterrupted progress, it’s (frankly) shocking to see declines like this. We’ve become almost expectant of the slow-but-steady increase in NAEP scores, and this release should shake that complacency. That said, we should not forget the two decades of progress when thinking about this two-year dip (nor should we forget that we still have yawning opportunity and achievement gaps and vast swaths of the population unprepared for success in college or career).
  2. Some folks are out with screeds blaming the results on particular policies that they don’t like (test-based accountability, unions, charters, Arne Duncan, etc.). Regardless of what the actual results were, they’d have made these same points. So these folks should be ignored in favor of actual research.[1] In general, people offering misNAEPery can only be of two types: (1) people who don’t know any better, or (2) people who know better but are shameless/irresponsible. Generally I would say anyone affiliated with a university who is blaming these results on particular policies at this point is highly likely to be in the latter camp.
  3. To a large extent, what actually caused these results (Common Core? Implementation? Teacher evaluation? Waivers? The economy? Opt-out? Something else?) is irrelevant in the court of public opinion. Perception is what matters. And the perception, fueled by charlatans and naifs, will be that Common Core is to blame. I wouldn’t be surprised if these results led to renewed repeal efforts for both the standards and the assessments in a number of states, even if there is, as yet, no evidence that these policies are harmful.

Overall, it’s a sad turn of events. And what makes it all the more sad is the knowledge that the results will be used in all kinds of perverse ways to score cheap political points and make policy decisions that may or may not help kids. We can’t do anything about the scores at this point. But we can do something about the misuse of the results. So, let’s.


[1] For instance, here’s a few summaries I’ve written on testing and accountability, and here’s a nice review chapter. These all conclude, rightly, that accountability has produced meaningful positive effects on student outcomes.

Advertisement

6 thoughts on “My quick thoughts on NAEP

  1. Solid post. Minor point on phrasing: can we say with certainty that test-based accountability has had a strong effect on student outcomes generally? Can agree there is causal inference for increased NAEP scores for math (even if all studies only use state level data instead of student level). But can we prove that it has increased other outcomes or would get same results if other tests used?

    Like

    • I think that’s a fair point. I view the evidence in math as pretty good (though would love to a version using student-level data, agreed). Quite limited evidence in other subjects, and no evidence on other tests (I’m not sure what other tests could be used).

      Like

  2. I was thinking of other international tests (PISA? PIRLS? TIMSS?) and perhaps tests of other subject areas. All of this analysis places too much faith in the average scores of one standardized exam of math and reading. I don’t believe (and I’m guessing you don’t either) that schools have gotten crappier over last couple of years

    Like

  3. The one other low-stakes test (other than NAEP) I’ve seen in NCLB research was Reback et al who used assessments from the Early Childhood Longitudinal Survey and found small positive, impacts of NCLB accountability pressure (statistically significant in science but not in reading or math): http://www.nber.org/papers/w16745.pdf

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s