Author: mpolikoff

Not playing around on play

mpolikoff , ,

This weekend’s hot opinion piece was the New York Times’ “Let the Kids Learn through Play,” by David Kohn. This piece set up the (fairly tired) play vs. academics dichotomy, citing a panoply of researchers and advocates who believe that kindergarten has suddenly become more academic (and Common Core is at least partly to blame).

There will undoubtedly be many takedowns of this piece. An early favorite is Sherman Dorn’s, which notes the ahistorical nature of Kohn’s argument. Another critique I noticed going around the Twittersphere centered on the fact that there’s far more variation in kindergarten instruction among classrooms than there is between time periods (almost undoubtedly true, though I don’t have a link handy).

Early childhood is not my area, so I can’t get too deep on this one, but I did have a few observations.

  1. I think the evidence is reasonably clear at this point that kindergarten is becoming more “academic.” Daphna Bassok has shown this using nationally representative data, and I have found it in my own analyses as well. This means both that kids are spending a greater proportion of their time on academic subjects, and also that instruction within subjects is becoming more concentrated on more “traditional” approaches (e.g., whole class, advanced content) and less concentrated on more student-directed approaches.
  2. Any time you read an op-ed and you think “if they just flipped the valance on all these quotes, I bet they could find equally prominent researchers who’d support them,” you know you don’t have an especially strong argument. To put it mildly, my read of this literature is that it is far more contested than is described here. For instance, Mimi Engel and colleagues have several studies demonstrating that some of the advanced instructional content that comes under fire in Kohn’s piece and in the anti-academic-kindergarten crowd is the content most associated with greater student learning and longer-term success. Now, that doesn’t mean there might not be some tradeoffs (though I’d like to see those demonstrated before I’m willing to acknowledge them), but the literature is clearly not as one-sided as was portrayed here (and it may even be that the bulk of the quality evidence falls on the other side of this argument).
  3. As Sherman points out, this is also a ridiculous false dichotomy that is quite unhelpful. I don’t think anyone envisions kindergarten classes where students are mindless drones, drilling their basic addition facts all day long. Rather, many believe, and I think evidence suggests, that kindergarten students can handle academic content and that early development of academic skills can have long-lasting effects. Daphna perhaps put it best in an EdWeek commentary (you should read the whole thing if you haven’t already):

Our own research shows that children get more out of kindergarten when teachers expose them to new and challenging academic content. We are not arguing that most kindergartners need more exposure to academic content. At the same time, exposure to academic content should not be viewed as inherently at odds with young children’s healthy development.

I think this is exactly the right view, and one that was missed in the Times over the weekend.

[1] Save that link! I’m sure if you change the policy under question you can apply the text almost verbatim to most education op-eds.

A note on Simpson’s Paradox and NAEP

mpolikoff , ,

A couple of weeks ago, before yours truly joined the blogosphere, the results for NAEP history, geography, and civics were released. Journalists and advocates around the nation reacted with their usual swift condemnation, noting the “flatlining”, “stagnant” performance. And it’s true, overall average scores on the newly released tests had not changed since their previous administration.

A few wise individuals, however, noticed that the scores had continued to increase when broken down by subgroup. Chad Aldeman penned the best defense, invoking Simpson’s Paradox to conclude that achievement is rising, and not by a trivial amount. In this case, Simpson’s Paradox means that the gains by individual subgroups (every subgroup is gaining in these subjects, and the largest gains are going to the historically most underserved groups) are masked when calculating overall averages because the typically lower-performing subgroups are increasing in numbers.

Jay Greene shot back in the comments section of Chad’s piece, arguing that Simpson’s Paradox was not an appropriate excuse here, because minority students are less difficult to educate now than minority students were 30 or 40 years ago, so making comparisons within groups is not necessarily appropriate.

I will actually take a middle ground here and say there is an element of truth to both arguments. This is because, in evaluating whether it’s better to focus on individual subgroups or the overall average in a case of Simpson’s Paradox, I find it useful to consider what the question of interest is.

As an example, consider the case of two airlines (American and United) operating at two airports (O’Hare and LAX). United flies 100 flights out of each airport with a 55% on-time rating from O’Hare and an 85% rating from LAX (thus, 70% overall). American flies 200 flights out of O’hare with a 60% on-time rating and 50 flights out of LAX with a 90% (thus, 66% overall). Now, if you were buying a ticket based on the aggregate statistics, you would choose United, because it has a higher overall on-time rate. But the overall average in this case is completely useless; it only applies to you if you pick your flights (including your departing airports) completely at random. If, instead, you pick your flights like a normal person by first choosing a departing airport and then choosing an airline, you are always better off choosing American. So in this case, the “subgroup” question is by far the more interesting one, and the “average” question is misleading and worthless.

To me, the primary question of interest with respect to NAEP is whether a given kid is likely to be better off now than he or she would have been 20 years ago. This is a subgroup question–we want to compare each kid to himself if he’d only been born 20 years earlier. Here, the answer is very clearly yes (with the possible exception of kids in extreme poverty). For all subgroups, NAEP achievement in all subjects continues to increase, as do high school graduation rates.

However, I can see the argument that the main question of interest is how the nation as a whole is doing, in which case it’s not overly relevant if the subgroups are making gains but the national average is not. The argument here basically says “the population is what it is, and we have to deal with that.”

Regardless of one’s view on Simpson’s Paradox in this particular case, I actually remain stunned and impressed by our students’ performance in subjects like geography and civics. Given that these are non-tested NCLB subjects (and thus have certainly seen reduced emphasis in classrooms), I find it remarkable that performance has not only not decreased, but actually has continued to tick up for all kinds of kids. This story, the nuanced version that includes attention to subgroups, is one that certainly needs to be told more often.

Wherein I get personal

mpolikoff ,

Every now and then, I will likely get personal on this blog. It’s my nature to share (some might say overshare) about my personal life, as it clearly shapes who I am as a scholar.

I’ve just wrapped up my fifth year as an assistant professor at the USC Rossier School of Education. Rossier is a small school of education. We’ve got fewer than two dozen tenure-track faculty members, and our PhD program is typically in the 10-15 student range per year. But what we lack in numbers we make up for in quality and collegiality.

In terms of quality, there’s no question that my colleagues are remarkably influential in both research and policy circles. We have notable experts in higher education, K-12 policy, educational psychology, and teacher education. Particularly in K-12 (the area about which I know the most), we have experts on virtually every major policy issue of the day: teacher evaluation and labor policies, accountability, privatization and markets, education finance, and standards-based reforms. And that’s great. We’re doing work that matters for kids.

But there’s another way in which Rossier is a special place. Often you hear about faculties where multiple people can’t be in the same room with each other because they hate each other so much. There’s even a name for this phenomenon, Sayre’s Law:

In any dispute the intensity of feeling is inversely proportional to the value of the issues at stake. That is why academic politics are so bitter.

We don’t have that at Rossier. There’s not one of my colleagues I don’t like, and (to my knowledge) none of them don’t like me. I don’t know if it’s just happenstance of who is here (that has to part of it; we don’t have any of the poisonous personalities you often see populating the twittersphere) or something about the institution, but it’s truly a delightful place to work.

This morning I’ll be having my faculty colleagues over for a brunch to celebrate the end of our academic year. All of ’em. It’s the rare institution where one could do that. I hope we all recognize how fortunate we are to work with such great people who happen to do such great research.

Everyone’s got an opinion about everything

mpolikoff , , ,

Okay, not everyone. And not everything. But surprisingly many people about surprisingly many things. This will be the first of many posts about public opinion polling data, something in which I have increasing interest (even if little technical expertise).

Today’s interesting nugget comes via NPR, which reports on a recent little exercise done by Public Policy Polling. It seems that after a random tweet from a TCU professor, PPP polled voters and found that they had stunningly negative views of this person (whom they could not possibly have heard of)–3% favorable to 20% unfavorable. The money quote:

The big lesson for Farris, who is already thinking about how she’ll work this experiment into her next political science class, is in “pseudo-opinions.”

“People will offer an opinion when they don’t actually have one,” she said. “There is social pressure to answer, and give some type of opinion, whether it’s right or wrong.”

There is a recent boom in public opinion polls on education, and I am willing to bet many of the same trends come into play. Despite their general lack of knowledge about education issues, Americans want to give their opinions. In particular, for example, polls suggest that Americans pretty strongly support local control and teachers while also supporting weakened labor protections and testing. I’m sure some of this support is real. But I’ll bet a good chunk of it is just pseudo-opinions. Hopefully well-crafted polling and research can be used to help discern the difference.

Gathering textbook adoption data (or: shouldn’t this be easier?)

mpolikoff , ,

Suppose you set out to study the impact of textbooks on teacher practice and student learning. The only way to begin such a study would be to pull together data on which textbooks were used in which schools.

You’d think this would be easy to do. After all, we live in a data-driven culture, and you can find just about any bit of information about your local school via a few seconds on Google (or the state department of ed website).

Well, you’d be wrong.

As I mentioned last post, I have a couple grants to study textbook adoptions. These grants are concentrated in the five largest US states by population (CA, TX, NY, FL, IL). Of these, only Florida keeps track of textbook adoptions at the district level. The other four states, comprising roughly 4,000 school districts, do not keep track at all [1].

This means that if you want to know which textbooks are being used in these 4,000 districts, you have to ask people. As far as I know, there’s no other way to do it. So that’s what we’ve done. We created a beautiful website where district personnel can go to report their textbooks. Then we gathered contact information for district personnel in all these districts and sent them a series of emails inviting them to participate and offering a chance at a $500 incentive to do so.

Suffice it to say the response rate was not what we hoped, even after several rounds. So we’re moving on to round two. We’re sending state-specific open-records request to every non-responsive school district in these states, pointing them to the website. And a couple of weeks after these requests arrive, a horde of USC undergraduate researchers will begin sending personalized emails and making phone calls to districts. Essentially, we hope to hammer all 3,000-ish non-California districts in our sample into submission.

I’m telling you all this not because it’s especially interesting (I probably should have picked a better topic for my early posts on the blog) but because it shows the absolutely absurd lengths one needs to go to in order to gather what should be a freely available, extremely basic piece of information about schools.

Of course my hope is that my projects are successful and that I can gather this information on almost all districts. But if I can’t, I at least hope I can convince some people that this is a piece of information we should be tracking. It costs essentially nothing to do, it does not endanger privacy in any way, and it’s very useful from both a research and an equity point of view.

[1] California actually does keep track to a certain extent; I’ll talk about the Golden State in a future post.

A textbook example of education research

mpolikoff , ,

One of my main research interests these days is the adoption and use of textbooks and other curriculum materials. Why would I possibly care about textbooks? Well, for starters, they’re incredibly cheap relative to other educational interventions, and they can have remarkably large causal effects (PDF) on student achievement. They also are just a skosh less politically treacherous than, say, radically altering teacher tenure policies.

This work began with a grant from an anonymous foundation to analyze the alignment of textbooks to the Common Core math standards. That investigation found overall weak alignment, with some common areas of misalignment across books (notably, they were excessively procedural relative to what’s in the standards) [1].

While that work was informative, it didn’t tell me much about who was using which textbooks, how, and to what effect. As a new set of standards rolls out, I’m guessing that curriculum materials may matter more than ever. So I set out to investigate these issues in a few different studies. The basic gist of this set of studies is to understand:

  • Which textbooks are being adopted in the core academic subjects in light of new standards?
  • What explains school and district textbook choices (qualitatively and quantitatively)?
  • How do teachers make use of textbooks in their teaching?
  • What are the impacts of textbook choices on student outcomes?

This work is funded by the National Science Foundation, the WT Grant Foundation (with co-PI Thad Domina), and by another anonymous foundation (with co-PI Cory Koedel).

In the coming days and months I’ll be talking quite a bit about this work and some of the lessons learned so far. The next post is going to highlight some of the things I’m learning as I’m trying to go through the (seemingly straightforward) task of simply gathering data on what textbooks schools and districts are using these days. Spoiler alert: it ain’t pretty.

[1] That work also identified some ways to make the process of analyzing textbooks (which turns out to be incredibly time- and labor-intensive) much simpler.

On Education Research

mpolikoff

Welcome to my blog. Over the past five years, since I joined the faculty at USC Rossier, I’ve started and stopped the creating-a-blog process probably a half-dozen times. Mostly, I’ve worried that I wouldn’t have anything interesting to add to the seemingly saturated edu-blogosphere. I’ve also thought that I probably didn’t have enough time to make a meaningful contribution.

But I’ve finally decided to give it a go. It may be a short-lived foray, or it may become a permanent fixture of my academic life, who knows? I think I have some things that are worth saying in a form longer than 140 characters, and I haven’t yet had anywhere to say them. So that’s what purpose this will serve.

As I currently see it, I expect this blog will have several main foci:

  • Chronicling my own academic experience on both the personal and professional sides;
  • Commenting on other edu-bloggers and education journalism (and perhaps even Twitter punditry);
  • Highlighting new research of import, particularly research that I feel isn’t being adequately recognized; and
  • Batting around new ideas about research and current events in education policy.

My hope is that this can become both a useful outlet for me and also a place that people go to read things that they aren’t reading elsewhere. I appreciate your stopping by, and I look forward to the discussion.

Morgan