Productive Struggle, Q&A

Answering your questions about our last post

My last post (“Lots of Research, No Evidence”) took a close look at an article. That article concerned an educational idea called “productive struggle” and claimed to offer evidence in support of its benefits. I argued that, despite many citations, it did not do this, and that this was typical in certain circles—using research to institutionalize consensus, rather than provide evidence.

Anyhoo, I got a lot of questions about that piece, and some criticism that probably would have been more productively posed as a question. Here, then, are some questions and answers about productive struggle.

---

Q: Is “productive struggle” the same thing as Manu Kapur’s “productive failure” idea? If so doesn’t his research count as support of productive struggle?

Kapur’s research is serious and interesting. It’s related to Daniel Schwartz’s idea of “preparation for future learning,” which is basically that discovery-style activities don’t directly cause learning, but prepare students to later learn from more explicit instruction. In a typical Kapur experiment, one group explores a problem in a group and then spends times learning through direct instruction; in the other group, the sequence is reversed. He always finds benefits from starting with the problem.

Anyway, is that “productive struggle”? It all depends on what productive struggle actually means, and nobody agrees on what it means.

Q: How is productive struggle defined in research?

A: Hiroko Warshauer’s dissertation is where the term originates. Here is what she is committed to:

• “struggling to make sense of mathematics is a necessary component of learning mathematics with understanding”

• “While the phenomenon we call struggle may be internal, it is also observable in most classrooms…students may voice confusion over directions, the wording of a problem, the question being asked or how to devise a strategy…students may voice a comment such as, “I don’t get it”….A student may be very engaged in working on a mathematics problem but then reach an impasse and get “stuck.”

In other words, she claims that student struggle is a necessary prerequisite of learning math concepts, and that being stuck, asking for help, saying “I don’t get it,” etc., are all signs of struggle. Teachers get nervous at signs of struggle, step in and remove the chance for learning.

That is many things—intriguging, debatable, partly true, perhaps contrary to existing evidence in other domains—but it is not productive failure, because Kapur has no commitment that students will struggle in any of those senses. He just wants you to do exploratory groupwork before the lecture.

Q: Aren’t you just splitting hairs because you’re biased?

A: Half of “productive struggle” is “struggle.” If you don’t have struggle—as indicated by those behaviors—it doesn’t count as productive struggle. If you wanted to tell teachers to do whatever Manu Kapur said, then write: “an important principle of teaching is doing what Manu Kapur said.”

What Kapur says is, precede direct instruction with exploration. If that’s your teaching idea, then say it. If your idea is that students need to get stuck before they can learn, that’s different.

Q: But are you actually just splitting hairs?

A: There is plenty of evidence out there that kids need to do things that are hard in order to maximize their learning. Some things are important and hard, I don’t need to tell you that! But I don’t think there’s evidence out there that the struggle itself is valuable for learning—and, in particular, that we should be creating more moments of struggle for students.

Yes, struggle creates opportunity for teaching.

Q: Fine. But that research definition is not what I thought “productive struggle” meant. I thought it meant that one should aim to challenge students with mathematics—not too hard, though, it should be productive—and that struggle is not a bad thing, and we shouldn’t be afraid of pushing students, and we should try to make the classroom a place that honors the difficulty of learning.

A: Everyone agrees with that, and I do too. It’s unclear to me if anyone would disagree with it, though certainly not everyone would put it on a list of “most important teaching ideas.”

Q: Is there evidence for productive struggle under that definition?

A: The issue is that radically different teaching ideas are compatible with productive struggle of that sort. Is it “supporting productive struggle” to help students who are struggling with a practice worksheet? Does it require (as the chapter I was reviewing suggested) giving students problems without instruction? Is it Kapur-style exploratory problems?

No, no, no, because any sort of classroom task can produce productive struggle under this definition. It’s just that you’re supposed to use those moments of struggle as learning opportunities. OK, fair enough.

So, is there evidence that you…should support kids in those moments? I don’t know, probably, though you don’t really need to convince me of that. There might be evidence (Warshauer’s dissertation) about how to support kids when they’re stuck.

Q: What are you, the evidence police or something?

A: I don’t care if you don’t have evidence for your teaching idea. Really! I don’t care. That is, unless you say something like: WE HAVE ALL THIS EVIDENCE FOR HOW TO TEACH PROPERLY AND WE JUST NEED TO SPREAD IT. HERE IS A DOCUMENT WITH SOME EXCELLENT PROVEN TECHNIQUES. DID I MENTION PROOF? EVIDENCE? RESEARCH? IT’S ALL THERE. ARE YOU ANTI-EVIDENCE? ARE YOU BIASED? I ASK BECAUSE THAT’S THE ONLY CONCEIVABLE REASON YOU WOULDN’T BE SATISFIED WITH OUR EVIDENCE OF WHICH, AGAIN, THERE IS COPIOUS REAMS. BUT I GUESS IT DOESN’T COUNT FOR YOU. BECAUSE OF YOUR WELL-KNOWN BIASES. SAD, REALLY.

Then, I care enough to write about it.

Q: Don’t you ever ask students to do things without first explaining them?

A: Oh yeah, absolutely.

Q: When?

A: I have a little sticky note near my desk. It says “Too Easy, Too Hard, Just Right.” It’s there because I sometimes forget to bring in questions that even my strongest students will find challenging during class. Variety and balance is good, and different kids need different things.

That said, the reason why my “strongest students” are ready for those problems is because they already know a lot about the topic. They’re ready to “struggle” a bit.

Q: Are there other times when you ask kids to “struggle”?

A: Sure. Kids are always struggling in my classes, because the math is hard for them. I mean roughly 100% of teachers experience this. I’m not afraid of challenging kids.

I wrote a book about teaching math with worked examples, and there’s a passage where I tangle a bit with Kapur etc. I’ve already shared this online, but here’s the relevant excerpt:

Even if we could guarantee that students analyzed every example with care – making sure they engaged in deep self-explanation – there would be times that learning would fail. That’s because frequently students will not be prepared to learn from the example.

The research concerning this question is somewhat confusing, with claims and counterclaims tossed back and forth. Some researchers have argued that working on the problem before studying a worked example is important for learning. Manu Kapur, for instance, has argued that this has benefits even if the student fails to correctly solve the problem – he calls this “productive failure.” Slava Kalyuga and Anne-Marie Singh, on the other hand, suggest that solving a problem before an example might be useful for helping a student understand what a problem is even asking and what its solution may look like. Likewise, Schwartz and Martin suggest that inventing an inefficient procedure prepares students to more quickly and effectively learn from examples and explanations.

Meanwhile, other researchers have performed experiments showing advantages to example-first teaching, and still others have found no differences between example-problem and problem-example approaches. It’s all very confusing, and researchers are currently trying to design studies that could explain these conflicting results.

Just as we did before, to make sense of this debate I think it’s helpful to dig deeper. There are at least two mental processes that are crucial prerequisites for learning from a solution:

Students must notice everything in the problem that will be involved in its solution

Students must remember previous knowledge and strategies that are taken for granted in the solution

I often precede a worked example with a short problem to solve. These problems tend to be brief, as I want my students to save their energy for working through a solution. When they work, I think it’s because they help my students notice and remember information crucial for the example.

Q: That passage was too long and I didn’t read it. What did it say?

A: That you can get past a lot of these annoying debates by just going a level deeper. Ask, why would struggle help? Why would failure help? And there are more specific mechanisms that are actually supposed to lead to the learning. One of these is remembering important relevant information. Another is understanding the meaning of the problem whose solution method the teacher wants students to learn.

As a teacher, those mechanisms are your principles. The things you ask people to do is engineering around those principles.

(Incidentally, one of Kapur’s ideas is that exploration does a good job of gathering prior knowledge and helping students understand the problem. I would find his work more convincing if he didn’t just swap the order of direct instruction/problem solving but tried to design direct instruction that asks students to recall prior knowledge. I’d also be curious how short the problem solving phase could be while still getting the same benefits.)

Q: How did this confusing situation arise?

A: I don’t think there’s someone in a room, cynically devising ways of confusing people by changing the meaning of terms.

It seems to me that it wasn’t Warshauer’s research that made people in math education excited. It was the words “productive struggle” and how they feel when you hear them. When you hear them, you probably just assume—as did I, at first—that it’s saying that you should put kids in situations where they struggle and that—very reasonable!—it’s not always productive. Problem solving, then, but done well.

And yet it seems to have a research pedigree!

In other words, this is exciting and novel language, and I can understand why it was attractive to an organization committed to the importance of promoting problem solving.

But this is a fundamental way in which research drifts away from its original meaning in our society. Terms of expertise that spread widely always have a technical meaning as well as an everyday sense. The everyday sense that lends it power can easily move away from its technical sense; it’s the technical sense that makes it “evidence-based.”

And I think we need to be sensitive to this dynamic in education, medicine, economics, public health, or any other field where research gets tossed around.