Understanding what doesn’t work in personalized learning

Everyone, including social scientists, is eager to highlight results that suggest an innovative education model is working for students. It might surprise you to learn, therefore, that I am excited by negative findings about blended learning (or technology-supported personalized learning)–findings that suggest interacting with technology might actually have a negative effect on student learning.

In recent years, there are two negative trends that have been consistently found across studies of blended or personalized learning. In a world where we are often not sure how to minimize unintended consequences, I am excited because these findings give us more information about how to proceed in ways that will not harm the very students we wish to help.

Educators are making decisions every day about what instructional practices to use to best support their students. In a situation where there is little opportunity to reduce the inherent risk of each decision, like innovation, every bit of information counts. Therefore, it is more critical to know what doesn’t work in a sector like blended learning than in other sectors. My goal here is to explain what these negative findings mean for practice.

Simple substitution of in-person learning with online learning is not effective

There are quite a few studies that have investigated the links between the presence of educational technology in a learning environment and student outcomes. While each study describes this evidence in a slightly different way, this finding has been replicated several times, most recently in a study from MIT. The finding from this study is summarized in the Hechinger Report this way: “Computers and internet access alone don’t boost learning” (emphasis mine). Previously, a study of information and communication technology (ICT) by the OECD found that “PISA results show no appreciable improvements in student achievement in reading, mathematics or science in the countries that had invested heavily in ICT for education.”

Both of these studies support earlier, yet limited, evidence from SRI, and CREDO, which suggested that blended environments (those with teachers and in person learning) produced greater academic gains than fully virtual environments (those without an in-person learning environment). In other words, putting devices into the hands of students–or even the hands of students and teachers–with no additional supports, does not improve learning. This may seem obvious when stated in this way, but may explain why many one-on-one initiatives did not have desired impacts on teaching and learning.

The SRI study, however, also found that the opposite is not true (i.e., avoiding or removing all technology does not boost learning either)–and did find blended environments significantly outperformed traditional face-to-face learning environments. So access and connectivity appears to be a necessary step, but just one step, in the path to improving outcomes using technology. For those who are seeking to implement blended learning, the exciting message is then: It’s not what you use, it’s how you use it that matters.

Students’ engagement with technology may hurt their engagement with school or feelings of belonging

This finding is supported by the OECD study mentioned above, along with RAND’s series of studies of personalized learning in Next Generation Learning Challenge schools (see Table 8 in the third of the series, and Chart 9 in the second). This is certainly the more puzzling and potentially troubling negative finding, and one that should carefully be considered when implementing blended learning. Both of the studies mentioned found a negative relationship between students’ use of technology in their learning environments, and their feelings of academic engagement or sense of belonging in school.

The explanations for such a relationship between technology use and engagement vary. It may be that already-less-engaged students spend more time in online learning environments compared to their peers. Or it may be that the way in which technology is being used is distancing students from each other, their teachers, the content, and their school. In a bit of a twist, previous research has found that providing students with agency (autonomy and choice in the setting and path to meeting their learning goals, for example) actually increases feelings of academic engagement. In any case, more research is needed here to figure out the mechanisms behind this potentially deal-breaking unintended consequence of blended learning.

Until that research is done, those seeking to implement blended learning should pay particular attention to the ways in which technology is being offered and used by students, and care should be taken to avoid the creation of new, isolating learning environments that can either amplify or create feelings of disengagement among students, and teachers.

Negative findings can stimulate positive change

Rather than being discouraged by these negative findings, I ask those implementing blended learning to instead embrace them and consider how one’s actions and decisions can earnestly prevent doing harm to one’s students. In the first case, truly understanding your teachers’ and students’ needs, and supporting them before and during implementation, can raise your chances of achieving intended objectives.

In the second, being mindful about the potential for technology to correlate with disengagement, and explicitly focusing on maintaining or increasing student engagement can likewise boost the odds of meeting your goals for implementation. In this second instance, there do exist some models for implementing blended learning that maximize students’ voice and choice–such as allowing students to determine their own learning goals, and methods for demonstrating mastery of content–which do also increase students’ engagement and sense of belonging. Somewhat reassuringly, these practices were reported to be less consistently implemented in the Next Generation Learning Challenge schools that formed the sample in the RAND study.

Overall, adopting more of a growth mindset when looking at what the evidence is telling us will ensure that we all do the best we can. It also provides some means for risk reduction while maintaining a highly flexible, innovative teaching and learning environment. Negative findings can be the guardrails we need as we navigate this new territory. They are a safety feature, not a failure, of research.