A portion of this analysis related to the author's interpretation of the Opportunity Culture study has been found to be flawed and will need to be corrected. An updated version of this piece will be posted soon.
As millions of students head back to school, families are probably wondering if those shiny new devices, apps, and even games that are becoming a typical part of the school day are good for learning. As an education researcher focused on blended learning, I am often asked if education technology “works.” The underlying question here for all of us, myself included, is: “Based on the current evidence, do I want my child’s educational experience to include ed tech?”
The answer to this question is often some variation of “it depends.” You’ve probably heard that the research base on ed tech is young, or that evidence is mixed, or perhaps you’ve read that asking if it “works” is not the right question to begin with. I recognize how frustrating and unhelpful these answers are to families and educators, and I have been noticing a trend that I think is quite stable and relevant to those of us working to provide effective learning experiences to all students.
Generally, the evidence base on ed tech supports existing evidence about in-person learning. In other words, teaching practices are the key to effective instruction and matter more than the presence or absence of specific tools (e.g., textbooks, blackboards, devices) in the learning environment. Therefore, ed tech is effective to the extent that it can support teachers in delivering, scaling, and sustaining effective teaching practices. This is unsurprising when you consider that the most effective teaching practices are those which require the most nuanced and sophisticated human expertise.
Contradictory findings tell the same story
I’d like to illustrate this trend by comparing two studies of blended learning, both published in 2018, that seem to come to opposing conclusions: Blended learning is more effective than “business as usual” versus blended learning is less effective than “business as usual.” In both of these studies, learning in the blended environment was compared to similar grade-level and content instruction that was more typical of the schools being studied (“business as usual” environments).
In the Tennessee Department of Education’s “2016-2017 Blended Learning Pilot Report,” researchers investigated whether blended learning increased student achievement. The authors reported, “The results, taken as a whole, show a positive relationship between participating in the blended learning pilot and increased test scores.” I should note here that these results are encouraging, but the authors point out that these results should be further substantiated through more research.
In contrast, in “Reaching Further and Learning More? Evaluating Public Impact’s Opportunity Culture Initiative,” Ben Backes and Michael Hansen investigated the effectiveness of different staffing models, including a blended learning model, and found: “Less optimistically, however, we also find … significant and negative results associated with Opportunity Culture’s blended learning model in some specifications” (emphasis added).
How can both of these studies be right? More importantly, is there a single, actionable takeaway from them? The answer lies in how the two programs used ed tech to blend instruction. In the first study, ed tech was used in a “deliberate, measured” way to increase students’ ownership of their learning, to differentiate instruction, and to allow teachers to focus on the students that needed the most support. In the second, ed tech was used to expose more students to “lecture-based instruction” led by effective teachers, and to deliver center-based activities facilitated by paraprofessionals to small groups of students.
From my perspective, the varied results demonstrate the importance of using ed tech to support what we already know to be good teaching as opposed to less-effective practices. The first study investigated blending that was used to prioritize known-effective practices and allowed teachers to spend more time on providing more personalized instruction for more students, thus resulting in positive impacts. Alternatively, the blended environment in the second study prioritized less-effective instructional practices (lecture) and actually took effective teachers away from working closely with small groups of students and differentiating instruction, resulting in no, or even negative, impacts.
Teaching > Tech
Perhaps the best kept secret of research on ed tech is the fact that teachers and teaching remain the most important influences on learning. When ed tech is used in ways that support good teaching practices, both in digital and face-to-face environments, effective learning happens. When ed tech is used in ways that propagate ineffective instructional strategies, learning is hindered.
Further support for this perspective can be found in another 2018 report on personalized learning by the Center for Reinventing Public Education. In “Personalized Learning at a Crossroads,” the authors studied two initiatives funded by the Bill & Melinda Gates Foundation to determine how educators design and implement personalized learning, and how school and district conditions support or impede the implementation and spread of innovation. This report is quite comprehensive and focuses on many broad issues, but one small nugget within their findings is particularly relevant to this discussion. What they found, among other things, was that the observed enthusiasm and even action from educators was not enough to scale and sustain innovations without an attendant focus on instructional practices—rather than classroom structures—supported by school and district administrators.
Simply adding ed tech to a learning environment doesn’t guarantee better learning. In fact, using ed tech to replace teachers or to scale ineffective practice guarantees poorer learning outcomes. When it comes to teaching and learning, it’s not what you use, but how you use it that counts.
Note: This post originally erroneously described the blended learning condition of Public Impact’s Opportunity Culture initiative as expanding lecture-based instruction through recorded videos. This description has been corrected to remove mention of recorded videos. The author sincerely apologizes for the error.
The Brown Center Chalkboard launched in January 2013 as a weekly series of new analyses of policy, research, and practice relevant to U.S. education.
In July 2015, the Chalkboard was re-launched as a Brookings blog in order to offer more frequent, timely, and diverse content. Contributors to both the original paper series and current blog are committed to bringing evidence to bear on the debates around education policy in America.