With online learning, ‘Let’s take a breath and see what worked and didn’t work’
Kameshwari Shankar watched for years as college and university courses were increasingly taught online instead of face to face, but without a definitive way of understanding which students benefited the most from them, or what if anything they learned.
As an associate professor of economics at City College in New York, Shankar knew that one of the most important requirements of scientific research was often missing from studies of the effectiveness of online higher education: a control group.
Then came the Covid-19 pandemic, forcing almost everyone on earth online and creating a randomized trial on a planetary scale with a control group so big, it was a researcher’s wildest dream.
“The pandemic and the lockdown – that’s a great natural experiment,” said Shankar. A study she co-authored called it “a gold mine of evidence.”
Now the results of this experiment are starting to come in. They suggest that online higher education may work better than pre-pandemic research suggested, and that it is evolving decisively toward a combination of in-person and online, or “blended,” classes.
“For two years we’ve had sort of a petri dish of experimenting with learning online,” said Anant Agarwal, chief platform officer of the online program management company 2U and former CEO of edX, the online provider created by the Massachusetts Institute of Technology and Harvard and sold last year to 2U for $800 million. “Now people are sitting down and saying, ‘Let’s take a breath. Let’s see what worked and didn’t work.’ ”
About a third of higher education was entirely online before the pandemic, and the rest continued to be delivered face to face in brick-and-mortar classrooms, according to the research company Bay View Analytics. The huge spike that occurred during the peak of lockdowns propelled not only unprecedented innovation and investment in online higher education, but also a spree of research into whether and how well it works.
“Initially when we were doing that research it was always on the class or the course level and very rarely were you able to see how online education worked across programs and across institutions,” never mind across the world, said Michael Brown, assistant professor of higher education and student affairs at the Iowa State University School of Education.
Scholars have the same chance to study how kindergartners through 12th graders fared online, though the results will take longer to see than they will for older students, said Carla Johnson, a professor of science education at North Carolina State University and co-author of a study underwritten by the US Department of Education that reviewed all of the preexisting research about this.
That earlier research largely found that primary- and secondary-school students who learned online before Covid – largely in rural and online charter schools – did less well in math and reading than their counterparts in brick-and-mortar classrooms. Based on this, Johnson and her colleagues suggested better teacher training, more equitable access to technology and more support for students to help them work independently.
As for how K-12 students did when almost everyone moved online, however, she said, “you’re looking at three to five years in terms of being able to measure academic impact” as they progress into subsequent grades and take standardized tests.
“There are a lot of people who are really interested in that and there are a lot of people investigating it,” Johnson said.
Online research grew
At this year’s meeting of the American Educational Research Association, 236 papers were presented with the word “online” in their titles, compared with 158 the year before the pandemic – a nearly 50% increase. Felice Levine, the association’s executive director, likened these projects to the real-time, rapid-response studies of the impact of natural disasters such as Hurricane Katrina.
“This is action research on steroids!” a university administrator enthused anonymously in a survey, referring to a type of scholarship that examines an activity in progress.
Just as with studies of the effectiveness of online education for students in kindergarten through 12th grade, much of the pre-pandemic research into online higher education concluded that students in online programs did worse than students in in-person courses. On average, they had lower grades, higher dropout rates and poorer performance in subsequent classes. Outcomes were especially bad for men, Black students and students who had fared poorly in their earlier educations.
But many of these studies suffered from what scholars describe as self-selection bias, said Shankar – in this case, the fact that people who chose to take courses online also shared other inherent characteristics, skewing the results. Students who enrolled in online higher education programs before Covid tended to be older and studying part time, for instance, and often had jobs and families. “We would expect a part-time student with other obligations to perform less well,” Shankar said.
During the pandemic, the variety of students learning online “absolutely increased,” said Di Xu, an associate professor of higher education and public policy at the University of California, Irvine, who began doing research into online learning before Covid. That makes it easier to objectively determine “who seems to benefit and who struggles the most,” Xu said.
Shankar and others have been combing through the massive amounts of newer information generated during the pandemic, when learning online was largely no longer a choice, eliminating self-selection bias. “We are optimistic that the post-pandemic data will show more favorable outcomes,” she said.
Already, in surveys conducted at City College, Shankar and her co-authors have found that online instruction appears particularly effective in large introductory classes with 100 or more students, since those students can use chats and virtual feedback to raise questions they aren’t able to ask in giant, impersonal lecture halls.
Students in small elective classes like it less, saying there is too little opportunity for face-to-face discussion.
Graduations sped up
Another study has found that providing required courses online helps students graduate more quickly, often because in-person versions of the courses are full or unavailable.
Students in general are also at least somewhat more interested in learning online now than they were before the pandemic. That’s the consensus of roughly 75% of university chief online officers in the Changing Landscape of Online Education, or CHLOE, survey by the nonprofit Quality Matters and higher education consulting firm Eduventures. College and university students will all now be online for at least part of their educations, these administrators almost unanimously say.
Not everyone loved online learning during the pandemic – especially in the early stages, when it was at its most haphazard. Nearly three in 10 students in a Strada Education survey in fall 2020 said their ability to learn was much worse online than in person.
Even now, acceptance is lukewarm, emerging research shows.
Only about a third of Americans say they are “probably” or “definitely” confident about the quality of online courses, another Strada survey found. Forty-seven percent of students and 43% of faculty members give online education an A for meeting their educational needs, according to Bay View Analytics.
But 57% of students are more optimistic about classes that are entirely online than they were before Covid. Nearly 99% of the university chief online officers in the CHLOE survey expect an increase in online instruction, up from 70% who expected one before Covid.
As for faculty, even if only 43% give a top grade to online higher education, in Agarwal’s opinion, that’s a win. “Really what happened during the pandemic is people tried it for the first time and many of them liked it,” he said. That has resulted in “a much bigger leap than in the normal course of evolution.”
Another thing that’s taken a big leap is blended learning, combining such things as short videos and frequent, computer-graded tests with in-person classroom lectures and discussion. Though it’s been practiced for a while, there’s now widespread consensus that online higher education, at least for undergraduates, will largely take this form.
“What we have now is an unprecedented agreement about the direction of higher education, in terms of teaching mode, that we have never seen before,” said Jeff Seaman, director of Bay View Analytics, who has tracked this field since 2003.
Faculty favored hybrid classes
By last year, more than half of all faculty said they “agreed” or “strongly agreed” that they wanted to combine online with face-to-face instruction, a Bay View Analytics survey found. A Harvard University task force found that 82% of faculty there were interested in adding digital tools they adopted while teaching remotely to their in-person classes.
That number likely includes many faculty who, before the pandemic, stuck to fully in-person teaching, Brown, of Iowa State, said. “Then this terrible thing happened that forced people to try online education who would have probably resisted it for another decade. If there is a takeaway that’s clear now, it’s that the shift in teaching culture has opened folks up to technology more.”
Agarwal said the pandemic also exposed bad practices, such as online-only classes that consisted almost exclusively of very long lectures.
“The world has not gone back to zero, but has settled somewhere in the middle where we’re coming into a blended future,” he said.
More than four out of five institutions built up their technical teams during the pandemic to support online learning, and are offering more faculty training programs in it, the CHLOE survey found – an infrastructure that will likely stick around.
Some scholars caution that the unprecedented nature of the pandemic could complicate research as much as self-selection bias did before it started. Faculty and students had to juggle outside stress, for instance.
“You have all of these factors going on with the pandemic affecting everything from personal health to family members’ health to employment,” said Cassandra Hart, an associate professor of education policy at the University of California, Davis, who is studying the effect of the pandemic on higher education.
Added Xu: “The shift to remote instruction because of the pandemic was accompanied by so many other things that changed at the same time. It’s really difficult to tease out the impact of instruction.”
Online higher education also took so many forms – from professors lecturing over Zoom to sophisticated multimedia courses developed by professional instructional designers – that it’s hard to generalize about.
“The variability is massive,” Seaman said. “I just see too many people lumping all of this stuff together, and you can’t do that.”
But Shankar said all of those varieties of online learning offer more, not less, opportunity for study. “That would be another gold mine of evidence,” she said: “not just online versus traditional, but also within online what works and what doesn’t.
One finding that has emerged from the pandemic could have the greatest effect of all on online higher education: Nearly half of employers said their opinion of the quality of online credentials has improved, according to a survey by the Northeastern University Center for the Future of Higher Education and Talent Strategy. Still, since so many students who participated in it haven’t yet applied for jobs or even graduated, conclusively measuring the effectiveness of online learning will take time. The academic semester now underway, researchers say, is a first test of new methods and acceptance.
“It’s going to take years for us to really be able to see, out of the things coming out of the pandemic, what works well, what works well in some settings and what works well for some students and not for others,” Hart said.
This article originally appeared in The New York Times.
