Colorado Study finds “No Significant Difference” in Online Science Courses
Published by: WCET | 10/18/2012
Tags: Data And Analytics, Lab Courses, Outcomes, Research, STEM
A big thank you to our guest blogger, Rhonda Epper, Assistant Provost at the Colorado Community College System and past chair of the WCET Executive Council, who shares with us today a new study of online science course outcomes in Colorado.
Just when we thought it was safe to move beyond the “no significant difference” phenomenon[i], knowing that scarcely any college class today is untouched by technology – we find ourselves in 2012 still engaged in debates about “online” versus “traditional” instruction. We don’t find this debate at conferences or annual meetings like WCET where conversations are centered on innovation and change. But within most colleges and universities, there are influential faculty members and groups who hold onto deeply held doubts about the quality of online learning. There is perhaps no discipline with more misgivings about online learning than Science.
Most of the genuinely held concerns among science faculty are rooted in the laboratory experience for students. The “hands-on” use of real lab equipment to gather and analyze real data is perceived as necessary for students in lab-based science courses such as Biology, Chemistry, and Physics. In the Colorado Community College System (CCCS), the lab portion of online science courses has been delivered since 2001 primarily through the use of commercially designed and assembled “kits” that are sent to the student’s home. The kits typically contain a lab manual along with the equipment and supplies needed to perform experiments, though often on a smaller scale or with lower-capability instrumentation than one might find in a traditional campus science lab.
Our online science faculty, many of whom teach both online and in the classroom, have worked with the kit manufacturer over the years to refine the lab experiments, and have continually improved the quality of the course content to ensure that students are mastering the statewide course objectives and competencies for lower division science courses.
Yet questions have persisted from traditional science faculty, and from within articulation committees considering whether students who have taken introductory science courses online are truly prepared to move into and be successful in science courses at the four-year level. Until now, we have not been able to refute these arguments.
In September 2012, the Colorado Department of Higher Education released a comparison study of CCCS students who took science courses online versus in traditional classrooms, and then tracked those students who transferred into four-year institutions in Colorado. The data set included students enrolled in first year Biology, Chemistry, and Physics for majors.[ii] Data were pulled for academic years Fall 2007 to Fall 2009. The sample of CCCS students totaled 4,585 (2,395 taking science courses online and 2,190 taking traditional science courses in the classroom). The study examined cumulative GPA, cumulative credit hours, and science-only GPA.
The first part of the study looked at online and traditional students only within the community college system. The greatest difference between students in the online versus traditional science courses were with grades earned in those specific science courses. Students in traditional Biology and Chemistry classes received, on average, higher course grades than students in online classes. Grades earned in Physics were similar for both online and traditional students. In spite of lower grades in Biology and Chemistry classes, the online students had either very similar or slightly higher overall GPAs than students completing these courses in traditional classrooms. Furthermore, the online students in all three science disciplines had similar or higher cumulative credit hours earned than their traditional counterparts. While the interpretations of these findings can be wide ranging, it is suggested that the higher GPAs and higher cumulative credit hours completed by online students could be the result of more experienced and academically prepared students self selecting online courses. It is also interesting to note that a widely held rumor that faculty in online courses have “easier” grading practices appears not to be the case.
The second part of the study tracked students who transferred and took science classes at one of four four-year public institutions in Colorado (CU-Boulder, CU-Denver, CU-Colorado Springs, and Colorado State University- Fort Collins). An average science GPA at the four-year institution was calculated for each student, along with Biology, Chemistry, and Physics GPAs. The bottom line: there were no statistically significant differences in students’ science GPAs based on the community college instructional delivery method (online versus traditional). The study suggests that students who took online science courses at the community college level perform just as well in science classes at four-year institutions as students who took traditional on-campus science classes.
In the CCCS, we believe it is part of our mission to address barriers that prevent underrepresented, low-income college students from completing their degrees due to challenges such as work and family obligations or living in rural areas that limit their access to traditional lab-based science courses (critical courses for many of our allied health and science programs that lead to lucrative jobs). Toward that end, we are continuing to push the envelope on innovative science education delivery by serving as a lead partner in the international NANSLO project — North American Network of Science Labs Online.[iii] In partnership with WICHE, BC Campus, North Island College, the Colorado School of Mines, and numerous colleges and universities throughout the western United States, we are implementing and developing remote web-based access to sophisticated lab equipment for science and allied health students. Through NANSLO, our online science students are able to use real lab equipment (e.g., high powered digital microscopes) to collect and analyze real data while working collaboratively in real time with their fellow students in a lab group. NANSLO is openly licensing how-to manuals and curricular materials for introductory science courses using remote labs. We are researching the pedagogical value and optimal combination of remote labs and lab kits mapped directly to scientific principles and laboratory skills. This work has been supported by the Next Generation Learning Challenges Program and a recently awarded U.S. Department of Labor TAACCCT grant.
Increasingly, the work of scientists is carried out via remote technology and robotics (think of engineering, space exploration, deep sea submersibles, and even medical procedures). But our educational model lags behind. As the Colorado study has shown, research once again supports the efficacy of online delivery, even in science courses. Now we must work together to ensure that the science education we are delivering is relevant, high quality, and accessible.
Assistant Provost
Colorado Community College System
[i] Thomas L. Russell, “The No Significant Difference Phenomenon.” 2001, IDECC, fifth edition.
[ii] The study is currently being expanded to include both Algebra-based and Calculus-based Physics courses. Only the Algebra-based courses were included in the current study.
30 replies on “Colorado Study finds “No Significant Difference” in Online Science Courses”
[…] Colorado Study finds “No Significant Difference” in Online Science Courses « WCET Frontiers. Posted in Teaching Effectiveness […]
Yes, science has been short-changed in online education at both secondary and post-secondary levels due to prejudice and bias among traditional faculty.
As a scientist and former chemistry professor at a large university, I understand this attitude toward science majors. I cannot understand requiring non-majors to learn to use arcane science equipment, much of it 19th century in vintage, if not manufacture. How many business majors have to know how to light a Bunsen burner or manipulate a microscope.
I do understand having some physically hands-on experience for everyone. I also understand the reticence of science educators to use online simulations as substitutes for labs. The National Research Council has agreed on this point.
These conclusions lead to a couple of questions. Must every lab experience be physically hands-on? If not, what should the mix be?
The answer to the first question is no with the qualification that some physical experiments add to the overall experience and should not be omitted entirely. However, this answer depends on having a decent alternative to virtual simulations masquerading as labs. That alternative is online hands-on labs with interactive point-by-point data collection from real experiments.
Given that there’s a third alternative rather than a dichotomy, the question comes down to the mix between physical hands-on experiments and online hands-on experiments. That decision must be in the hands on instructors and curriculum creators. I’d start with 50-50 and work from there.
So, instead of students paying $200 or more for a lab kit that often contains modeling or verification labs or procedure labs instead of investigation labs, why not have students assemble the $30 to $50 worth of materials necessary to do simple experiments and fill in with $30 worth of online hands-on labs? It’s a very large savings for the students. It involves technology in an important way, and it does not use simulations as labs.
In the world of liberal arts education, our goal is not to teach business majors to be master surgeons with state-of-the-art equipment, but to educate our citizens to think beyond their current interests and be well-rounded. This allows them to be better prepared for the next 5 careers they will have before they can retire.
Just as science majors learn history from professionals who practice in their field, non-science majors can learn much more than how to work in a lab during chemistry. However, learning how to safely work with chemicals that would not be safe to dispose of in the public water supply, and to engage in the process is still a topic for organized labs and in-person instruction. Even non-science majors learn more about themselves when they learn the limits of their power in controlling a reaction they have started.
Your first paragraph is spot on.
Then you meander away from the point and propose specious reasons for foisting traditional, physically present science labs from the 18th century on all who come calling.
I say that the primary purposes of a science lab are to develop scientific thinking skills and to understand the nature of science. Safe chemical disposal, reaction control, and the rest are just kind of nice but nowhere near to being necessary.
Yet, simulations do not fulfill the requirements either, which means that I do understand the motivation of many who oppose dropping any of the traditional labs from their courses. After all, I am a former chemistry professor from a large university. I taught a course of 350 students with labs that I designed to go with the course.
These days, we can do better. I believe that we can cut the number of physical-facility labs in half without harming the experience at all if we substitute and even add online hands-on labs to the mix. I’d rather just do the adding on for science majors, but for the rest, they’ll save time and money. So will the school. The learning outcomes will be at least equal and likely better than the old way.
You see, I still think that there’s benefit in doing some stuff physically hands-on just for the experience, having that kinesthetic feel, and for the range of experimental design options in this mode. However, these labs are expensive and time-consuming, often require special space, and may pose unforeseen safety hazards. We shouldn’t do more than are really necessary.
If these traditional labs were the only option other than simulations, I’d hold my breath and say that we should stick with them. That situation, however, no longer holds. Welcome to a new era in science education.
Dr. Taylor,
This WAS SUPPOSED to be what the second large NSF grant at ASCC was to be doing. They could have had the World in terms of potential distance learning enrollment in other states and countries. The Pulp, Paper, and Chemical Technology Lab was planned to have complete remote access and manipulation with numerous cameras and sensors – Auburn University steered the project away from such a process during the grant implementation. They said it was not practical. ASCC faculty were against it as well. WE thought about it before the science world thought about such applications. About two years after the fact, I found out that the Auburn Chem Department was working on remote sensing systems for use by their graduate assistant students –for high paying contracts with pulp and paper companies and their faculty were getting big research dollars. I was on the Auburn Campus with Dr. Johnson and I wondered into a lab in their brand new chem lab and saw the signal receivers from remote pulp and paper plants connected to computers, etc. At any rate, the article is interesting. Perhaps we could go for an NSF grant for science, etc. — someday when accreditation is not on our minds.
Good job Dr. Taylor on the Outcomes.
Take care,
John
_______________________________________________
Description: Description: Description: Amridge University
Dr. John H. White Vice President of Institutional Research and Legal Affairs 1200 Taylor Road Montgomery, AL 36117 Local: 334.387.7450 Toll Free: 800.351.4040 Ext. 7507 Fax: 801.340.2583
E-Mail: johnwhite@amridgeuniversity.edu Web Site: http://www.amridgeuniversity.edu
I hesitated to approve the comment from Harry Keller once I figured out that he represents a private concern (Smart Science) that sells lab experiences. I wanted the readers to understand his connection with that company when considering his point-of-view.
Russ
http://www.paracompusa.com/SmartScience/SmartScienceMedia.html
My apologies to Russ and other readers for not identifying myself more clearly. I’m not as well known as I would like to be. Neither is my service. Some forums do not allow commercial mention or websites. Thanks, Russ.
[…] morning Inside Higher Ed reports on a new study finding that students who took introductory science courses online did as well in more advanced […]
Well, if the students are actually doing the labs with home kits, then the results aren’t surprising to me: Lower average grades, but equal learning among the students who do pass.
This agrees with my own personal data collection for hybrid and online math classes. In any decently designed online class, those who succeed in the online version have really mastered the material. However, purely online classes do have lower pass rates, possibly because students are presented with less personal interaction and fewer learning modalities.
What concerns me is that in some high schools, out of some combination of budget cuts and liability fears, students never get to use real lab equipment at all and do ALL their labs in simulation. Even folks who are’t science majors should be know what it’s like to actually perform experiments, see the variation between expected results and actual results, and see some experiments fail due to mistakes, faulty equipment, or bad luck.
I agree generally with the first two paragraphs by Young CC Prof. I can almost agree with the last paragraph, except that it makes the incorrect assumption of a dichotomy: physically hands-on labs and online simulations.
YES, students should have some opportunity actually to perform live, physically hands-on, immediately available experiments. INDEED, students should never be subjected to simulations as a substitute for labs (although you can use them for learning in other contexts). You CAN do real experiments online with true student-mediated point-by-point data collection.
Disclaimer: I invented online hands-on labs and am president of Smart Science Education Inc. I am a scientist trained in the best institutions our nation has to offer. These online labs are really good.
Interesting Study.
I don’t necessarily accept the argument for “no-difference” , as we need to compare a typical online vs typical traditional program.
While the results indicate that the two groups (online vs traditional class) are roughly the same down the road, I’d argue that this online classroom isn’t the necessarily the “norm” in that the students were given science kits that “Our online science faculty, many of whom teach both online and in the classroom, have worked with the kit manufacturer over the years to refine the lab experiments, and have continually improved the quality of the course content to ensure that students are mastering the statewide course objectives and competencies for lower division science courses.”
Compare that online course preparation, delivery and outcome, to a more common one (i.e a traditional instructor is told to “place their course online next semester”) and compare those results.
Yes, online courses can compete, but that is not the norm.
[…] “No Significant Difference” in Online Science Courses […]
Dr. Epper, I am not surprised of the results of the study. Quality online learning is equal to quality classroom learning. Only a caveman living under a rock for the last 30 years would be surprised. What I am amazed by is the skepticism and refusal for some who still refuse to accept the study’s findings. How biased does one have be?
Also, I take exception to the “typical online” compared to the “typical traditional” classes. Please define what typical is. My recollection of a typical college biology class was boring lecture, read your textbook, and meet me in the lab. My typical online classes (3 degrees earned online) were of much higher quality, professional designed, and pedagogically implemented.
Unfortunately, the quality of online courses varies enormously, not to say that traditional course quality does not also vary. Anecdotal evidence notwithstanding, the online can be better or worse than the classroom version.
Let’s hope that online continues to improve because classroom versions are about as good as they’re going to get, and the average is not great.
Personally, I think that the potential for online blows away the classroom version. We just don’t see that potential fully developed yet.
No doubt on line education can do “as well” as large lectures. And simulations can work for some types of learning, and kits for some types of experiments.
To me, simulations should not be called labs but a valid extension of the course. I am not sure how I can be convinced that not handling a microscope, not doing actual stains, not dropping a tube of bacteria on the floor and having an instructor help can be duplicated. Instead, perhaps there could be Saturday labs or a week on site (perhaps during spring break).
The working full time problem is actually not solved by this other than helping with transportation and daycare. You still need time to study and think.
How can we explore this from an unbiased manner?
Steven, let me just clarify that we are not using simulations in our courses to replace any experiments. The experiments using remote labs allow students to use real equipment to produce real data. The lab kits are hands-on lab experiments that students conduct in their homes. In neither case are there simulated data or outcomes.
Thanks Rhonda. I see the point. I guess it still depends on the type of experiment and learning outcomes desired. But I could see where in some areas where this is fine and emphasizes the key thing of how to think.
Steven’s first paragraph is exactly correct. He begins the second paragraph accurately and then strays. Perhaps, he teaches biology. Then, the rest would be correct for science majors taking his courses but not for the business majors.
Why should a business major care about how work a microscope or how to do a slide stain? How do these lab equipment skills translate into later value?
Yet, simulations do not substitute for labs as Steven correctly asserts. Neither the supervised labs nor the kit labs are absolutely necessary for learning from lab investigation experience. Either is a useful adjunct to a course. In the end, it’s all about balance.
How much of a course can be provided as online hands-on labs? How much must be done in a supervised facility? How much can be done “wet” at home?
I am familiar with online science courses in which the students are literally thousands of miles away from the school. The CTY courses of Johns Hopkins are an example. I corresponded with one student taking AP Biology who as a daughter of the Saudi royal family and was in Saudi Arabia. This was a fully audited (by the College Board) course that used lots of online labs but not simulations.
In these instances, Saturday labs are not an option. Neither is a week on campus. Online courses are becoming more and more geographically independent. If you’re planning on a career in a science laboratory, you must find ways to get to a supervised facility for your college courses, definitely after your freshman year and as much as possible that year as well.
For those not majoring in science and medicine, supervised facilities are an expensive extravagance. Even lab kits are a tax on a great many students’ budgets, especially those who attend community colleges for cost reasons. Why not do as CTY did?
I see what you are saying Harry. Hmmm… then I might accept it for a different level course or what might be called non-majors. A business student perhaps would not need it at all. Of course, they would lose those skills and understanding that might be helpful in a biotech or health business but then perhaps they might never have taken any labs at all in that area. As long as they are not “misled.” Still I see your point.
A few years ago students always commented how much they loved using the microscopes and see the bacteria and that impacted me as a kid. Seeing pond water under microscope changed me in a way that a video would not. But times have changed.
Yes, Steven. You have it right when you say earlier, “… emphasizes the key thing of how to think.”
For me (I’m a chemist, BTW), the issues are items 2, 3, and 5 on the list of lab goals in America’s Lab Report.
• Developing scientific thinking skills
• Appreciating the complexity and ambiguity of empirical work
• Understanding the nature of science.
These are harder to get outside of a lab experience than when doing real investigations in the real world. I hope they are the major take-away from a science course, especially for the non-majors. In my mind, these are the basis for the how-to-think that goes with science.
There are other modes of thinking that you’ll learn about in other courses. All are valid, but scientific thinking should not be short-changed, even for business majors — at least as I see it.
Steven, this study did not compare online instruction to large lectures. It compared online instruction to traditional community college classes, which are sometimes smaller than the online class sections. This is significant because there have been doubts for many years that students taking online science courses are as well prepared for the 4-year college experience as traditional community college students are.
Thank you Dan. So much to think about in terms of these issues.
[…] See on wcetblog.wordpress.com […]
[…] Colorado Study finds “No Significant Difference” in Online Science Courses […]
The ETC Journal posted a critique of the Colorado study that opined that there is “No Satisfaction in finding on Online vs. Traditional Science Classes” and then talks mostly about how all college science is lacking:
http://etcjournal.com/2012/10/22/no-satisfaction-in-wcet-finding-on-online-vs-traditional-science-classes/
Dan Brannan of the Colorado Community College System wrote a response blog posting that says that the critique misses the point:
http://etcjournal.com/2012/10/24/not-satisfied-but-hopeful-about-online-science/
Russ
[…] full blog post has more about the study, the methodology used, and the […]
[…] Read a study synopsis and more information on the WCET Blog. […]
[…] http://wcetblog.wordpress.com/2012/10/18/co_nsd/ […]
[…] courses for students to choose from. And the data backs us up: a Colorado study recently found no difference in outcomes between online courses and in-classroom learning. The focus now is not on place; […]
[…] http://wcetblog.wordpress.com/2012/10/18/co_nsd/ […]