Since 2017, Codio has provided services to a supplemental project of the Catalyzing Computing and Cybersecurity in Community Colleges (C5) grant* funded by the National Science Foundation (NSF). Under the leadership of project Co-PI Dr. Melissa Dark of Dark Enterprises, and with support from project PI Dr. Vera Zdravkovich and Dr. Beth Hawthorne, this project was an independent research study to compare the use of a Codio version of a Cryptography module to “traditional materials” where instruction was primarily delivered through PowerPoint slides.
Specifically, Codio’s role included adapting materials developed by C5 contributors for the Codio platform. C5 project team members organized the comparative study that included 19 instructors and nearly 700 students from a mix of high schools and community colleges across the US. Instructors were assigned to one of the two conditions. The results of the entire study were presented at the 2019 Community College Cyber Summit (3CS) and some key highlights are below.
Quantitative Findings
Students took a pre and post test to measure learning gains. An ANOVA analysis revealed a statistically significant (p<.021) difference between learning gains of students using Codio and students using traditional materials.
“Students who learned applied cryptography using Codio learned more than the students who learned in the ‘traditional’ way.”
A stepwise regression was conducted to explore which variables of the study significantly predicted—and to what extent—the learning outcomes. The figure below shows which variables were considered for each outcome.
The High School vs. Community College variable came back as the predictor of pre-test scores.
The use of Codio vs. traditional materials came “back as the strongest predictor of Post-Test Scores.”
Both High School vs. Community College and Codio vs. traditional materials “come back as the strongest predictors of learning gains.”
Qualitative Findings
Following classroom implementation, the instructors were placed into one of four focus groups separated by condition type—Codio users or traditional material users. Each focus group was led by a researcher unaffiliated with the creation of the materials.
Instructors in the Codio condition “reported that Codio provided the best seamless lab environment that students had encountered and their reaction was overwhelmingly positive.” From their perspective, “the Codio platform was a well-functioning, stable virtual lab environment.”
“For what it’s worth 1) I very much enjoyed introducing cybersecurity into my AP Computer Science Principles curriculum, 2) the kids benefit from that exposure, and 3) the Codio mechanism provided a sturdy mechanism to deliver that learning material and thereby achieve my mini-learning targets for this unit.” - High School Teacher
A professor noted that they “appreciated that the students did their own registration, and the instructors did not need to register the students.”
Turning towards the presentation of content on the platform, the “high school instructors appreciated the variety of materials included with Codio.” One instructor shared that “I liked the little short interview things, questions. It was almost like a dating site. ... I know, my advanced class, we watched all of those, and we really had fun watching those. Those were good.”
Acknowledgements
Codio would like to acknowledge the wonderful research leadership of Vera Zdravkovich, Melissa Dark, and Elizabeth Hawthorne as well as Candiya Mann for leading the qualitative portion of the study and John Sener for serving as an assessment advisor. Additionally, Codio is grateful to the 19 faculty who were both partners and participants in this project. Finally, Codio would like to acknowledge the NSF for funding this project (#DGE-1548315, Amendment 001) and Whatcom Community College for serving as the academic home for this study.
*The C5 project is based at Whatcom Community College in Bellingham, Washington.
