Levelling Up Learning: Exploring the Impact of Gamification in Flipped Classrooms: Discussion

Table of Links

• Abstract and Introduction
• Course information
• Motivation and intervention
• Findings
• Discussion
• Conclusion and References

5 Discussion

The purpose of this study was to investigate the effects of a gamification element in a mathematical sciences course taught in flipped format. The study compared levels of engagement with pre-lecture quizzes before and after the gamification intervention was introduced. Comparisons were made across semesters (different courses, same students) and across academic years (same course, different students). Our study found that in both cases the level of engagement (measured by average percentage of students completing pre-lecture quizzes prior to the associated in-person sessions) was significantly higher when compared to a course where Level Up! was not implemented. Although engagement levels did drop over the course of each semester, they remained substantially higher for each week when compared to the previous semester or year. These results provide convincing evidence that introducing a gamification element to a course taught in flipped format increases students motivation and engagement to complete work outside of scheduled class time.

When comparing engagement levels within students between two semesters, we found a general increase in engagement across both semesters but more interestingly found that some students with very low engagement levels in semester 1 went on the have some of the highest engagement levels in semester 2. For these students it seems that the gamification element drastically improved their motivation and engagement levels and is evidence that including a gamification element to a course may encourage engagement from students who would otherwise not engage.

Our results also found that this increase in engagement could lead to an increase in performance with 82% of students receiving a grade of A, B or C in the semester 1 course of 2023-24 compared to 60% in 2022-23. Although we cannot say for sure what led to this increase in achievement and there are many factors which could influence this (such as cohort effect, different exam paper, etc.), an increase in engagement throughout a semester is compatible with an increase in performance since students are more likely to be distributing their learning over the course of a semester. This is especially beneficial for mathematical sciences courses where learning is often thought of as linear with each topic building on the previous one and poor understanding of early material can have a detrimental effect later in the course [Ireland and Mouthaan, 2020].

This study focuses on Year 1 undergraduate students who typically fall into a post-secondary education age range. The transition to university for students can be overwhelming, with changes in teaching delivery, workload and cognitive demands ([Blair, 2017]). Our results show that gamification can improve the learning experience, which may help with this transition, in line with observations by [Zaric et al., 2017]. This being said, [Kim and Castelli, 2021] looked at the effects of gamification across three age cohorts and found that university level participants found the least significant results in terms of positive interaction when compared to secondary school and adult learners, indicating that younger and older learners may have more interest in gamified factors. Currently, gamification is only present within the Year 1 modules of the undergraduate programme, and is not implemented within other modules on the programme. There is scope to explore the effects of gamification on a more mature cohort to observe if we see similar patterns of engagement.

This study has considered student engagement levels in terms of completing pre-lecture quizzes, though it is important to note that we see meaningful engagement instead of cases where students are simply trying to ‘game’ the leaderboard. Our results are based solely on participation on formative reading quizzes where participation is identified through criteria being met, as this material was found to show that genuine participation took place. There are other activities in the course, such as textbook readings, which simply require a manual completion indicator and could be exposed to non-meaningful engagement. We feel that a reasonable performance on the quizzes indicates engagement with the other activities available (or indeed, of students independently finding their own resources for study).

The results presented here are for a self-selected subset of students. However, course instructors were able to view results for the entire cohort of students to inform their future decision making on the learning and teaching approach for this course and others. The results presented here were found to be similar to the results for the entire cohort and the general conclusions made would not change if data on the entire cohort of students were considered.

Although studies in gamification have reflected positive outcomes in terms of increased motivation and engagement in tasks, this teaching strategy has potential downsides. Effects such as increased competition between students ([Hakulinen et al., 2013]) and task evaluation difficulties, where certain tasks may not be suitable for a game based element or may have ways of being marked as complete without proper engagement in the learning materials ([Domínguez et al., 2013]) may become issues. The design of game based elements within a course must also be considered with a balance of reward and level of work and engagement required. Tasks which are overly complex can lead to a lack of engagement from learners ([Dong et al., 2012]). There is also potential for demotivation through the full gamification process, particulaly if students find themselves placed low on the points leaderboard. The competency levels of students may lead to negative consequences, including frustration and loss of self confidence ([Alomari et al., 2019]). A study by [Ding et al., 2018] showed that some students in a gamified learning activity may require more time to obtain the necessary understanding to progress through learning process.

Engagement in gamification not only requires active participation from students, but time and attention from staff to implement the system effectively. From our experience using Level Up!, it takes time to set up the system. Several considerations have to be made at the set-up stage, including the allocation of experience points to certain tasks, and allocating points in such a way as to reflect the level of engagement required from the student. Levels were attainable for students which were split by weekly tasks, which requires manual intervention to tally the available points over a given week. For example, students could receive half points for completing certain tasks after a deadline which requires manual intervention from the course lecturer to reset the points awarded to a different total after each deadline. Though our study explored gamification using Level Up!, there are a host of other plug-ins which can be adapted in virtual learning environments such as Quizventure[4] and Block Game[5]. [Heilbrunn et al., 2017] provide a detailed review of tools for gamification analytics.

Finally, we highlight the importance of using learning analytics, where possible, to access data from students on a course when measuring the success of a new teaching method or intervention. If the instructors of the two courses under consideration here had relied on feedback from students to inform teaching strategy - which was consistently positive even before the implementation of Level Up! - the true extent of the lack of engagement in the pre-lecture materials may never have been identified.

[4] https://github.com/xow/moodle-mod_quizgame

[5] https://moodle.org/plugins/block_game