Authors:
(1) Raquel Blanco, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (rblanco@uniovi.es);
(2) Manuel Trinidad, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (manuel.trinidad@uca.es);
(3) María José Suárez-Cabal, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (cabal@uniovi.es);
(4) Alejandro Calderón, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (alejandro.calderon@uca.es);
(5) Mercedes Ruiz, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (mercedes.ruiz@uca.es);
(6) Javier Tuya, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (tuya@uniovi.es). Editor's note: This is part 4 of 7 of a study detailing attempts by researchers to create effective tests using gamification. Read the rest below. Table of Links Abstract and 1 Introduction
2 Gamification and software testing education

2.1 Software testing
2.2 Gamification
2.3 Gamification in software testing education


3 Experiment design and 3.1 Course design


3.2 Participants


3.3 Materials


3.4 Procedure


3.5 Metrics



4 Results and 4.1 RQ1: Student engagement

4.2 RQ2: Student performance


5 Discussion
6 Threats to validity
7 Conclusions, Acknowledgements and References 4 Results This section presents the experiment results and answers the research questions introduced in Section 1. The data collected to carry out the analysis are available at http://dx.doi.org/10.17811/ruo_datasets.64866. 4.1 RQ1: Student engagement RQ1: Is the engagement of the software testing students who carry out gamified activities higher than the ones who carry out them in a non-gamified environment? Analyzing each individual exercise, it can be observed that the results for the Number of Test Suite Executions and the Active Time are similar to the ones obtained in the Olympic race. In general, significant differences are found and the mean of the experimental group is greater than the mean of the control group. However, in the last exercise, the mean of the control group for both metrics is slightly higher (there is no significant difference for the Number of Test Suite Executions, whereas significant difference is found for the Active Time). Regarding the Participation Rate, there is a significant difference between both groups in the first three exercises, with a small size effect, and the mean of the experimental group is greater than the mean of the control group. In the last exercise, there is no significant difference and the mean of the experimental group is slightly lower. On the other hand, no significant differences are found in any exercise for the Dropout Rate. The students in both groups started to drop out of the seminar exercises in the middle of the semester, which corresponds with exercise 3 (3% in the control group and 1% in the experimental group). In the last exercise, the dropout was slightly higher (6% in the control group and 7% in the experimental group). Therefore, our findings from the Olympic race are in line with the first three exercises: the students in the experimental group worked more on the exercises and dropped out of them less than the control group. Only in the last exercise the control group seemed to be more engaged; however, no significant differences are found in three out of four metrics. Overall, the null hypothesis is rejected in favor of the gamification experience when the engagement is measured with the four metrics. So, the engagement of the students who perform gamified software testing activities is higher than the ones who perform them in a non-gamified environment. 4.2 RQ2: Student performance RQ2. Is the performance of the software testing students who carry out gamified activities higher than the ones who carry out them in a non-gamified environment? For both metrics, the p-value obtained for the Olympic race is smaller than a, so once again, it can be assumed that there is significant difference between the control and experimental groups, although the effect size is small. Moreover, the mean of the Effectiveness, as well as the mean of the Effectiveness Increase, are higher in the experimental group. Therefore, the students in the experimental group achieved better performance and they worked harder in the test improvement activity to increase the effectiveness. The analysis of each individual exercise also reveals that the experimental group performs better: the difference is significant in both metrics in exercises 2 and 3, with small size effect, and the mean of both metrics is higher in the experimental group in all exercises. Despite the benefits of the gamification experience to improve both the Effectiveness and the Effectiveness Increase, we can observe a downward trend, mainly in the last two exercises. Therefore, the null hypothesis is also rejected in favor of the gamification experience when the performance is measured with both Effectiveness and Effectiveness Increase. So, the performance of the students who carried out gamified software testing activities is higher than the ones who carried them out in a non-gamified environment. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Raquel Blanco, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (rblanco@uniovi.es); (2) Manuel Trinidad, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (manuel.trinidad@uca.es); (3) María José Suárez-Cabal, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (cabal@uniovi.es); (4) Alejandro Calderón, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (alejandro.calderon@uca.es); (5) Mercedes Ruiz, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (mercedes.ruiz@uca.es); (6) Javier Tuya, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (tuya@uniovi.es). Authors: Authors: (1) Raquel Blanco, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (rblanco@uniovi.es); (2) Manuel Trinidad, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (manuel.trinidad@uca.es); (3) María José Suárez-Cabal, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (cabal@uniovi.es); (4) Alejandro Calderón, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (alejandro.calderon@uca.es); (5) Mercedes Ruiz, Software Process Improvement and Formal Methods Research Group, University of Cadiz, Department of Computer Science and Engineering, Cádiz, Spain (mercedes.ruiz@uca.es); (6) Javier Tuya, Software Engineering Research Group, University of Oviedo, Department of Computer Science, Gijón, Spain (tuya@uniovi.es). Editor's note: This is part 4 of 7 of a study detailing attempts by researchers to create effective tests using gamification. Read the rest below. Editor's note: This is part 4 of 7 of a study detailing attempts by researchers to create effective tests using gamification. Read the rest below. Editor's note: This is part 4 of 7 of a study detailing attempts by researchers to create effective tests using gamification. Read the rest below. Editor's note: This is part 4 of 7 of a study detailing attempts by researchers to create effective tests using gamification. Read the rest below. Table of Links Abstract and 1 Introduction 2 Gamification and software testing education

2.1 Software testing
2.2 Gamification
2.3 Gamification in software testing education 3 Experiment design and 3.1 Course design


3.2 Participants


3.3 Materials


3.4 Procedure


3.5 Metrics 4 Results and 4.1 RQ1: Student engagement

4.2 RQ2: Student performance 5 Discussion 6 Threats to validity 7 Conclusions, Acknowledgements and References Abstract and 1 Introduction Abstract and 1 Introduction 2 Gamification and software testing education 2.1 Software testing
2.2 Gamification
2.3 Gamification in software testing education 2 Gamification and software testing education 2.1 Software testing 2.2 Gamification 2.3 Gamification in software testing education 2.1 Software testing 2.2 Gamification 2.3 Gamification in software testing education 3 Experiment design and 3.1 Course design 3.2 Participants


3.3 Materials


3.4 Procedure


3.5 Metrics 3 Experiment design and 3.1 Course design 3.2 Participants 3.3 Materials 3.4 Procedure 3.5 Metrics 3.2 Participants 3.2 Participants 3.3 Materials 3.3 Materials 3.4 Procedure 3.4 Procedure 3.5 Metrics 3.5 Metrics 4 Results and 4.1 RQ1: Student engagement 4.2 RQ2: Student performance 4 Results and 4.1 RQ1: Student engagement 4.2 RQ2: Student performance 4.2 RQ2: Student performance 5 Discussion 5 Discussion 6 Threats to validity 6 Threats to validity 7 Conclusions, Acknowledgements and References 7 Conclusions, Acknowledgements and References 4 Results This section presents the experiment results and answers the research questions introduced in Section 1. The data collected to carry out the analysis are available at http://dx.doi.org/10.17811/ruo_datasets.64866. 4.1 RQ1: Student engagement RQ1: Is the engagement of the software testing students who carry out gamified activities higher than the ones who carry out them in a non-gamified environment? Analyzing each individual exercise, it can be observed that the results for the Number of Test Suite Executions and the Active Time are similar to the ones obtained in the Olympic race. In general, significant differences are found and the mean of the experimental group is greater than the mean of the control group. However, in the last exercise, the mean of the control group for both metrics is slightly higher (there is no significant difference for the Number of Test Suite Executions , whereas significant difference is found for the Active Time ). Number of Test Suite Executions Active Time Number of Test Suite Executions Active Time Regarding the Participation Rate , there is a significant difference between both groups in the first three exercises, with a small size effect, and the mean of the experimental group is greater than the mean of the control group. In the last exercise, there is no significant difference and the mean of the experimental group is slightly lower. On the other hand, no significant differences are found in any exercise for the Dropout Rate . The students in both groups started to drop out of the seminar exercises in the middle of the semester, which corresponds with exercise 3 (3% in the control group and 1% in the experimental group). In the last exercise, the dropout was slightly higher (6% in the control group and 7% in the experimental group). Participation Rate Dropout Rate Therefore, our findings from the Olympic race are in line with the first three exercises: the students in the experimental group worked more on the exercises and dropped out of them less than the control group. Only in the last exercise the control group seemed to be more engaged; however, no significant differences are found in three out of four metrics. Overall, the null hypothesis is rejected in favor of the gamification experience when the engagement is measured with the four metrics. So, the engagement of the students who perform gamified software testing activities is higher than the ones who perform them in a non-gamified environment. 4.2 RQ2: Student performance RQ2. Is the performance of the software testing students who carry out gamified activities higher than the ones who carry out them in a non-gamified environment? For both metrics, the p-value obtained for the Olympic race is smaller than a, so once again, it can be assumed that there is significant difference between the control and experimental groups, although the effect size is small. Moreover, the mean of the Effectiveness, as well as the mean of the Effectiveness Increase, are higher in the experimental group. Therefore, the students in the experimental group achieved better performance and they worked harder in the test improvement activity to increase the effectiveness. The analysis of each individual exercise also reveals that the experimental group performs better: the difference is significant in both metrics in exercises 2 and 3, with small size effect, and the mean of both metrics is higher in the experimental group in all exercises. Despite the benefits of the gamification experience to improve both the Effectiveness and the Effectiveness Increase , we can observe a downward trend, mainly in the last two exercises. Effectiveness Effectiveness Increase Therefore, the null hypothesis is also rejected in favor of the gamification experience when the performance is measured with both Effectiveness and Effectiveness Increase . So, the performance of the students who carried out gamified software testing activities is higher than the ones who carried them out in a non-gamified environment. Effectiveness Effectiveness Increase This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Researchers Say Gamified Learning Boosts Student Engagement and Performance in Software Testing

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Case Study on Gamifying Software Engineering Workplaces

3 Less-Known NFT Games You Can Play Right Now

3 Ways Gamification in Social Media Can Increase User Engagement

5 Best Fitness Games for Your New Years Resolution

Gamification: A New Era in Workplace Experiences

A Case Study on Gamifying Software Engineering Workplaces

3 Less-Known NFT Games You Can Play Right Now

3 Ways Gamification in Social Media Can Increase User Engagement

5 Best Fitness Games for Your New Years Resolution

Gamification: A New Era in Workplace Experiences

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps