Authors:
(1) Philipp Straubinger, University of Passau, Germany;
(2) Gordon Fraser, University of Passau, Germany. Table of Links Abstract and 1 Introduction 2.1 Software Testing 2.2 Gamification of Software Testing 3 Gamifying Continuous Integration and 3.1 Challenges in Teaching Software Testing 3.2 Gamification Elements of Gamekins 3.3 Gamified Elements and the Testing Curriculum 4 Experiment Setup and 4.1 Software Testing Course 4.2 Integration of Gamekins and 4.3 Participants 4.4 Data Analysis 4.5 Threats to Validity 5.1 RQ1: How did the students use Gamekins during the course? 5.2 RQ2: What testing behavior did the students exhibit? 5.3 RQ3: How did the students perceive the integration of Gamekins into their projects? 6 Related Work 7 Conclusions, Acknowledgments, and References ABSTRACT Testing plays a crucial role in software development, and it is essential for software engineering students to receive proper testing education. However, motivating students to write tests and use automated testing during software development can be challenging. To address this issue and enhance student engagement in testing when they write code, we propose to incentivize students to test more by gamifying continuous integration. For this we use Gamekins, a tool that is seamlessly integrated into the Jenkins continuous integration platform and uses game elements based on commits to the source code repository: Developers can earn points by completing test challenges and quests generated by Gamekins, compete with other developers or teams on a leaderboard, and receive achievements for their test-related accomplishments. In this paper, we present our integration of Gamekins into an undergraduate-level course on software testing. We observe a correlation between how students test their code and their use of Gamekins, as well as a significant improvement in the accuracy of their results compared to a previous iteration of the course without gamification. As a further indicator of how this approach improves testing behavior, the students reported enjoyment in writing tests with Gamekins. CCS CONCEPTS • Software and its engineering → Software testing and debugging; • Social and professional topics → Software engineering education KEYWORDS Software Testing, Gamification, Continuous Integration, Education ACM Reference Format: Philipp Straubinger and Gordon Fraser. 2024. Gamifying a Software Testing Course with Continuous Integration. In 46th International Conference on Software Engineering: : Software Engineering Education and Training (ICSESEET ’24), April 14–20, 2024, Lisbon, Portugal. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3639474.3640054 1 INTRODUCTION Software testing is a well-established concept and is extensively used in the industry [13]. However, despite the availability of various tools that facilitate the testing process, such as testing support in integrated development environments (IDE) [26] and automated test execution in continuous integration (CI) platforms [8], testing is often overlooked in practice. Speculated reasons for this include developers’ lack of motivation to engage in testing activities and a lack of education in software testing [14, 30, 41, 49, 50]. Although the often inadequate treatment of testing in higher education [41] has recently been countered with a growing recognition of the importance of testing [9, 29, 35], this is nevertheless hampered by learners just like developers tending to perceive testing as tedious and boring [32]. As a result, the industry suffers significant losses due to inadequate software quality and insufficient testing [31]. Gamification techniques provide an opportunity to motivate students to test despite their perception of it being boring or tedious. Gamification involves incorporating elements commonly found in games, such as leaderboards and achievements, into nongame contexts [15]. It has been demonstrated that this approach helps engage students in software engineering education [24], for example by gamifying lectures or tools to teach software testing [7, 12, 16, 22, 36]. However, in order for students to internalize testing and establish it as an integral part of their development approach, we argue that seamless integration of gamification of testing into the software development process is necessary. In order to achieve this, we integrate gamification into continuous integration (CI) systems used by students. Specifically, we use Gamekins [47], a plugin designed for the widely used CI platform Jenkins[1], which seamlessly integrates gamification into the software development workflow. By analyzing source code and test results, Gamekins identifies areas where testing can be improved based on code coverage [53] or mutation analysis [28] and then incentivizes improvement using gamification concepts such as challenges and quests, rewarding good testing with points, achievements, and leaderboard rankings. We evaluated the benefits of integrating Gamekins in an undergraduate software testing course, investigating its impact on the course’ learning objectives and student behavior. In detail, the contributions of this paper are as follows: • We propose the use of gamification as a means to incentivize and reward testing activities in CI. • We introduceGamekins as a tool to be used in an undergraduate-level software testing course. • We empirically evaluate the effects of integrating Gamekins into the software testing course, comparing to a previous cohort without gamification and surveying the students. The results of the study demonstrate a correlation between students’ testing behavior and the use of Gamekins, as well as a significant improvement in correct results compared to the previous edition of the course, where no gamification was used on the same assignments. While not all aspects of Gamekins were universally liked by students, overall, they enjoyed its use throughout the course, confirming that Gamekins is a viable teaching tool. This paper is available on arxiv under CC BY-SA 4.0 DEED license. [1] https://www.jenkins.io/ Authors: (1) Philipp Straubinger, University of Passau, Germany; (2) Gordon Fraser, University of Passau, Germany. Authors: Authors: (1) Philipp Straubinger, University of Passau, Germany; (2) Gordon Fraser, University of Passau, Germany. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2.1 Software Testing 2.1 Software Testing 2.2 Gamification of Software Testing 2.2 Gamification of Software Testing 3 Gamifying Continuous Integration and 3.1 Challenges in Teaching Software Testing 3 Gamifying Continuous Integration and 3.1 Challenges in Teaching Software Testing 3.2 Gamification Elements of Gamekins 3.2 Gamification Elements of Gamekins 3.3 Gamified Elements and the Testing Curriculum 3.3 Gamified Elements and the Testing Curriculum 4 Experiment Setup and 4.1 Software Testing Course 4 Experiment Setup and 4.1 Software Testing Course 4.2 Integration of Gamekins and 4.3 Participants 4.2 Integration of Gamekins and 4.3 Participants 4.4 Data Analysis 4.4 Data Analysis 4.5 Threats to Validity 4.5 Threats to Validity 5.1 RQ1: How did the students use Gamekins during the course? 5.1 RQ1: How did the students use Gamekins during the course? 5.2 RQ2: What testing behavior did the students exhibit? 5.2 RQ2: What testing behavior did the students exhibit? 5.3 RQ3: How did the students perceive the integration of Gamekins into their projects? 5.3 RQ3: How did the students perceive the integration of Gamekins into their projects? 6 Related Work 6 Related Work 7 Conclusions, Acknowledgments, and References 7 Conclusions, Acknowledgments, and References ABSTRACT Testing plays a crucial role in software development, and it is essential for software engineering students to receive proper testing education. However, motivating students to write tests and use automated testing during software development can be challenging. To address this issue and enhance student engagement in testing when they write code, we propose to incentivize students to test more by gamifying continuous integration. For this we use Gamekins, a tool that is seamlessly integrated into the Jenkins continuous integration platform and uses game elements based on commits to the source code repository: Developers can earn points by completing test challenges and quests generated by Gamekins, compete with other developers or teams on a leaderboard, and receive achievements for their test-related accomplishments. In this paper, we present our integration of Gamekins into an undergraduate-level course on software testing. We observe a correlation between how students test their code and their use of Gamekins, as well as a significant improvement in the accuracy of their results compared to a previous iteration of the course without gamification. As a further indicator of how this approach improves testing behavior, the students reported enjoyment in writing tests with Gamekins. CCS CONCEPTS CCS CONCEPTS • Software and its engineering → Software testing and debugging; • Social and professional topics → Software engineering education • Software and its engineering → Software testing and debugging; • Social and professional topics → Software engineering education KEYWORDS KEYWORDS Software Testing, Gamification, Continuous Integration, Education ACM Reference Format: ACM Reference Format: Philipp Straubinger and Gordon Fraser. 2024. Gamifying a Software Testing Course with Continuous Integration. In 46th International Conference on Software Engineering: : Software Engineering Education and Training (ICSESEET ’24), April 14–20, 2024, Lisbon, Portugal. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3639474.3640054 1 INTRODUCTION Software testing is a well-established concept and is extensively used in the industry [13]. However, despite the availability of various tools that facilitate the testing process, such as testing support in integrated development environments (IDE) [26] and automated test execution in continuous integration (CI) platforms [8], testing is often overlooked in practice. Speculated reasons for this include developers’ lack of motivation to engage in testing activities and a lack of education in software testing [14, 30, 41, 49, 50]. Although the often inadequate treatment of testing in higher education [41] has recently been countered with a growing recognition of the importance of testing [9, 29, 35], this is nevertheless hampered by learners just like developers tending to perceive testing as tedious and boring [32]. As a result, the industry suffers significant losses due to inadequate software quality and insufficient testing [31]. Gamification techniques provide an opportunity to motivate students to test despite their perception of it being boring or tedious. Gamification involves incorporating elements commonly found in games, such as leaderboards and achievements, into nongame contexts [15]. It has been demonstrated that this approach helps engage students in software engineering education [24], for example by gamifying lectures or tools to teach software testing [7, 12, 16, 22, 36]. However, in order for students to internalize testing and establish it as an integral part of their development approach, we argue that seamless integration of gamification of testing into the software development process is necessary. In order to achieve this, we integrate gamification into continuous integration (CI) systems used by students. Specifically, we use Gamekins [47], a plugin designed for the widely used CI platform Jenkins[1], which seamlessly integrates gamification into the software development workflow. By analyzing source code and test results, Gamekins identifies areas where testing can be improved based on code coverage [53] or mutation analysis [28] and then incentivizes improvement using gamification concepts such as challenges and quests, rewarding good testing with points, achievements, and leaderboard rankings. We evaluated the benefits of integrating Gamekins in an undergraduate software testing course, investigating its impact on the course’ learning objectives and student behavior. In detail, the contributions of this paper are as follows: • We propose the use of gamification as a means to incentivize and reward testing activities in CI. • We introduceGamekins as a tool to be used in an undergraduate-level software testing course. • We empirically evaluate the effects of integrating Gamekins into the software testing course, comparing to a previous cohort without gamification and surveying the students. The results of the study demonstrate a correlation between students’ testing behavior and the use of Gamekins, as well as a significant improvement in correct results compared to the previous edition of the course, where no gamification was used on the same assignments. While not all aspects of Gamekins were universally liked by students, overall, they enjoyed its use throughout the course, confirming that Gamekins is a viable teaching tool. This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv [1] https://www.jenkins.io/

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Gamification Makes It Easier to Teach Software Testing – New Study Shows

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Case Study on Gamifying Software Engineering Workplaces

Boosting Developer Engagement in Software Testing with Gamification

How to Gamify Continuous Integration

How Gamekins Integrates Gamification into Continuous Integration for Better Testing

What Happened When Students Used Gamekins for Testing?

A Case Study on Gamifying Software Engineering Workplaces

Boosting Developer Engagement in Software Testing with Gamification

How to Gamify Continuous Integration

How Gamekins Integrates Gamification into Continuous Integration for Better Testing

What Happened When Students Used Gamekins for Testing?

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps