IntelliGame in Action: Gamifying JavaScript Unit Tests - Conclusions, and References

Table of Links

Abstract and Introduction

Background and Related Work

Implementation

Experiment

Results

Conclusions, Acknowledgement, and References

6 CONCLUSIONS

This paper presents the integration and empirical evaluation of IntelliGame, a gamification plugin originally designed for Java development, in the realm of JavaScript unit testing. The study aimed to validate IntelliGame’s effectiveness in JavaScript, adapting it to the popular Jest testing framework. Despite challenges such as transitioning from IntelliJ Community Edition to Ultimate for JavaScript support and addressing differences in coverage information, we successfully ported 19 out of 26 achievements. The controlled experiment with 152 participants revealed mixed perceptions of IntelliGame’s impact, with achievements influencing testing behaviour and participants’ motivation. We will continue with an in-depth analysis of the experimental measures and the code written by the participants to gain more insights and to broaden our knowledge of IntelliGame.

ACKNOWLEDGEMENT

This study was carried out within the “EndGame - Improving Endto-End Testing of Web and Mobile Apps through Gamification” project (2022PCCMLF) – funded by European Union – Next Generation EU within the PRIN 2022 program (D.D.104 - 02/02/2022 Ministero dell’Università e della Ricerca). This manuscript reflects only the authors’ views and opinions and the Ministry cannot be considered responsible for them. This work is also supported by the DFG under grant FR 2955/2-1, “QuestWare: Gamifying the Quest for Software Tests”.

REFERENCES

[1] Riccardo Coppola, Tommaso Fulcini, Luca Ardito, Marco Torchiano, and Emil Alègroth. 2023. On Effectiveness and Efficiency of Gamified Exploratory GUI Testing. IEEE Transactions on Software Engineering (2023), 1–16. https://doi.org/ 10.1109/TSE.2023.3348036

[2] Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From Game Design Elements to Gamefulness: Defining "Gamification". In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (Tampere, Finland) (MindTrek ’11). Association for Computing Machinery, New York, NY, USA, 9–15. https://doi.org/10.1145/2181037.2181040

[3] Tommaso Fulcini, Riccardo Coppola, Luca Ardito, and Marco Torchiano. 2023. A Review on Tools, Mechanics, Benefits, and Challenges of Gamified Software Testing. ACM Comput. Surv. 55, 14s, Article 310 (jul 2023), 37 pages. https: //doi.org/10.1145/3582273

[4] Andreas Jedlitschka and Dietmar Pfahl. 2005. Reporting guidelines for controlled experiments in software engineering. In 2005 International Symposium on Empirical Software Engineering, 2005. IEEE, 10–pp.

[5] José Miguel Rojas and Gordon Fraser. 2016. Code Defenders: A Mutation Testing Game. In 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 162–167. https://doi.org/10.1109/ICSTW. 2016.43

[6] Statista. 2023. Most used programming languages among developers worldwide as of 2023. https://www.statista.com/statistics/793628/worldwide-developersurvey-most-used-languages/ Accessed: 2023-12-05.

[7] Philipp Straubinger and Gordon Fraser. 2023. A Survey on What Developers Think About Testing. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). 80–90. https://doi.org/10.1109/ISSRE59848.2023.00075

[8] Philipp Straubinger and Gordon Fraser. 2024. Improving Testing Behavior by Gamifying IntelliJ. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14-20, 2024. ACM, 49:1–49:13. https://doi.org/10.1145/3597503.3623339

[9] Kwangwon Sun and Sukyoung Ryu. 2017. Analysis of JavaScript programs: Challenges and research trends. ACM Computing Surveys (CSUR) 50, 4 (2017), 1–34.

[10] Bo Wang, Aashish Kolluri, Ivica Nikolic, Teodora Baluta, and Prateek Saxena. 2023. User-Customizable Transpilation of Scripting Languages. Proc. ACM Program. Lang. 7, OOPSLA1 (2023), 201–229. https://doi.org/10.1145/3586034