Table of Links
II. Approach
B. Proof of Concept Implementation
III. Envisioned Usage Scenarios
IV. Experiment Design and Demographics
VII. Conclusions and Future Work, Acknowledgment, and References
C. Procedure
In the following, we present the experiment’s procedure. For additional information, we refer readers to the second prepared video8 that demonstrates an exemplary experiment run. Overall, the experiment is divided into pre-questionnaire, mid-questionnaires, i.e., questions that had to be solved after each task completion, and post-questionnaire.
The user study took place at Kiel University and included one instructor who also co-authored this paper. The instructor designed and implemented the approach as well as conducted the user study. Although our approach can be used remotely, we decided to have the study take place in one locality, so that the instructor could intervene if necessary. In each experimental run, the participants were first informed about the data that would be recorded and used for publication. After signing a consent form, the instructor gave a brief introduction to VS Code and the embedded SV. It was mentioned that all introduced features were additionally described on a cheat sheet, which was placed on the table in front of the subjects. Afterwards, the participants were told to openly ask questions if they had a problem. Furthermore, they were told that they could pause or abort the experiment at any time. They draw their login token for the survey tool LimeSurvey9 and started with the pre-questionnaire. Then T1 was introduced and all participants were redirected to browser-based VS Code instances by clicking a button inside of the LimeSurvey form. Each VS Code instance was specifically prepared for a given task and ready to use. It did not require any setup, so that the participants could completely focus on the task itself. They began by reading a markdown file that introduced the target system, controls, and the task itself. After answering T1 in LimeSurvey, all participants gave their feedback to the just used approach. T2 was introduced in the same way as T1. However, here participants were instructed to test the collaborative features first and then work together on solving T2. Again, the subjects gave their feedback and concluded with the post-questionnaire. During each experiment run, the instructor made a note of noticeable mentions stated by the participants.
Authors:
(1) Alexander Krause-Glau, Software Engineering Group, Kiel University, Kiel, Germany ([email protected]);
(2) Wilhelm Hasselbring, Software Engineering Group, Kiel University, Kiel, Germany ([email protected]).
This paper is
