Detailed Execution Plan for Remote Pair Programming Study

Table of Links

Abstract and 1 Introduction

1.1 The twincode platform

1.2 Related Work

2 Research Questions

3 Variables

3.1 Independent Variables

3.2 Dependent Variables

3.3 Confounding Variables

4 Participants

5 Execution Plan and 5.1 Recruitment

5.2 Training and 5.3 Experiment Execution

5.4 Data Analysis

Acknowledgments and References

5.2 Training

One week or so before the experiment execution, we plan to provide a short training session about the twincode platform, so students become familiar with it and any concerns about how to use the platform were dispelled, thus reducing any potential anxiety for using a new platform.

5.3 Experiment Execution

For experiment execution, all registered students must log into the twincode platform, which will automatically allocate them into the control and experimental groups. This allocation is random and gender–balanced, i.e., the proportion of men and women in each group is as close as possible. Once all students are allocated to groups, they are randomly allocated into control–experimental pairs, as shown in Figure 2.

After subject allocation, the pairs are presented a programming exercise that they must solve collaboratively (labeled as Task #1 in Figure 3) in the twincode platform. They are given 20 minutes to do it, and in case they finished earlier, another exercise of increasing difficulty is presented. At the end of the 20–minute period, they are asked to individually fill in a questionnaire about the perceived productivity compared to solo programming and about the perceived partner’s technical competency compared to their own. They are given 10 minutes to fill in the questionnaire.

After filling the first questionnaire, the students are presented another programming exercise to be solved individually in 10 minutes (Task #2 in Figure 3). As in the previous task, another exercise of increasing difficulty is presented if they finish earlier. The main purpose of this individual task is to make students forget about their first partners, i.e. their way of writing dialog messages or source code, so they do not recognize them in the second in–pair task.

After the individual task, pairs are presented again a new collaborative programming exercise that they must solve in similar conditions to the exercise in Task #1. In this second in–pair exercise, the avatar gender (which in this initial study will always indicate binary gender) is swapped with respect to the first exercise for the subjects in the experimental group. Note that pairs are kept the same in order to reduce the variability due to the subjects themselves, which could possibly have had a confounding effect in case of a new pair allocation for Task #3.

Once Task #3 is finished, students are asked to fill in the same questionnaire than they filled after Task #1, and another questionnaire comparing the perceived genders and skills of the first and second partners. They are given 15 minutes for responding both questionnaires.

Finally, they are informed about the actual goal of the study and that the pairs remained the same during the experiment. At this moment, they are allowed to withdraw their data if they wish to do so.