Quantifying Gender Bias: Key Variables in Remote Pair Programming

Table of Links

Abstract and 1 Introduction

1.1 The twincode platform

1.2 Related Work

2 Research Questions

3 Variables

3.1 Independent Variables

3.2 Dependent Variables

3.3 Confounding Variables

4 Participants

5 Execution Plan and 5.1 Recruitment

5.2 Training and 5.3 Experiment Execution

5.4 Data Analysis

Acknowledgments and References

3.2 Dependent Variables

The response variables measured using questionnaires containing 0–10 linear numerical response items are the following:

Perceived productivity compared to solo programming (pp) interval variable measuring the subject’s perceived productivity compared to solo programming after each in–pair task (see RQ1). Low values correspond to better solo programming productivity, i.e., “solo programming would have been more productive than pair programming”, whereas high values correspond to better pair programming productivity, i.e. “pair programming has been more productive than solo programming”.

Perceived partner’s technical competency compared to their own (pptc) interval variable measuring the subject’s partner’s perceived technical competency compared to their own after each in–pair task (see RQ2). Low values correspond to higher subject’s productivity, i.e., “I have been more productive than my partner”, whereas higher values correspond to high partner’s productivity, i.e. “My partner has been more productive than me”.

Compared partners’ skills (cps) interval variable measuring whether the subject perceived better skills in their first or second partner in the in–pair tasks (see RQ3). Low values correspond to the first partner, i.e., “My first partner was a better partner than my second partner”, whereas high values correspond to the second partner, i.e. “My second partner was a better partner than my first partner”. In the case of the experimental group only, this variable is transformed after collection using an R script in such a way that low values correspond to the partner perceived as a man, and high values to the partner perceived as a woman, in order to analyze whether there is a gender bias in the scoring.

Apart from the variables described above, the questionnaires will also include questions about the perceived gender of their parners at each task. The corresponding variable is described below:

perceived partner’s gender (ppgender) nominal factor representing the subject’s perception of their partner’s gender (woman, man, I don’t know, or I don’t remember) at each in–pair task.

source code additions (sca) Ratio scale variable representing the count of characters added by a subject to the source code window during an in–pair task.

source code deletions (scd) Ratio scale variable representing the count of characters deleted by a subject from the source code window during an in–pair task.

successful validations (okv) Ratio scale variable representing the count of successful validations of the source code performed by a subject during an in–pair task.

unsuccessful validations (kov) Ratio scale variable representing the count of unsuccessful validations of the source code performed by a subject during an in–pair task.

dialog messages (dm) Ratio scale variable representing the count of dialog messages sent by a subject during an in–pair task.

The response variables related to the manual tagging of the dialog messages (see RQ5 and RQ6) correspond to the tags in Table 2 and are listed below. Every variable represents a frequency, i.e., a count, and its associated relative frequency is computed with respect to the number of dialog messages generated by the subject during an in–pair task, which is defined by the dm variable specified above.

i Ratio scale variable representing the count of informal messages generated by a subject during an in–pair task.

f Ratio scale variable representing the count of non–informal, i.e. formal, messages generated by a subject during an in–pair task.

s Ratio scale variable representing the count of statement of information or explanation messages generated by a subject during an in–pair task.

u Ratio scale variable representing the count of opinion or indication of uncertainty messages generated by a subject during an in–pair task.

d Ratio scale variable representing the count of explicit instruction messages generated by a subject during an in–pair task.

su Ratio scale variable representing the count of polite or indirect instruction messages generated by a subject during an in– pair task.

ack Ratio scale variable representing the count of acknowledgment messages generated by a subject during an in–pair task.

m Ratio scale variable representing the count of meta–comment or reflection messages generated by a subject during an in–pair task.

qyn Ratio scale variable representing the count of yes/no question messages generated by a subject during an in–pair task.

qwh Ratio scale variable representing the count of wh- question (who, what, where, when, why, and how) messages generated by a subject during an in–pair task.

ayn Ratio scale variable representing the count of answer to yes/no question messages generated by a subject during an in–pair task.

awh Ratio scale variable representing the count of answer to wh-question messages generated by a subject during an in–pair task.

fp Ratio scale variable representing the count of positive task feedback messages generated by a subject during an in–pair task.

fnon Ratio scale variable representing the count of non–positive task feedback messages generated by a subject during an in–pair task.

o Ratio scale variable representing the count of off–task messages generated by a subject during an in–pair task.