Table of Links Abstract and 1 Introduction 1.1 The twincode platform 1.2 Related Work 2 Research Questions 3 Variables 3.1 Independent Variables 3.2 Dependent Variables 3.3 Confounding Variables 4 Participants 5 Execution Plan and 5.1 Recruitment 5.2 Training and 5.3 Experiment Execution 5.4 Data Analysis Acknowledgments and References 3.2 Dependent Variables The response variables measured using questionnaires containing 0–10 linear numerical response items are the following: Perceived productivity compared to solo programming (pp) interval variable measuring the subject’s perceived productivity compared to solo programming after each in–pair task (see RQ1). Low values correspond to better solo programming productivity, i.e., “solo programming would have been more productive than pair programming”, whereas high values correspond to better pair programming productivity, i.e. “pair programming has been more productive than solo programming”. Perceived partner’s technical competency compared to their own (pptc) interval variable measuring the subject’s partner’s perceived technical competency compared to their own after each in–pair task (see RQ2). Low values correspond to higher subject’s productivity, i.e., “I have been more productive than my partner”, whereas higher values correspond to high partner’s productivity, i.e. “My partner has been more productive than me”. Compared partners’ skills (cps) interval variable measuring whether the subject perceived better skills in their first or second partner in the in–pair tasks (see RQ3). Low values correspond to the first partner, i.e., “My first partner was a better partner than my second partner”, whereas high values correspond to the second partner, i.e. “My second partner was a better partner than my first partner”. In the case of the experimental group only, this variable is transformed after collection using an R script in such a way that low values correspond to the partner perceived as a man, and high values to the partner perceived as a woman, in order to analyze whether there is a gender bias in the scoring. Apart from the variables described above, the questionnaires will also include questions about the perceived gender of their parners at each task. The corresponding variable is described below: perceived partner’s gender (ppgender) nominal factor representing the subject’s perception of their partner’s gender (woman, man, I don’t know, or I don’t remember) at each in–pair task. source code additions (sca) Ratio scale variable representing the count of characters added by a subject to the source code window during an in–pair task. source code deletions (scd) Ratio scale variable representing the count of characters deleted by a subject from the source code window during an in–pair task. successful validations (okv) Ratio scale variable representing the count of successful validations of the source code performed by a subject during an in–pair task. unsuccessful validations (kov) Ratio scale variable representing the count of unsuccessful validations of the source code performed by a subject during an in–pair task. dialog messages (dm) Ratio scale variable representing the count of dialog messages sent by a subject during an in–pair task. The response variables related to the manual tagging of the dialog messages (see RQ5 and RQ6) correspond to the tags in Table 2 and are listed below. Every variable represents a frequency, i.e., a count, and its associated relative frequency is computed with respect to the number of dialog messages generated by the subject during an in–pair task, which is defined by the dm variable specified above. i Ratio scale variable representing the count of informal messages generated by a subject during an in–pair task. f Ratio scale variable representing the count of non–informal, i.e. formal, messages generated by a subject during an in–pair task. s Ratio scale variable representing the count of statement of information or explanation messages generated by a subject during an in–pair task. u Ratio scale variable representing the count of opinion or indication of uncertainty messages generated by a subject during an in–pair task. d Ratio scale variable representing the count of explicit instruction messages generated by a subject during an in–pair task. su Ratio scale variable representing the count of polite or indirect instruction messages generated by a subject during an in– pair task. ack Ratio scale variable representing the count of acknowledgment messages generated by a subject during an in–pair task. m Ratio scale variable representing the count of meta–comment or reflection messages generated by a subject during an in–pair task. qyn Ratio scale variable representing the count of yes/no question messages generated by a subject during an in–pair task. qwh Ratio scale variable representing the count of wh- question (who, what, where, when, why, and how) messages generated by a subject during an in–pair task. ayn Ratio scale variable representing the count of answer to yes/no question messages generated by a subject during an in–pair task. awh Ratio scale variable representing the count of answer to wh-question messages generated by a subject during an in–pair task. fp Ratio scale variable representing the count of positive task feedback messages generated by a subject during an in–pair task. fnon Ratio scale variable representing the count of non–positive task feedback messages generated by a subject during an in–pair task. o Ratio scale variable representing the count of off–task messages generated by a subject during an in–pair task. Authors:
(1) Amador Durán, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (amador@us.es);
(2) Pablo Fernández, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es);
(3) Beatriz Bernárdez, I3US Institute, Universidad de Sevilla, Sevilla, Spain (beat@us.es);
(4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (nweinman@berkeley.edu);
(5) Aslı Akalın, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (asliakalin@berkeley.edu);
(6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (fox@berkeley.edu). This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 1.1 The twincode platform 1.1 The twincode platform 1.2 Related Work 1.2 Related Work 2 Research Questions 2 Research Questions 3 Variables 3 Variables 3.1 Independent Variables 3.1 Independent Variables 3.2 Dependent Variables 3.2 Dependent Variables 3.3 Confounding Variables 3.3 Confounding Variables 4 Participants 4 Participants 5 Execution Plan and 5.1 Recruitment 5 Execution Plan and 5.1 Recruitment 5.2 Training and 5.3 Experiment Execution 5.2 Training and 5.3 Experiment Execution 5.4 Data Analysis 5.4 Data Analysis Acknowledgments and References Acknowledgments and References 3.2 Dependent Variables The response variables measured using questionnaires containing 0–10 linear numerical response items are the following: Perceived productivity compared to solo programming (pp) interval variable measuring the subject’s perceived productivity compared to solo programming after each in–pair task (see RQ1). Low values correspond to better solo programming productivity, i.e., “solo programming would have been more productive than pair programming”, whereas high values correspond to better pair programming productivity, i.e. “pair programming has been more productive than solo programming”. Perceived productivity compared to solo programming (pp) Perceived partner’s technical competency compared to their own (pptc) interval variable measuring the subject’s partner’s perceived technical competency compared to their own after each in–pair task (see RQ2). Low values correspond to higher subject’s productivity, i.e., “I have been more productive than my partner”, whereas higher values correspond to high partner’s productivity, i.e. “My partner has been more productive than me”. Perceived partner’s technical competency compared to their own (pptc) Compared partners’ skills (cps) interval variable measuring whether the subject perceived better skills in their first or second partner in the in–pair tasks (see RQ3). Low values correspond to the first partner, i.e., “My first partner was a better partner than my second partner”, whereas high values correspond to the second partner, i.e. “My second partner was a better partner than my first partner”. In the case of the experimental group only, this variable is transformed after collection using an R script in such a way that low values correspond to the partner perceived as a man, and high values to the partner perceived as a woman, in order to analyze whether there is a gender bias in the scoring. Compared partners’ skills (cps) Apart from the variables described above, the questionnaires will also include questions about the perceived gender of their parners at each task. The corresponding variable is described below: perceived partner’s gender (ppgender) nominal factor representing the subject’s perception of their partner’s gender (woman, man, I don’t know, or I don’t remember) at each in–pair task. perceived partner’s gender (ppgender) source code additions (sca) Ratio scale variable representing the count of characters added by a subject to the source code window during an in–pair task. source code additions (sca) source code deletions (scd) Ratio scale variable representing the count of characters deleted by a subject from the source code window during an in–pair task. source code deletions (scd) successful validations (okv) Ratio scale variable representing the count of successful validations of the source code performed by a subject during an in–pair task. successful validations (okv) unsuccessful validations (kov) Ratio scale variable representing the count of unsuccessful validations of the source code performed by a subject during an in–pair task. unsuccessful validations (kov) dialog messages (dm) Ratio scale variable representing the count of dialog messages sent by a subject during an in–pair task. dialog messages (dm) The response variables related to the manual tagging of the dialog messages (see RQ5 and RQ6) correspond to the tags in Table 2 and are listed below. Every variable represents a frequency, i.e., a count, and its associated relative frequency is computed with respect to the number of dialog messages generated by the subject during an in–pair task, which is defined by the dm variable specified above. i Ratio scale variable representing the count of informal messages generated by a subject during an in–pair task. i f Ratio scale variable representing the count of non–informal, i.e. formal, messages generated by a subject during an in–pair task. f s Ratio scale variable representing the count of statement of information or explanation messages generated by a subject during an in–pair task. s u Ratio scale variable representing the count of opinion or indication of uncertainty messages generated by a subject during an in–pair task. u d Ratio scale variable representing the count of explicit instruction messages generated by a subject during an in–pair task. d su Ratio scale variable representing the count of polite or indirect instruction messages generated by a subject during an in– pair task. su ack Ratio scale variable representing the count of acknowledgment messages generated by a subject during an in–pair task. ack m Ratio scale variable representing the count of meta–comment or reflection messages generated by a subject during an in–pair task. m qyn Ratio scale variable representing the count of yes/no question messages generated by a subject during an in–pair task. qyn qwh Ratio scale variable representing the count of wh- question (who, what, where, when, why, and how) messages generated by a subject during an in–pair task. qwh ayn Ratio scale variable representing the count of answer to yes/no question messages generated by a subject during an in–pair task. ayn awh Ratio scale variable representing the count of answer to wh-question messages generated by a subject during an in–pair task. awh fp Ratio scale variable representing the count of positive task feedback messages generated by a subject during an in–pair task. fp fnon Ratio scale variable representing the count of non–positive task feedback messages generated by a subject during an in–pair task. fnon o Ratio scale variable representing the count of off–task messages generated by a subject during an in–pair task. o Authors: (1) Amador Durán, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (amador@us.es); (2) Pablo Fernández, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es); (3) Beatriz Bernárdez, I3US Institute, Universidad de Sevilla, Sevilla, Spain (beat@us.es); (4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (nweinman@berkeley.edu); (5) Aslı Akalın, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (asliakalin@berkeley.edu); (6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (fox@berkeley.edu). Authors: Authors: (1) Amador Durán, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (amador@us.es); (2) Pablo Fernández, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es); (3) Beatriz Bernárdez, I3US Institute, Universidad de Sevilla, Sevilla, Spain (beat@us.es); (4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (nweinman@berkeley.edu); (5) Aslı Akalın, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (asliakalin@berkeley.edu); (6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (fox@berkeley.edu). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

How Twincode Uncovers Gender Bias in Remote Pair Programming

Detailed Execution Plan for Remote Pair Programming Study

Pair Programming AI Agent 

Too Long; Didn't Read

Make resilience your competitive advantage

Quantifying Gender Bias: Key Variables in Remote Pair Programming

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

The Final Verdict on Pair vs. Solo Programming: A Summary

5 Innovations In Tech That Are Transforming Higher Education

Crucial Features for Customized Virtual Classroom Software

Four Tips For The Newly Minted Remote Student

How Big Data is Shaping Adaptive Learning

The Final Verdict on Pair vs. Solo Programming: A Summary

5 Innovations In Tech That Are Transforming Higher Education

Crucial Features for Customized Virtual Classroom Software

Four Tips For The Newly Minted Remote Student

How Big Data is Shaping Adaptive Learning

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps