Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 1.1 The twincode platform 1.1 The twincode platform 1.2 Pilot Studies 1.2 Pilot Studies 1.3 Other Gender Identities and 1.4 Structure of the Paper 1.3 Other Gender Identities and 1.4 Structure of the Paper 2 Related Work 2 Related Work 3 Original Study (Seville Dec, 2021) and 3.1 Participants 3 Original Study (Seville Dec, 2021) and 3.1 Participants 3.2 Experiment Execution 3.2 Experiment Execution 3.3 Factors (Independent Variables) 3.3 Factors (Independent Variables) 3.4 Response Variables (Dependent Variables) 3.4 Response Variables (Dependent Variables) 3.5 Confounding Variables 3.5 Confounding Variables 3.6 Data Analysis 3.6 Data Analysis 4 First Replication (Berkeley May, 2022) 4 First Replication (Berkeley May, 2022) 4.1 Participants 4.1 Participants 4.2 Experiment Execution 4.2 Experiment Execution 4.3 Data Analysis 4.3 Data Analysis 5 Discussion and Threats to Validity and 5.1 Operationalization of the Cause Construct — Treatment 5 Discussion and Threats to Validity and 5.1 Operationalization of the Cause Construct — Treatment 5.2 Operationalization of the Effect Construct — Metrics 5.2 Operationalization of the Effect Construct — Metrics 5.3 Sampling the Population — Participants 5.3 Sampling the Population — Participants 6 Conclusions and Future Work 6 Conclusions and Future Work 6.1 Replication in Different Cultural Background 6.1 Replication in Different Cultural Background 6.2 Using Chatbots as Partners and AI-based Utterance Coding 6.2 Using Chatbots as Partners and AI-based Utterance Coding Datasets, Compliance with Ethical Standards, Acknowledgements, and References Datasets, Compliance with Ethical Standards, Acknowledgements, and References A. Questionnaire #1 and #2 response items A. Questionnaire #1 and #2 response items B. Evolution of the twincode User Interface B. Evolution of the twincode User Interface C. User Interface of tag-a-chat C. User Interface of tag-a-chat 4.3 Data Analysis The data analysis was performed only for those subjects considered as valid according to the same criteria than in the original experiment. This resulted in 23 pairs, i.e. 46 valid subjects, as previously mentioned in Section 4.1. 4.3.1 Correlation between Induced and Perceived Gender 4.3.1 Correlation between Induced and Perceived Gender As in the original experiment, the correlation of the induced and perceived gender in both groups was analyzed to check treatment effectiveness, especially after having changed the gendered avatars and included a gendered message at the top of the chat window, as described in Section 4.2.4. As shown in Table 6, the man/man and woman/woman effectiveness was close to 40% in the replication whereas was close to 60% in the original experiment (see Table 4 in Section 3.6.1). Although Cramer’s V for Table 6 showed also a large effect (0.530), we applied the same strict criteria than in the original experiment and decided to discard those subjects in the experimental group for whom the induced and perceived gender did not match. For the subjects in the control group, we kept those who did not perceived any gendered avatar or did not remember it. As a result, we kept 22 subjects in the control group (10 men, 12 women) but only 9 (3 men, 6 women) in the experimental group. 4.3.2 Between-groups Analysis 4.3.2 Between-groups Analysis As in the original experiment, and contrary to our research hypothesis, no significant differences were observed at α=0.05 between the control and experimental groups in the replication for any of the 45 response variables[11] described in Section 3.4, including cps. The corresponding boxplots are depicted in Figure 10. 4.3.3 Within-groups Analysis 4.3.3 Within-groups Analysis Within the experimental group (see Figure 11 for the corresponding boxplots), we performed the same analysis than in the original experiment, finding statistically significant differences at α=0.05 in the following four response variables when using the perceived partner’s gender (ppgender) as a within-subjects variable. The four variables passed the Shapiro-Wilk normality test and were therefore analyzed using a two-sided paired t-test. Their effect sizes were computed using Cohen’s d. • scd (source code deletions): the test detected (p = 0.0485) that subjects deleted more source characters when they perceived their partners as a woman, with a moderate effect size (d = −0.775). scd • i_rf (relative frequency of informal messages): the test detected (p = 0.0138) that subjects increased the relative frequency of informal messages when they perceived their partners as a man, with a large effect size (d = 1.050). • i_rf • m_rf (relative frequency of meta-comments or reflections): the test detected (p = 0.0377) that subjects increased the relative frequency of meta-comments or reflections when they perceived their partners as a man, with a large effect size (d = 0.829). • m_rf • qyn_rf (relative frequency of yes/no questions): the test detected (p = 0.0297) that subjects increased the relative frequency of yes/no questions when they perceived their partners as a man, with a large effect size (d = 0.880). • qyn_rf Note that these results must be considered carefully because of the small number of selected subjects (n=9), and because when false discovery rate (FDR) adjustments are applied [5], only the hypothesis test corresponding to the i_rf variable remains significant. i_rf No significant interactions were detected between the perceived partner’s gender and the subject’s gender for the same response variables than in the original study. Authors:
(1) Amador Duran, I3US Institute, Universidad de Sevilla, Sevilla, Spain and SCORE Lab, Universidad de Sevilla, Sevilla, Spain (amador@us.es);
(2) Pablo Fernandez, I3US Institute, Universidad de Sevilla, Sevilla, Spain and SCORE Lab, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es);
(3) Beatriz Bernardez, I3US Institute, Universidad de Sevilla, Sevilla, Spain and SCORE Lab, Universidad de Sevilla, Sevilla, Spain (beat@us.es);
(4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, USA (nweinman@berkeley.edu);
(5) Aslıhan Akalın, Computer Science Division, University of California, Berkeley, Berkeley, USA (asliakalin@berkeley.edu);
(6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, USA (fox@berkeley.edu). Authors: Authors: (1) Amador Duran, I3US Institute, Universidad de Sevilla, Sevilla, Spain and SCORE Lab, Universidad de Sevilla, Sevilla, Spain (amador@us.es); (2) Pablo Fernandez, I3US Institute, Universidad de Sevilla, Sevilla, Spain and SCORE Lab, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es); (3) Beatriz Bernardez, I3US Institute, Universidad de Sevilla, Sevilla, Spain and SCORE Lab, Universidad de Sevilla, Sevilla, Spain (beat@us.es); (4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, USA (nweinman@berkeley.edu); (5) Aslıhan Akalın, Computer Science Division, University of California, Berkeley, Berkeley, USA (asliakalin@berkeley.edu); (6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, USA (fox@berkeley.edu). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv [11] Actually, only 41 variables were analyzed in the replication due to technical problems with the Python server used by the twincode platform. As a result, okv, okv rf, kov, and kov rf could not be measured and, therefore, analyzed. As can be seen in Figure 10 and 11, their means are 0 in all cases.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

How Gender Perception Affects Developer Communication

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

11 Proven Solutions to Common GitHub Copilot Problems

How Does Perceived Gender Impact Pair Programming Dynamics?

Researching Coding Bias with Twincode

What We Learned From 92 Students in a Spanish Software Class

Inside a Study on Gender Bias in Remote Pair Programming

11 Proven Solutions to Common GitHub Copilot Problems

How Does Perceived Gender Impact Pair Programming Dynamics?

Researching Coding Bias with Twincode

What We Learned From 92 Students in a Spanish Software Class

Inside a Study on Gender Bias in Remote Pair Programming

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps