Table of Links Abstract and 1 Introduction 1.1 The twincode platform 1.2 Related Work 2 Research Questions 3 Variables 3.1 Independent Variables 3.2 Dependent Variables 3.3 Confounding Variables 4 Participants 5 Execution Plan and 5.1 Recruitment 5.2 Training and 5.3 Experiment Execution 5.4 Data Analysis Acknowledgments and References ACKNOWLEDGMENTS We would like to thank the students who volunteered to participate in the pilot studies at the University of California, Berkeley and the University of Seville. We particularly acknowledge Vron Vance’s assistance regarding inclusive language around gender identity. This work has been partially supported by FEDER/Ministerio de Ciencia e Innovación – Agencia Estatal de Investigación under projects OPHELIA (RTI2018–101204–B–C22), HORATIO (RTI2018- 101204–B–C21), and Junta de Andalucía under project EKIPMENTPLUS (P18–FR–2895). REFERENCES [1] Ahmad Al-Jarrah and Enrico Pontelli. 2016. On the effectiveness of a collaborative virtual pair-programming environment. In International Conference on Learning and Collaboration Technologies. Springer, 583–595. [2] E. A. Chaparro, Aybala Yuksel, Pablo Romero, and Sallyann Bryant. 2005. Factors Affecting the Perceived Effectiveness of Pair Programming in Higher Education. In Proceedings of the 17th Workshop of the Psychology of Programming Interest Group. [3] Kyungsub S. Choi. 2013. Evaluating Gender Significance within a Pair Programming Context. In Proceedings of the 2013 46th Hawaii International Conference on System Sciences (HICSS ’13). IEEE Computer Society, USA, 4817–4825. https://doi.org/10.1109/HICSS.2013.209 [4] Kyungsub Stephen Choi. 2015. A comparative analysis of different gender pair combinations in pair programming. Behaviour & Information Technology 34, 8 (2015), 825–837. https://doi.org/10.1080/0144929X.2014.937460 arXiv:https://doi.org/10.1080/0144929X.2014.937460 [5] Louis Cohen, Lawrence Manion, and Keith Morrison. 2018. Research Methods in Education (8th ed.). Routledge. [6] Bernardo José da Silva Estácio and Rafael Prikladnicki. 2015. Distributed Pair Programming: A Systematic Literature Review. Information and Software Technology 63 (2015), 1–10. [7] N.K. Denzin. 2006. Sociological Methods: A Sourcebook (5th ed.). Aldine Transaction. [8] Dean Eckles, René F. Kizilcec, and Eytan Bakshy. 2016. Estimating peer effects in networks with peer encouragement designs. Proceedings of the National Academy of Sciences 113, 27 (2016), 7316–7322. https://doi.org/10.1073/pnas.1511201113 arXiv:https://www.pnas.org/content/113/27/7316.full.pdf [9] O. Gómez, M. Solari, C. Calvache, and A. Ledezma-Carrizalez. 2017. A Controlled Experiment on Productivity of Pair Programming Gender Combinations: Preliminary Results. In Proceedings of the XX Ibero–American Conference on Software Engineering. 197–210. [10] Brian Hanks, Sue Fitzgerald, Renée McCauley, Laurie Murphy, and Carol Zander. 2011. Pair programming in education: A literature review. Computer Science Education 21, 2 (2011), 135–173. [11] Jo E. Hannay, Erik Arisholm, Harald Engvik, and Dag I.K. Sjoberg. 2010. Effects of Personality on Pair Programming. IEEE Transactions on Software Engineering 36, 1 (2010), 61–80. https://doi.org/10.1109/TSE.2009.41 [12] Sarah I. Hofer. 2015. Studying Gender Bias in Physics Grading: The role of teaching experience and country. International Journal of Science Education 37, 17 (2015), 2879–2905. https://doi.org/10.1080/09500693.2015.1114190 arXiv:https://doi.org/10.1080/09500693.2015.1114190 [13] Lindsay Jarratt, Nicholas A. Bowman, K.C. Culver, and Alberto Maria Segre. 2019. A Large-Scale Experimental Study of Gender and Pair Composition in Pair Programming. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (Aberdeen, Scotland Uk) (ITiCSE ’19). Association for Computing Machinery, New York, NY, USA, 176–181. https: //doi.org/10.1145/3304221.3319782 [14] N. Katira, L. Williams, and J. Osborne. 2005. Towards increasing the compatibility of student pair programmers. In Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. 625–626. https://doi.org/10.1109/ICSE. 2005.1553618 [15] K. Kaur Chahal, A. Kaur, and M. Saini. 2021. Research and Evidence in Software Engineering: From Empirical Studies to Open Source Artifacts. Taylor & Francis Group, Chapter Empirical Studies on Using Pair Programming as a Pedagogical Tool in Higher Education Courses: A Systematic Literature Review, 251–287. [16] Sandeep Kaur Kuttal, Kevin Gerstner, and Alexandra Bejarano. 2019. Remote Pair Programming in Online CS Education: Investigating through a Gender Lens. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 75–85. https://doi.org/10.1109/VLHCC.2019.8818790 [17] J.R. Landis and G.G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 1 (1977), 159–174. [18] Martell, D. M. Lane, and C. Emrich. 1996. Male-female differences: A computer simulation. American Psychologist 2, 51 (7 1996), 157–158. https://doi.org/10. 1037/0003-066X.51.2.157 [19] Fernando J. Rodríguez, Kimberly Michelle Price, and Kristy Elizabeth Boyer. 2017. Exploring the Pair Programming Process: Characteristics of Effective Collaboration. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (Seattle, Washington, USA) (SIGCSE ’17). Association for Computing Machinery, New York, NY, USA, 507–512. https://doi.org/10. 1145/3017680.3017748 [20] Norsaremah Salleh, Emilia Mendes, and John Grundy. 2014. Investigating the effects of personality traits on pair programming in a higher education setting through a family of experiments. Empirical Software Engineering 19, 3 (2014), 714–752. [21] Norsaremah Salleh, Emilia Mendes, John Grundy, and Giles St J Burch. 2010. The effects of neuroticism on pair programming: an empirical study in the higher education context. In Proceedings of the 2010 ACM-IEEE international symposium on empirical software engineering and measurement. 1–10. [22] David Stotts, Laurie Williams, Nachiappan Nagappan, Prashant Baheti, Dennis Jen, and Anne Jackson. 2003. Virtual teaming: Experiments and experiences with distributed pair programming. In Conference on Extreme Programming and Agile Methods. Springer, 129–141. [23] Lynda Thomas, Mark Ratcliffe, and Ann Robertson. 2003. Code Warriors and Code-a-Phobes: A Study in Attitude and Pair Programming. In Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education (Reno, Navada, USA) (SIGCSE ’03). Association for Computing Machinery, New York, NY, USA, 363–367. https://doi.org/10.1145/611892.612007 [24] Linda L. Werner, Brian Hanks, and Charlie McDowell. 2004. Pair-Programming Helps Female Computer Science Students. J. Educ. Resour. Comput. 4, 1 (March 2004), 4–es. https://doi.org/10.1145/1060071.1060075 [25] S. Xinogalos, M. Satratzemi, A. Chatzigeorgiou, and D. Tsompanoudi. 2017. Student perceptions on the benefits and shortcomings of distributed pair programming assignments. 2017 IEEE Global Engineering Education Conference (EDUCON) (2017), 1513–1521. Authors:
(1) Amador Durán, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (amador@us.es);
(2) Pablo Fernández, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es);
(3) Beatriz Bernárdez, I3US Institute, Universidad de Sevilla, Sevilla, Spain (beat@us.es);
(4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (nweinman@berkeley.edu);
(5) Aslı Akalın, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (asliakalin@berkeley.edu);
(6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (fox@berkeley.edu). This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 1.1 The twincode platform 1.1 The twincode platform 1.2 Related Work 1.2 Related Work 2 Research Questions 2 Research Questions 3 Variables 3 Variables 3.1 Independent Variables 3.1 Independent Variables 3.2 Dependent Variables 3.2 Dependent Variables 3.3 Confounding Variables 3.3 Confounding Variables 4 Participants 4 Participants 5 Execution Plan and 5.1 Recruitment 5 Execution Plan and 5.1 Recruitment 5.2 Training and 5.3 Experiment Execution 5.2 Training and 5.3 Experiment Execution 5.4 Data Analysis 5.4 Data Analysis Acknowledgments and References Acknowledgments and References ACKNOWLEDGMENTS We would like to thank the students who volunteered to participate in the pilot studies at the University of California, Berkeley and the University of Seville. We particularly acknowledge Vron Vance’s assistance regarding inclusive language around gender identity. This work has been partially supported by FEDER/Ministerio de Ciencia e Innovación – Agencia Estatal de Investigación under projects OPHELIA (RTI2018–101204–B–C22), HORATIO (RTI2018- 101204–B–C21), and Junta de Andalucía under project EKIPMENTPLUS (P18–FR–2895). REFERENCES [1] Ahmad Al-Jarrah and Enrico Pontelli. 2016. On the effectiveness of a collaborative virtual pair-programming environment. In International Conference on Learning and Collaboration Technologies. Springer, 583–595. [2] E. A. Chaparro, Aybala Yuksel, Pablo Romero, and Sallyann Bryant. 2005. Factors Affecting the Perceived Effectiveness of Pair Programming in Higher Education. In Proceedings of the 17th Workshop of the Psychology of Programming Interest Group. [3] Kyungsub S. Choi. 2013. Evaluating Gender Significance within a Pair Programming Context. In Proceedings of the 2013 46th Hawaii International Conference on System Sciences (HICSS ’13). IEEE Computer Society, USA, 4817–4825. https://doi.org/10.1109/HICSS.2013.209 [4] Kyungsub Stephen Choi. 2015. A comparative analysis of different gender pair combinations in pair programming. Behaviour & Information Technology 34, 8 (2015), 825–837. https://doi.org/10.1080/0144929X.2014.937460 arXiv:https://doi.org/10.1080/0144929X.2014.937460 [5] Louis Cohen, Lawrence Manion, and Keith Morrison. 2018. Research Methods in Education (8th ed.). Routledge. [6] Bernardo José da Silva Estácio and Rafael Prikladnicki. 2015. Distributed Pair Programming: A Systematic Literature Review. Information and Software Technology 63 (2015), 1–10. [7] N.K. Denzin. 2006. Sociological Methods: A Sourcebook (5th ed.). Aldine Transaction. [8] Dean Eckles, René F. Kizilcec, and Eytan Bakshy. 2016. Estimating peer effects in networks with peer encouragement designs. Proceedings of the National Academy of Sciences 113, 27 (2016), 7316–7322. https://doi.org/10.1073/pnas.1511201113 arXiv:https://www.pnas.org/content/113/27/7316.full.pdf [9] O. Gómez, M. Solari, C. Calvache, and A. Ledezma-Carrizalez. 2017. A Controlled Experiment on Productivity of Pair Programming Gender Combinations: Preliminary Results. In Proceedings of the XX Ibero–American Conference on Software Engineering. 197–210. [10] Brian Hanks, Sue Fitzgerald, Renée McCauley, Laurie Murphy, and Carol Zander. 2011. Pair programming in education: A literature review. Computer Science Education 21, 2 (2011), 135–173. [11] Jo E. Hannay, Erik Arisholm, Harald Engvik, and Dag I.K. Sjoberg. 2010. Effects of Personality on Pair Programming. IEEE Transactions on Software Engineering 36, 1 (2010), 61–80. https://doi.org/10.1109/TSE.2009.41 [12] Sarah I. Hofer. 2015. Studying Gender Bias in Physics Grading: The role of teaching experience and country. International Journal of Science Education 37, 17 (2015), 2879–2905. https://doi.org/10.1080/09500693.2015.1114190 arXiv:https://doi.org/10.1080/09500693.2015.1114190 [13] Lindsay Jarratt, Nicholas A. Bowman, K.C. Culver, and Alberto Maria Segre. 2019. A Large-Scale Experimental Study of Gender and Pair Composition in Pair Programming. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (Aberdeen, Scotland Uk) (ITiCSE ’19). Association for Computing Machinery, New York, NY, USA, 176–181. https: //doi.org/10.1145/3304221.3319782 [14] N. Katira, L. Williams, and J. Osborne. 2005. Towards increasing the compatibility of student pair programmers. In Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. 625–626. https://doi.org/10.1109/ICSE. 2005.1553618 [15] K. Kaur Chahal, A. Kaur, and M. Saini. 2021. Research and Evidence in Software Engineering: From Empirical Studies to Open Source Artifacts. Taylor & Francis Group, Chapter Empirical Studies on Using Pair Programming as a Pedagogical Tool in Higher Education Courses: A Systematic Literature Review, 251–287. [16] Sandeep Kaur Kuttal, Kevin Gerstner, and Alexandra Bejarano. 2019. Remote Pair Programming in Online CS Education: Investigating through a Gender Lens. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 75–85. https://doi.org/10.1109/VLHCC.2019.8818790 [17] J.R. Landis and G.G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 1 (1977), 159–174. [18] Martell, D. M. Lane, and C. Emrich. 1996. Male-female differences: A computer simulation. American Psychologist 2, 51 (7 1996), 157–158. https://doi.org/10. 1037/0003-066X.51.2.157 [19] Fernando J. Rodríguez, Kimberly Michelle Price, and Kristy Elizabeth Boyer. 2017. Exploring the Pair Programming Process: Characteristics of Effective Collaboration. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (Seattle, Washington, USA) (SIGCSE ’17). Association for Computing Machinery, New York, NY, USA, 507–512. https://doi.org/10. 1145/3017680.3017748 [20] Norsaremah Salleh, Emilia Mendes, and John Grundy. 2014. Investigating the effects of personality traits on pair programming in a higher education setting through a family of experiments. Empirical Software Engineering 19, 3 (2014), 714–752. [21] Norsaremah Salleh, Emilia Mendes, John Grundy, and Giles St J Burch. 2010. The effects of neuroticism on pair programming: an empirical study in the higher education context. In Proceedings of the 2010 ACM-IEEE international symposium on empirical software engineering and measurement. 1–10. [22] David Stotts, Laurie Williams, Nachiappan Nagappan, Prashant Baheti, Dennis Jen, and Anne Jackson. 2003. Virtual teaming: Experiments and experiences with distributed pair programming. In Conference on Extreme Programming and Agile Methods. Springer, 129–141. [23] Lynda Thomas, Mark Ratcliffe, and Ann Robertson. 2003. Code Warriors and Code-a-Phobes: A Study in Attitude and Pair Programming. In Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education (Reno, Navada, USA) (SIGCSE ’03). Association for Computing Machinery, New York, NY, USA, 363–367. https://doi.org/10.1145/611892.612007 [24] Linda L. Werner, Brian Hanks, and Charlie McDowell. 2004. Pair-Programming Helps Female Computer Science Students. J. Educ. Resour. Comput. 4, 1 (March 2004), 4–es. https://doi.org/10.1145/1060071.1060075 [25] S. Xinogalos, M. Satratzemi, A. Chatzigeorgiou, and D. Tsompanoudi. 2017. Student perceptions on the benefits and shortcomings of distributed pair programming assignments. 2017 IEEE Global Engineering Education Conference (EDUCON) (2017), 1513–1521. Authors: (1) Amador Durán, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (amador@us.es); (2) Pablo Fernández, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es); (3) Beatriz Bernárdez, I3US Institute, Universidad de Sevilla, Sevilla, Spain (beat@us.es); (4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (nweinman@berkeley.edu); (5) Aslı Akalın, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (asliakalin@berkeley.edu); (6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (fox@berkeley.edu). Authors: Authors: (1) Amador Durán, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (amador@us.es); (2) Pablo Fernández, SCORE Lab, I3US Institute, Universidad de Sevilla, Sevilla, Spain (pablofm@us.es); (3) Beatriz Bernárdez, I3US Institute, Universidad de Sevilla, Sevilla, Spain (beat@us.es); (4) Nathaniel Weinman, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (nweinman@berkeley.edu); (5) Aslı Akalın, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (asliakalin@berkeley.edu); (6) Armando Fox, Computer Science Division, University of California, Berkeley, Berkeley, CA, USA (fox@berkeley.edu). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Two Devs, One Keyboard: A Bold Experiment in Computer Science Education 

How Twincode Uncovers Gender Bias in Remote Pair Programming

Pair Programming AI Agent 

Too Long; Didn't Read

Make resilience your competitive advantage

Gender Bias in Remote Pair Programming: Acknowledgments and References

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

The Final Verdict on Pair vs. Solo Programming: A Summary

5 Innovations In Tech That Are Transforming Higher Education

Crucial Features for Customized Virtual Classroom Software

Four Tips For The Newly Minted Remote Student

How Big Data is Shaping Adaptive Learning

The Final Verdict on Pair vs. Solo Programming: A Summary

5 Innovations In Tech That Are Transforming Higher Education

Crucial Features for Customized Virtual Classroom Software

Four Tips For The Newly Minted Remote Student

How Big Data is Shaping Adaptive Learning

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps