What Science Says About Learning to Code in Pairs

Table of Links Abstract and 1. Introduction 2 Research Settings 3 Methods 3.1 Instruments and 3.2 Quantitative Analysis 3.3 Thematic Analysis 4 Results 4.1 Quantitative Results 4.2 Qualitative Results 5 Discussion and 5.1 Answering the research questions 5.2 Threats to validity and 5.3 Limitations and generalizability 6 Conclusion, Acknowledgments, and References 5 DISCUSSION The main aim of our preliminary study was to shed light on the psychological aspects of pairing in software engineering and to find ways of improving its utilization in educational settings while building a knowledge base for AI pairing software research. Prior experimental designs on the topic were extended with novel approaches, incorporating the Self-determination framework [16] and distinguishing all effects per individual, as opposed to by pair. 5.1 Answering the research questions The following discussion will provide answers to the research questions and recapitulate the observed themes, codes, and subthemes. For more insight, the reader should refer to Table II. 5.1.1 RQ1: Are both pair programming roles intrinsically more motivating to students in university classrooms than solo? By quantitative results, the pilot-navigator roles are more intrinsically motivating (μ = 28.41 and 27.87, respectively) than both members coding simultaneously (μ = 25.11), which is a vital sign and implies that universities should include pair programming in software engineering courses. When people are autonomously motivated, their interests and values align with their actions, which become biologically distinct from controlled behaviors. Consequently, autonomous motivation leads to higher creativity, better problem-solving (e.g., thinking outside the box), increased performance (particularly in heuristic activities like programming), positive emotions, and psychological and physical wellness [16]. Noteworthy, some prior research has shown that there are exceptions, and some students prefer pairing in only one of the roles over solo and would rather be solo than in the other pairing role [2]. Our quantitative data, however, did not show that. 5.1.2 RQ2: What are the psychological aspects of pair programming? The discussion about the answers to the second research question will be divided into seven parts by its areas: pairing constellations, feedback, soft and social skills practice, psychological aspects, role-specific insights, task complexity and time constraint, and discovering the “perfect programmer’s” traits. 5.1.2.1. Pairing Constellations. The richest of the identified themes is pairing constellations with its three subthemes: skill level, personal compatibilities, and partner familiarity, which all significantly influence motivation and productivity in pairs. First, social compatibility is vital and allows pairing longer. Great pairing constellations are characterized as those where the partners' skill level and pace of work are the same or similar. Such pairing constellations lead to new friendships. On the other hand, if partners are familiar upfront, it opens ways to communicate more directly and effectively, thereby producing better results. Concerns have been raised about forming detrimental pairs where members score extreme values on specific big five dimensions, such as pairing someone with high sensitivity to negative emotion (neuroticism) with someone low on agreeableness. Similar extraversion and also skill levels are vital. A speed-dating-like system was proposed as a solution to help form the pairs at the beginning of the semester and solve the “bit of an alchemy” problem of pairing constellations. The students could test their character compatibilities in brief talk sessions, and the instructor would assign the pairs on resulting preference cards. 5.1.2.2. Feedback. The second theme is about feedback, which is suggested by [14] to play an essential role in software engineering, especially in motivation, and our experimental participants also stressed it. They listed a variety of positive effects stemming from providing feedback during pairings, such as the valuation of the other, the facilitation of their improvement, connective therapy, or enablement of progress. On another note, participant P6 expressed that his dominant partner had to be suppressed: "Yes, I can do this on my own.” 5.1.2.3. Soft and Social Skills. The training of soft and social skills was found to be of crucial importance and a valued benefit yet hard to master, as participant P1 voiced: “Programming consists of two skills: programming something and asking questions perfectly. The latter is really difficult.” Participants felt they were simulating real-world settings in classrooms. 5.1.2.4. Psychological Aspects. Participants felt more empowered in pairs, stating that “Almost everything can be solved in pairs, from programming to your emotional state.” (P2) They also enjoyed the experimental psychological introspections, concluding that the experimental pair programming was emotional and fun and that they believe personality traits profoundly affect role preferences. For example, four interview participants' voices were marked by the code “roles and being observed push you out of your comfort zone.” They expressed that in such cases, they performed better, which is commonly known as “the Hawthorne effect.” Also, participants P3 and P7 found it very interesting to be able to compare their own thoughts with the thoughts of someone else, “Very interesting to compare what you would plan to do with what your partner is suggesting.” (P7). Such insights are transferrable to the AI pairing software, suggesting that the user experience of comparing is essential and should receive particular attention. 5.1.2.5. Role-specific insights. Participants differed in their views on the benefits and negatives of the three distinct programming roles. For example, some enjoyed the “free space in their head” that pairing provided by splitting the activities per person in half, while others suffered from “being confined” in just a subset of the activities of software engineering, which is inherently multitasking in its nature. Contrarily, they all praised attaining responsibilities, feeling of control, and perceived importance. Of particular interest is that splitting roles allows for developing novel, otherwise inaccessible solutions, as the code “found ways I wouldn’t have found in the pilot role” explains. Also, a command hierarchy originates in role splitting, where “the navigator has the final say.” Lastly, it is essential to say that the participants found that people of some personality traits are fitter for one role and not the other, e.g., code “problem: introverts taking on the navigator role and not talking much.” 5.1.2.6. Task Complexity and Time Constraint. The participants voiced opposing opinions on the strict time limit for staying in their roles. While it provided a fair share of “seat time,” helped with efficiency, and prevented attention deficiencies, it also “confused” those who had not finished their tasks within the limit. Some proposed to make the time limit longer, e.g., 15 minutes per task, to allow for some research time if the pair get lost in the task. Proposed improvements also included revealing the right solution at the end of each round so that everyone could proceed together. Task type and difficulty were found not to affect pair programming. 5.1.2.7. Last theme was about discovering the traits of the perfect programmer and, in effect, finding ways to be a better pairing partner or construct or parametrize a better AI pairing software in ways that it “behaves” toward the user. The perfect pairing partner was described as someone humble, tolerant of mistakes, friendly, makes others feel good, and has the tendency to show self-discipline. In the Big five terms, he would score high on extraversion, agreeableness, and openness but low on the neuroticism scale. This is in accord with the results of a previous study [2], where members of the personality cluster scoring high on extraversion and agreeableness were found to be great navigators, and members of the cluster characterized by high openness strongly preferred to be in the pilot role. 5.2 Threats to validity The subjective nature of interpretation poses threats to the validity of our qualitative results. However, the data have been processed systematically and in an epistemological way that introduces as little subjective bias as possible. Internal validity in our research context mainly refers to the suitability of our data sets for the application of the selected statistical methods. This discussion was covered in the methods. 5.3 Limitations and generalizability More research is needed to fully understand the psychological aspects of pairing and pair programming to understand how members of different personality clusters pair with different AI software representing the roles of pilot or navigator. However, some results, such as the “perfect programmer’s” big five dimensions, can already be reused in the development of AI pairing software. Unfortunately, the results of studies on pair programming in university classrooms may not necessarily generalize to professional software development settings, where different setup is more commonly used, such as switching roles deliberately or after finishing a task and also pairing with someone else after a switch of the role. Author: (1) Marcel Valový, Department of Information Technologies, Prague, Czech Republic (marcel.valovy@vse.cz). This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Research Settings 2 Research Settings 3 Methods 3 Methods 3.1 Instruments and 3.2 Quantitative Analysis 3.1 Instruments and 3.2 Quantitative Analysis 3.3 Thematic Analysis 3.3 Thematic Analysis 4 Results 4 Results 4.1 Quantitative Results 4.1 Quantitative Results 4.2 Qualitative Results 4.2 Qualitative Results 5 Discussion and 5.1 Answering the research questions 5 Discussion and 5.1 Answering the research questions 5.2 Threats to validity and 5.3 Limitations and generalizability 5.2 Threats to validity and 5.3 Limitations and generalizability 6 Conclusion, Acknowledgments, and References 6 Conclusion, Acknowledgments, and References 5 DISCUSSION The main aim of our preliminary study was to shed light on the psychological aspects of pairing in software engineering and to find ways of improving its utilization in educational settings while building a knowledge base for AI pairing software research. Prior experimental designs on the topic were extended with novel approaches, incorporating the Self-determination framework [16] and distinguishing all effects per individual, as opposed to by pair. 5.1 Answering the research questions The following discussion will provide answers to the research questions and recapitulate the observed themes, codes, and subthemes. For more insight, the reader should refer to Table II. 5.1.1 RQ1: Are both pair programming roles intrinsically more motivating to students in university classrooms than solo? By quantitative results, the pilot-navigator roles are more intrinsically motivating (μ = 28.41 and 27.87, respectively) than both members coding simultaneously (μ = 25.11), which is a vital sign and implies that universities should include pair programming in software engineering courses. When people are autonomously motivated, their interests and values align with their actions, which become biologically distinct from controlled behaviors. Consequently, autonomous motivation leads to higher creativity, better problem-solving (e.g., thinking outside the box), increased performance (particularly in heuristic activities like programming), positive emotions, and psychological and physical wellness [16]. Noteworthy, some prior research has shown that there are exceptions, and some students prefer pairing in only one of the roles over solo and would rather be solo than in the other pairing role [2]. Our quantitative data, however, did not show that. 5.1.1 RQ1: Are both pair programming roles intrinsically more motivating to students in university classrooms than solo? 5.1.2 RQ2: What are the psychological aspects of pair programming? 5.1.2 RQ2: What are the psychological aspects of pair programming? The discussion about the answers to the second research question will be divided into seven parts by its areas: pairing constellations, feedback, soft and social skills practice, psychological aspects, role-specific insights, task complexity and time constraint, and discovering the “perfect programmer’s” traits. 5.1.2.1. Pairing Constellations. The richest of the identified themes is pairing constellations with its three subthemes: skill level, personal compatibilities, and partner familiarity, which all significantly influence motivation and productivity in pairs. 5.1.2.1. Pairing Constellations. First, social compatibility is vital and allows pairing longer. Great pairing constellations are characterized as those where the partners' skill level and pace of work are the same or similar. Such pairing constellations lead to new friendships. On the other hand, if partners are familiar upfront, it opens ways to communicate more directly and effectively, thereby producing better results. Concerns have been raised about forming detrimental pairs where members score extreme values on specific big five dimensions, such as pairing someone with high sensitivity to negative emotion (neuroticism) with someone low on agreeableness. Similar extraversion and also skill levels are vital. A speed-dating-like system was proposed as a solution to help form the pairs at the beginning of the semester and solve the “bit of an alchemy” problem of pairing constellations. The students could test their character compatibilities in brief talk sessions, and the instructor would assign the pairs on resulting preference cards. 5.1.2.2. Feedback. The second theme is about feedback, which is suggested by [14] to play an essential role in software engineering, especially in motivation, and our experimental participants also stressed it. They listed a variety of positive effects stemming from providing feedback during pairings, such as the valuation of the other, the facilitation of their improvement, connective therapy, or enablement of progress. On another note, participant P6 expressed that his dominant partner had to be suppressed: "Yes, I can do this on my own.” 5.1.2.2. Feedback. 5.1.2.3. Soft and Social Skills. The training of soft and social skills was found to be of crucial importance and a valued benefit yet hard to master, as participant P1 voiced: “Programming consists of two skills: programming something and asking questions perfectly. The latter is really difficult.” Participants felt they were simulating real-world settings in classrooms. 5.1.2.3. Soft and Social Skills. 5.1.2.4. Psychological Aspects. Participants felt more empowered in pairs, stating that “Almost everything can be solved in pairs, from programming to your emotional state.” (P2) 5.1.2.4. Psychological Aspects. They also enjoyed the experimental psychological introspections, concluding that the experimental pair programming was emotional and fun and that they believe personality traits profoundly affect role preferences. For example, four interview participants' voices were marked by the code “roles and being observed push you out of your comfort zone.” They expressed that in such cases, they performed better, which is commonly known as “the Hawthorne effect.” Also, participants P3 and P7 found it very interesting to be able to compare their own thoughts with the thoughts of someone else, “Very interesting to compare what you would plan to do with what your partner is suggesting.” (P7). Such insights are transferrable to the AI pairing software, suggesting that the user experience of comparing is essential and should receive particular attention. 5.1.2.5. Role-specific insights. Participants differed in their views on the benefits and negatives of the three distinct programming roles. For example, some enjoyed the “free space in their head” that pairing provided by splitting the activities per person in half, while others suffered from “being confined” in just a subset of the activities of software engineering, which is inherently multitasking in its nature. Contrarily, they all praised attaining responsibilities, feeling of control, and perceived importance. 5.1.2.5. Role-specific insights. Of particular interest is that splitting roles allows for developing novel, otherwise inaccessible solutions, as the code “found ways I wouldn’t have found in the pilot role” explains. Also, a command hierarchy originates in role splitting, where “the navigator has the final say.” Lastly, it is essential to say that the participants found that people of some personality traits are fitter for one role and not the other, e.g., code “problem: introverts taking on the navigator role and not talking much.” 5.1.2.6. Task Complexity and Time Constraint. The participants voiced opposing opinions on the strict time limit for staying in their roles. While it provided a fair share of “seat time,” helped with efficiency, and prevented attention deficiencies, it also “confused” those who had not finished their tasks within the limit. Some proposed to make the time limit longer, e.g., 15 minutes per task, to allow for some research time if the pair get lost in the task. Proposed improvements also included revealing the right solution at the end of each round so that everyone could proceed together. Task type and difficulty were found not to affect pair programming. 5.1.2.6. Task Complexity and Time Constraint. 5.1.2.7. Last theme was about discovering the traits of the perfect programmer and, in effect, finding ways to be a better pairing partner or construct or parametrize a better AI pairing software in ways that it “behaves” toward the user. The perfect pairing partner was described as someone humble, tolerant of mistakes, friendly, makes others feel good, and has the tendency to show self-discipline. In the Big five terms, he would score high on extraversion, agreeableness, and openness but low on the neuroticism scale. This is in accord with the results of a previous study [2], where members of the personality cluster scoring high on extraversion and agreeableness were found to be great navigators, and members of the cluster characterized by high openness strongly preferred to be in the pilot role. 5.2 Threats to validity The subjective nature of interpretation poses threats to the validity of our qualitative results. However, the data have been processed systematically and in an epistemological way that introduces as little subjective bias as possible. Internal validity in our research context mainly refers to the suitability of our data sets for the application of the selected statistical methods. This discussion was covered in the methods. 5.3 Limitations and generalizability More research is needed to fully understand the psychological aspects of pairing and pair programming to understand how members of different personality clusters pair with different AI software representing the roles of pilot or navigator. However, some results, such as the “perfect programmer’s” big five dimensions, can already be reused in the development of AI pairing software. Unfortunately, the results of studies on pair programming in university classrooms may not necessarily generalize to professional software development settings, where different setup is more commonly used, such as switching roles deliberately or after finishing a task and also pairing with someone else after a switch of the role. Author: (1) Marcel Valový, Department of Information Technologies, Prague, Czech Republic (marcel.valovy@vse.cz). Author: Author: (1) Marcel Valový, Department of Information Technologies, Prague, Czech Republic (marcel.valovy@vse.cz). This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv available on arxiv