The Impact of Remote Pair Programming in an Upper-Level CS Course: Results

Table of Links

Abstract and 1 Introduction

2 Previous Research

3 Course Description

4 Methods

5 Results

6 Discussion

7 Conclusion and Future Work, Acknowledgments, and References

5 RESULTS

A total of 88 students completed both surveys: 64 male students and 24 female students. Pair programming (n=56) and individual sections (n=32) were similar at both at baseline and end of the term with respect to responses given on the survey with the exception of gender. There were more women in the pair programming section than in the individual programming section (20 versus 4).

5.1 Individual vs. Pair Programming

In Table 1, repeated measures ANOVA on programming assignments showed both a within-subject (p = .003) and a between subject effect (p = .001). The pair programming group consistently scored higher than the individuals over time, with the difference ranging from 8–14% (Figure 1). Additionally, the pair programming group scored nearly perfectly on assignments 3 and 4.

Repeated measures ANOVA on exam score showed a withinsubject effect (p < .001) but no between subject effect. The pair programming group scored higher than the individual programming group as the semester progressed (Figure 2). Final scores revealed a 7% higher score among students in the pair programming group (87.4 vs. 80.4; p = .013) compared to students in the individual programming group (Table 1).

From the survey, for both paired and individual students, there was a slight decrease in the number of students planning “to continue to pursue a career in computing/programming" (Figure 3). This effect was mitigated in the pair programming group, which saw a total increase of 1.8% in those selecting “no" or “unsure", as opposed to an increase of 12.3% in the individual group.

Finally, individual students were initially more comfortable asking questions in class than pair programming students (Figure 4). However, by the end of the semester, pair programming students had slightly surpassed the individuals in number of students who “strongly agreed" that “I am comfortable asking questions in class" (44.6% vs. 43.8%). This suggests that pair programming may positively affect student confidence related to speaking up in class.

5.2 In Person vs. Remote

Students in the pair programming group were asked to respond to the statement “my pair programming partner and I always worked remotely” on the end of semester survey. Overall, 34 (59%) strongly agreed, 21 (36%) somewhat agreed, 2 (3%) somewhat disagreed, and 1 (2%) strongly disagreed. It appears one pair had a student who

somewhat agreed with the statement, while the other somewhat disagreed. Ultimately, while the course itself was offered in person, most students chose to pair program remotely. While we did not ask the students why they chose to do so, we suspect it was due to convenience (several students mentioned something similar to struggling with finding a common time to meet) or safety concerns (e.g., to protect against the transmission of Covid-19).

Programming assignments and final scores were compared between those who agreed or strongly agreed to working remotely and those who disagreed or strongly disagreed (i.e., worked in person). An interesting trend emerged (Table 2). The T-tests showed no statistically significant difference between both groups on any assessment, except for assignment 4, in which the group that worked remotely did better (by 2.9 points) than those that worked in person (p = .012). In addition, the actual difference between remote and in person groups narrowed over time, suggesting that those who pair programmed remotely found a routine that worked as the semester progressed. This means that no adverse effects of remote pair programming on student outcomes were observed in this study.

5.3 Gender Differences

Women in the pair programming group scored almost 12% higher than women in the individual programming group on their final

score in the course. While this difference did not achieve statistical significance (p = .091) due to the low sample size of women in the individual (non-pair) programming group, a 12% difference is practically significant in that it could result in a difference of 2 letter grades (e.g., from a 79 to a 91).

There were also several gender differences in responses on the end of semester survey from students in the pair programming section. First, women reported an increase in comfort asking questions in class, when compared to men (Figure 5). At the beginning of the semester, only 15% strongly agreed with the statement, “I am comfortable asking questions in class". At the end of the semester, 40% of women students agreed with the statement, and fewer women than men disagreed with the statement. This is important, because it is an indicator of confidence and implies the ability to handle graceful failure in front of peers. By participating in classroom discussion, there is greater opportunity for learning.

Both men and women saw a similar decrease in confidence in their programming abilities by the end of the semester (a reduction of 10% in “strongly agree", Figure 6). In contrast to a first year course, where many students are programming for the first time and seeing marked strides in learning a new language, students in a CS-3 course like Data Structures have prior experience, but are then exposed to significantly difficult assignments. The learning curve is much higher. Thus, it is not surprising that some students adjusted their confidence after experiencing the rigor of solving large, real-world problems.

No male student indicated that he had a definitive plan to pursue a career outside of computer science (Figure 7). Some women, however, noted definitively that they did not intend to continue work in computer science. A representative comment that explains the sentiment behind taking Data Structures, but not desiring to continue in the field is the following from a woman in the first section: “I feel as though I am a people person and I would not enjoy sitting behind a computer computing/programming all day. I’d rather sell the product [of] the computation to people, but have the understanding of what it is doing."

Finally, when asked to respond to the statement “I prefer pair programming to working individually” women had more extreme responses than men. Almost one-quarter (23.8%) of female students strongly agreed compared to 8.1% of male students. Similarly, onethird of women (33.3%) strongly disagreed compared to 18.9% of men. The differences in responses to this question were marginally significant with p = .052.