Interaction of Leaderboard Size, Rank, and Prior Activity Levels

Table of Links

Abstract and 1 Introduction

2. Background and 2.1. Leaderboards

3. Effect of Leaderboards on Healthful Physical Activity and 3.1. Competition

3.2. Social Influence

3.3. Moderating Effects of Prior Activity Levels and Leaderboard Size

4. Data and Model

4.1. Data

4.2. Model

5. Estimation and Robustness of the Main Effects of Leaderboards

5.2. Robustness Check for Leaderboard Initiation

5.3. Fitbit Compliance

5.4. Fitbit Attrition, Leaderboard De-Adoption, and Additional Robustness Checks

6. Heterogeneous Effect of Leaderboards

6.1. Heterogeneity by Prior Activity Levels

6.2. Interaction of Leaderboard Size, Rank, and Prior Activity Levels

6.3. Summary of Findings from Heterogeneous Effect Analysis

7. Conclusions and Discussion, Endnotes, and References

6.2. Interaction of Leaderboard Size, Rank, and Prior Activity Levels

Last, we consider the intersection of all the prior factors in an effort to understand some of the heterogeneity in leaderboard benefit for sedentary versus active users.

6.2.1. Leaderboard Size and Rank. We start by evaluating the motivational impact of leaderboard rank and size separately by prior activity levels. We find that both previously sedentary and highly active individuals see substantial increases in physical activity during the week after they ranked first on their leaderboard (Table 4, columns 4 and 5). For sedentary individuals, being first in the prior week unlocks even more value for them and increases the treatment effect for leaderboards by 746 steps to nearly 2,000 steps. Highly active individuals see slightly less value (522 steps versus 746 steps) from being first in the prior week but this benefit counteracts part of the negative main effect they observe. Extending this analysis to also include leaderboard size and the interaction of leaderboard size and prior week’s performance reveals further richness in these results. In column 6, we observe that previously sedentary individuals still see substantial benefit when leaderboards are small and when they are not first (1,016 steps, p < 0.1) and that this benefit increases further when they rank first and leaderboards are larger. In column 7, we observe that previously active individuals are harmed when they are on small leaderboards; these users only start to see positive impacts from leaderboards if they rank first on leaderboards with more than four individuals or if they are on relatively large leaderboards (more than six active users).

The finding that sedentary individuals observe leaderboard benefit despite unsuccessfully competing on small leaderboards suggests that these individuals accrue significant benefit from noncompetition mechanisms (i.e., social influence). In contrast, the negative impact of the same type of leaderboard for highly active individuals suggests that social influence is harming these individuals, or that its benefit is not sufficient to overcome any de-motivational effects of not ranking first. We probe this conclusion further using two additional analyses that leverage certain leaderboard instances which could be telling of the impact of these noncompetition mechanisms on physical activity. The first analysis seeks to identify instances of leaderboards where competitive dynamics are arguably weakened but the potential for mutual accountability and changes in reference points is still present. Specifically, we create NoCompetitionLB, which is an indicator of leaderboard instances where the focal user is sandwiched between two other users such that there is not a credible threat to their rank from either user (see the Online Appendix, Section H.3 for details). Table 5, columns 1 and 2, shows a positive and significant impact (534 steps, p < 0.05) of these types of leaderboards for previously sedentary users but a small and insignificant impact of these same types of leaderboards for highly active users (-39 steps, p = 0.9). We examine this conjecture further by identifying instances of leaderboards at the other end of the

6.2.2. Implication of Findings. Although only suggestive evidence, these results lend some credence to the notion that previously sedentary individuals are benefiting from mutual accountability and positive impacts on their exercise reference point and do not need competition to benefit. In contrast, highly active individuals seemed to be harmed by the noncompetition mechanisms, but this harm can be offset if leaderboards are sufficiently competitive.

6.2.3. Nonlinear Impacts of Leaderboard Size. Finally, we consider whether nonlinear impacts of leaderboard size are similar based on prior activity level. In Table 5, columns 5 and 6, we find that, although both groups have diminishing returns from additional users, the negative coefficient on the quadratic term (ActiveLB2) for the sedentary group is thrice that for the highly active group. These results suggest that for sedentary

6.2.4. Implications of Findings. The smaller optimal size for sedentary individuals suggest that they accrue benefits (e.g., positive impacts on their exercise reference points) even when leaderboards are small. However, benefits diminish as leaderboards become larger, and leaderboards can even be harmful if they become excessively large (leaderboard sizes that become harmful to sedentary users were uncommon in our data). In contrast, highly active individuals seem to be demotivated by leaderboards with too few individuals and only start to benefit on larger, more competitive leaderboards. These dynamics for highly active individuals reinforce the notion that these users require large leaderboards to derive benefit.