This story draft by @escholar has not been reviewed by an editor, YET.
Authors:
(1) Muhammad Zia Hydari, Katz Graduate School of Business, University of Pittsburgh and Corresponding author;
(2) Idris Adjerid, Pamplin College of Business;
(3) AAaron D. Striegel, Department of Computer Science and Engineering, University of Notre Dame.
2. Background and 2.1. Leaderboards
3. Effect of Leaderboards on Healthful Physical Activity and 3.1. Competition
3.3. Moderating Effects of Prior Activity Levels and Leaderboard Size
4. Data and Model
5. Estimation and Robustness of the Main Effects of Leaderboards
5.2. Robustness Check for Leaderboard Initiation
5.4. Fitbit Attrition, Leaderboard De-Adoption, and Additional Robustness Checks
6. Heterogeneous Effect of Leaderboards
6.1. Heterogeneity by Prior Activity Levels
6.2. Interaction of Leaderboard Size, Rank, and Prior Activity Levels
6.3. Summary of Findings from Heterogeneous Effect Analysis
7. Conclusions and Discussion, Endnotes, and References
To evaluate potential heterogeneity in leaderboard benefit by prior activity, we estimate Specification (1) on samples stratified by preleaderboard activity levels. Specifically, we stratify our sample into two groups based on their preleaderboard activity levels: the top quartile by daily step count comprise the highly active group whereas the bottom quartile by daily step count comprise the sedentary group.[21] The differences in step count between these two groups were meaningful in terms of magnitude and were statistically significant (13,000 versus 8,000, p < 0.01)—see the Online Appendix, Table H.9, for summary statistics on these groups.
Before examining the impact of leaderboards on physical activity levels, we evaluated the correlation between activity levels and relevant preadoption survey measures (see the Online Appendix, Table H.10). We found correlations consistent with our expectations of the key differences between highly active and sedentary individuals. Sedentary individuals reported lower levels of self-efficacy and self-regulation for exercise and reported being more likely to exercise alone. In addition, sedentary individuals reported higher levels of anxiety and depression and lower levels of trust. These correlations suggest that sedentary individuals may need these interventions more than highly active individuals but that they could also be prone to de-motivational impacts if these leaderboards exacerbate mental health barriers to improving health (e.g., increase their anxiety).
Turning to the examination of the effect on steps, we find stark differences in the effect of leaderboards on the highly active group vs. the sedentary group. For sedentary users, the adoption of leaderboards has large and significant impacts on their daily step counts (1,365, p < 0.05; see Table 4, column 1). In contrast, we find that the highly active group, instead of benefiting from leaderboards, experienced a significant decrease in their daily physical activity after leaderboard adoption (–631, p < 0.05; see Table 4, column 2). For the sake of completion, we also estimate the effect for the middle group, i.e., individuals whose activity levels were between the 25th and 75th percentile. We find that the middle group benefited from leaderboards, but the benefit was less than for those who were in the bottom quartile (859, p < :05; see Table 4, column 3). Overall, these results suggest significant heterogeneity in the effect of leaderboard based on prior activity levels, with particularly stark, negative effects for those who were previously highly active.[22]
6.1.1. Leads-Lags Model by Prior Activity. To examine the presence of any trends before leaderboard adoption, we plotted the coefficients from a leads-lags model for the full sample and subsamples by prior activity levels in Figure 2 (see Section 5.1.2 for details about the specification). The first point to note is that the lead coefficients are relatively small in magnitude and statistically insignificant, which increases the plausibility of the parallel trends assumption. Second, the lag coefficients shift to larger magnitudes with the effect persisting for more than two months after adoption. Third, there is some attenuation in the effect for the full sample in later time periods, which can be plausibly explained by the opposite direction of effect within the sedentary and middle groups versus the highly active subsample.
6.1.2. Implication of Findings. The divergence of leaderboard effects for sedentary versus highly active individuals substantiates our conjecture that leaderboards can introduce both motivational and de-motivational dynamics with respect to physical activity. Specifically, these results are suggestive evidence of different impacts on reference points for sedentary versus highly active users and differences in the potential of leaderboards to provide accountability for lapses in physical activity. We explore this difference and its implications for mechanisms underlying leaderboard value further by evaluating the impact of rank and leaderboard size on the physical activity of sedentary versus highly active users in the next section.
This paper is available on arxiv under CC BY 4.0 DEED license.