Countering Mainstream Bias via End-to-End Adaptive Local Learning: Debiasing Performance

Table of Links

Abstract and 1 Introduction

2 Preliminaries

3 End-to-End Adaptive Local Learning

3.1 Loss-Driven Mixture-of-Experts

3.2 Synchronized Learning via Adaptive Weight

4 Debiasing Experiments and 4.1 Experimental Setup

4.2 Debiasing Performance

4.3 Ablation Study

4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study

5 Related Work

6 Conclusion, Acknowledgements, and References

4.2 Debiasing Performance

First, we conduct a comparative analysis to show the effectiveness of the proposed TALL. In Table 2, we evaluate the overall NDCG@20 and average NDCG@20 for five user subgroups with varying mainstream levels for all methods and datasets. The best results of each metric and subgroup for all datasets are marked in bold, and the improvement rate of the proposed TALL over the best baseline MultVAE, LOCA, and LFT is exhibited as well. The user subgroups are categorized based on their mainstream scores.

TALL vs. MultVAE & WL. First, we can observe that the utilities for users of all five subgroups are greatly promoted by our proposed TALL compared to the widely used model MultVAE. Moreover, we can see that although the global debiasing method WL can alleviate the mainstream bias to a certain degree compared to MultVAE, our proposed TALL can produce higher NDCG@20 for all five user subgroups than WL, depicting that the proposed TALL exhibits a more outstanding debiasing ability over WL.

TALL vs. EnLFT & LOCA. Hence, we next have a fair comparison between models of the same complexity and compare directly across local learning methods. From Table 2, we observe that LOCA and EnLFT are more effective in mitigating the mainstream bias than WL, as they remarkably enhance utility across all five groups on all datasets. Meanwhile, our TALL significantly outperforms LOCA and EnLFT across all user groups and datasets. The improvement is especially prominent for niche users: TALL improves NDCG@20 of the ‘low’ user group by 6.07% on average over LOCA and 10% over EnLFT. This shows that with the same model complexity, the proposed end-to-end adaptive local learning model is more effective than heuristic-based local learning models.

TALL vs. LFT. Last, we compare TALL with the state-of-the-art local learning baseline LFT, which is heavily computationally intense and time-consuming. But due to its special design that every user gets their own customized model trained by their local data, LFT can effectively address the discrepancy modeling problem. From Table 2, we observe that, for most of the time, LFT achieves the best performance for niche users among all baselines. And LFT can perform especially effectively for dense datasets (i.e., ML1M). In fact, given that the model complexity of LFT is much higher than TALL, it is unfair to compare them only based on recommendation accuracy (i.e., NDCG). For example, in the ML1M dataset, our TALL contains 100 expert models (MultVAE), while LFT trains 6,040 (#users) models separately, which is over 60 times larger than TALL. Although it is not a fair comparison, we can still observe in Table 2 that TALL can outperform LFT in most cases. Especially for the two sparse datasets Yelp and CDs&Vinyl, TALL produces significantly higher utilities for all types of users. This demonstrates the efficacy and necessity of an end-to-end local learning method compared to a heuristic-based one.

In sum, from Table 2, we see that for all datasets, the proposed TALL produces the greatest NDCG@20 improvement for each subgroup of different mainstream levels and leads to the state-of-the-art overall model performance. TALL can outperform baselines with lower and the same model complexity, and it can even outperform the baseline model that is way more complex than it.