Moonstream

moonstream

Why The Metaverse Is Already Broken (And How To Fix It — Chapter Two)

A guide to finding good web3 games and avoiding scams in 10 steps So you decided to look into NFT gaming and play-to-earn games on the blockchain. Is it really possible to make money just by playing? Is it still possible during a market crash? Once you start getting into web3 gaming, sometimes you get more questions than answers. So I want to help you with a series of articles on the topic. Rug pulls Possibly the first thing you come across after starting your research is an abundance of scams, rug pulls and pump-and-dump schemes. What is that? Rug pulls and pump-and-dumps are similar in meaning. It describes a project that drives up its value with marketing to attract investors. Then the founders sell all their assets, dropping the value to zero and leaving investors with nothing. Do your own research How to avoid scams? You might hear a lot of advice to (do your own research). But how do you DYOR when you just entered the scene? Below are 10 things we suggest you start with. Check their previous projects. Are they being truthful about their background? Were any of their projects scams? How did they deal with their previous projects failing? If they just left without saying anything or providing any meaningful resolution — it’s a red flag. Look up the founders. Are there any big names? Google them. Check their investors. If a gaming project doesn’t have one — it’s a red flag. See what the community is talking about. How are the developers responding if at all? Join their Discord server. or are there only investors waiting to cash out when the time comes? Are there players in the community who build game strategies If they aren’t — are they providing explanations and support to their community or are they staying silent? See if they’re delivering on their promises stated on the roadmap. Is anyone buying their tokens? Does the price change a lot? You can check that on , for example. Check their token activity. CoinMarketCap The truth is, the cheaper the token stays the better. Low game token prices mean that the GameFi project was designed as a game and not as an investment opportunity. https://twitter.com/Moonstreamto/status/1540062155914891264?s=20&t=ocC7HtXDqLILu1kDkjx8Nw Some blockchains have higher gas fees and slower processing times — that will become important during gameplay. Ethereum has high gas (at least before ), Ethereum’s Level 2 chains have low gas fees. Check what blockchain the game is on. the Merge Is it too expensive? How easy is it to start participating? , how successful are they? Do they have 10k NFTs and have sold only 10 of them? Check prices on OpenSea Are they all words and Discord/Twitter (a.k.a. off-chain) giveaways and do not have any blockchain development going on? There’s a big chance they want to sell you tokens/NFTs with no actual in-game utility. See if the project has any blockchain activity. Look for on-chain loyalty programs, leaderboards, airdrops, minigames, in-game items, and consumables. Think of web3 games not as an investment but as an actual game. When you can tell that the developers put emphasis on gameplay, on the fun players are supposed to be having — that’s a good sign. If you want to get rich fast — there’s a higher chance of investing in the wrong kind of project that overpromises and flops after launch. Stay tuned for more articles on how to enter and participate in blockchain gaming. If you found this post useful, leave a comment, and share it with a friend! I’d really appreciate that.

How to Find Promising NFT Games and Avoid Scams

No one can deny it anymore; we are finally on the verge of Bitcoin Endgame, and no one is ready for what is coming. The mainstream..? Find out here. For years, Bitcoiners have been trying to tell as many people as they possibly could about Bitcoin. How it is not only a great way to preserve your wealth but also to preserve your freedom as well. These same Bitcoiners have long predicted that someday Bitcoin would cause the largest transfer of wealth that the world has ever seen. The people who have been heavily investing in BTC would reap the rewards and see their wealth grow beyond what they ever imagined possible. And that those who had been sitting on the sideline not investing in Bitcoin would feel extreme pain. The question has always been When would Bitcoin go mainstream? Would there ever be hyper-bitcoinization? A moment when everyone wakes up to how scarce Bitcoin is, the benefits it offers over fiat, and that they need to own as much as they possibly can. Causing a stampede rush toward Bitcoin that would cause it to go parabolic. The theory has always been that each day Bitcoin survives, remains decentralized, and produces blocks. That result becomes more and more likely. 3 Bitcoin News/Events That Storm the World Over the last 3 years, there have been 4 significant events that have taken the world by storm and also dramatically sped up Bitcoin's path to dominance. Bitcoin Mission to El Salvador The first was in June 2021, at the Bitcoin Conference in Miami. The price of BTC was fluctuating around the $35k range at that time. It was at this conference that the President of El Salvador, Nayib Bukele, announced that El Salvador would officially be making Bitcoin legal tender in the country. Up until this point, this was something that not even the most optimistic Bitcoiners had on their chessboard. A handful of months before Michael Saylor had announced that Microstrategy had put Bitcoin on their balance sheet, and most of us were only thinking about company adoption. But now the possibilities had become limitless and it was truly the shot heard around the world. However, that was only the beginning... BlackRock & Bitcoin ETF The next giant move happened in 2023 when BlackRock announced that they were filing for a spot in Bitcoin ETF.  BlackRock is the largest financial fund manager in the world and arguably has more power than governments. When BlackRock speaks, everyone listens. Up until that point, most agreed that a spot ETF approval in the US would be nearly impossible with the current government. But BlackRock always gets what they want. The bullishness from the Bitcoin spot ETFs going live in January of 2024 was enough of a catalyst to make Bitcoin do things that it had never done before. In March, Bitcoin soared to $73k, and for the first time in its history, it made a new all-time high price before the Bitcoin halving. This led many to speculate just how high the price could go during this cycle. Now, finally fast-forwarding to today, there has been a 3rd Bitcoin announcement that has sent shockwaves across the world. Donald Trump Presidency: The Bitcoin President In recent months (earlier this year) Donald Trump has done a complete reversal of his opinions about Bitcoin and crypto. Even going so far as to be a speaker at the Bitcoin Conference 2024 in Nashville. During this speech, he announced that if elected President he would be making a national Bitcoin reserve using the Bitcoin the US already owned and adding to it. There were also proposals by other government leaders to accumulate as much as one million Bitcoins. What this all means is that the Bitcoin End Game is officially here. Nation-state adoption is here. It was incredible news in 2021 when El Salvador announced their Bitcoin adoption as legal tender. The US possibly making these moves is something on a much larger scale. Like it or not, all countries watch what the US does, and often follow suit. It is a safe bet that after Trump made this announcement of his plans to make a US Bitcoin reserve, you can bet that nearly every other country was having meetings on what their Bitcoin strategy would be. Trump Presidential Victory: The All-Time Highs (ATHs) Donald Trump's victory in the US presidential election has proved to be a good omen for Bitcoin (including the crypto market generally), with BTC price rising above $75,000 for the first time, breaking all-time high (ATH) multiple times. During his election campaign, Donald Trump promised to make the US the world's crypto capital. So, with his victory, the prices of the most popular and important digital currency have skyrocketed, and one bitcoin is worth over $90,000. Bitcoin surpassed Meta (Facebook) and Silver in the global market capitalization and now ranked 8th, with the rise triggered by the optimism created by Donald Trump's victory in the 2024 presidential elections. To round up here... It is important to understand that since the last bull cycle, the long-prophesied Bitcoin Endgame to mainstream adoption has finally arrived, with BTC to $100k anytime soon. In this new phase, the promise of hyper-bitcoinization has been fulfilled. Bitcoin has successfully become the reserve asset of choice, for stability and collaboration among nations. For individuals, it represents hope; a decentralized future where wealth, freedom, and opportunity are within reach for all. This is not an end but a beginning. No one can deny it anymore; we are finally on the verge of Bitcoin Endgame, and no one is ready for what is coming. The mainstream..? The mainstream..? The mainstream..? Find out here. Find out here. For years, Bitcoiners have been trying to tell as many people as they possibly could about Bitcoin. How it is not only a great way to preserve your wealth but also to preserve your freedom as well. These same Bitcoiners have long predicted that someday Bitcoin would cause the largest transfer of wealth that the world has ever seen. The people who have been heavily investing in BTC would reap the rewards and see their wealth grow beyond what they ever imagined possible. And that those who had been sitting on the sideline not investing in Bitcoin would feel extreme pain. The question has always been When would Bitcoin go mainstream? When would Bitcoin go mainstream? When would Bitcoin go mainstream? When would Bitcoin go mainstream? Would there ever be hyper-bitcoinization? Would there ever be hyper-bitcoinization? Would there ever be hyper-bitcoinization? Would there ever be hyper-bitcoinization? A moment when everyone wakes up to how scarce Bitcoin is, the benefits it offers over fiat, and that they need to own as much as they possibly can. Causing a stampede rush toward Bitcoin that would cause it to go parabolic. The theory has always been that each day Bitcoin survives, remains decentralized, and produces blocks. That result becomes more and more likely. 3 Bitcoin News/Events That Storm the World 3 Bitcoin News/Events That Storm the World Over the last 3 years, there have been 4 significant events that have taken the world by storm and also dramatically sped up Bitcoin's path to dominance. Bitcoin Mission to El Salvador Bitcoin Mission to El Salvador The first was in June 2021, at the Bitcoin Conference in Miami. The price of BTC was fluctuating around the $35k range at that time. It was at this conference that the President of El Salvador, Nayib Bukele, announced that El Salvador would officially be making Bitcoin legal tender in the country. Up until this point, this was something that not even the most optimistic Bitcoiners had on their chessboard. A handful of months before Michael Saylor had announced that Microstrategy had put Bitcoin on their balance sheet, and most of us were only thinking about company adoption. But now the possibilities had become limitless and it was truly the shot heard around the world. However, that was only the beginning... However, that was only the beginning... However, that was only the beginning... BlackRock & Bitcoin ETF The next giant move happened in 2023 when BlackRock announced that they were filing for a spot in Bitcoin ETF. BlackRock is the largest financial fund manager in the world and arguably has more power than governments. When BlackRock speaks, everyone listens. Up until that point, most agreed that a spot ETF approval in the US would be nearly impossible with the current government. But BlackRock always gets what they want. The bullishness from the Bitcoin spot ETFs going live in January of 2024 was enough of a catalyst to make Bitcoin do things that it had never done before. In March, Bitcoin soared to $73k, and for the first time in its history, it made a new all-time high price before the Bitcoin halving. This led many to speculate just how high the price could go during this cycle. Now, finally fast-forwarding to today, there has been a 3rd Bitcoin announcement that has sent shockwaves across the world. Now, finally fast-forwarding to today, there has been a 3rd Bitcoin announcement that has sent shockwaves across the world. Now, finally fast-forwarding to today, there has been a 3rd Bitcoin announcement that has sent shockwaves across the world. Donald Trump Presidency: The Bitcoin President In recent months (earlier this year) Donald Trump has done a complete reversal of his opinions about Bitcoin and crypto. Even going so far as to be a speaker at the Bitcoin Conference 2024 in Nashville. (earlier this year) During this speech, he announced that if elected President he would be making a national Bitcoin reserve using the Bitcoin the US already owned and adding to it. There were also proposals by other government leaders to accumulate as much as one million Bitcoins. What this all means is that the Bitcoin End Game is officially here. What this all means is that the Bitcoin End Game is officially here. What this all means is that the Bitcoin End Game is officially here. Nation-state adoption is here. It was incredible news in 2021 when El Salvador announced their Bitcoin adoption as legal tender. The US possibly making these moves is something on a much larger scale. Like it or not, all countries watch what the US does, and often follow suit. It is a safe bet that after Trump made this announcement of his plans to make a US Bitcoin reserve, you can bet that nearly every other country was having meetings on what their Bitcoin strategy would be. Trump Presidential Victory: The All-Time Highs (ATHs) Donald Trump's victory in the US presidential election has proved to be a good omen for Bitcoin (including the crypto market generally), with BTC price rising above $75,000 for the first time, breaking all-time high (ATH) multiple times. (including the crypto market generally), (ATH) During his election campaign, Donald Trump promised to make the US the world's crypto capital. So, with his victory, the prices of the most popular and important digital currency have skyrocketed, and one bitcoin is worth over $90,000. the world's crypto capital. the world's crypto capital. Bitcoin surpassed Meta (Facebook) and Silver in the global market capitalization and now ranked 8th, with the rise triggered by the optimism created by Donald Trump's victory in the 2024 presidential elections. (Facebook) To round up here... To round up here... To round up here... It is important to understand that since the last bull cycle, the long-prophesied Bitcoin Endgame to mainstream adoption has finally arrived, with BTC to $100k anytime soon. In this new phase, the promise of hyper-bitcoinization has been fulfilled. Bitcoin has successfully become the reserve asset of choice, for stability and collaboration among nations. For individuals, it represents hope; a decentralized future where wealth, freedom, and opportunity are within reach for all. Bitcoin has successfully become the reserve asset of choice, for stability and collaboration among nations. For individuals, it represents hope; a decentralized future where wealth, freedom, and opportunity are within reach for all. This is not an end but a beginning. This is not an end but a beginning. This is not an end but a beginning.

Bitcoin Endgame Is Here: From Satoshi to Mainstream Adoption

Countering Mainstream Bias via End-to-End Adaptive Local Learning: Conclusion and References

Table of Links Abstract and 1 Introduction 2 Preliminaries 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.3 Ablation Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 6 Conclusion, Acknowledgements, and References 5 Related Work Fairness and bias issues in recommender systems have attracted increasing attention recently. Popularity bias [2,3,9,34,37,38], exposure bias [5,21,30,31], and item fairness [6,7,11,17,27,35] exemplify significant item-side biases. Besides prior works mainly focusing on the item perspective, several research studies have explored user biases, analyzing utility differences among diverse user groups based on user demographic attributes, like age or gender [10,15,16,25,32,33,36]. For instance, Ekstrand et al. [15] empirically investigated multiple recommendation models and demonstrated utility differences across user demographic groups. Schedl et al. [32] examined music preference differences among user age groups, revealing variations in recommendation performance. To address these issues, Fu et al. [16] proposed to leverage rich information from knowledge graphs, Li et al. [25] developed a re-ranking to narrow the utility gap between different user groups, and Chen et al. [10] implemented data augmentation by generating “fake” data to achieve a balanced distribution. However, demographic attributes may not comprehensively capture user interests and behaviors. Unlike the aforementioned works focusing on bias analysis based on demographic groups, mainstream bias poses a critical challenge in recommender systems. Previous works [4,18,24] acknowledge mainstream bias as the “grey-sheep” problem, where “grey-sheep users” with niche interests lead to challenges in finding similar peers and result in poor recommendations. However, they do not propose robust bias measurements and debiasing methods. A more aligned study with better mainstream bias evaluations to this paper is [39], which also addresses mainstream bias and enhances utility for niche users using global and local methods. Prior existing local methods [12,13,22,23,39] and global methods [39] can mitigate the bias to some degree by improving the utility for niche users. The recently proposed Local Fine Tuning (LFT) [39] and local collaborative autoencoder (LOCA) [12] produce state-of-the-art performance by employing multiple multinomial variational autoencoders (MultVAE) [26] as base models and generating customized local models to capture special patterns of different types of user. Nonetheless, prior methods have a key limitation: their reliance on heuristics impacts performance, necessitating meticulous hyper-parameters tuning by practitioners. Thus, the performance of these prior heuristic-based local learning methods is limited. This work targets the mainstream bias problem by proposing an end-to-end adaptive local learning framework to automatically and adaptively learn customized local models for different users, overcoming the limitations of heuristic-based methods to mitigate mainstream bias. Authors:
(1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA;
(2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA;
(3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA;
(4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA;
(5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3 End-to-End Adaptive Local Learning 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.2 Debiasing Performance 4.3 Ablation Study Ablation 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 5 Related Work 6 Conclusion, Acknowledgements, and References 6 Conclusion, Acknowledgements, and References 5 Related Work Fairness and bias issues in recommender systems have attracted increasing attention recently. Popularity bias [2,3,9,34,37,38], exposure bias [5,21,30,31], and item fairness [6,7,11,17,27,35] exemplify significant item-side biases. Besides prior works mainly focusing on the item perspective, several research studies have explored user biases, analyzing utility differences among diverse user groups based on user demographic attributes, like age or gender [10,15,16,25,32,33,36]. For instance, Ekstrand et al. [15] empirically investigated multiple recommendation models and demonstrated utility differences across user demographic groups. Schedl et al. [32] examined music preference differences among user age groups, revealing variations in recommendation performance. To address these issues, Fu et al. [16] proposed to leverage rich information from knowledge graphs, Li et al. [25] developed a re-ranking to narrow the utility gap between different user groups, and Chen et al. [10] implemented data augmentation by generating “fake” data to achieve a balanced distribution. However, demographic attributes may not comprehensively capture user interests and behaviors. Unlike the aforementioned works focusing on bias analysis based on demographic groups, mainstream bias poses a critical challenge in recommender systems. Previous works [4,18,24] acknowledge mainstream bias as the “grey-sheep” problem, where “grey-sheep users” with niche interests lead to challenges in finding similar peers and result in poor recommendations. However, they do not propose robust bias measurements and debiasing methods. A more aligned study with better mainstream bias evaluations to this paper is [39], which also addresses mainstream bias and enhances utility for niche users using global and local methods. Prior existing local methods [12,13,22,23,39] and global methods [39] can mitigate the bias to some degree by improving the utility for niche users. The recently proposed Local Fine Tuning (LFT) [39] and local collaborative autoencoder (LOCA) [12] produce state-of-the-art performance by employing multiple multinomial variational autoencoders (MultVAE) [26] as base models and generating customized local models to capture special patterns of different types of user. Nonetheless, prior methods have a key limitation: their reliance on heuristics impacts performance, necessitating meticulous hyper-parameters tuning by practitioners. Thus, the performance of these prior heuristic-based local learning methods is limited. This work targets the mainstream bias problem by proposing an end-to-end adaptive local learning framework to automatically and adaptively learn customized local models for different users, overcoming the limitations of heuristic-based methods to mitigate mainstream bias. mainstream bias Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. Authors: Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Countering Mainstream Bias via End-to-End Adaptive Local Learning: Related Work

Table of Links Abstract and 1 Introduction 2 Preliminaries 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.3 Ablation Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 6 Conclusion, Acknowledgements, and References 4.4 Effect of the Adaptive Weight Module Last, we turn our attention to investigating the effect of the adaptive weight module, studying how it synchronizes the learning paces of different users. We run TALL on the ML1M dataset and present the average weights for the five subgroups with the gap window (#gap = 40) in Figure 3. It can be observed that the adaptive weight module assigns weights dynamically to different types of users to synchronize their learning paces. Initially, mainstream users receive higher weights because they are easier to learn and have a higher upper bound of performance than niche users. Then, when mainstream users reach the peak, the model switches the attention to niche users who are more difficult to learn, gradually increasing the weights for ‘low’, ‘med-low’, and ‘medium’ users until the end of the training procedure. However, ‘med-high’ and ‘high’ users, approaching converged, need a slower learning pace to avoid overfitting, leading to a decrease in the weights. Figure 3 illuminates the effectiveness and dynamic nature of the proposed adaptive weight module in synchronizing the learning procedures for different types of users. 4.5 Hyper-parameter Study Additionally, we have also conducted a comprehensive hyper-parameter study investigating the impacts of three hyper-parameters in TALL: (1) the gap window in the adaptive weight module; (2) α in the adaptive weight module; and (3) the number of experts. The complete results are in https://github.com/JP-25/ end-To-end-Adaptive-Local-Leanring-TALL-/blob/main/Hyperparameter Study. pdf. Authors:
(1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA;
(2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA;
(3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA;
(4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA;
(5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3 End-to-End Adaptive Local Learning 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.2 Debiasing Performance 4.3 Ablation Study Ablation 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 5 Related Work 6 Conclusion, Acknowledgements, and References 6 Conclusion, Acknowledgements, and References 4.4 Effect of the Adaptive Weight Module Last, we turn our attention to investigating the effect of the adaptive weight module, studying how it synchronizes the learning paces of different users. We run TALL on the ML1M dataset and present the average weights for the five subgroups with the gap window (#gap = 40) in Figure 3. It can be observed that the adaptive weight module assigns weights dynamically to different types of users to synchronize their learning paces. Initially, mainstream users receive higher weights because they are easier to learn and have a higher upper bound of performance than niche users. Then, when mainstream users reach the peak, the model switches the attention to niche users who are more difficult to learn, gradually increasing the weights for ‘low’, ‘med-low’, and ‘medium’ users until the end of the training procedure. However, ‘med-high’ and ‘high’ users, approaching converged, need a slower learning pace to avoid overfitting, leading to a decrease in the weights. Figure 3 illuminates the effectiveness and dynamic nature of the proposed adaptive weight module in synchronizing the learning procedures for different types of users. 4.5 Hyper-parameter Study Additionally, we have also conducted a comprehensive hyper-parameter study investigating the impacts of three hyper-parameters in TALL: (1) the gap window in the adaptive weight module; (2) α in the adaptive weight module; and (3) the number of experts. The complete results are in https://github.com/JP-25/ end-To-end-Adaptive-Local-Leanring-TALL-/blob/main/Hyperparameter Study. pdf. Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. Authors: Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Countering Mainstream Bias via End-to-End Adaptive Local Learning: Hyper-parameter Study

Table of Links Abstract and 1 Introduction 2 Preliminaries 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.3 Ablation Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 6 Conclusion, Acknowledgements, and References 4.3 Ablation Study Next, we aim to investigate the effectiveness of different components in the proposed framework, including the proposed adaptive loss-driven gate module, the adaptive weight module, the gap mechanism in the adaptive weight module, and the loss change mechanism in the adaptive weight module. Adaptive Loss-Driven Gate. To verify the effectiveness of the proposed adaptive loss-driven gate module in the proposed TALL, we compare the MoE component with the adaptive loss-driven gate (denoted as LMoE, which is the TALL model without the adaptive weight module) to a conventional MoE (denoted as MoE) with the standard multilayer perceptron (MLP) as the gate learning from the dataset. By comparing LMoE and MoE, we can justify the effect of the proposed adaptive loss-driven gate. The results are present in Table 3, where we also include the result of MultVAE as a baseline. From the table, we can observe that MoE produces better performance for niche users compared to MultVAE but worse results for other users. This is caused by the MLP-based gate model in MoE, which cannot precisely allocate gate values to expert models to ensemble a strong customized model for different users. Conversely, we can see that even without the adaptive weight module, the LMoE can deliver greatly higher utilities for all types of users compared to MultVAE and MoE, showing the strong capability of the proposed adaptive loss-driven gate module in distributing gate values across expert models. And this result also demonstrates the efficacy of the proposed LMoE module in TALL in terms of addressing the discrepancy modeling problem. Adaptive Weight. To address the unsynchronized learning problem, we develop the adaptive weight module to dynamically adjust the learning paces of different users. To verify the effectiveness of the proposed adaptive weight, we compare the complete TALL algorithm (with both the loss-driven MoE module and the adaptive weight module) to the loss-driven MoE module (LMoE). The results on ML1M are shown in Table 4, from which we can see that TALL outperforms LMoE for all types of users, manifesting the effectiveness of the proposed adaptive weight method. Furthermore, we explore the effectiveness of two special mechanisms, the gap mechanism and the loss change mechanism within the adaptive weight module, in the subsequent sections. Gap Mechanism. To avoid the unstable loss problem at the initial training stage, we propose to have a gap for the adaptive weight method, i.e., we wait for a certain number of epochs at the initial training stage until the loss is stable and then apply the adaptive weight method. To verify the effectiveness of such a gap strategy, we compare the complete TALL (with a full version of the adaptive weight module including both the gap mechanism and the loss change mechanism) to a variation of TALL with the adaptive weight module without the gap mechanism. The comparison is presented in Table 4 as LMoE+LC vs. TALL. We can observe that the gap mechanism does have a significant influence on the model performance that the model with the gap mechanism (TALL) delivers better utilities for all types of users than the model without the gap mechanism (LMoE+LC). Loss Change Mechanism. Last, we aim to verify the effectiveness of the proposed loss change mechanism in the adaptive weight module. The goal of loss change is to counter the scale diversity problem when applying the adaptive weight module. Here, we compare the complete TALL (with a full version of the adaptive weight module including both the gap mechanism and the loss change mechanism) to a variation of TALL (LMoE+gap+L) with the adaptive weight module that uses original loss as introduced in Section 3.2. From the comparison result shown in Table 4, we see that the model using loss change (TALL) performs better than the model using original loss for calculating weights (LMoE+gap+L). TALL outperforms LMoE+gap+L for all types of users. In sum, by a series of comparative analyses, we show that the proposed adaptive loss-driven gate module, adaptive weight module, the gap mechanism in adaptive weight, and the loss change mechanism in adaptive weight are effective and play imperative roles in the proposed TALL framework. Authors:
(1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA;
(2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA;
(3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA;
(4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA;
(5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3 End-to-End Adaptive Local Learning 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.2 Debiasing Performance 4.3 Ablation Study Ablation 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 5 Related Work 6 Conclusion, Acknowledgements, and References 6 Conclusion, Acknowledgements, and References 4.3 Ablation Study Next, we aim to investigate the effectiveness of different components in the proposed framework, including the proposed adaptive loss-driven gate module, the adaptive weight module, the gap mechanism in the adaptive weight module, and the loss change mechanism in the adaptive weight module. Adaptive Loss-Driven Gate. To verify the effectiveness of the proposed adaptive loss-driven gate module in the proposed TALL, we compare the MoE component with the adaptive loss-driven gate (denoted as LMoE, which is the TALL model without the adaptive weight module) to a conventional MoE (denoted as MoE) with the standard multilayer perceptron (MLP) as the gate learning from Adaptive Loss-Driven Gate. the dataset. By comparing LMoE and MoE, we can justify the effect of the proposed adaptive loss-driven gate. The results are present in Table 3, where we also include the result of MultVAE as a baseline. From the table, we can observe that MoE produces better performance for niche users compared to MultVAE but worse results for other users. This is caused by the MLP-based gate model in MoE, which cannot precisely allocate gate values to expert models to ensemble a strong customized model for different users. Conversely, we can see that even without the adaptive weight module, the LMoE can deliver greatly higher utilities for all types of users compared to MultVAE and MoE, showing the strong capability of the proposed adaptive loss-driven gate module in distributing gate values across expert models. And this result also demonstrates the efficacy of the proposed LMoE module in TALL in terms of addressing the discrepancy modeling problem. Adaptive Weight. To address the unsynchronized learning problem, we develop the adaptive weight module to dynamically adjust the learning paces of different users. To verify the effectiveness of the proposed adaptive weight, we compare the complete TALL algorithm (with both the loss-driven MoE module and the adaptive weight module) to the loss-driven MoE module (LMoE). The results on ML1M are shown in Table 4, from which we can see that TALL outperforms LMoE for all types of users, manifesting the effectiveness of the proposed adaptive weight method. Furthermore, we explore the effectiveness of two special mechanisms, the gap mechanism and the loss change mechanism within the adaptive weight module, in the subsequent sections. Adaptive Weight. Gap Mechanism . To avoid the unstable loss problem at the initial training stage, we propose to have a gap for the adaptive weight method, i.e., we wait for a certain number of epochs at the initial training stage until the loss is stable and then apply the adaptive weight method. To verify the effectiveness of such a gap strategy, we compare the complete TALL (with a full version of the adaptive weight module including both the gap mechanism and the loss change mechanism) to a variation of TALL with the adaptive weight module without the gap mechanism. The comparison is presented in Table 4 as LMoE+LC vs. TALL. We can observe that the gap mechanism does have a significant influence on the model performance that the model with the gap mechanism (TALL) delivers better utilities for all types of users than the model without the gap mechanism (LMoE+LC). Gap Mechanism Loss Change Mechanism. Last, we aim to verify the effectiveness of the proposed loss change mechanism in the adaptive weight module. The goal of loss change is to counter the scale diversity problem when applying the adaptive weight module. Here, we compare the complete TALL (with a full version of the adaptive weight module including both the gap mechanism and the loss change mechanism) to a variation of TALL (LMoE+gap+L) with the adaptive weight module that uses original loss as introduced in Section 3.2. From the comparison result shown in Table 4, we see that the model using loss change (TALL) performs better than the model using original loss for calculating weights (LMoE+gap+L). TALL outperforms LMoE+gap+L for all types of users. Loss Change Mechanism. In sum, by a series of comparative analyses, we show that the proposed adaptive loss-driven gate module, adaptive weight module, the gap mechanism in adaptive weight, and the loss change mechanism in adaptive weight are effective and play imperative roles in the proposed TALL framework. Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. Authors: Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Countering Mainstream Bias via End-to-End Adaptive Local Learning: Ablation Study

Table of Links Abstract and 1 Introduction 2 Preliminaries 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.3 Ablation Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 6 Conclusion, Acknowledgements, and References 4.2 Debiasing Performance First, we conduct a comparative analysis to show the effectiveness of the proposed TALL. In Table 2, we evaluate the overall NDCG@20 and average NDCG@20 for five user subgroups with varying mainstream levels for all methods and datasets. The best results of each metric and subgroup for all datasets are marked in bold, and the improvement rate of the proposed TALL over the best baseline MultVAE, LOCA, and LFT is exhibited as well. The user subgroups are categorized based on their mainstream scores. TALL vs. MultVAE & WL. First, we can observe that the utilities for users of all five subgroups are greatly promoted by our proposed TALL compared to the widely used model MultVAE. Moreover, we can see that although the global debiasing method WL can alleviate the mainstream bias to a certain degree compared to MultVAE, our proposed TALL can produce higher NDCG@20 for all five user subgroups than WL, depicting that the proposed TALL exhibits a more outstanding debiasing ability over WL. TALL vs. EnLFT & LOCA. Hence, we next have a fair comparison between models of the same complexity and compare directly across local learning methods. From Table 2, we observe that LOCA and EnLFT are more effective in mitigating the mainstream bias than WL, as they remarkably enhance utility across all five groups on all datasets. Meanwhile, our TALL significantly outperforms LOCA and EnLFT across all user groups and datasets. The improvement is especially prominent for niche users: TALL improves NDCG@20 of the ‘low’ user group by 6.07% on average over LOCA and 10% over EnLFT. This shows that with the same model complexity, the proposed end-to-end adaptive local learning model is more effective than heuristic-based local learning models. TALL vs. LFT. Last, we compare TALL with the state-of-the-art local learning baseline LFT, which is heavily computationally intense and time-consuming. But due to its special design that every user gets their own customized model trained by their local data, LFT can effectively address the discrepancy modeling problem. From Table 2, we observe that, for most of the time, LFT achieves the best performance for niche users among all baselines. And LFT can perform especially effectively for dense datasets (i.e., ML1M). In fact, given that the model complexity of LFT is much higher than TALL, it is unfair to compare them only based on recommendation accuracy (i.e., NDCG). For example, in the ML1M dataset, our TALL contains 100 expert models (MultVAE), while LFT trains 6,040 (#users) models separately, which is over 60 times larger than TALL. Although it is not a fair comparison, we can still observe in Table 2 that TALL can outperform LFT in most cases. Especially for the two sparse datasets Yelp and CDs&Vinyl, TALL produces significantly higher utilities for all types of users. This demonstrates the efficacy and necessity of an end-to-end local learning method compared to a heuristic-based one. In sum, from Table 2, we see that for all datasets, the proposed TALL produces the greatest NDCG@20 improvement for each subgroup of different mainstream levels and leads to the state-of-the-art overall model performance. TALL can outperform baselines with lower and the same model complexity, and it can even outperform the baseline model that is way more complex than it. Authors:
(1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA;
(2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA;
(3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA;
(4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA;
(5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3 End-to-End Adaptive Local Learning 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.2 Debiasing Performance 4.3 Ablation Study Ablation 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 5 Related Work 6 Conclusion, Acknowledgements, and References 6 Conclusion, Acknowledgements, and References 4.2 Debiasing Performance First, we conduct a comparative analysis to show the effectiveness of the proposed TALL. In Table 2, we evaluate the overall NDCG@20 and average NDCG@20 for five user subgroups with varying mainstream levels for all methods and datasets. The best results of each metric and subgroup for all datasets are marked in bold, and the improvement rate of the proposed TALL over the best baseline MultVAE, LOCA, and LFT is exhibited as well. The user subgroups are categorized based on their mainstream scores. TALL vs. MultVAE & WL. First, we can observe that the utilities for users of all five subgroups are greatly promoted by our proposed TALL compared to the widely used model MultVAE. Moreover, we can see that although the global debiasing method WL can alleviate the mainstream bias to a certain degree compared to MultVAE, our proposed TALL can produce higher NDCG@20 for all five user subgroups than WL, depicting that the proposed TALL exhibits a more outstanding debiasing ability over WL. TALL vs. MultVAE & WL. TALL vs. EnLFT & LOCA. Hence, we next have a fair comparison between models of the same complexity and compare directly across local learning methods. From Table 2, we observe that LOCA and EnLFT are more effective in mitigating the mainstream bias than WL, as they remarkably enhance utility across all five groups on all datasets. Meanwhile, our TALL significantly outperforms LOCA and EnLFT across all user groups and datasets. The improvement is especially prominent for niche users: TALL improves NDCG@20 of the ‘low’ user group by 6.07% on average over LOCA and 10% over EnLFT. This shows that with the same model complexity, the proposed end-to-end adaptive local learning model is more effective than heuristic-based local learning models. TALL vs. EnLFT & LOCA. TALL vs. LFT. Last, we compare TALL with the state-of-the-art local learning baseline LFT, which is heavily computationally intense and time-consuming. But due to its special design that every user gets their own customized model trained by their local data, LFT can effectively address the discrepancy modeling problem. From Table 2, we observe that, for most of the time, LFT achieves the best performance for niche users among all baselines. And LFT can perform especially effectively for dense datasets (i.e., ML1M). In fact, given that the model complexity of LFT is much higher than TALL, it is unfair to compare them only based on recommendation accuracy (i.e., NDCG). For example, in the ML1M dataset, our TALL contains 100 expert models (MultVAE), while LFT trains 6,040 (#users) models separately, which is over 60 times larger than TALL. Although it is not a fair comparison, we can still observe in Table 2 that TALL can outperform LFT in most cases. Especially for the two sparse datasets Yelp and CDs&Vinyl, TALL produces significantly higher utilities for all types of users. This demonstrates the efficacy and necessity of an end-to-end local learning method compared to a heuristic-based one. TALL vs. LFT. In sum, from Table 2, we see that for all datasets, the proposed TALL produces the greatest NDCG@20 improvement for each subgroup of different mainstream levels and leads to the state-of-the-art overall model performance. TALL can outperform baselines with lower and the same model complexity, and it can even outperform the baseline model that is way more complex than it. Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. Authors: Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Countering Mainstream Bias via End-to-End Adaptive Local Learning: Debiasing Performance

Table of Links Abstract and 1 Introduction 2 Preliminaries 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.3 Ablation Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 6 Conclusion, Acknowledgements, and References 4 Debiasing Experiments In this section, we present a comprehensive set of experiments to highlight the strong debiasing performance of the proposed method, validate the effectiveness of various model components, assess the impact of the proposed adaptive weight module, and examine the impact of hyperparameters. 4.1 Experimental Setup Considering the Rawlsian Max-Min fairness principle [29], the goal of debiasing is to promote the average NDCG@20 for subgroups with low mainstream scores while preserving or even improving the utility for subgroups with high mainstream scores at the same time. Hence, we also anticipate an increase in the overall NDCG@20 of the model. Baselines. In the experiments, we compare the proposed TALL with MultVAE and four state-of-art debiasing methods: (1) MultVAE [26] is the widely used vanilla recommendation model without debiasing. (2) WL [39] is a global method designed to assign more weights to niche users in the training loss of MultVAE. (3) LOCA [12] is a local learning model that trains multiple anchor models corresponding to identified anchor users and aggregates the outputs from anchor models based on the similarity between the target user and anchor users. (4) LFT [39] is the SOTA local learning model that first trains a global model with all data and then fine-tunes a customized local model for each user using their local data for the target user. (5) EnLFT [39] is the ensembled version of LFT, which is similar to LOCA but trains the anchor models by the approach of LFT. To fairly compare the performance of different baselines and our proposed model, for all four debiasing baselines and our proposed TALL, we adopt the MultVAE as the base model (or the expert model in TALL). LOCA, EnLFT, and TALL have the same complexity with a fixed number of MultVAE in them. And owing to the end-to-end training paradigm, TALL takes less training time than other local learning baselines. Last, LFT has the largest complexity, which has an independent MultVAE for each user. Reproducibility. All models are implemented in PyTorch and optimized by the Adam algorithm [20]. For the baseline MultVAE and the MultVAE component in other models, we set one hidden layer of size 100. And we maintain the number of local models at 100 for LOCA, EnLFT, and TALL for all datasets to ensure a fair comparison. All other hyper-parameters are grid searched by the validation sets. All code and data can be found at https:// github.com/ JP-25/ end-To-end-Adaptive-Local-Leanring-TALL-. Authors:
(1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA;
(2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA;
(3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA;
(4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA;
(5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3 End-to-End Adaptive Local Learning 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.2 Debiasing Performance 4.3 Ablation Study Ablation 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 5 Related Work 6 Conclusion, Acknowledgements, and References 6 Conclusion, Acknowledgements, and References 4 Debiasing Experiments In this section, we present a comprehensive set of experiments to highlight the strong debiasing performance of the proposed method, validate the effectiveness of various model components, assess the impact of the proposed adaptive weight module, and examine the impact of hyperparameters. 4.1 Experimental Setup Considering the Rawlsian Max-Min fairness principle [29], the goal of debiasing is to promote the average NDCG@20 for subgroups with low mainstream scores while preserving or even improving the utility for subgroups with high mainstream scores at the same time. Hence, we also anticipate an increase in the overall NDCG@20 of the model. the goal of debiasing is to promote the average NDCG@20 for subgroups with low mainstream scores while preserving or even improving the utility for subgroups with high mainstream scores at the same time. Hence, we also anticipate an increase in the overall NDCG@20 of the model. Baselines . In the experiments, we compare the proposed TALL with MultVAE and four state-of-art debiasing methods: (1) MultVAE [26] is the widely used vanilla recommendation model without debiasing. (2) WL [39] is a global method designed to assign more weights to niche users in the training loss of MultVAE. Baselines MultVAE WL (3) LOCA [12] is a local learning model that trains multiple anchor models corresponding to identified anchor users and aggregates the outputs from anchor models based on the similarity between the target user and anchor users. (4) LFT [39] is the SOTA local learning model that first trains a global model with all data and then fine-tunes a customized local model for each user using their local data for the target user. (5) EnLFT [39] is the ensembled version of LFT, which is similar to LOCA but trains the anchor models by the approach of LFT. LOCA LFT EnLFT To fairly compare the performance of different baselines and our proposed model, for all four debiasing baselines and our proposed TALL, we adopt the MultVAE as the base model (or the expert model in TALL). LOCA, EnLFT, and TALL have the same complexity with a fixed number of MultVAE in them. And owing to the end-to-end training paradigm, TALL takes less training time than other local learning baselines. Last, LFT has the largest complexity, which has an independent MultVAE for each user. Reproducibility . All models are implemented in PyTorch and optimized by the Adam algorithm [20]. For the baseline MultVAE and the MultVAE component in other models, we set one hidden layer of size 100. And we maintain the number of local models at 100 for LOCA, EnLFT, and TALL for all datasets to ensure a fair comparison. All other hyper-parameters are grid searched by the validation sets. All code and data can be found at https:// github.com/ JP-25/ end-To-end-Adaptive-Local-Leanring-TALL-. Reproducibility Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. Authors: Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Countering Mainstream Bias via End-to-End Adaptive Local Learning: Debiasing Experiments and Setup

Table of Links Abstract and 1 Introduction 2 Preliminaries 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.3 Ablation Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 6 Conclusion, Acknowledgements, and References 3.2 Synchronized Learning via Adaptive Weight After addressing the discrepancy modeling problem, another core cause of the mainstream bias is the unsynchronized learning problem – the learning difficulties vary for different users, and users reach the performance peak during training at different speeds (refer to Figure 1 for an example). Thus, a method to synchronize the learning paces of users is desired. We devise an adaptive weight approach to achieve learning synchronization so that the dilemma of performance trade-off between mainstream users and niche users can be overcome. Adaptive Weight. The fundamental motivation of the proposed method lies in linking the learning status of a user to the loss function of the user at the current epoch. A high loss for a user signifies ineffective learning by the model, necessitating more epochs for accurate predictions. Conversely, a low loss indicates successful modeling, requiring less or even no further training. Moreover, we can use a weight in the loss function to control the learning pace for the user: a small weight induces slow updating, and a large weight incurs fast updating. Hence, we propose to synchronize different users by applying weights to the objective function based on losses users get currently – a user with a high loss should receive a large weight, and vice versa. We aim to achieve this intuition by solving the following optimization problem: Loss Change Mechanism and Gap Mechanism. With the proposed adaptive weight approach, we take a solid step toward learning synchronization. However, two critical issues remain unaddressed. First, the scale of the loss function is innately different across different users. Usually, mainstream users possess a lower loss value than niche users because the algorithm can achieve better utility for mainstream users who are easier to model. Due to this scale diversity problem, computing weights by exact values of the loss can lead to the undesired situation that mainstream users get overly low weights and niche users get overly high weights, disturbing the learning process. On the other hand, the loss is not always stable, especially at the early stage of training. This unstable loss problem can deteriorate the efficacy of the problem adaptive weight method too. At last, since the losses are excessively unstable at the initial stage of training and the proposed adaptive weight module heavily relies on the stability of the loss value, the proposed method cannot perform well at the initial stage of training. Hence, we propose to have a gap at the beginning for our adaptive weight method. That is, we do not apply the proposed adaptive weight method to the framework at the first T epochs. Since at the early stage of the training, all users will be at a fast learning status (consider the first 50 epochs in Figure 1), as there is no demand for learning synchronization. After T epochs of ordinary training, when the learning procedure is more stable and the loss is more reliable, we apply the adaptive weight method to synchronize the learning for different users. And this time synchronization is desired and plays an important role. The gap window T is a hyper-parameter and needs to be predefined. Authors:
(1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA;
(2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA;
(3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA;
(4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA;
(5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3 End-to-End Adaptive Local Learning 3 End-to-End Adaptive Local Learning 3.1 Loss-Driven Mixture-of-Experts 3.1 Loss-Driven Mixture-of-Experts 3.2 Synchronized Learning via Adaptive Weight 3.2 Synchronized Learning via Adaptive Weight 4 Debiasing Experiments and 4.1 Experimental Setup 4 Debiasing Experiments and 4.1 Experimental Setup 4.2 Debiasing Performance 4.2 Debiasing Performance 4.3 Ablation Study Ablation 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 4.4 Effect of the Adaptive Weight Module and 4.5 Hyper-parameter Study 5 Related Work 5 Related Work 6 Conclusion, Acknowledgements, and References 6 Conclusion, Acknowledgements, and References 3.2 Synchronized Learning via Adaptive Weight After addressing the discrepancy modeling problem, another core cause of the mainstream bias is the unsynchronized learning problem – the learning difficulties vary for different users, and users reach the performance peak during training at different speeds (refer to Figure 1 for an example). Thus, a method to synchronize the learning paces of users is desired. We devise an adaptive weight approach to achieve learning synchronization so that the dilemma of performance trade-off between mainstream users and niche users can be overcome. Adaptive Weight. The fundamental motivation of the proposed method lies in linking the learning status of a user to the loss function of the user at the current epoch. A high loss for a user signifies ineffective learning by the model, necessitating more epochs for accurate predictions. Conversely, a low loss indicates successful modeling, requiring less or even no further training. Moreover, we can use a weight in the loss function to control the learning pace for the user: a small weight induces slow updating, and a large weight incurs fast updating. Hence, we propose to synchronize different users by applying weights to the objective function based on losses users get currently – a user with a high loss should receive a large weight, and vice versa. We aim to achieve this intuition by solving the following optimization problem: Adaptive Weight. Loss Change Mechanism and Gap Mechanism. With the proposed adaptive weight approach, we take a solid step toward learning synchronization. However, two critical issues remain unaddressed. First, the scale of the loss function is innately different across different users. Usually, mainstream users possess a lower loss value than niche users because the algorithm can achieve better utility for mainstream users who are easier to model. Due to this scale diversity problem , computing weights by exact values of the loss can lead to the undesired situation that mainstream users get overly low weights and niche users get overly high weights, disturbing the learning process. On the other hand, the loss is not always stable, especially at the early stage of training. This unstable loss problem can deteriorate the efficacy of the problem adaptive weight method too. Loss Change Mechanism and Gap Mechanism. scale diversity problem unstable loss problem At last, since the losses are excessively unstable at the initial stage of training and the proposed adaptive weight module heavily relies on the stability of the loss value, the proposed method cannot perform well at the initial stage of training. Hence, we propose to have a gap at the beginning for our adaptive weight method. That is, we do not apply the proposed adaptive weight method to the framework at the first T epochs. Since at the early stage of the training, all users will be at a fast learning status (consider the first 50 epochs in Figure 1), as there is no demand for learning synchronization. After T epochs of ordinary training, when the learning procedure is more stable and the loss is more reliable, we apply the adaptive weight method to synchronize the learning for different users. And this time synchronization is desired and plays an important role. The gap window T is a hyper-parameter and needs to be predefined. Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. Authors: Authors: (1) Jinhao Pan [0009 −0006 −1574 −6376], Texas A&M University, College Station, TX, USA; (2) Ziwei Zhu [0000 −0002 −3990 −4774], George Mason University, Fairfax, VA, USA; (3) Jianling Wang [0000 −0001 −9916 −0976], Texas A&M University, College Station, TX, USA; (4) Allen Lin [0000 −0003 −0980 −4323], Texas A&M University, College Station, TX, USA; (5) James Caverlee [0000 −0001 −8350 −8528]. Texas A&M University, College Station, TX, USA. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Stories, News & Mentions about MOONSTREAM

Moonstream’s Stories on HackerNoon