Moment

moment

Telegram started as a messaging service, but it has transformed into a hub of mini-games that has made entertainment faster and more reachable for the users of this application. While this milestone was reached in record time, it has continued to face user retention challenges among others. This article dwells on the major challenges that are preventing Telegram-based mini games from enjoying long-term active user engagement and how incentive-based gamification as a retention strategy is key to sustaining users’ interest. User Engagement and Retention One of the biggest challenges facing Telegram mini-games is offering lifetime value. Mini-games on the platform are mostly designed for casual use in short sessions, after which a "novelty curve" very quickly sets in. Users engaging with these kinds of games to always have quick entertainment often move on afterward once they've completed a few sessions. Unlike most mobile or console games, Telegram mini-games don't usually have depth, long-term goals, and complicated features like immersive graphics or sound. To address this, developers can add gamification elements that foster continuous participation. It includes daily challenges, achievements, and leaderboards, which raise users' interest in returning regularly to the game. Multiplayer features, which capitalize on Telegram's strength as a social platform, will also help gain more engagement by fostering competition and cooperation among users. Social interactions, like team-based competitions or friendly challenges, are ways to extend the life cycle of such games. Otherwise, only regular updating and the addition of new content will keep the users interested and the experience relevant. Game Discoverability Its biggest drawback is that Telegram does not have one marketplace or mechanism of discovery whereby users can discover new games without word-of-mouth or specific gaming channels. This really makes discovery hard, so even extremely well-designed games just don't have a big reach. Mini-games come across as basically invisible in an ecosystem with thousands of bots and services. To overcome this, the developers should make use of partnerships with popular channels or groups to promote their games on Telegram. The creation of more effective collaborations with influential or famous communities within the app will raise awareness. Additionally, Telegram can enhance its gaming ecosystem by providing a game discovery system where users easily navigate to trending or recommended games, similar to the store in which applications are sold. This will allow users to find new games more easily and, therefore, spend more time with the service, which gives developers a wider reach. Competition from Alternative Platforms The competition is also high from the feature-rich rivals: mobile app stores, including iOS/Android, and social media networks, such as Facebook and Discord. The games are more complex and have better graphics and sound, and depth in the mechanics; hence, they can overshadow the simpler games on Telegram. In general, mobile games do let one get into an immersive experience where only advanced gameplay can be played. This is highly limited in the telegram mini-games owing to the limitation of the platform. In this regard, Telegram can win for itself a niche of improving the experience of online game playing. It can embed games directly into group chats and channels to give users an opportunity to create new forms of communicative and entertaining activity. That would allow casual gaming right from within the user's existing flow of social interactions, making them more frequent. Also, adding casual multiplayer gameplay, where people can actually play together with their friends in real-time, might position Telegram strongly alongside most of the big players. Novelty Fatigue and Replayability Most Telegram mini-games are facing the problem of novelty fatigue: users get bored in a few playthroughs. Both these games are designed for short, casual sessions; each will allow limited replay value. Once users have completed the game or mastered its mechanics, there's little reason to return, especially in cases when a game doesn't update with new content. However, most of the Telegram mini-games are extremely simple, and they also lack other complex features such as high-quality graphics or sound, which complicates things. Novelty fatigue can be compensated for by adding more complexity and variety as more time has passed. Seasonal events, daily or weekly challenges, and new game modes or levels add freshness to the way one plays. Competitive elements, such as leaderboards or user-given content as custom levels or challenges, increase replayability. Updates added periodically by developers enable continued resetting of new objectives for players, helping maintain longer-term interests. Integration of Blockchain and Decentralized Features The rising Blockchain technology gives Telegram room to extend its mini-game ecosystem. Introducing the platform's proprietary blockchain, TON, may dramatically change the perspective in terms of gaming on the platform. TON can allow fast and secure in-game transactions, opening different revenue streams and building a decentralized game economy. Therefore, developers can use rewards in tangible form by introducing blockchain elements like NFT and in-game tokens to keep users engaged for a long period of time. One could visualize players earning, trading, or even staking tokens to give them real value and incentivize them to continue contributing. It is a whole decentralized gaming approach, with an additional investment layering in the users themselves, who will not only engage in gameplay but also earn ownership of in-game assets. Yet these features have been rather slow to be adopted since users may be leery about engaging with the blockchain features of the site due to unfamiliarity or simply technical bottlenecks. For the developers, the integration process should be smoothed out to make blockchain features in games more accessible to users. Besides, users should be educated about the benefits of gaming on a blockchain, and seamless onboarding processes provided for better adoption. In a continuously maturing Telegram mini-game ecosystem, blockchain technology may give just what is needed by maintaining user interest through unique and rich gaming experiences. Case Study: Telegram-based Mini-Game App TapShot The maker of StrayShot, a third person shooter game on mobile, PC and consoles, recently launched its Telegram-based clicker game, TapShot, a fun idle game that allows users to upgrade weapons, rank up, and unlock new hunters--as a long-term user retention strategy to counter the prevalence of crypto airdrops and its short-term effect. This approach could enhance the retention strategies that other Telegram mini-games have explored. With a more meaningful gameplay experience and a clear link to a larger game, TapShot could keep players engaged for the long haul. It is a smart marketing strategy that not only builds an audience for StrayShot, but also tackles the challenge of user drop-off seen in some other crypto games. Additionally, TapShot’s user retention strategy introduces a more targeted approach to airdrop rewards. Since the game encourages real gameplay and engagement, it’s likely that airdrops will be connected to actual progress within the game, making rewards larger and more valuable for dedicated players in the long-term. Conclusion The momentum of Telegram mini-games should be regarded as multi-factorial, and along the way, more creativity and fitting are required. Developers should be worried about enriching user engagement by adding more gamification, making it more discoverable, and somehow distinguishing the platform from other competitors. Introducing blockchain and going decentralized opens a wide door to possibilities for making games even more engaging and rewarding. These challenges being addressed, Telegram will be able to further develop its mini-game ecosystem and keep users interested in the long term. Telegram started as a messaging service, but it has transformed into a hub of mini-games that has made entertainment faster and more reachable for the users of this application. While this milestone was reached in record time, it has continued to face user retention challenges among others. This article dwells on the major challenges that are preventing Telegram-based mini games from enjoying long-term active user engagement and how incentive-based gamification as a retention strategy is key to sustaining users’ interest. User Engagement and Retention One of the biggest challenges facing Telegram mini-games is offering lifetime value. Mini-games on the platform are mostly designed for casual use in short sessions, after which a "novelty curve" very quickly sets in. Users engaging with these kinds of games to always have quick entertainment often move on afterward once they've completed a few sessions. Unlike most mobile or console games, Telegram mini-games don't usually have depth, long-term goals, and complicated features like immersive graphics or sound. To address this, developers can add gamification elements that foster continuous participation. It includes daily challenges, achievements, and leaderboards, which raise users' interest in returning regularly to the game. Multiplayer features, which capitalize on Telegram's strength as a social platform, will also help gain more engagement by fostering competition and cooperation among users. Social interactions, like team-based competitions or friendly challenges, are ways to extend the life cycle of such games. Otherwise, only regular updating and the addition of new content will keep the users interested and the experience relevant. Game Discoverability Its biggest drawback is that Telegram does not have one marketplace or mechanism of discovery whereby users can discover new games without word-of-mouth or specific gaming channels. This really makes discovery hard, so even extremely well-designed games just don't have a big reach. Mini-games come across as basically invisible in an ecosystem with thousands of bots and services. To overcome this, the developers should make use of partnerships with popular channels or groups to promote their games on Telegram. The creation of more effective collaborations with influential or famous communities within the app will raise awareness. Additionally, Telegram can enhance its gaming ecosystem by providing a game discovery system where users easily navigate to trending or recommended games, similar to the store in which applications are sold. This will allow users to find new games more easily and, therefore, spend more time with the service, which gives developers a wider reach. game discovery system game discovery system Competition from Alternative Platforms The competition is also high from the feature-rich rivals: mobile app stores, including iOS/Android, and social media networks, such as Facebook and Discord. The games are more complex and have better graphics and sound, and depth in the mechanics; hence, they can overshadow the simpler games on Telegram. In general, mobile games do let one get into an immersive experience where only advanced gameplay can be played. This is highly limited in the telegram mini-games owing to the limitation of the platform. In this regard, Telegram can win for itself a niche of improving the experience of online game playing. It can embed games directly into group chats and channels to give users an opportunity to create new forms of communicative and entertaining activity. can embed games directly into group can embed games directly into group That would allow casual gaming right from within the user's existing flow of social interactions, making them more frequent. Also, adding casual multiplayer gameplay, where people can actually play together with their friends in real-time, might position Telegram strongly alongside most of the big players. Novelty Fatigue and Replayability Most Telegram mini-games are facing the problem of novelty fatigue: users get bored in a few playthroughs. Both these games are designed for short, casual sessions; each will allow limited replay value. Once users have completed the game or mastered its mechanics, there's little reason to return, especially in cases when a game doesn't update with new content. However, most of the Telegram mini-games are extremely simple, and they also lack other complex features such as high-quality graphics or sound, which complicates things. Novelty fatigue can be compensated for by adding more complexity and variety as more time has passed. Seasonal events, daily or weekly challenges, and new game modes or levels add freshness to the way one plays. Competitive elements, such as leaderboards or user-given content as custom levels or challenges, increase replayability. Updates added periodically by developers enable continued resetting of new objectives for players, helping maintain longer-term interests. Integration of Blockchain and Decentralized Features The rising Blockchain technology gives Telegram room to extend its mini-game ecosystem. Introducing the platform's proprietary blockchain, TON, may dramatically change the perspective in terms of gaming on the platform. TON can allow fast and secure in-game transactions, opening different revenue streams and building a decentralized game economy. Therefore, developers can use rewards in tangible form by introducing blockchain elements like NFT and in-game tokens to keep users engaged for a long period of time. One could visualize players earning, trading, or even staking tokens to give them real value and incentivize them to continue contributing. It is a whole decentralized gaming approach, with an additional investment layering in the users themselves, who will not only engage in gameplay but also earn ownership of in-game assets. Yet these features have been rather slow to be adopted since users may be leery about engaging with the blockchain features of the site due to unfamiliarity or simply technical bottlenecks. For the developers, the integration process should be smoothed out to make blockchain features in games more accessible to users. Besides, users should be educated about the benefits of gaming on a blockchain, and seamless onboarding processes provided for better adoption. In a continuously maturing Telegram mini-game ecosystem, blockchain technology may give just what is needed by maintaining user interest through unique and rich gaming experiences. Case Study: Telegram-based Mini-Game App TapShot The maker of StrayShot, a third person shooter game on mobile, PC and consoles, recently launched its Telegram-based clicker game, TapShot, a fun idle game that allows users to upgrade weapons, rank up, and unlock new hunters--as a long-term user retention strategy to counter the prevalence of crypto airdrops and its short-term effect. This approach could enhance the retention strategies that other Telegram mini-games have explored. With a more meaningful gameplay experience and a clear link to a larger game, TapShot could keep players engaged for the long haul. It is a smart marketing strategy that not only builds an audience for StrayShot, but also tackles the challenge of user drop-off seen in some other crypto games. Additionally, TapShot’s user retention strategy introduces a more targeted approach to airdrop rewards. Since the game encourages real gameplay and engagement, it’s likely that airdrops will be connected to actual progress within the game, making rewards larger and more valuable for dedicated players in the long-term. Conclusion The momentum of Telegram mini-games should be regarded as multi-factorial, and along the way, more creativity and fitting are required. Developers should be worried about enriching user engagement by adding more gamification, making it more discoverable, and somehow distinguishing the platform from other competitors. Introducing blockchain and going decentralized opens a wide door to possibilities for making games even more engaging and rewarding. These challenges being addressed, Telegram will be able to further develop its mini-game ecosystem and keep users interested in the long term.

The Challenge Of Sustaining Mini-Game Momentum On Telegram

Exploring "Clarity Windows" in AI: The Unpredictable Moments of Perceived Consciousness

Εξερευνώντας τα "Clarity Windows" στο AI: Οι απρόβλεπτες στιγμές της αντιληπτής συνείδησης

Explorando las "ventanas de claridad" en la IA: los momentos impredecibles de la conciencia percibida

AI-n "Argitasun-leihoak" arakatzea: hautematen den kontzientziaren une ezustekoak

Exploration des « fenêtres de clarté » en IA : les moments imprévisibles de la conscience perçue

Istraživanje "prozora jasnoće" u umjetnoj inteligenciji: nepredvidivi trenuci percipirane svijesti

AI における「明瞭性の窓」を探る: 知覚された意識の予測不可能な瞬間

Истражување на „Clarity Windows“ во вештачката интелигенција: непредвидливите моменти на перцепирана свест

په AI کې د "وضاحت وینډوز" سپړنه: د پوه شوي شعور غیر متوقع شیبې

AI nisqapi "Clarity Windows" nisqamanta maskay: Conciencia Percibida nisqamanta mana yachay atina pachakuna

Eksplorimi i "Dritareve të qartësisë" në AI: Momentet e paparashikueshme të ndërgjegjes së perceptuar

Истраживање „прозора јасноће“ у АИ: непредвидиви тренуци перципиране свести

AI இல் "தெளிவு விண்டோஸ்" ஆய்வு: உணரப்பட்ட நனவின் கணிக்க முடியாத தருணங்கள்

Beacon, a leading web3 accelerator founded by Polygon co-founder Sandeep Nailwal, has unveiled its fourth and largest cohort to date. The program has selected 17 promising startups for its latest three-month accelerator, marking a significant expansion from previous cohorts. The new group includes some of the most high-profile companies in Beacon's history, with several already backed by prominent venture capital firms such as Andreessen Horowitz (a16z), Founders Fund, Pantera Capital, and Polychain Capital. The startups span various sectors within the web3 space, including decentralized finance (DeFi), infrastructure, and artificial intelligence (AI). “We're thrilled to welcome our largest and most diverse cohort so far,” said Sandeep Nailwal, co-founder of Polygon and Beacon. “The caliber of companies joining us is truly exceptional, and we're excited to see how, with the aid of our mentorship and resources, they will shape the future of web3 and beyond.” Notable Participants and Their Innovations Among the standout companies in the cohort are: Stakestone: An omni-chain liquid staking token protocol enhancing staking yields and liquidity on Layer 2s.


EXO: Enables users to run their own AI clusters at home using everyday devices.


Nubit: A scalable, cost-efficient data availability layer secured by Bitcoin, empowering applications like Ordinals and Layer 2s.


Aligned Layer: A decentralized network that verifies zero-knowledge proofs and posts the results to Ethereum.


eoracle: A programmable data layer that connects smart contracts with real-world data by extending Ethereum's Proof of Stake. Building on Past Successes The announcement comes after the graduation of Beacon's successful S23 cohort, which comprised 10 companies. Alumni such as scalable data availability layer 0G and verifiable oracle protocol ORA have achieved unicorn status since completing the program. ORA secured $20 million in funding led by Polychain Capital, while Inco, a universal confidentiality layer, closed a $4.5 million seed round led by 1kx. “With the level of talent and innovation we're witnessing in this new group, we are entirely confident that they can emulate, if not surpass, their predecessors,” Nailwal added. The Full Cohort Lineup The complete list of companies in Beacon's S24 cohort includes: Avantis: Provides infrastructure for traders to access up to 100x leverage on major cryptocurrencies, forex, and commodities.


Compute Labs: Utilizes a Compute Tokenization Protocol to financialize AI and create compute derivatives.


Crynux: Founded by former Google engineers, building a permissionless and trustless protocol for decentralized AI.


EZKL: Offers a developer-friendly system for verifiable AI and analytics, powering various decentralized systems.


Lorenzo: A Bitcoin liquidity finance layer designed to meet global demand for Bitcoin liquidity across Layer 2s and DeFi platforms.


Nodekit: Developing Javelin, a solution aimed at restoring composability to rollups.


Othentic: Provides infrastructure for building distributed services, simplifying development on EigenLayer.


PADO: A decentralized zkAttestation and computation network aiming to unlock internet data for Web3.


Pallet: A hub for digital culture, community, and ownership built on the Sei network.


Pin AI: An open platform for personal AI, leveraging open-source technology and crypto for data ownership and privacy.


TAC: TON App Chain, an interoperability layer for the TON blockchain and EVM-based applications.


Warlock: Creating new primitives to disrupt and disintermediate the MEV (Miner Extractable Value) landscape. Beacon's decision to expand its cohort size may be a strategic move to capitalize on the growing momentum in the web3 sector. By supporting a larger and more diverse group of startups, the accelerator is positioning itself as a key player in shaping the future of decentralized technologies. The inclusion of companies backed by major venture capital firms also adds credibility to the program and could attract more high-quality applicants in the future. This move could foster a more collaborative environment where startups benefit not only from Beacon's resources but also from each other's experiences and networks. Don’t forget to like and share the story! Vested Interest Disclosure: This author is an independent contributor publishing via our business blogging program. HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR Beacon , a leading web3 accelerator founded by Polygon co-founder Sandeep Nailwal, has unveiled its fourth and largest cohort to date. The program has selected 17 promising startups for its latest three-month accelerator, marking a significant expansion from previous cohorts. Beacon The new group includes some of the most high-profile companies in Beacon's history, with several already backed by prominent venture capital firms such as Andreessen Horowitz (a16z), Founders Fund, Pantera Capital, and Polychain Capital. The startups span various sectors within the web3 space, including decentralized finance (DeFi), infrastructure, and artificial intelligence (AI). “We're thrilled to welcome our largest and most diverse cohort so far,” said Sandeep Nailwal, co-founder of Polygon and Beacon. “The caliber of companies joining us is truly exceptional, and we're excited to see how, with the aid of our mentorship and resources, they will shape the future of web3 and beyond.” Notable Participants and Their Innovations Notable Participants and Their Innovations Among the standout companies in the cohort are: Stakestone: An omni-chain liquid staking token protocol enhancing staking yields and liquidity on Layer 2s. EXO: Enables users to run their own AI clusters at home using everyday devices. Nubit: A scalable, cost-efficient data availability layer secured by Bitcoin, empowering applications like Ordinals and Layer 2s. Aligned Layer: A decentralized network that verifies zero-knowledge proofs and posts the results to Ethereum. eoracle: A programmable data layer that connects smart contracts with real-world data by extending Ethereum's Proof of Stake. Stakestone: An omni-chain liquid staking token protocol enhancing staking yields and liquidity on Layer 2s. Stakestone : An omni-chain liquid staking token protocol enhancing staking yields and liquidity on Layer 2s. Stakestone EXO: Enables users to run their own AI clusters at home using everyday devices. EXO : Enables users to run their own AI clusters at home using everyday devices. EXO Nubit: A scalable, cost-efficient data availability layer secured by Bitcoin, empowering applications like Ordinals and Layer 2s. Nubit : A scalable, cost-efficient data availability layer secured by Bitcoin, empowering applications like Ordinals and Layer 2s. Nubit Aligned Layer: A decentralized network that verifies zero-knowledge proofs and posts the results to Ethereum. Aligned Layer : A decentralized network that verifies zero-knowledge proofs and posts the results to Ethereum. Aligned Layer eoracle: A programmable data layer that connects smart contracts with real-world data by extending Ethereum's Proof of Stake. eoracle : A programmable data layer that connects smart contracts with real-world data by extending Ethereum's Proof of Stake. eoracle Building on Past Successes Building on Past Successes The announcement comes after the graduation of Beacon's successful S23 cohort, which comprised 10 companies. Alumni such as scalable data availability layer 0G and verifiable oracle protocol ORA have achieved unicorn status since completing the program. ORA secured $20 million in funding led by Polychain Capital, while Inco, a universal confidentiality layer, closed a $4.5 million seed round led by 1kx. “With the level of talent and innovation we're witnessing in this new group, we are entirely confident that they can emulate, if not surpass, their predecessors,” Nailwal added. The Full Cohort Lineup The Full Cohort Lineup The complete list of companies in Beacon's S24 cohort includes: Avantis: Provides infrastructure for traders to access up to 100x leverage on major cryptocurrencies, forex, and commodities. Compute Labs: Utilizes a Compute Tokenization Protocol to financialize AI and create compute derivatives. Crynux: Founded by former Google engineers, building a permissionless and trustless protocol for decentralized AI. EZKL: Offers a developer-friendly system for verifiable AI and analytics, powering various decentralized systems. Lorenzo: A Bitcoin liquidity finance layer designed to meet global demand for Bitcoin liquidity across Layer 2s and DeFi platforms. Nodekit: Developing Javelin, a solution aimed at restoring composability to rollups. Othentic: Provides infrastructure for building distributed services, simplifying development on EigenLayer. PADO: A decentralized zkAttestation and computation network aiming to unlock internet data for Web3. Pallet: A hub for digital culture, community, and ownership built on the Sei network. Pin AI: An open platform for personal AI, leveraging open-source technology and crypto for data ownership and privacy. TAC: TON App Chain, an interoperability layer for the TON blockchain and EVM-based applications. Warlock: Creating new primitives to disrupt and disintermediate the MEV (Miner Extractable Value) landscape. Avantis: Provides infrastructure for traders to access up to 100x leverage on major cryptocurrencies, forex, and commodities. Avantis : Provides infrastructure for traders to access up to 100x leverage on major cryptocurrencies, forex, and commodities. Avantis Compute Labs: Utilizes a Compute Tokenization Protocol to financialize AI and create compute derivatives. Compute Labs : Utilizes a Compute Tokenization Protocol to financialize AI and create compute derivatives. Compute Labs Crynux: Founded by former Google engineers, building a permissionless and trustless protocol for decentralized AI. Crynux : Founded by former Google engineers, building a permissionless and trustless protocol for decentralized AI. Crynux EZKL: Offers a developer-friendly system for verifiable AI and analytics, powering various decentralized systems. EZKL : Offers a developer-friendly system for verifiable AI and analytics, powering various decentralized systems. EZKL Lorenzo: A Bitcoin liquidity finance layer designed to meet global demand for Bitcoin liquidity across Layer 2s and DeFi platforms. Lorenzo : A Bitcoin liquidity finance layer designed to meet global demand for Bitcoin liquidity across Layer 2s and DeFi platforms. Lorenzo Nodekit: Developing Javelin, a solution aimed at restoring composability to rollups. Nodekit : Developing Javelin, a solution aimed at restoring composability to rollups. Nodekit Othentic: Provides infrastructure for building distributed services, simplifying development on EigenLayer. Othentic : Provides infrastructure for building distributed services, simplifying development on EigenLayer. Othentic PADO: A decentralized zkAttestation and computation network aiming to unlock internet data for Web3. PADO : A decentralized zkAttestation and computation network aiming to unlock internet data for Web3. PADO Pallet: A hub for digital culture, community, and ownership built on the Sei network. Pallet : A hub for digital culture, community, and ownership built on the Sei network. Pallet Pin AI: An open platform for personal AI, leveraging open-source technology and crypto for data ownership and privacy. Pin AI : An open platform for personal AI, leveraging open-source technology and crypto for data ownership and privacy. Pin AI TAC: TON App Chain, an interoperability layer for the TON blockchain and EVM-based applications. TAC : TON App Chain, an interoperability layer for the TON blockchain and EVM-based applications. TAC Warlock: Creating new primitives to disrupt and disintermediate the MEV (Miner Extractable Value) landscape. Warlock : Creating new primitives to disrupt and disintermediate the MEV (Miner Extractable Value) landscape. Warlock Beacon's decision to expand its cohort size may be a strategic move to capitalize on the growing momentum in the web3 sector. By supporting a larger and more diverse group of startups, the accelerator is positioning itself as a key player in shaping the future of decentralized technologies. The inclusion of companies backed by major venture capital firms also adds credibility to the program and could attract more high-quality applicants in the future. This move could foster a more collaborative environment where startups benefit not only from Beacon's resources but also from each other's experiences and networks. Don’t forget to like and share the story! Vested Interest Disclosure: This author is an independent contributor publishing via our business blogging program. HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR Vested Interest Disclosure: This author is an independent contributor publishing via our business blogging program . HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR Vested Interest Disclosure: business blogging program business blogging program

Beacon's Largest Cohort Signals Growing Momentum in Web3 Innovation

Η μεγαλύτερη κοόρτη της Beacon σηματοδοτεί αυξανόμενη ορμή στην καινοτομία Web3

La cohorte más grande de Beacon indica un impulso creciente en la innovación de la Web3

Kelompok Terbesar Beacon Menandakan Meningkatnya Momentum Inovasi Web3

ビーコンの最大規模のコホートがWeb3イノベーションの勢いの高まりを示唆

សញ្ញានៃក្រុមដ៏ធំបំផុតរបស់ Beacon ដែលកំពុងកើនឡើងនៅក្នុងការច្នៃប្រឌិត Web3

बीकनको सबैभन्दा ठूलो कोहोर्ट संकेतहरू Web3 नवाचारमा बढ्दो गति

Cea mai mare cohortă a lui Beacon semnalează creșterea impulsului în inovarea Web3

Крупнейшая когорта Beacon сигнализирует о растущем импульсе инноваций Web3

Беацон-ова највећа кохорта сигнали растући замах у Веб3 иновацијама

Web3 இன்னோவேஷனில் பெக்கனின் மிகப்பெரிய கோஹார்ட் சிக்னல்கள் வளரும் வேகம்

“Beacon” -yň “Web3” innowasiýasynda ösýän pursatdygyny görkezýär

Mayoqning eng katta kohort signallari Web3 innovatsiyasida o'sib borayotgan momentum

Robinhood has made waves in the casual investment community for being a commission-free, mobile-friendly app that’s made trading accessible to everyone — not just the wealthy or experienced traders. It’s only natural, then, that a similar solution be made available for would-be crypto investors — and it’s vital that this solution is made specifically for the casual investor. After all, even with more people trading in decentralized digital currencies than ever before, the majority of Americans still don’t have a strong grasp of crypto concepts. This can prevent new and inexperienced traders from integrating cryptocurrency into their everyday transactions and investments, and it risks concentrating knowledge and trading power in the hands of a few individuals or groups. And sure, there are solutions like Coinbase and Kraken available, but these platforms are still bogged down by unnecessary features and confusing terminologies. This is where Bolide comes in. This easy-to-use mobile trading app aims to simplify crypto investments (specifically memecoins) for everyone, even those without any prior crypto experience. Learn more about the common barriers of crypto investing, how Bolide is breaking these down, and how Bolide CEO Ox Bid is giving memecoins their Robinhood moment — pushing to make social, on-chain memecoin trading easy and fun for everyone. Memecoin Trading: Navigating the Wild West of Crypto Memecoin trading is the buying and selling of digital currencies that are often inspired by internet jokes or cultural memes. However, unlike more established cryptocurrencies like Bitcoin or Ethereum, memecoins are highly volatile and lack additional built-in uses, making them a risky investment. The value of memecoins typically stems from viral marketing campaigns, speculation, and hype. While this adds to the risky charm of memecoins, it also makes it challenging for newcomers to distinguish between legitimate investments and fleeting trends. Additionally, the market's lack of regulation makes it ripe for scams, where promoters might inflate a memecoin's value and sell off their holdings at a peak, leaving others with devalued assets. The cryptocurrency market itself is also volatile. For instance, during the crypto winter of 2022, the market saw a major downturn, resulting in a $2 trillion loss in market value and impacting many investors. Such slumps can be triggered by withdrawing from large investors, significant frauds or security breaches, or tighter government regulations. Finally, navigating crypto trading platforms also requires a high level of digital and financial literacy. In order to maintain their investment and stay safe, beginners need to invest time into learning which risks avoiding and deciphering which coins are worth investing in — all while managing complex trading interfaces. Due to these barriers, a lot of investors forgo crypto trading altogether. Bolide: Memecoin Trading for the Masses Bolide is a mobile app that makes it easy and affordable for beginners to start trading memecoins. It uses Pimlico’s Account Abstraction technology to allow gasless transactions and embedded wallets from Privy for social logins. In doing so, it gives users affordable transactions, secure account access, and lets them store their customer keys safely. This way, users can manage their investments with ease and confidence, knowing their data and digital assets are well-protected. And thanks to a simplistic user interface, users can navigate Bolide as easily as they would a standard bank account — even if they’ve never used the on-chain market before “As an early investor and trader in crypto,” says Ox Bid, CEO and co-founder of Bolide, “my dream has always been to provide access and teach people how to navigate this space. The crypto market is undoubtedly risky, but it can be less so if customers have access to straightforward tools, as complexity often leads to mistakes and barriers.” Bolide is now available on the App Store, and its initial traction is notable. “Though we've been on the market for only a few weeks,” Ox says, “our trading volume has nearly reached $157,000, with an average of 13 trades per user and an average trade value of $30.” Clearly, Ox’s strategy of appealing to a lay audience of retail investors is paying off. Bolide’s Social Finance Future According to Ox Bid, it's time for memecoins to have their Robinhood moment. To this end, she believes that the secret to making memecoin trading accessible to everyone is to take Robinhood's affordable, user-friendly approach, apply it to crypto, and add a social twist. She notes that modern consumers are deeply influenced by social media in everything they do, from choosing where to eat to deciding where to live — and she sees this trend as vital to the future of retail crypto trading. “People often make decisions based on recommendations from influencers or friends,” Ox explains. “My goal is to create a similar influence-driven recommendation system, where users can see which tokens to buy based on the on-chain actions of top influencers, friends, and traders.” In doing so, Ox hopes to return to what made memecoins great in the first place: fun and levity. “I aim to make the crypto market not only accessible to the masses but also exciting and social,” she says. Trade Memecoins with Confidence Using Bolide Newcomers should be able to confidently trade crypto without having to sift through pages of research. They should feel at ease knowing that their investments are protected and that they have access to the tools and support required to make informed decisions. This is where Bolide shines. By blending the user-friendliness of Robinhood and the engagement of popular social platforms, Bolide gives users the freedom to navigate crypto trading in a fun and informative way. To learn how Bolide is making the most out of its Robinhood moment, check out the app for yourself. You can also follow Ox Bid on her Farcaster at @oxb, and on X at @Oxxbid. Robinhood has made waves in the casual investment community for being a commission-free, mobile-friendly app that’s made trading accessible to everyone — not just the wealthy or experienced traders. It’s only natural, then, that a similar solution be made available for would-be crypto investors — and it’s vital that this solution is made specifically for the casual investor. After all, even with more people trading in decentralized digital currencies than ever before, the majority of Americans still don’t have a strong grasp of crypto concepts. This can prevent new and inexperienced traders from integrating cryptocurrency into their everyday transactions and investments, and it risks concentrating knowledge and trading power in the hands of a few individuals or groups. And sure, there are solutions like Coinbase and Kraken available, but these platforms are still bogged down by unnecessary features and confusing terminologies. majority of Americans majority of Americans This is where Bolide comes in. This easy-to-use mobile trading app aims to simplify crypto investments (specifically memecoins) for everyone, even those without any prior crypto experience. Learn more about the common barriers of crypto investing, how Bolide is breaking these down, and how Bolide CEO Ox Bid is giving memecoins their Robinhood moment — pushing to make social, on-chain memecoin trading easy and fun for everyone. Memecoin Trading: Navigating the Wild West of Crypto Memecoin trading is the buying and selling of digital currencies that are often inspired by internet jokes or cultural memes. However, unlike more established cryptocurrencies like Bitcoin or Ethereum, memecoins are highly volatile and lack additional built-in uses, making them a risky investment. The value of memecoins typically stems from viral marketing campaigns, speculation, and hype. While this adds to the risky charm of memecoins, it also makes it challenging for newcomers to distinguish between legitimate investments and fleeting trends. Additionally, the market's lack of regulation makes it ripe for scams, where promoters might inflate a memecoin's value and sell off their holdings at a peak, leaving others with devalued assets. The cryptocurrency market itself is also volatile. For instance, during the crypto winter of 2022 , the market saw a major downturn, resulting in a $2 trillion loss in market value and impacting many investors. Such slumps can be triggered by withdrawing from large investors, significant frauds or security breaches, or tighter government regulations. crypto winter of 2022 crypto winter of 2022 Finally, navigating crypto trading platforms also requires a high level of digital and financial literacy . In order to maintain their investment and stay safe, beginners need to invest time into learning which risks avoiding and deciphering which coins are worth investing in — all while managing complex trading interfaces. financial literacy financial literacy Due to these barriers, a lot of investors forgo crypto trading altogether. Bolide: Memecoin Trading for the Masses Bolide is a mobile app that makes it easy and affordable for beginners to start trading memecoins. It uses Pimlico ’s Account Abstraction technology to allow gasless transactions and embedded wallets from Privy for social logins. In doing so, it gives users affordable transactions, secure account access, and lets them store their customer keys safely. This way, users can manage their investments with ease and confidence, knowing their data and digital assets are well-protected. Pimlico Pimlico Privy Privy And thanks to a simplistic user interface, users can navigate Bolide as easily as they would a standard bank account — even if they’ve never used the on-chain market before “As an early investor and trader in crypto,” says Ox Bid, CEO and co-founder of Bolide, “my dream has always been to provide access and teach people how to navigate this space. The crypto market is undoubtedly risky, but it can be less so if customers have access to straightforward tools, as complexity often leads to mistakes and barriers.” Bolide is now available on the App Store, and its initial traction is notable. “Though we've been on the market for only a few weeks,” Ox says, “our trading volume has nearly reached $157,000, with an average of 13 trades per user and an average trade value of $30.” Clearly, Ox’s strategy of appealing to a lay audience of retail investors is paying off. Bolide’s Social Finance Future According to Ox Bid, it's time for memecoins to have their Robinhood moment. To this end, she believes that the secret to making memecoin trading accessible to everyone is to take Robinhood's affordable, user-friendly approach, apply it to crypto, and add a social twist. She notes that modern consumers are deeply influenced by social media in everything they do, from choosing where to eat to deciding where to live — and she sees this trend as vital to the future of retail crypto trading. “People often make decisions based on recommendations from influencers or friends,” Ox explains. “My goal is to create a similar influence-driven recommendation system, where users can see which tokens to buy based on the on-chain actions of top influencers, friends, and traders.” In doing so, Ox hopes to return to what made memecoins great in the first place: fun and levity. “I aim to make the crypto market not only accessible to the masses but also exciting and social,” she says. Trade Memecoins with Confidence Using Bolide Newcomers should be able to confidently trade crypto without having to sift through pages of research. They should feel at ease knowing that their investments are protected and that they have access to the tools and support required to make informed decisions. This is where Bolide shines. By blending the user-friendliness of Robinhood and the engagement of popular social platforms, Bolide gives users the freedom to navigate crypto trading in a fun and informative way. To learn how Bolide is making the most out of its Robinhood moment, check out the app for yourself. You can also follow Ox Bid on her Farcaster at @oxb , and on X at @Oxxbid. the app the app at @oxb @oxb @Oxxbid . @Oxxbid

Is This the Robinhood Moment for Memecoins? Discovering the Bolide App

এটি কি মেমেকয়েনের জন্য রবিনহুড মুহূর্ত? বোলাইড অ্যাপ আবিষ্কার করা হচ্ছে

Ist dies der Robinhood-Moment für Memecoins? Entdeckung der Bolide-App

¿Es este el momento Robinhood para las Memecoins? Descubriendo la aplicación Bolide

Est-ce le moment Robinhood pour les Memecoins ? À la découverte de l'application Bolide

क्या यह मेमेकॉइन्स के लिए रॉबिनहुड मोमेंट है? बोलाइड ऐप की खोज

これはミームコインのロビンフッドの瞬間か？Bolideアプリを発見

지금이 Memecoin의 로빈후드 순간인가요? Bolide 앱 발견

Este é o momento Robinhood para Memecoins? Conhecendo o aplicativo Bolide

Это момент Робинхуда для мемкоинов? Знакомство с приложением Bolide

Bu Memecoin'ler için Robinhood Anı mı? Bolide Uygulamasını Keşfetmek

Đây có phải là khoảnh khắc Robinhood dành cho Memecoin? Khám phá ứng dụng Bolide

这是 Memecoins 的罗宾汉时刻吗？探索 Bolide 应用程序

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk;
(2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk;
(3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction
Related work
Background
Methods
Experiments
"Conclusion, Limitations, and References"
Derivations
Algorithms
Experimental Datasets And Model Experimental Datasets And Models MovieLens Dataset The MovieLens100K1 (Harper and Konstan 2015) dataset contains 100k ratings of N=1682 films from among M=943 users. The original user ratings run from 0 to 5. Following Geffner and Domke (2022), we binarise the ratings into just likes/dislikes, by taking user-ratings of {0, 1, 2, 3} as a binarised-rating of 0 (dislikes) and user-ratings of {4, 5} as a binarised-rating of 1 (likes). We assume binarised ratings of 0 for films which users have not previously rated. The probabilistic graphical model is given in Fig 3. We use n to index films, and m to index users. Each film has a known feature vector, xn, while each user has a latent weight-vector, zm, of the same length, describing whether or not they like any given feature. There are 18 features, indicating which genre tags the film has (Action, Adventure, Animation, Childrens,...). Each film may have more than one tag. The probability of the user liking a film is given by taking the dot-product of the film’s feature vector with the latent weight-vector, and applying a sigmoid (σ(·)) (final line) Additionally, we have latent vectors for the global mean, µ, and variance, ψ, of the weight vectors. We use a random subset of N = 20 films and M = 450 users for our experiment to ensure high levels of uncertainty. We use equally sized but disjoint subset of users held aside for calculation of the predictive log-likelihood. In Fig. 1c,d we consider the variance and MSE of estimators of zm. Bus Delay Datase In our second experiment, we model the length of delays to New York school bus journeys,2 working with a dataset supplied by the City of New York (DOE 2023). Our goal is to predict the length of a delay, based on the year, y, borough, b, bus company, c and journey type, j. Specifically, our data includes the years 2015 − 2022 inclusive and covers the five New York boroughs (Brooklyn, Manhatten, The Bronx, Queens, Staten Island) as well as some surrounding areas (Nassau County, New Jersey, Connecticut, Rockland County, Westchester). There are 57 bus companies, and 6 different journey types (e.g. pre-K/elementary school route, general education AM/PM route etc.) We take I = 60 delayed buses in each borough and year, and take Y = 3 years and B = 3 boroughs. We then split along the I dimension to get two equally sized train and test sets. Thus, each delay is uniquely identified by the year, y, the borough, b, and the index, i, giving delayybi The delays are recorded as an integer number of minutes and we discard any entries greater than 130 minutes. We have a hierarchical latent variable model describing the impact of each of the features (year, borough, bus company and journey type) on the length of delays. Specifically, the integer delay is modelled with a Negative Binomial distribution, with fixed total count of 130, as we discard entries with delays longer than this. The expected delay length is controlled by a logits latent variable, logitsybi, with one logits for each delayed bus. The logits is a sum of three terms: one for the borough and year jointly, one for the bus company and one for the journey type. Each of these three terms is themselves a latent variable that must be inferred. First, we have a term for the year and borough, YearBoroughWeightyb, which has a hierarchical prior. Specifically, we begin by sampling a global mean and variance, GlobalMean and GlobalVariance. Then for each year, we use GlobalMean and GlobalVariance to sample a mean for each year, YearMeany. Additionally, we sample a variance for each borough, BoroughVarianceb. Then we sample a YearBoroughWeightyb from a Gaussian distribution with a year-dependent mean, YearMeany, and a borough-dependent variance, BoroughVarianceb. Next, the weights for the bus company and journey type are very similar. Specifically, we have one latent weight for each bus company, CompanyWeightc, with c ∈ {1, . . . , 57}, and for each journey type, JourneyTypeWeightj, with j ∈ {1, . . . , 6}. We have a table identifying the bus company, bybi, and journey type, jybi, for each delayed bus journey (remember that a particular delayed bus journey is uniquely identified by the year, y, borough, b and index i). In logitsybi we use these tables to pick out the right company and journey type weight for that particular delayed bus journey, CompanyWeightcybi and JourneyTypeWeight cybi. The final generative model is, and the graphical model is given in Fig. 4 Experiment Details We ran all experiments on an 80GB NVIDIA A100 GPU and averaged the results over three different dataset splits generated using different random seeds. On each dataset split, the MovieLens experiments required a maximum 26.9 GB of memory at any one time. For the NYC Bus Breakdown experiments this value was 0.9 GB. For each experiment using a single dataset  split, we used global IS, MP IS, VI and HMC methods to obtain (for each value of K in the IS methods, or each iteration of VI/HMC) 1000 values of the ELBO, predictive log-likelihood (on a test set that was disjoint from the training set), and posterior mean estimates for each latent variable in the model. These 1000 values were averaged for the ELBO and predictive log-likelihood, whilst the 1000 mean estimates were used to calculate mean squared error or variance per variable, based on whether the observed data had been sampled from the model itself—in which case the true variable values were known—or not. We present error bars in the top rows of Fig. 1 and Fig. 2 representing the standard deviation over the three dataset splits. For VI, we optimize our approximate posterior using Adam with learning rates of 10−2, 10−3 and 10−4 (we found that learning rates faster than 10−2 were unstable), and ran 100 optimization steps for both for the MovieLens and Bus Breakdown models. For HMC, we ran the No U-Turn Sampler on our GPU using JAX (Bradbury et al. 2018) to obtain 10 consecutive samples after a single step with PyMC’s (Salvatier, Wiecki, and Fonnesbeck 2016) pre-tuning and with all other parameters set to their default values. We present results of the first 5 samples (not including the tuning sample) in order to align as best we can with the small time-scale of the IS methods. This paper is available on arxiv under CC 4.0 license. Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction Related work Background Methods Experiments "Conclusion, Limitations, and References" Derivations Algorithms Experimental Datasets And Model Abstract & Introduction Abstract & Introduction Related work Related work Background Background Methods Methods Experiments Experiments "Conclusion, Limitations, and References" "Conclusion, Limitations, and References" Derivations Derivations Algorithms Algorithms Experimental Datasets And Model Experimental Datasets And Model Experimental Datasets And Models MovieLens Dataset The MovieLens100K1 (Harper and Konstan 2015) dataset contains 100k ratings of N=1682 films from among M=943 users. The original user ratings run from 0 to 5. Following Geffner and Domke (2022), we binarise the ratings into just likes/dislikes, by taking user-ratings of {0, 1, 2, 3} as a binarised-rating of 0 (dislikes) and user-ratings of {4, 5} as a binarised-rating of 1 (likes). We assume binarised ratings of 0 for films which users have not previously rated. The probabilistic graphical model is given in Fig 3. We use n to index films, and m to index users. Each film has a known feature vector, xn, while each user has a latent weight-vector, zm, of the same length, describing whether or not they like any given feature. There are 18 features, indicating which genre tags the film has (Action, Adventure, Animation, Childrens,...). Each film may have more than one tag. The probability of the user liking a film is given by taking the dot-product of the film’s feature vector with the latent weight-vector, and applying a sigmoid (σ(·)) (final line) Additionally, we have latent vectors for the global mean, µ, and variance, ψ, of the weight vectors. We use a random subset of N = 20 films and M = 450 users for our experiment to ensure high levels of uncertainty. We use equally sized but disjoint subset of users held aside for calculation of the predictive log-likelihood. In Fig. 1c,d we consider the variance and MSE of estimators of zm. Bus Delay Datase In our second experiment, we model the length of delays to New York school bus journeys,2 working with a dataset supplied by the City of New York (DOE 2023). Our goal is to predict the length of a delay, based on the year, y, borough, b, bus company, c and journey type, j. Specifically, our data includes the years 2015 − 2022 inclusive and covers the five New York boroughs (Brooklyn, Manhatten, The Bronx, Queens, Staten Island) as well as some surrounding areas (Nassau County, New Jersey, Connecticut, Rockland County, Westchester). There are 57 bus companies, and 6 different journey types (e.g. pre-K/elementary school route, general education AM/PM route etc.) We take I = 60 delayed buses in each borough and year, and take Y = 3 years and B = 3 boroughs. We then split along the I dimension to get two equally sized train and test sets. Thus, each delay is uniquely identified by the year, y, the borough, b, and the index, i, giving delayybi The delays are recorded as an integer number of minutes and we discard any entries greater than 130 minutes. We have a hierarchical latent variable model describing the impact of each of the features (year, borough, bus company and journey type) on the length of delays. Specifically, the integer delay is modelled with a Negative Binomial distribution, with fixed total count of 130, as we discard entries with delays longer than this. The expected delay length is controlled by a logits latent variable, logitsybi, with one logits for each delayed bus. The logits is a sum of three terms: one for the borough and year jointly, one for the bus company and one for the journey type. Each of these three terms is themselves a latent variable that must be inferred. First, we have a term for the year and borough, YearBoroughWeightyb, which has a hierarchical prior. Specifically, we begin by sampling a global mean and variance, GlobalMean and GlobalVariance. Then for each year, we use GlobalMean and GlobalVariance to sample a mean for each year, YearMeany. Additionally, we sample a variance for each borough, BoroughVarianceb. Then we sample a YearBoroughWeightyb from a Gaussian distribution with a year-dependent mean, YearMeany, and a borough-dependent variance, BoroughVarianceb. Next, the weights for the bus company and journey type are very similar. Specifically, we have one latent weight for each bus company, CompanyWeightc, with c ∈ {1, . . . , 57}, and for each journey type, JourneyTypeWeightj, with j ∈ {1, . . . , 6}. We have a table identifying the bus company, bybi, and journey type, jybi, for each delayed bus journey (remember that a particular delayed bus journey is uniquely identified by the year, y, borough, b and index i). In logitsybi we use these tables to pick out the right company and journey type weight for that particular delayed bus journey, CompanyWeightcybi and JourneyTypeWeight cybi. The final generative model is, and the graphical model is given in Fig. 4 Experiment Details We ran all experiments on an 80GB NVIDIA A100 GPU and averaged the results over three different dataset splits generated using different random seeds. On each dataset split, the MovieLens experiments required a maximum 26.9 GB of memory at any one time. For the NYC Bus Breakdown experiments this value was 0.9 GB. For each experiment using a single dataset  split, we used global IS, MP IS, VI and HMC methods to obtain (for each value of K in the IS methods, or each iteration of VI/HMC) 1000 values of the ELBO, predictive log-likelihood (on a test set that was disjoint from the training set), and posterior mean estimates for each latent variable in the model. These 1000 values were averaged for the ELBO and predictive log-likelihood, whilst the 1000 mean estimates were used to calculate mean squared error or variance per variable, based on whether the observed data had been sampled from the model itself—in which case the true variable values were known—or not. We present error bars in the top rows of Fig. 1 and Fig. 2 representing the standard deviation over the three dataset splits. For VI, we optimize our approximate posterior using Adam with learning rates of 10−2, 10−3 and 10−4 (we found that learning rates faster than 10−2 were unstable), and ran 100 optimization steps for both for the MovieLens and Bus Breakdown models. For HMC, we ran the No U-Turn Sampler on our GPU using JAX (Bradbury et al. 2018) to obtain 10 consecutive samples after a single step with PyMC’s (Salvatier, Wiecki, and Fonnesbeck 2016) pre-tuning and with all other parameters set to their default values. We present results of the first 5 samples (not including the tuning sample) in order to align as best we can with the small time-scale of the IS methods.

Using Autodiff to Estimate Posterior Moments, Marginals and Samples: Experimental Datasets and Model

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk;
(2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk;
(3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction
Related work
Background
Methods
Experiments
"Conclusion, Limitations, and References"
Derivations
Algorithms
Experimental Datasets And Model Algorithms This paper is available on arxiv under CC 4.0 license. Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction Related work Background Methods Experiments "Conclusion, Limitations, and References" Derivations Algorithms Experimental Datasets And Model Abstract & Introduction Abstract & Introduction Related work Related work Background Background Methods Methods Experiments Experiments "Conclusion, Limitations, and References" "Conclusion, Limitations, and References" Derivations Derivations Algorithms Algorithms Experimental Datasets And Model Experimental Datasets And Model Algorithms

Using Autodiff to Estimate Posterior Moments, Marginals and Samples: Algorithms

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk;
(2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk;
(3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction
Related work
Background
Methods
Experiments
"Conclusion, Limitations, and References"
Derivations
Algorithms
Experimental Datasets And Model Conclusion and Limitations We gave a new and far simpler method for computing posterior moments, marginals and samples in massively parallel importance sampling based on differentiating a slightly modified marginal likelihood estimator. The method has limitations, in that while it is considerably more effective than e.g. VI, HMC and global importance sampling, it is more complex. Additionally, at least a naive implementation may be quite costly in terms of memory consumption, limiting how the number of importance samples we can draw for each variable. That said, it should be possible to eliminate almost all of this overhead by careful optimizations to avoid allocating large intermediate tensors, following the strategy in KeOps (Charlier et al. 2021). References Aitchison, L. 2019. Tensor Monte Carlo: particle methods for the GPU era. Advances in Neural Information Processing Systems, 32. Albarghouthi, A.; D’Antoni, L.; Drews, S.; and Nori, A. V. 2017. Fairsquare: probabilistic verification of program fairness. Proceedings of the ACM on Programming Languages, 1(OOPSLA): 1–30. Andrieu, C.; Doucet, A.; and Holenstein, R. 2010. Particle markov chain monte carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3): 269–342. Bidyuk, B.; and Dechter, R. 2007. Cutset sampling for Bayesian networks. Journal of Artificial Intelligence Research, 28: 1–48. Bornschein, J.; and Bengio, Y. 2014. Reweighted wakesleep. arXiv preprint arXiv:1406.2751. Bradbury, J.; Frostig, R.; Hawkins, P.; Johnson, M. J.; Leary, C.; Maclaurin, D.; Necula, G.; Paszke, A.; VanderPlas, J.; Wanderman-Milne, S.; and Zhang, Q. 2018. JAX: composable transformations of Python+NumPy programs. Burda, Y.; Grosse, R.; and Salakhutdinov, R. 2015. Importance weighted autoencoders. arXiv preprint arXiv:1509.00519. Charlier, B.; Feydy, J.; Glaunes, J. A.; Collin, F.-D.; and Du- ` rif, G. 2021. Kernel Operations on the GPU, with Autodiff,  without Memory Overflows. Journal of Machine Learning Research. Chatterjee, S.; and Diaconis, P. 2018. The sample size required in importance sampling. The Annals of Applied Probability, 28(2): 1099–1135. Claret, G.; Rajamani, S. K.; Nori, A. V.; Gordon, A. D.; and Borgstrom, J. 2013. Bayesian inference using data flow anal- ¨ ysis. In Proceedings of the 2013 9th joint meeting on foundations of software engineering, 92–102. Corenflos, A.; Chopin, N.; and Sarkk ¨ a, S. 2022. De- ¨ Sequentialized Monte Carlo: a parallel-in time particle smoother. arXiv preprint arXiv:2202.02264. Crucinio, F. R.; and Johansen, A. M. 2023. Properties of Marginal Sequential Monte Carlo Methods. ArXiv:2303.03498 [math, stat]. Dawid, A. P. 1992. Applications of a general propagation algorithm for probabilistic expert systems. Statistics and computing, 2(1): 25–36. Dechter, R.; Broka, F.; Kask, K.; and Ihler, A. 2018. Abstraction Sampling in Graphical Models. AAAI. Del Moral, P.; Doucet, A.; and Jasra, A. 2006. Sequential Monte Carlo Samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(3): 411–436. DOE. 2023. Bus Breakdown and Delays. url https://data.cityofnewyork.us/Transportation/BusBreakdown-and-Delays/ez4e-fazm Accessed on: 05.05.2023. Doucet, A.; Johansen, A. M.; et al. 2009. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of nonlinear filtering, 12(656-704): 3. Geffner, T.; and Domke, J. 2022. Variational Inference with Locally Enhanced Bounds for Hierarchical Models. arXiv preprint arXiv:2203.04432. Gehr, T.; Misailovic, S.; and Vechev, M. 2016. PSI: Exact symbolic inference for probabilistic programs. In International Conference on Computer Aided Verification, 62–83. Springer. Geldenhuys, J.; Dwyer, M. B.; and Visser, W. 2012. Probabilistic symbolic execution. In Proceedings of the 2012 International Symposium on Software Testing and Analysis, 166–176. Gogate, V.; and Dechter, R. 2012. Importance samplingbased estimation over and/or search spaces for graphical models. Artificial Intelligence, 184: 38–77. Goodman, N. D.; and Stuhlmuller, A. 2014. The design and implementation of probabilistic programming languages. Gordon, N. J.; Salmond, D. J.; and Smith, A. F. 1993. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. In IEE proceedings F (radar and signal processing), volume 140, 107–113. IET. Harper, F. M.; and Konstan, J. A. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4): 1–19. Heap, T.; and Laurence, A. 2023. Massively parallel reweighted wake-sleep. UAI. Hoffman, M. D.; Gelman, A.; et al. 2014. The No-UTurn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 15(1): 1593–1623. Holtzen, S.; Van den Broeck, G.; and Millstein, T. 2020. Scaling exact inference for discrete probabilistic programs. Proceedings of the ACM on Programming Languages, 4(OOPSLA): 1–31. Jordan, M. I.; Ghahramani, Z.; Jaakkola, T. S.; and Saul, L. K. 1999. An introduction to variational methods for graphical models. Machine learning, 37(2): 183–233. Kingma, D. P.; Welling, M.; et al. 2019. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning, 12(4): 307–392. Kuntz, J.; Crucinio, F. R.; and Johansen, A. M. 2023. The divide-and-conquer sequential Monte Carlo algorithm: theoretical properties and limit theorems. ArXiv:2110.15782 [math, stat]. Lai, J.; Domke, J.; and Sheldon, D. 2022. Variational Marginal Particle Filters. In International Conference on Artificial Intelligence and Statistics, 875–895. PMLR. Le, T. A.; Igl, M.; Rainforth, T.; Jin, T.; and Wood, F. 2017. Auto-encoding sequential monte carlo. arXiv preprint arXiv:1705.10306. Le, T. A.; Kosiorek, A. R.; Siddharth, N.; Teh, Y. W.; and Wood, F. 2020. Revisiting reweighted wake sleep for models with stochastic control flow. In Uncertainty in Artificial Intelligence, 1039–1049. PMLR. Lindsten, F.; Johansen, A. M.; Naesseth, C. A.; Kirkpatrick, B.; Schon, T. B.; Aston, J.; and Bouchard-Cotˆ e, A. 2017. ´ Divide-and-conquer with sequential Monte Carlo. Journal of Computational and Graphical Statistics, 26(2): 445–458. Maddison, C. J.; Lawson, J.; Tucker, G.; Heess, N.; Norouzi, M.; Mnih, A.; Doucet, A.; and Teh, Y. 2017. Filtering variational objectives. Advances in Neural Information Processing Systems, 30. Naesseth, C.; Linderman, S.; Ranganath, R.; and Blei, D. 2018. Variational sequential monte carlo. In International conference on artificial intelligence and statistics, 968–977. PMLR. Narayanan, P.; Carette, J.; Romano, W.; Shan, C.-c.; and Zinkov, R. 2016. Probabilistic inference by program transformation in Hakaru (system description). In International Symposium on Functional and Logic Programming, 62–79. Springer Obermeyer, F.; Bingham, E.; Jankowiak, M.; Pradhan, N.; Chiu, J.; Rush, A.; and Goodman, N. 2019. Tensor variable elimination for plated factor graphs. In International Conference on Machine Learning, 4871–4880. PMLR. Pfeffer, A. 2005. The design and implementation of IBAL: A generalpurpose probabilistic programming language. In Harvard Univesity. Citeseer. Salvatier, J.; Wiecki, T. V.; and Fonnesbeck, C. 2016. Probabilistic programming in Python using PyMC3. PeerJ Computer Science, 2: e55. Publisher: PeerJ Inc. Sankaranarayanan, S.; Chakarov, A.; and Gulwani, S. 2013. Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation, 447–458. Wang, D.; Hoffmann, J.; and Reps, T. 2018. PMAF: an algebraic framework for static analysis of probabilistic programs. ACM SIGPLAN Notices, 53(4): 513–528. Weinberg, S. 1995. The quantum theory of fields, volume 2. Cambridge university press. Zavatone-Veth, J.; Canatar, A.; Ruben, B.; and Pehlevan, C. 2021. Asymptotics of representation learning in finite Bayesian neural networks. Advances in neural information processing systems, 34: 24765–24777. This paper is available on arxiv under CC 4.0 license. Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction Related work Background Methods Experiments "Conclusion, Limitations, and References" Derivations Algorithms Experimental Datasets And Model Abstract & Introduction Abstract & Introduction Related work Related work Background Background Methods Methods Experiments Experiments "Conclusion, Limitations, and References" "Conclusion, Limitations, and References" Derivations Derivations Algorithms Algorithms Experimental Datasets And Model Experimental Datasets And Model Conclusion and Limitations We gave a new and far simpler method for computing posterior moments, marginals and samples in massively parallel importance sampling based on differentiating a slightly modified marginal likelihood estimator. The method has limitations, in that while it is considerably more effective than e.g. VI, HMC and global importance sampling, it is more complex. Additionally, at least a naive implementation may be quite costly in terms of memory consumption, limiting how the number of importance samples we can draw for each variable. That said, it should be possible to eliminate almost all of this overhead by careful optimizations to avoid allocating large intermediate tensors, following the strategy in KeOps (Charlier et al. 2021). References Aitchison, L. 2019. Tensor Monte Carlo: particle methods for the GPU era. Advances in Neural Information Processing Systems, 32. Albarghouthi, A.; D’Antoni, L.; Drews, S.; and Nori, A. V. 2017. Fairsquare: probabilistic verification of program fairness. Proceedings of the ACM on Programming Languages, 1(OOPSLA): 1–30. Andrieu, C.; Doucet, A.; and Holenstein, R. 2010. Particle markov chain monte carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3): 269–342. Bidyuk, B.; and Dechter, R. 2007. Cutset sampling for Bayesian networks. Journal of Artificial Intelligence Research, 28: 1–48. Bornschein, J.; and Bengio, Y. 2014. Reweighted wakesleep. arXiv preprint arXiv:1406.2751. Bradbury, J.; Frostig, R.; Hawkins, P.; Johnson, M. J.; Leary, C.; Maclaurin, D.; Necula, G.; Paszke, A.; VanderPlas, J.; Wanderman-Milne, S.; and Zhang, Q. 2018. JAX: composable transformations of Python+NumPy programs. Burda, Y.; Grosse, R.; and Salakhutdinov, R. 2015. Importance weighted autoencoders. arXiv preprint arXiv:1509.00519. Charlier, B.; Feydy, J.; Glaunes, J. A.; Collin, F.-D.; and Du- ` rif, G. 2021. Kernel Operations on the GPU, with Autodiff,  without Memory Overflows. Journal of Machine Learning Research. Chatterjee, S.; and Diaconis, P. 2018. The sample size required in importance sampling. The Annals of Applied Probability, 28(2): 1099–1135. Claret, G.; Rajamani, S. K.; Nori, A. V.; Gordon, A. D.; and Borgstrom, J. 2013. Bayesian inference using data flow anal- ¨ ysis. In Proceedings of the 2013 9th joint meeting on foundations of software engineering, 92–102. Corenflos, A.; Chopin, N.; and Sarkk ¨ a, S. 2022. De- ¨ Sequentialized Monte Carlo: a parallel-in time particle smoother. arXiv preprint arXiv:2202.02264. Crucinio, F. R.; and Johansen, A. M. 2023. Properties of Marginal Sequential Monte Carlo Methods. ArXiv:2303.03498 [math, stat]. Dawid, A. P. 1992. Applications of a general propagation algorithm for probabilistic expert systems. Statistics and computing, 2(1): 25–36. Dechter, R.; Broka, F.; Kask, K.; and Ihler, A. 2018. Abstraction Sampling in Graphical Models. AAAI. Del Moral, P.; Doucet, A.; and Jasra, A. 2006. Sequential Monte Carlo Samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(3): 411–436. DOE. 2023. Bus Breakdown and Delays. url https://data.cityofnewyork.us/Transportation/BusBreakdown-and-Delays/ez4e-fazm Accessed on: 05.05.2023. Doucet, A.; Johansen, A. M.; et al. 2009. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of nonlinear filtering, 12(656-704): 3. Geffner, T.; and Domke, J. 2022. Variational Inference with Locally Enhanced Bounds for Hierarchical Models. arXiv preprint arXiv:2203.04432. Gehr, T.; Misailovic, S.; and Vechev, M. 2016. PSI: Exact symbolic inference for probabilistic programs. In International Conference on Computer Aided Verification, 62–83. Springer. Geldenhuys, J.; Dwyer, M. B.; and Visser, W. 2012. Probabilistic symbolic execution. In Proceedings of the 2012 International Symposium on Software Testing and Analysis, 166–176. Gogate, V.; and Dechter, R. 2012. Importance samplingbased estimation over and/or search spaces for graphical models. Artificial Intelligence, 184: 38–77. Goodman, N. D.; and Stuhlmuller, A. 2014. The design and implementation of probabilistic programming languages. Gordon, N. J.; Salmond, D. J.; and Smith, A. F. 1993. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. In IEE proceedings F (radar and signal processing), volume 140, 107–113. IET. Harper, F. M.; and Konstan, J. A. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4): 1–19. Heap, T.; and Laurence, A. 2023. Massively parallel reweighted wake-sleep. UAI. Hoffman, M. D.; Gelman, A.; et al. 2014. The No-UTurn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 15(1): 1593–1623. Holtzen, S.; Van den Broeck, G.; and Millstein, T. 2020. Scaling exact inference for discrete probabilistic programs. Proceedings of the ACM on Programming Languages, 4(OOPSLA): 1–31. Jordan, M. I.; Ghahramani, Z.; Jaakkola, T. S.; and Saul, L. K. 1999. An introduction to variational methods for graphical models. Machine learning, 37(2): 183–233. Kingma, D. P.; Welling, M.; et al. 2019. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning, 12(4): 307–392. Kuntz, J.; Crucinio, F. R.; and Johansen, A. M. 2023. The divide-and-conquer sequential Monte Carlo algorithm: theoretical properties and limit theorems. ArXiv:2110.15782 [math, stat]. Lai, J.; Domke, J.; and Sheldon, D. 2022. Variational Marginal Particle Filters. In International Conference on Artificial Intelligence and Statistics, 875–895. PMLR. Le, T. A.; Igl, M.; Rainforth, T.; Jin, T.; and Wood, F. 2017. Auto-encoding sequential monte carlo. arXiv preprint arXiv:1705.10306. Le, T. A.; Kosiorek, A. R.; Siddharth, N.; Teh, Y. W.; and Wood, F. 2020. Revisiting reweighted wake sleep for models with stochastic control flow. In Uncertainty in Artificial Intelligence, 1039–1049. PMLR. Lindsten, F.; Johansen, A. M.; Naesseth, C. A.; Kirkpatrick, B.; Schon, T. B.; Aston, J.; and Bouchard-Cotˆ e, A. 2017. ´ Divide-and-conquer with sequential Monte Carlo. Journal of Computational and Graphical Statistics, 26(2): 445–458. Maddison, C. J.; Lawson, J.; Tucker, G.; Heess, N.; Norouzi, M.; Mnih, A.; Doucet, A.; and Teh, Y. 2017. Filtering variational objectives. Advances in Neural Information Processing Systems, 30. Naesseth, C.; Linderman, S.; Ranganath, R.; and Blei, D. 2018. Variational sequential monte carlo. In International conference on artificial intelligence and statistics, 968–977. PMLR. Narayanan, P.; Carette, J.; Romano, W.; Shan, C.-c.; and Zinkov, R. 2016. Probabilistic inference by program transformation in Hakaru (system description). In International Symposium on Functional and Logic Programming, 62–79. Springer Obermeyer, F.; Bingham, E.; Jankowiak, M.; Pradhan, N.; Chiu, J.; Rush, A.; and Goodman, N. 2019. Tensor variable elimination for plated factor graphs. In International Conference on Machine Learning, 4871–4880. PMLR. Pfeffer, A. 2005. The design and implementation of IBAL: A generalpurpose probabilistic programming language. In Harvard Univesity. Citeseer. Salvatier, J.; Wiecki, T. V.; and Fonnesbeck, C. 2016. Probabilistic programming in Python using PyMC3. PeerJ Computer Science, 2: e55. Publisher: PeerJ Inc. Sankaranarayanan, S.; Chakarov, A.; and Gulwani, S. 2013. Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation, 447–458. Wang, D.; Hoffmann, J.; and Reps, T. 2018. PMAF: an algebraic framework for static analysis of probabilistic programs. ACM SIGPLAN Notices, 53(4): 513–528. Weinberg, S. 1995. The quantum theory of fields, volume 2. Cambridge university press. Zavatone-Veth, J.; Canatar, A.; Ruben, B.; and Pehlevan, C. 2021. Asymptotics of representation learning in finite Bayesian neural networks. Advances in neural information processing systems, 34: 24765–24777.

Using Autodiff to Estimate Posterior Moments: Conclusion, Limitations, and References

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk;
(2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk;
(3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction
Related work
Background
Methods
Experiments
"Conclusion, Limitations, and References"
Derivations
Algorithms
Experimental Datasets And Model Derivations Global Importance Sampling Massively Parallel Importance Sampling This paper is available on arxiv under CC 4.0 license. Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction Related work Background Methods Experiments "Conclusion, Limitations, and References" Derivations Algorithms Experimental Datasets And Model Abstract & Introduction Abstract & Introduction Related work Related work Background Background Methods Methods Experiments Experiments "Conclusion, Limitations, and References" "Conclusion, Limitations, and References" Derivations Derivations Algorithms Algorithms Experimental Datasets And Model Experimental Datasets And Model Derivations Global Importance Sampling Massively Parallel Importance Sampling

Using Autodiff to Estimate Posterior Moments, Marginals and Samples: Derivations

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk;
(2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk;
(3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction
Related work
Background
Methods
Experiments
"Conclusion, Limitations, and References"
Derivations
Algorithms
Experimental Datasets And Model Methods Of course, the contributions of this paper are not in computing the unbiased marginal likelihood estimator, which previously has been used in learning general probabilistic models, but instead our major contribution is a novel approach to computing key quantities of interest in Bayesian computation by applying the source term trick to the massively parallel marginal likelihood estimator. In particular, in the following sections, we outline in turn how to compute posterior expectations, marginals and samples. This paper is available on arxiv under CC 4.0 license. Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. This paper is available on arxiv under CC 4.0 license. Authors : Authors (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction Related work Background Methods Experiments "Conclusion, Limitations, and References" Derivations Algorithms Experimental Datasets And Model Abstract & Introduction Abstract & Introduction Related work Related work Background Background Methods Methods Experiments Experiments "Conclusion, Limitations, and References" "Conclusion, Limitations, and References" Derivations Derivations Algorithms Algorithms Experimental Datasets And Model Experimental Datasets And Model Methods Of course, the contributions of this paper are not in computing the unbiased marginal likelihood estimator, which previously has been used in learning general probabilistic models, but instead our major contribution is a novel approach to computing key quantities of interest in Bayesian computation by applying the source term trick to the massively parallel marginal likelihood estimator. In particular, in the following sections, we outline in turn how to compute posterior expectations, marginals and samples.

Using Autodiff to Estimate Posterior Moments, Marginals and Samples: Methods

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk;
(2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk;
(3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction
Related work
Background
Methods
Experiments
"Conclusion, Limitations, and References"
Derivations
Algorithms
Experimental Datasets And Model Experiment We provide empirical results comparing the global and massively parallel importance weighting/sampling methods, as well as VI and HMC baselines. We considered two datasets: MovieLens100K (Fig. 1) and NYC Bus Breakdown (Fig. 2). MovieLens100K (Harper and Konstan 2015) contains 100k ratings of N=1682 films from among M=943 users. NYC Bus Breakdown describes the length of around 150,000 delays to New York school bus journeys, segregated by year, borough, bus company and journey type. We use hierarchical probabilistic graphical models for these datasets, which are described in more detail in Appendix section Experimental Datasets And Models. For global and massively parallel importance sampling, we use the prior as a proposal. For VI, we used a factorised approximate posterior and optimized using Adam with learning rates ranging from 10−2 to 10−4  (learning rates faster than 10−2 were unstable). For HMC, we used the NUTS (Hoffman, Gelman et al. 2014) implementation from PyMC (Salvatier, Wiecki, and Fonnesbeck 2016). We consider four quantities in Fig. 1 and Fig. 2. We begin by looking at the ELBO (Fig. 1a,2a). While massively parallel estimates of the ELBO can be computed using previous methods (e.g. Heap and Laurence 2023), it is nonetheless useful as it is a good measure of the quality of our approximate posteriors (as the ELBO can be written as the sum of the marginal likelihood and the KL-divergence between the true and approximate posterior Jordan et al. 1999; Kingma, Welling et al. 2019). Next, we plot the predictive log-likelihood, based on posterior samples obtained using the approach in the Methods section. In both of these plots, we see dramatic improvements against standard “global” importance sampling for a fixed K (top row), and for a fixed time against other baselines such as VI and HMC (bottom row). Finally, we consider the quality of the posterior moment estimates computed using the approach in the Methods section. In particular we consider two measures of the quality of our posterior mean estimator. First, we consider the variance of the estimator (Fig. 1c,2c); in the ideal case, when we have exactly computed the posterior mean, this variance should be zero. While this variance can be zero in other cases (e.g. when the mean estimator always returns a constant), it does provide a lower-bound on the expected squared error between the estimator and the true value. Second, we directly considered the MSE between our posterior mean estimator and the true latent variable (Fig. 1d,2d). However, this quantity requires us to know the true value for the latent variable, which requires us to generate data from the model (we only generated data for d, columns a–c use real data). Again, in both of these cases, we see dramatic improvements for massively parallel against standard “global” importance sampling for a fixed K (top row), and for a fixed time against other methods such as VI and HMC (bottom row). Note the extremely poor performance of VI and HMC is because these are iterative methods that take many steps, and hence a long time to reach good performance. HMC is especially problematic on these short timescales, because it needs many gradient steps before it returns even a single posterior sample (and this is ignoring burn-in and adaptation phases). In contrast, importance sampling gets a reasonable answer much more quickly, as it just a single, non-iterative computation. This paper is available on arxiv under CC 4.0 license. Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk; (2)  Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk; (3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk. Table of Links Abstract & Introduction Related work Background Methods Experiments "Conclusion, Limitations, and References" Derivations Algorithms Experimental Datasets And Model Abstract & Introduction Abstract & Introduction Related work Related work Background Background Methods Methods Experiments Experiments "Conclusion, Limitations, and References" "Conclusion, Limitations, and References" Derivations Derivations Algorithms Algorithms Experimental Datasets And Model Experimental Datasets And Model Experiment We provide empirical results comparing the global and massively parallel importance weighting/sampling methods, as well as VI and HMC baselines. We considered two datasets: MovieLens100K (Fig. 1) and NYC Bus Breakdown (Fig. 2). MovieLens100K (Harper and Konstan 2015) contains 100k ratings of N=1682 films from among M=943 users. NYC Bus Breakdown describes the length of around 150,000 delays to New York school bus journeys, segregated by year, borough, bus company and journey type. We use hierarchical probabilistic graphical models for these datasets, which are described in more detail in Appendix section Experimental Datasets And Models. For global and massively parallel importance sampling, we use the prior as a proposal. For VI, we used a factorised approximate posterior and optimized using Adam with learning rates ranging from 10−2 to 10−4  (learning rates faster than 10−2 were unstable). For HMC, we used the NUTS (Hoffman, Gelman et al. 2014) implementation from PyMC (Salvatier, Wiecki, and Fonnesbeck 2016). We consider four quantities in Fig. 1 and Fig. 2. We begin by looking at the ELBO (Fig. 1a,2a). While massively parallel estimates of the ELBO can be computed using previous methods (e.g. Heap and Laurence 2023), it is nonetheless useful as it is a good measure of the quality of our approximate posteriors (as the ELBO can be written as the sum of the marginal likelihood and the KL-divergence between the true and approximate posterior Jordan et al. 1999; Kingma, Welling et al. 2019). Next, we plot the predictive log-likelihood, based on posterior samples obtained using the approach in the Methods section. In both of these plots, we see dramatic improvements against standard “global” importance sampling for a fixed K (top row), and for a fixed time against other baselines such as VI and HMC (bottom row). Finally, we consider the quality of the posterior moment estimates computed using the approach in the Methods section. In particular we consider two measures of the quality of our posterior mean estimator. First, we consider the variance of the estimator (Fig. 1c,2c); in the ideal case, when we have exactly computed the posterior mean, this variance should be zero. While this variance can be zero in other cases (e.g. when the mean estimator always returns a constant), it does provide a lower-bound on the expected squared error between the estimator and the true value. Second, we directly considered the MSE between our posterior mean estimator and the true latent variable (Fig. 1d,2d). However, this quantity requires us to know the true value for the latent variable, which requires us to generate data from the model (we only generated data for d, columns a–c use real data). Again, in both of these cases, we see dramatic improvements for massively parallel against standard “global” importance sampling for a fixed K (top row), and for a fixed time against other methods such as VI and HMC (bottom row). Note the extremely poor performance of VI and HMC is because these are iterative methods that take many steps, and hence a long time to reach good performance. HMC is especially problematic on these short timescales, because it needs many gradient steps before it returns even a single posterior sample (and this is ignoring burn-in and adaptation phases). In contrast, importance sampling gets a reasonable answer much more quickly, as it just a single, non-iterative computation.

Using Autodiff to Estimate Posterior Moments, Marginals and Samples: Experiments

Stories, News & Mentions about MOMENT

Moment’s Stories on HackerNoon