This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Yici Zhong, Department of Physics, Graduate School of Science, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan;
(2) Kazumi Kashiyama, Research Center for the Early Universe, Graduate School of Science, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan and Kavli Institute for the Physics and Mathematics of the Universe (Kavli IPMU,WPI), The University of Tokyo, Chiba 277-8582, Japan;
(3) Shinsuke Takasao, Department of Earth and Space Science, Graduate School of Science, Osaka University, Toyonaka, Osaka 560-0043, Japan;
(4) Toshikazu Shigeyama, Research Center for the Early Universe (RESCEU), School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan and Department of Astronomy, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan;
(5) Kotaro Fujisawa, Research Center for the Early Universe (RESCEU), School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan and Department of Liberal Arts, Tokyo University of Technology, Ota-ku, Tokyo 144-0051, Japan. Table of Links Abstract and Intro Setup Result Summary and Discussion Appendix A. Dual Energy Formalism B. Convergence of Results C. Change of the Mass Loss Rate in MHD Regime References REFERENCES Briggs, G. P., Ferrario, L., Tout, C. A., Wickramasinghe, D. T., & Hurley, J. R. 2015, MNRAS, 447, 1713, doi: 10.1093/mnras/stu2539 Bryan, G. L., Norman, M. L., Stone, J. M., Cen, R., & Ostriker, J. P. 1995, Computer Physics Communications, 89, 149, doi: 10.1016/0010-4655(94)00191-4 Bucciantini, N., Thompson, T. A., Arons, J., Quataert, E., & Del Zanna, L. 2006, MNRAS, 368, 1717, doi: 10.1111/j.1365-2966.2006.10217.x Caiazzo, I., Burdge, K. B., Fuller, J., et al. 2021, Nature, 595, 39, doi: 10.1038/s41586-021-03615-y Cheng, S., Cummings, J. D., M´enard, B., & Toonen, S. 2020, ApJ, 891, 160, doi: 10.3847/1538-4357/ab733c Contopoulos, I., & Spitkovsky, A. 2006, ApJ, 643, 1139, doi: 10.1086/501161 Ferreira, J. M. 2000, MNRAS, 316, 647, doi: 10.1046/j.1365-8711.2000.03540.x Garc´ıa-Berro, E., Lor´en-Aguilar, P., Aznar-Sigu´an, G., et al. 2012, ApJ, 749, 25, doi: 10.1088/0004-637X/749/1/25 Gvaramadze, V. V., Gr¨afener, G., Langer, N., et al. 2019, Nature, 569, 684, doi: 10.1038/s41586-019-1216-1 Iben, I., J., & Tutukov, A. V. 1984, ApJS, 54, 335, doi: 10.1086/190932 Iijima, H. 2016, PhD thesis, University of Tokyo, Department of Earth and Planetary Environmental Science Jardine, M., & Collier Cameron, A. 2019, MNRAS, 482, 2853, doi: 10.1093/mnras/sty2872 Kashiyama, K., Fujisawa, K., & Shigeyama, T. 2019, ApJ, 887, 39, doi: 10.3847/1538-4357/ab4e97 Kashiyama, K., Ioka, K., & Kawanaka, N. 2011, PhRvD, 83, 023002, doi: 10.1103/PhysRevD.83.023002 Kashiyama, K., & Murase, K. 2017, ApJL, 839, L3, doi: 10.3847/2041-8213/aa68e1 Kilic, M., Kosakowski, A., Moss, A. G., Bergeron, P., & Conly, A. A. 2021, ApJL, 923, L6, doi: 10.3847/2041-8213/ac3b60 Kirsten, F., Marcote, B., Nimmo, K., et al. 2022, Nature, 602, 585, doi: 10.1038/s41586-021-04354-w Ko, T., Suzuki, H., Kashiyama, K., et al. 2023, arXiv e-prints, arXiv:2304.14669, doi: 10.48550/arXiv.2304.14669 Kremer, K., Piro, A. L., & Li, D. 2021, ApJL, 917, L11, doi: 10.3847/2041-8213/ac13a0 Lu, W., Beniamini, P., & Kumar, P. 2022, MNRAS, 510, 1867, doi: 10.1093/mnras/stab3500 Lykou, F., Parker, Q. A., Ritter, A., et al. 2022, arXiv e-prints, arXiv:2208.03946, doi: 10.48550/arXiv.2208.03946 Matt, S. P., MacGregor, K. B., Pinsonneault, M. H., & Greene, T. P. 2012, ApJL, 754, L26, doi: 10.1088/2041-8205/754/2/L26 Michel, F. C. 1969, ApJ, 158, 727, doi: 10.1086/150233 Miyoshi, T., & Kusano, K. 2005, Journal of Computational Physics, 208, 315, doi: 10.1016/j.jcp.2005.02.017 Nomoto, K., & Iben, I., J. 1985, ApJ, 297, 531, doi: 10.1086/163547 Oskinova, L. M., Gvaramadze, V. V., Gr¨afener, G., Langer, N., & Todt, H. 2020, A&A, 644, L8, doi: 10.1051/0004-6361/202039232 Parker, E. N. 1965, SSRv, 4, 666, doi: 10.1007/BF00216273 Raives, M. J., Coleman, M. S. B., & Thompson, T. A. 2023, arXiv e-prints, arXiv:2302.05462, doi: 10.48550/arXiv.2302.05462 Ritter, A., Parker, Q. A., Lykou, F., et al. 2021, ApJL, 918, L33, doi: 10.3847/2041-8213/ac2253 Saio, H., & Nomoto, K. 2004, ApJ, 615, 444, doi: 10.1086/423976 Schwab, J. 2021, ApJ, 906, 53, doi: 10.3847/1538-4357/abc87e Schwab, J., Quataert, E., & Kasen, D. 2016, MNRAS, 463, 3461, doi: 10.1093/mnras/stw2249 Stone, J. M., Tomida, K., White, C. J., & Felker, K. G. 2020, The Astrophysical Journal Supplement Series, 249, 4, doi: 10.3847/1538-4365/ab929b Takasao, S., Tomida, K., Iwasaki, K., & Suzuki, T. K. 2019, ApJL, 878, L10, doi: 10.3847/2041-8213/ab22bb Tout, C. A., Wickramasinghe, D. T., Liebert, J., Ferrario, L., & Pringle, J. E. 2008, MNRAS, 387, 897, doi: 10.1111/j.1365-2966.2008.13291.x Townsend, R. H. D., & Owocki, S. P. 2005, MNRAS, 357, 251, doi: 10.1111/j.1365-2966.2005.08642.x Ud-Doula, A., Owocki, S. P., & Townsend, R. H. D. 2009, MNRAS, 392, 1022, doi: 10.1111/j.1365-2966.2008.14134.x Webbink, R. F. 1984, ApJ, 277, 355, doi: 10.1086/161701 Weber, E. J., & Davis, Leverett, J. 1967, ApJ, 148, 217, doi: 10.1086/149138 Wu, C., Xiong, H., Lin, J., et al. 2023, ApJL, 944, L54, doi: 10.3847/2041-8213/acb6f3 Yao, P. Z., Quataert, E., & Goulding, A. 2023, MNRAS, doi: 10.1093/mnras/stad1934 This paper is available on arxiv under CC 4.0 license. Authors: (1) Yici Zhong, Department of Physics, Graduate School of Science, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan; (2) Kazumi Kashiyama, Research Center for the Early Universe, Graduate School of Science, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan and Kavli Institute for the Physics and Mathematics of the Universe (Kavli IPMU,WPI), The University of Tokyo, Chiba 277-8582, Japan; (3) Shinsuke Takasao, Department of Earth and Space Science, Graduate School of Science, Osaka University, Toyonaka, Osaka 560-0043, Japan; (4) Toshikazu Shigeyama, Research Center for the Early Universe (RESCEU), School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan and Department of Astronomy, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan; (5) Kotaro Fujisawa, Research Center for the Early Universe (RESCEU), School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan and Department of Liberal Arts, Tokyo University of Technology, Ota-ku, Tokyo 144-0051, Japan. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Yici Zhong, Department of Physics, Graduate School of Science, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan; (2) Kazumi Kashiyama, Research Center for the Early Universe, Graduate School of Science, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan and Kavli Institute for the Physics and Mathematics of the Universe (Kavli IPMU,WPI), The University of Tokyo, Chiba 277-8582, Japan; (3) Shinsuke Takasao, Department of Earth and Space Science, Graduate School of Science, Osaka University, Toyonaka, Osaka 560-0043, Japan; (4) Toshikazu Shigeyama, Research Center for the Early Universe (RESCEU), School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan and Department of Astronomy, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan; (5) Kotaro Fujisawa, Research Center for the Early Universe (RESCEU), School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan and Department of Liberal Arts, Tokyo University of Technology, Ota-ku, Tokyo 144-0051, Japan. Table of Links Abstract and Intro Abstract and Intro Setup Setup Result Result Summary and Discussion Summary and Discussion Appendix Appendix A. Dual Energy Formalism A. Dual Energy Formalism B. Convergence of Results B. Convergence of Results C. Change of the Mass Loss Rate in MHD Regime C. Change of the Mass Loss Rate in MHD Regime References References REFERENCES Briggs, G. P., Ferrario, L., Tout, C. A., Wickramasinghe, D. T., & Hurley, J. R. 2015, MNRAS, 447, 1713, doi: 10.1093/mnras/stu2539 Bryan, G. L., Norman, M. L., Stone, J. M., Cen, R., & Ostriker, J. P. 1995, Computer Physics Communications, 89, 149, doi: 10.1016/0010-4655(94)00191-4 Bucciantini, N., Thompson, T. A., Arons, J., Quataert, E., & Del Zanna, L. 2006, MNRAS, 368, 1717, doi: 10.1111/j.1365-2966.2006.10217.x Caiazzo, I., Burdge, K. B., Fuller, J., et al. 2021, Nature, 595, 39, doi: 10.1038/s41586-021-03615-y Cheng, S., Cummings, J. D., M´enard, B., & Toonen, S. 2020, ApJ, 891, 160, doi: 10.3847/1538-4357/ab733c Contopoulos, I., & Spitkovsky, A. 2006, ApJ, 643, 1139, doi: 10.1086/501161 Ferreira, J. M. 2000, MNRAS, 316, 647, doi: 10.1046/j.1365-8711.2000.03540.x Garc´ıa-Berro, E., Lor´en-Aguilar, P., Aznar-Sigu´an, G., et al. 2012, ApJ, 749, 25, doi: 10.1088/0004-637X/749/1/25 Gvaramadze, V. V., Gr¨afener, G., Langer, N., et al. 2019, Nature, 569, 684, doi: 10.1038/s41586-019-1216-1 Iben, I., J., & Tutukov, A. V. 1984, ApJS, 54, 335, doi: 10.1086/190932 Iijima, H. 2016, PhD thesis, University of Tokyo, Department of Earth and Planetary Environmental Science Jardine, M., & Collier Cameron, A. 2019, MNRAS, 482, 2853, doi: 10.1093/mnras/sty2872 Kashiyama, K., Fujisawa, K., & Shigeyama, T. 2019, ApJ, 887, 39, doi: 10.3847/1538-4357/ab4e97 Kashiyama, K., Ioka, K., & Kawanaka, N. 2011, PhRvD, 83, 023002, doi: 10.1103/PhysRevD.83.023002 Kashiyama, K., & Murase, K. 2017, ApJL, 839, L3, doi: 10.3847/2041-8213/aa68e1 Kilic, M., Kosakowski, A., Moss, A. G., Bergeron, P., & Conly, A. A. 2021, ApJL, 923, L6, doi: 10.3847/2041-8213/ac3b60 Kirsten, F., Marcote, B., Nimmo, K., et al. 2022, Nature, 602, 585, doi: 10.1038/s41586-021-04354-w Ko, T., Suzuki, H., Kashiyama, K., et al. 2023, arXiv e-prints, arXiv:2304.14669, doi: 10.48550/arXiv.2304.14669 Kremer, K., Piro, A. L., & Li, D. 2021, ApJL, 917, L11, doi: 10.3847/2041-8213/ac13a0 Lu, W., Beniamini, P., & Kumar, P. 2022, MNRAS, 510, 1867, doi: 10.1093/mnras/stab3500 Lykou, F., Parker, Q. A., Ritter, A., et al. 2022, arXiv e-prints, arXiv:2208.03946, doi: 10.48550/arXiv.2208.03946 Matt, S. P., MacGregor, K. B., Pinsonneault, M. H., & Greene, T. P. 2012, ApJL, 754, L26, doi: 10.1088/2041-8205/754/2/L26 Michel, F. C. 1969, ApJ, 158, 727, doi: 10.1086/150233 Miyoshi, T., & Kusano, K. 2005, Journal of Computational Physics, 208, 315, doi: 10.1016/j.jcp.2005.02.017 Nomoto, K., & Iben, I., J. 1985, ApJ, 297, 531, doi: 10.1086/163547 Oskinova, L. M., Gvaramadze, V. V., Gr¨afener, G., Langer, N., & Todt, H. 2020, A&A, 644, L8, doi: 10.1051/0004-6361/202039232 Parker, E. N. 1965, SSRv, 4, 666, doi: 10.1007/BF00216273 Raives, M. J., Coleman, M. S. B., & Thompson, T. A. 2023, arXiv e-prints, arXiv:2302.05462, doi: 10.48550/arXiv.2302.05462 Ritter, A., Parker, Q. A., Lykou, F., et al. 2021, ApJL, 918, L33, doi: 10.3847/2041-8213/ac2253 Saio, H., & Nomoto, K. 2004, ApJ, 615, 444, doi: 10.1086/423976 Schwab, J. 2021, ApJ, 906, 53, doi: 10.3847/1538-4357/abc87e Schwab, J., Quataert, E., & Kasen, D. 2016, MNRAS, 463, 3461, doi: 10.1093/mnras/stw2249 Stone, J. M., Tomida, K., White, C. J., & Felker, K. G. 2020, The Astrophysical Journal Supplement Series, 249, 4, doi: 10.3847/1538-4365/ab929b Takasao, S., Tomida, K., Iwasaki, K., & Suzuki, T. K. 2019, ApJL, 878, L10, doi: 10.3847/2041-8213/ab22bb Tout, C. A., Wickramasinghe, D. T., Liebert, J., Ferrario, L., & Pringle, J. E. 2008, MNRAS, 387, 897, doi: 10.1111/j.1365-2966.2008.13291.x Townsend, R. H. D., & Owocki, S. P. 2005, MNRAS, 357, 251, doi: 10.1111/j.1365-2966.2005.08642.x Ud-Doula, A., Owocki, S. P., & Townsend, R. H. D. 2009, MNRAS, 392, 1022, doi: 10.1111/j.1365-2966.2008.14134.x Webbink, R. F. 1984, ApJ, 277, 355, doi: 10.1086/161701 Weber, E. J., & Davis, Leverett, J. 1967, ApJ, 148, 217, doi: 10.1086/149138 Wu, C., Xiong, H., Lin, J., et al. 2023, ApJL, 944, L54, doi: 10.3847/2041-8213/acb6f3 Yao, P. Z., Quataert, E., & Goulding, A. 2023, MNRAS, doi: 10.1093/mnras/stad1934

Magnetic Wind From a Massive White Dwarf Merger: References

The fight between OpenAI and Google to win the generative AI arms race has all the intensity of the war between Kendrick Lamar and Drake but without the low-blow trash talk. This week, both companies unleashed a dizzying array of updates that say a lot about their strategies: OpenAI: A More Human Gen AI On May 13, OpenAI announced several improvements to ChatGPT including multimodal capabilities. https://www.youtube.com/live/DQacCB9tDaw?si=RyXrifz6jnHbYWCW&embedable=true One feature that has captured the imagination of industry watchers is how ChatGPT is becoming more human. OpenAI demonstrated how the latest iteration of ChatGPT (GPT-4o) can detect emotions in users’ voices, analyze their facial expressions, and adjust its own tone and cadence to match user preferences. For instance, if you request a bedtime story, it can adopt a gentle, whispering tone. It can respond in a playful, sarcastic manner if you want it to. It even has the capability to sing when asked. https://youtu.be/D9byh4MAsUQ?si=DnmC0PteXYgj833j&embedable=true The demonstration immediately drew comparisons to Her, the 2013 movie in which a virtual assistant seduces an introvert… But making OpenAI more human is not in and of itself nothing new. There is a growing trend, especially among Gen Z, to turn to AI for personal companionship. Apps such as Pi offer life coaching in a helpful and conversational manner. Replika is a virtual friend and mental health lifeline. Blush is an AI-generated dating simulator. And there are many, many more examples. OpenAI knows that humanlike AI is the future, especially with Gen Z. Recently, Npj Mental Health Research studied how college students use Replika. The study observed, “Many used Replika in multiple, overlapping ways—as a friend, a therapist, and an intellectual mirror. Many also held overlapping and often conflicting beliefs about Replika—calling it a machine, an intelligence, and a human. Critically, 3% reported that Replika halted their suicidal ideation.” The significance of how ChatGPT is evolving is that OpenAI is making an incredibly popular everyday utility more human. There is no turning back now. Google: Doubling Down on AI-Powered Search At Google’s annual I/O conference on May 14, Google showed that it is leaning into its core strengths with gen AI-powered search. For instance, the AI Overview feature provides a brief, AI-generated answer to searches, along with a series of links to the sources cited in the response. Google is also introducing multistep reasoning (refining search results without requiring multiple searches in the same search session). Let’s say you are searching for a new yoga or pilates studio that is popular, conveniently located for your daily commute, and also offers a discount for new members. Soon, you will be able to ask Google Search to “find the best yoga or pilates studios in Boston and show details on their intro offers and walking time from Beacon Hill.” https://youtu.be/s4InWsd-J6g?si=X429BeoiivA6taTQ&embedable=true If you are an avid user of Google search for wayfinding, you can appreciate how important and valuable multistep reasoning will be when rolled out. The company is also introducing updates to its Gemini gen AI chatbot, including the ability to respond to longer prompts, engage in audio-based conversations, and provide more powerful functionality, including a tool summarizing 1,500 pages of the text uploaded by a user. These are important updates to a tool that I personally find more useful than ChatGPT in many ways. But for Google’s business, it is more critical that the company inject gen AI across its powerful search ecosystem, including Google Search, YouTube, and Email. Search is Google’s bread-and-butter business. And Google is fortifying its moat. Photo by Mohamed Nohassi via Unsplash The fight between OpenAI and Google to win the generative AI arms race has all the intensity of the war between Kendrick Lamar and Drake but without the low-blow trash talk. This week, both companies unleashed a dizzying array of updates that say a lot about their strategies: OpenAI: A More Human Gen AI On May 13, OpenAI announced several improvements to ChatGPT including multimodal capabilities. https://www.youtube.com/live/DQacCB9tDaw?si=RyXrifz6jnHbYWCW&embedable=true https://www.youtube.com/live/DQacCB9tDaw?si=RyXrifz6jnHbYWCW&embedable=true One feature that has captured the imagination of industry watchers is how ChatGPT is becoming more human. OpenAI demonstrated how the latest iteration of ChatGPT (GPT-4o) can detect emotions in users’ voices, analyze their facial expressions, and adjust its own tone and cadence to match user preferences. For instance, if you request a bedtime story, it can adopt a gentle, whispering tone. It can respond in a playful, sarcastic manner if you want it to. It even has the capability to sing when asked. https://youtu.be/D9byh4MAsUQ?si=DnmC0PteXYgj833j&embedable=true https://youtu.be/D9byh4MAsUQ?si=DnmC0PteXYgj833j&embedable=true The demonstration immediately drew comparisons to Her , the 2013 movie in which a virtual assistant seduces an introvert… But making OpenAI more human is not in and of itself nothing new. There is a growing trend, especially among Gen Z, to turn to AI for personal companionship. Apps such as Pi offer life coaching in a helpful and conversational manner. Replika is a virtual friend and mental health lifeline. Blush is an AI-generated dating simulator. And there are many, many more examples. The demonstration immediately drew comparisons to Her Her OpenAI knows that humanlike AI is the future, especially with Gen Z. Recently, Npj Mental Health Research studied how college students use Replika. The study observed , “Many used Replika in multiple, overlapping ways—as a friend, a therapist, and an intellectual mirror. Many also held overlapping and often conflicting beliefs about Replika—calling it a machine, an intelligence, and a human. Critically, 3% reported that Replika halted their suicidal ideation.” The study observed The significance of how ChatGPT is evolving is that OpenAI is making an incredibly popular everyday utility more human. There is no turning back now. Google: Doubling Down on AI-Powered Search At Google’s annual I/O conference on May 14, Google showed that it is leaning into its core strengths with gen AI-powered search . For instance, the AI Overview feature provides a brief, AI-generated answer to searches, along with a series of links to the sources cited in the response. Google is also introducing multistep reasoning (refining search results without requiring multiple searches in the same search session). Google showed that it is leaning into its core strengths with gen AI-powered search Let’s say you are searching for a new yoga or pilates studio that is popular, conveniently located for your daily commute, and also offers a discount for new members. Soon, you will be able to ask Google Search to “find the best yoga or pilates studios in Boston and show details on their intro offers and walking time from Beacon Hill.” https://youtu.be/s4InWsd-J6g?si=X429BeoiivA6taTQ&embedable=true https://youtu.be/s4InWsd-J6g?si=X429BeoiivA6taTQ&embedable=true If you are an avid user of Google search for wayfinding, you can appreciate how important and valuable multistep reasoning will be when rolled out. The company is also introducing updates to its Gemini gen AI chatbot, including the ability to respond to longer prompts, engage in audio-based conversations, and provide more powerful functionality, including a tool summarizing 1,500 pages of the text uploaded by a user. These are important updates to a tool that I personally find more useful than ChatGPT in many ways. But for Google’s business, it is more critical that the company inject gen AI across its powerful search ecosystem, including Google Search, YouTube, and Email. Search is Google’s bread-and-butter business. And Google is fortifying its moat. Photo by Mohamed Nohassi via Unsplash Photo by Mohamed Nohassi via Unsplash

Generative AI Clash: OpenAI’s Emotional AI vs. Google’s Enhanced Search

Authors: (1) Jonathan H. Rystrøm. Table of Links Abstract and Introduction Previous Literature Methods and Data Results Discussions Conclusions and References A. Validation of Assumptions B. Other Models C. Pre-processing steps 6 Conclusion Understanding how recommender systems shape our behaviour is essential to avoid manipulation. In this paper, we investigated the Amazon recommender system concerning whether it has made it harder to change preferences. By analysing the Barrier-to-Exit (Rakova & Chowdhury, 2019) of more than 50,000 users, we found a highly significant growth in Barrier-to-Exit over time, which indicates that it has indeed become harder to change preferences for the analysed users. However, sampling bias induced by the calculation of Barrier-to-Exit makes it difficult to draw conclusions about the general population of Amazon customers. This highlights the dilemma of portability in measuring socio-technical systems (Selbst et al., 2019): accurately evaluating a concept like ”changing preferences” requires adapting to the context of the system, which makes it more difficult to generalise (and compare) to other systems. Comparing recommender systems is necessary for ensuring that these respect human autonomy (Varshney, 2020) and live up to new regulations such as the EU AI Act (Kop, 2021). Further work, should aim to create auditing procedures and metrics that allow third parties to measure potential preference manipulation in a way that fits within the context of the industry and allows for comparisons between different systems. This will help assess the pressures of Surveillance Capitalism (Zuboff, 2019) on human autonomy. References Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of memory and language, 59 (4), 390–412. (Publisher: Elsevier) Barto´n, K. (2022, September). MuMIn: Multi-Model Inference. Retrieved 2023-01-08, from https://CRAN.R-project .org/package=MuMIn Bates, D., M¨achler, M., Bolker, B., & Walker, S. (2014, June). Fitting Linear Mixed-Effects Models using lme4. arXiv. Retrieved 2023-01-12, from http://arxiv.org/abs/1406.5823 (arXiv:1406.5823 [stat]) Bennett, J., & Lanning, S. (2007). The netflix prize. In Proceedings of KDD cup and workshop (Vol. 2007, p. 35). New York. urrell, J. (2016, June). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3 (1), 2053951715622512. Retrieved 2022-10-11, from https://ezproxy-prd.bodleian.ox.ac.uk:2246/doi/full/10.1177/2053951715622512 (Publisher: SAGE Publications Ltd) doi: 10.1177/ 2053951715622512 Calvo, R. A., Peters, D., Vold, K., & Ryan, R. M. (2020). Supporting Human Autonomy in AI Systems: A Framework for Ethical Enquiry. In C. Burr & L. Floridi (Eds.), Ethics of Digital Well-Being: A Multidisciplinary Approach (pp. 31–54). Cham: Springer International Publishing. Retrieved 2022-08-16, from https://doi.org/10.1007/ 978-3-030-50585-1 2 doi: 10.1007/978-3-030-50585-1 2 Carroll, M. D., Dragan, A., Russell, S., & Hadfield-Menell, D. (2022). Estimating and penalizing induced preference shifts in recommender systems. In International Conference on Machine Learning (pp. 2686–2708). PMLR. Feng, C., Wang, H., Lu, N., Chen, T., He, H., Lu, Y., & Tu, X. M. (2014, April). Log-transformation and its implications for data analysis. Shanghai Archives of Psychiatry, 26 (2), 105–109. Retrieved 2023-01-10, from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4120293/ doi: 10.3969/j.issn.1002-0829.2014.02.009 Floridi, L., Holweg, M., Taddeo, M., Amaya Silva, J., M¨okander, J., & Wen, Y. (2022, March). capAI - A Procedure for Conducting Conformity Assessment of AI Systems in Line with the EU Artificial Intelligence Act (SSRN Scholarly Paper No. ID 4064091). Rochester, NY: Social Science Research Network. Retrieved 2022-04-01, from https://papers.ssrn.com/abstract=4064091 doi: 10.2139/ssrn.4064091 Fox, J. (2003). Effect Displays in R for Generalised Linear Models. Journal of Statistical Software, 8 (15). Retrieved 2023-01-10, from http://www.jstatsoft.org/v08/i15/ doi: 10.18637/jss.v008.i15 Fox, J. (2015). Applied regression analysis and generalized linear models. Sage Publications. Franklin, M., Ashton, H., Gorman, R., & Armstrong, S. (2022, May). Missing Mechanisms of Manipulation in the EU AI Act. The International FLAIRS Conference Proceedings, 35 . Retrieved 2022-12-31, from https:// journals.flvc.org/FLAIRS/article/view/130723 doi: 10.32473/flairs.v35i.130723 Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daum´e III, H., & Crawford, K. (2021, December). Datasheets for Datasets. arXiv. Retrieved 2022-10-20, from http://arxiv.org/abs/1803.09010 (arXiv:1803.09010 [cs]) Ghasemi, A., & Zahediasl, S. (2012, December). Normality Tests for Statistical Analysis: A Guide for NonStatisticians. International Journal of Endocrinology and Metabolism, 10 (2), 486–489. Retrieved 2023-01-11, from https://brief.land/ijem/articles/71904.html doi: 10.5812/ijem.3505 Gorelick, M., & Ozsvald, I. (2020). High performance Python: practical performant programming for humans (Second edition ed.). Beijing [China] ; Boston [MA]: O’Reilly. Harper, F. M., & Konstan, J. A. (2016, January). The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, 5 (4), 1–19. Retrieved 2022-12-29, from https://dl.acm.org/doi/10.1145/ 2827872 doi: 10.1145/2827872 Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., . . . Oliphant, T. E. (2020, September). Array programming with NumPy. Nature, 585 (7825), 357–362. Retrieved 2021-12-27, from https://www.nature.com/articles/s41586-020-2649-2 doi: 10.1038/s41586-020-2649-2 Jannach, D., & Adomavicius, G. (2016, September). Recommendations with a Purpose. In Proceedings of the 10th ACM Conference on Recommender Systems (pp. 7–10). Boston Massachusetts USA: ACM. Retrieved 2022-09-01, from https://dl.acm.org/doi/10.1145/2959100.2959186 doi: 10.1145/2959100.2959186 Jiang, R., Chiappa, S., Lattimore, T., Gy¨orgy, A., & Kohli, P. (2019, January). Degenerate Feedback Loops in Recommender Systems. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 383–390). Honolulu HI USA: ACM. Retrieved 2022-09-07, from https://dl.acm.org/doi/10.1145/3306618 .3314288 doi: 10.1145/3306618.3314288 Kipf, T. N., & Welling, M. (2017, February). Semi-Supervised Classification with Graph Convolutional Networks. arXiv. Retrieved 2022-12-14, from http://arxiv.org/abs/1609.02907 (arXiv:1609.02907 [cs, stat]) Knijnenburg, B. P., Reijmer, N. J., & Willemsen, M. C. (2011, October). Each to his own: how different users call for different interaction methods in recommender systems. In Proceedings of the fifth ACM conference on Recommender systems (pp. 141–148). New York, NY, USA: Association for Computing Machinery. Retrieved 2023-01-11, from https://doi.org/10.1145/2043932.2043960 doi: 10.1145/2043932.2043960 Kop, M. (2021, November). EU Artificial Intelligence Act: The European Approach to AI. , 11. Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: tests in linear mixed effects models. Journal of statistical software, 82 (13). (Publisher: The Foundation for Open Access Statistics) Lam, S. K., Pitrou, A., & Seibert, S. (2015). Numba: A llvm-based python jit compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC (pp. 1–6). Ledwich, M., & Zaitsev, A. (2019). Algorithmic extremism: Examining YouTube’s rabbit hole of radicalization. arXiv preprint arXiv:1912.11211 . Leino, J., & R¨aih¨a, K.-J. (2007, October). Case amazon: ratings and reviews as part of recommendations. In Proceedings of the 2007 ACM conference on Recommender systems (pp. 137–140). New York, NY, USA: Association for Computing Machinery. Retrieved 2023-01-08, from https://doi.org/10.1145/1297231.1297255 doi: 10.1145/1297231.1297255 Linden, G., Smith, B., & York, J. (2003). Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, 7 (1), 76–80. (Publisher: Ieee) Maddala, G. S. (1971, March). The Use of Variance Components Models in Pooling Cross Section and Time Series Data. Econometrica, 39 (2), 341. Retrieved 2023-01-11, from https://www.jstor.org/stable/1913349 ?origin=crossref doi: 10.2307/1913349 McAuley, J., Targett, C., Shi, Q., & Hengel, A. v. d. (2015, June). Image-based Recommendations on Styles and Substitutes. arXiv. Retrieved 2022-12-29, from http://arxiv.org/abs/1506.04757 (arXiv:1506.04757 [cs]) McKinney, W. (2011). pandas: a foundational Python library for data analysis and statistics. Python for high performance and scientific computing, 14 (9), 1–9. (Publisher: Seattle) Millecamp, M., Htun, N. N., Jin, Y., & Verbert, K. (2018). Controlling spotify recommendations: effects of personal characteristics on music recommender user interfaces. In Proceedings of the 26th Conference on user modeling, adaptation and personalization (pp. 101–109). Nakagawa, S., Johnson, P. C., & Schielzeth, H. (2017). The coefficient of determination R 2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded. Journal of the Royal Society Interface, 14 (134), 20170213. (Publisher: The Royal Society) Nguyen, T. T., Hui, P.-M., Harper, F. M., Terveen, L., & Konstan, J. A. (2014). Exploring the filter bubble: the effect of using recommender systems on content diversity. In Proceedings of the 23rd international conference on World wide web - WWW ’14 (pp. 677–686). Seoul, Korea: ACM Press. Retrieved 2022-12-31, from http:// dl.acm.org/citation.cfm?doid=2566486.2568012 doi: 10.1145/2566486.2568012 Ni, J., Li, J., & McAuley, J. (2019). Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 188–197). Nonnecke, B., & Preece, J. (2000). Lurker demographics: Counting the silent. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 73–80). O’Hara, R., & Kotze, J. (2010, January). Do not log-transform count data. Nature Precedings. Retrieved 2023-01-11, from https://www.nature.com/articles/npre.2010.4136.1 doi: 10.1038/npre.2010.4136.1 Pandita, R. (2017). Internet: A change agent an overview of internet penetration & growth across the world. International Journal of Information Dissemination and Technology, 7 (2), 83–91. Retrieved 2023-01-11, from https://www.proquest.com/docview/1920419032/abstract/14B253C0DD6A4277PQ/1 (Num Pages: 83-91 Place: Ambala, India Publisher: Maharishi Markandehwar University) Papakyriakopoulos, O., Serrano, J. C. M., & Hegelich, S. (2020). Political communication on social media: A tale of hyperactive users and bias in recommender systems. Online Social Networks and Media, 15 , 100058. (Publisher: Elsevier) Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Dubourg, V. (2011). Scikit-learn: Machine learning in Python. the Journal of machine Learning research, 12 , 2825–2830. (Publisher: JMLR. org) Poole, M. A., & O’Farrell, P. N. (1971). The assumptions of the linear regression model. Transactions of the Institute of British Geographers, 145–158. (Publisher: JSTOR) Popper, K. R. (1970). Normal science and its dangers. Cambridge University Press Cambridge. Rakova, B., & Chowdhury, R. (2019, September). Human self-determination within algorithmic sociotechnical systems. arXiv. Retrieved 2022-09-07, from http://arxiv.org/abs/1909.06713 (arXiv:1909.06713 [cs]) Raschka, S., Patterson, J., & Nolet, C. (2020). Machine learning in python: Main developments and technology trends in data science, machine learning, and artificial intelligence. Information, 11 (4), 193. (Publisher: Multidisciplinary Digital Publishing Institute) Roski, J., Maier, E. J., Vigilante, K., Kane, E. A., & Matheny, M. E. (2021). Enhancing trust in AI through industry self-governance. Journal of the American Medical Informatics Association, 28 (7), 1582–1590. (Publisher: Oxford University Press) Sandvig, C., Hamilton, K., Karahalios, K., & Langbort, C. (2014). Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry, 22 , 4349–4357. Satterthwaite, F. E. (1946, December). An Approximate Distribution of Estimates of Variance Components. Biometrics Bulletin, 2 (6), 110. Retrieved 2023-01-11, from https://www.jstor.org/stable/10.2307/3002019 ?origin=crossref doi: 10.2307/3002019 Schielzeth, H., Dingemanse, N. J., Nakagawa, S., Westneat, D. F., Allegue, H., Teplitsky, C., . . . Araya-Ajoy, Y. G. (2020, September). Robustness of linear mixed-effects models to violations of distributional assumptions. Methods in Ecology and Evolution, 11 (9), 1141–1152. Retrieved 2022-12-27, from https://onlinelibrary.wiley.com/ doi/10.1111/2041-210X.13434 doi: 10.1111/2041-210X.13434 Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019, January). Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency (pp. 59–68). Atlanta GA USA: ACM. Retrieved 2022-09-07, from https://dl.acm.org/doi/10.1145/3287560.3287598 doi: 10.1145/3287560.3287598 Smith, B., & Linden, G. (2017, May). Two Decades of Recommender Systems at Amazon.com. IEEE Internet Computing, 21 (3), 12–18. Retrieved 2022-12-20, from http://ieeexplore.ieee.org/document/7927889/ doi: 10.1109/MIC.2017.72 Timmers, H., & Wagenaar, W. A. (1977, November). Inverse statistics and misperception of exponential growth. Perception & Psychophysics, 21 (6), 558–562. Retrieved 2023-01-10, from https://doi.org/10.3758/BF03198737 doi: 10.3758/BF03198737 Van Rossum, G. (2007). Python Programming language. In USENIX annual technical conference (Vol. 41, pp. 1–36). (Issue: 1) Villadsen, A. R., & Wulff, J. N. (2021, July). Statistical Myths About Log-Transformed Dependent Variables and How to Better Estimate Exponential Models. British Journal of Management, 32 (3), 779–796. Retrieved 2023-01-12, from https://onlinelibrary.wiley.com/doi/10.1111/1467-8551.12431 doi: 10.1111/1467-8551.12431 Wells, J. R., Danskin, G., & Ellsworth, G. (2018). Amazon. com, 2018. Harvard Business School Case Study(716-402). Wood, S. N., Pya, N., & S¨afken, B. (2016, October). Smoothing Parameter and Model Selection for General Smooth Models. Journal of the American Statistical Association, 111 (516), 1548–1563. Retrieved 2023- 01-12, from https://doi.org/10.1080/01621459.2016.1180986 (Publisher: Taylor & Francis eprint: https://doi.org/10.1080/01621459.2016.1180986) doi: 10.1080/01621459.2016.1180986 Zhu, F., & Liu, Q. (2018). Competing with complementors: An empirical look at Amazon.com. Strategic Management Journal, 39 (10), 2618–2642. Retrieved 2023-01-11, from https://onlinelibrary.wiley.com/doi/ abs/10.1002/smj.2932 ( eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/smj.2932) doi: 10.1002/ smj.2932 Zuboff, S. (2019). The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power: Barack Obama’s Books of 2019. Profile Books. This paper is under CC 4.0 license. available on arxiv

Ensuring Ethical AI: Lessons from Amazon's Barrier-to-Exit Analysis

This paper is available on arxiv under CC BY-SA 4.0 DEED license.
Authors:
(1) Lana L. Blaschke, Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, Potsdam Institute for Climate Impact Research, Potsdam, Germany and lana.blaschke@pik-potsdam.de;
(2) Da Nian, Potsdam Institute for Climate Impact Research, Potsdam, Germany;
(3) Sebastian Bathiany,  Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, and Potsdam Institute for Climate Impact Research, Potsdam, Germany;
(4)  Maya Ben-Yami,  Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, and Potsdam Institute for Climate Impact Research, Potsdam, Germany;
(5) Taylor Smith, Institute of Geosciences, University of Potsdam, 14476 Potsdam, Germany;
(6)  Chris A. Boulton, Global Systems Institute, University of Exeter, EX4 4QE Exeter, UK ;
(7) Niklas Boers, Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, Potsdam Institute for Climate Impact Research, Potsdam, Germany, Global Systems Institute, University of Exeter, EX4 4QE Exeter, UK, and Department of Mathematics, University of Exeter, EX4 4QF Exeter, UK. Table of Links Abstract and Introduction
Materials and Methods
Data
Results
Conclusions, Open Research Section and Conflict of Interest/Competing Interests
Author Contributions, Supplementary Information, Acknowledgments and References
Additional Insight from the Conceptual Model 
AMSR2 until 2022 Excluded Due to Missing Human Land Use Data
Investigation of Resilience Loss in AMSR2’s X-Band
Results Proving Robustness
Potential Driving Forces and Sources of Bias ADDITIONAL INSIGHT FROM THE CONCEPTUAL MODEL Text S1. Stability in the conceptual model The interaction of vegetation and precipitation is plotted in Figure S1. In the conceputal model, vegetation is a function of absolute incoming precipitation only and is not dependent on the cell’s location. Hence, it is the same for each cell and plotted as the single black solid line. Precipitation on the other hand is a function of the vegetation of the cell, but also of the vegetation of the cell’s eastern neighbors. Hence, it differs for all cells, which are encoded by the colors as in Figure 2. For the calculation of one cell’s precipitation as a function of vegetation, its eastern neighbors are considered to be in equilibrium. Furthermore, precipitation depends on the amount of moisture coming in from the Atlantic ocean, or in other words on the control parameter B. The solid lines represent the state in the beginning of the experiment with B = 1010, and the dashed lines in the end with B = 990. Where the black and coloured lines cross at some (V∗, P∗), the corresponding value of vegetation supports the value of precipitation P∗, but also is V∗ supported by P∗. Hence, this point is considered a fixed point, or equilibrium. As the moisture inflow is reduced in the simulations from 1010 (solid lines) to 990 (dashed lines), the fixed points with high vegetation values vanish, explaining the tipping in the Figures S4 and S5 . Text S2. Precipitation decrease in the conceptual model The results of simulations with decreasing moisture inflow from the Atlantic ocean (decreasing B(t)) were presented in the main text in Figure 4. Figure S2 zooms in into cell-wise total precipitation and highlights its stochasticity as well as the almost linear decrease up to the Tipping Point of vegetation. Text S3. Deforestation scenario leads to Tipping in the conceptual model The results of simulations with decreasing moisture inflow from the Atlantic ocean (decreasingB(t)) were presented in the main text in Figure 4. Figure S3 is identical but presents the results from simulations with unchanging inflow from the Atlantic, i.e. B(t) = 1000 for all t. Instead, the amount vegetation in the east-most cell is considered the bifurcation parameter and constantly decreased from 1 to 0, thereby mimicking deforestation. Text S4. Distribution of the trends for the three resilience indicators In the conceptual model, all indicators show clear trends when approaching the Tipping Point of vegetation, as illustrated in Figure S4. There, clearly all three indicators show a strong shift towards positive values with only very few negative ones for AC1. In a spatially extended system like the ARF, one would thus expect all three indicators to increase when the corresponding grid cell loses resilience. Text S5. Distribution of the trends for the three resilience indicators In Figure 4 the time of emergence (ToE) was defined as the first time when a trend becomes significant. Yet, as more conservative approach one can define the ToE as the time when a trend becomes permanently significant, i.e. when all trends later in time are significant as well. The results with this definition are displayed in Figure S5. The uncertainty in the timing of this type of ToE differs much less for AC1 compared to the definition in the main text. This indicates that AC1 might be more prone to false alarms. This paper is available on arxiv under CC BY-SA 4.0 DEED license. Authors: (1) Lana L. Blaschke, Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, Potsdam Institute for Climate Impact Research, Potsdam, Germany and lana.blaschke@pik-potsdam.de; (2) Da Nian, Potsdam Institute for Climate Impact Research, Potsdam, Germany; (3) Sebastian Bathiany,  Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, and Potsdam Institute for Climate Impact Research, Potsdam, Germany; (4)  Maya Ben-Yami,  Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, and Potsdam Institute for Climate Impact Research, Potsdam, Germany; (5) Taylor Smith, Institute of Geosciences, University of Potsdam, 14476 Potsdam, Germany; (6)  Chris A. Boulton, Global Systems Institute, University of Exeter, EX4 4QE Exeter, UK ; (7) Niklas Boers, Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, Potsdam Institute for Climate Impact Research, Potsdam, Germany, Global Systems Institute, University of Exeter, EX4 4QE Exeter, UK, and Department of Mathematics, University of Exeter, EX4 4QF Exeter, UK. This paper is available on arxiv under CC BY-SA 4.0 DEED license. Authors: Authors: (1) Lana L. Blaschke, Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, Potsdam Institute for Climate Impact Research, Potsdam, Germany and lana.blaschke@pik-potsdam.de; (2) Da Nian, Potsdam Institute for Climate Impact Research, Potsdam, Germany; (3) Sebastian Bathiany,  Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, and Potsdam Institute for Climate Impact Research, Potsdam, Germany; (4)  Maya Ben-Yami,  Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, and Potsdam Institute for Climate Impact Research, Potsdam, Germany; (5) Taylor Smith, Institute of Geosciences, University of Potsdam, 14476 Potsdam, Germany; (6)  Chris A. Boulton, Global Systems Institute, University of Exeter, EX4 4QE Exeter, UK ; (7) Niklas Boers, Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany, Potsdam Institute for Climate Impact Research, Potsdam, Germany, Global Systems Institute, University of Exeter, EX4 4QE Exeter, UK, and Department of Mathematics, University of Exeter, EX4 4QF Exeter, UK. Table of Links Abstract and Introduction Materials and Methods Data Results Conclusions, Open Research Section and Conflict of Interest/Competing Interests Author Contributions, Supplementary Information, Acknowledgments and References Additional Insight from the Conceptual Model AMSR2 until 2022 Excluded Due to Missing Human Land Use Data Investigation of Resilience Loss in AMSR2’s X-Band Results Proving Robustness Potential Driving Forces and Sources of Bias Abstract and Introduction Abstract and Introduction Materials and Methods Materials and Methods Data Data Results Results Conclusions, Open Research Section and Conflict of Interest/Competing Interests Conclusions, Open Research Section and Conflict of Interest/Competing Interests Author Contributions, Supplementary Information, Acknowledgments and References Author Contributions, Supplementary Information, Acknowledgments and References Additional Insight from the Conceptual Model Additional Insight from the Conceptual Model AMSR2 until 2022 Excluded Due to Missing Human Land Use Data AMSR2 until 2022 Excluded Due to Missing Human Land Use Data Investigation of Resilience Loss in AMSR2’s X-Band Investigation of Resilience Loss in AMSR2’s X-Band Results Proving Robustness Results Proving Robustness Potential Driving Forces and Sources of Bias Potential Driving Forces and Sources of Bias ADDITIONAL INSIGHT FROM THE CONCEPTUAL MODEL Text S1. Stability in the conceptual model The interaction of vegetation and precipitation is plotted in Figure S1. In the conceputal model, vegetation is a function of absolute incoming precipitation only and is not dependent on the cell’s location. Hence, it is the same for each cell and plotted as the single black solid line. Precipitation on the other hand is a function of the vegetation of the cell, but also of the vegetation of the cell’s eastern neighbors. Hence, it differs for all cells, which are encoded by the colors as in Figure 2. For the calculation of one cell’s precipitation as a function of vegetation, its eastern neighbors are considered to be in equilibrium. Furthermore, precipitation depends on the amount of moisture coming in from the Atlantic ocean, or in other words on the control parameter B. The solid lines represent the state in the beginning of the experiment with B = 1010, and the dashed lines in the end with B = 990. Where the black and coloured lines cross at some (V∗, P∗), the corresponding value of vegetation supports the value of precipitation P∗, but also is V∗ supported by P∗. Hence, this point is considered a fixed point, or equilibrium. As the moisture inflow is reduced in the simulations from 1010 (solid lines) to 990 (dashed lines), the fixed points with high vegetation values vanish, explaining the tipping in the Figures S4 and S5 . Text S2. Precipitation decrease in the conceptual model The results of simulations with decreasing moisture inflow from the Atlantic ocean (decreasing B(t)) were presented in the main text in Figure 4. Figure S2 zooms in into cell-wise total precipitation and highlights its stochasticity as well as the almost linear decrease up to the Tipping Point of vegetation. Text S3. Deforestation scenario leads to Tipping in the conceptual model The results of simulations with decreasing moisture inflow from the Atlantic ocean (decreasingB(t)) were presented in the main text in Figure 4. Figure S3 is identical but presents the results from simulations with unchanging inflow from the Atlantic, i.e. B(t) = 1000 for all t. Instead, the amount vegetation in the east-most cell is considered the bifurcation parameter and constantly decreased from 1 to 0, thereby mimicking deforestation. Text S4. Distribution of the trends for the three resilience indicators In the conceptual model, all indicators show clear trends when approaching the Tipping Point of vegetation, as illustrated in Figure S4. There, clearly all three indicators show a strong shift towards positive values with only very few negative ones for AC1. In a spatially extended system like the ARF, one would thus expect all three indicators to increase when the corresponding grid cell loses resilience. Text S5. Distribution of the trends for the three resilience indicators In Figure 4 the time of emergence (ToE) was defined as the first time when a trend becomes significant. Yet, as more conservative approach one can define the ToE as the time when a trend becomes permanently significant, i.e. when all trends later in time are significant as well. The results with this definition are displayed in Figure S5. The uncertainty in the timing of this type of ToE differs much less for AC1 compared to the definition in the main text. This indicates that AC1 might be more prone to false alarms.

Loss of Amazon Rainforest Resilience: Additional Insight from the Conceptual Model

Loss of Amazon Rainforest Resilience: Author Contributions

Loss of Amazon Rainforest Resilience: Results

How Useful are Computational Models to Traders in Their Decision Making?

In a landmark paper published March 29, 2024, in the journal, Entropy, a team of visionary researchers from VERSES AI has unveiled a groundbreaking framework that could pave the way for the emergence of a new form of artificial intelligence , operating on the principles of natural intelligence— one that arises from the dense interactions of networked intelligent agents, rather than from a single, monolithic system. based on a first-principles approach to multi-scale artificial intelligence The study, titled “Shared Protentions in Multi-Agent Active Inference,” represents a major milestone in VERSES AI’s ambitious roadmap towards “shared intelligence” and could have profound implications for the future of AI and human-machine cooperation. The Power of Active Inference and Shared Protentions The research, led by Mahault Albarracin and co-authored by Riddhi Jain, Toby St Clere Smithe, Daniel Friedman, Maxwell Ramstead, and renowned neuroscientist Dr. Karl Friston, pioneer of and the , draws upon insights from philosophy, biology, and mathematics to explore how shared goals and collective intelligence can emerge from the interactions of individual intelligent agents. Central to their framework is the concept of “shared protentions” — mutually attuned expectations about future states and actions that enable agents to coordinate their behaviors towards common objectives. Active Inference Free Energy Principle What are "Protentions?" The philosophical foundation of the work lies in the concept of “inner time consciousness” developed by the early 20th-century thinker Edmund Husserl. Husserl proposed that our subjective experience of the present moment always has a certain “temporal thickness” — it encompasses not just the immediate instant, but also traces of the just-past (which he called “retentions”) and anticipations of the near-future (termed “protentions”). The authors of the new study take this idea of an individual’s temporal experience and extend it to the level of a social group. They introduce the notion of “shared protentions” — the idea that members of a group can develop mutually attuned expectations about future states and actions, which then allow them to coordinate their behaviors. How Does it Work? To translate this philosophical concept into a scientific model, the researchers employ a framework from theoretical biology known as “active inference.” This approach views living organisms as continuously making inferences about the state of their environment and selecting actions that minimize surprises and uncertainty. The authors show how Husserl’s ideas can be mapped onto the components of an active inference model. Sensory inputs correspond to raw “hyletic data,” (impressions that form perceptions), while the inferred causes of those sensations make up what we perceive as our experience. Crucially, the model equips each individual with an internal representation that includes not just their own states but also those of other group members. This allows the model to capture how individuals infer each other’s mental states and anticipate their actions. Leveraging Category Theory for Unified Representation The most innovative aspect of the new framework is its integration of mathematical tools from a field known as category theory. The researchers now express how individuals make decisions using abstract math concepts known as “polynomial functors,” which represent their potential interactions and observations. They then use a technique called “sheaf theory” to stitch together the compatible parts of these individual models. This results in a unified picture of the group’s shared understanding, termed a “consensus topos” by the authors. Within this formal framework, the “shared protentions” that enable coordinated group action emerge spontaneously as a consequence of each individual’s understanding and reasoning being influenced by the need to maintain consistency with the group. In other words, the underlying alignment of expectations that allows individuals to work together towards common goals arises naturally from the mathematics of the model. An Example from Nature Think of it like a flock of birds flying together. Each bird is constantly adjusting its own movements based on what it sees the other birds doing. It doesn’t need to have a detailed plan or a leader telling it what to do — it just naturally falls into formation by following simple rules and paying attention to its neighbors. A Human Example - Alignment of Understanding and Anticipation We can also use the example of a sports team, like a basketball team, playing together on the court. Each player on the team has a mental model of the game — they understand the rules, the objectives, and the general strategy. But more importantly, they also have a sense of what their teammates are likely to do in any given situation. This understanding is built up through hours of practice and gameplay together. During a fast-paced game, there’s no time for the players to stop and discuss each play in detail. Instead, they rely on their shared understanding and anticipation of each other’s actions. The point guard, for instance, might make a split-second decision to pass the ball to a particular teammate, not because they discussed it beforehand, but because they anticipate that the teammate will be in the right position to receive the pass. This is a prime example of “shared protentions” in action. Each player’s individual actions are guided by their expectations of what their teammates will do. And these expectations are not just one-way — they’re reciprocal and self-reinforcing. The teammate anticipates the pass because they know the point guard is likely to make it, and the point guard makes the pass because they anticipate the teammate being ready for it. Over time, this interplay of expectations and actions leads to a highly coordinated style of play, where the team seems to move and think as one unit. And importantly, this coordination emerges not from any one player’s explicit instructions, but from the implicit alignment of each player’s understanding and anticipations. This is the essence of what the researchers’ mathematical model captures — It’s a powerful insight into the cognitive underpinnings of teamwork and collective action. how shared goals and coordinated action can arise naturally from the interplay of individual minds, guided by the need to maintain a coherent, mutually aligned understanding of the world. VERSES AI's Vision for Shared Intelligence The significance of this work extends far beyond the realm of academic theory. In a post on LinkedIn, Albarracin explained that the paper “sets the stage for .” This stage, which roughly corresponds to what is popularly known as “Artificial Super-Intelligence,” envisions the emergence of planetary-scale, distributed intelligences from the highly coordinated interactions of networked Intelligent Agents. The key insight of the “Shared Protentions” paper is that such collective intelligences are not just a distant dream but a natural consequence of the principles of Active Inference and the mathematical tools of category theory. VERSES’ Active Inference intelligence roadmap stage S4 of Shared Intelligence The Mathematics of Shared Intelligence To understand how this works, it’s helpful to step back and consider the basic premise of Active Inference. This framework, rooted in neuroscience and theoretical biology, views Intelligent Agents as constantly making deductions and drawing conclusions about the state of their environment and choosing to act in a way that closes the gap between expectation and reality — refining understanding to better anticipate what will happen. In the “Shared Protentions” paper, the authors show how this process of inference and action selection can be formally modeled using abstract mathematical objects called “polynomial functors” and “sheaves,” as explained above. Coordinating Intelligent Agents through Implicit Alignment By applying these tools to networks of interacting Intelligent Agents, the researchers demonstrate how individual agents can intuitively align their expectations and coordinate their behaviors, even without explicit communication. In essence, they provide a mathematical recipe for the unspoken emergence of shared goals and collective intelligence from the bottom up. The Role of HSML in Enabling Shared Intelligence But what makes this research truly groundbreaking is its integration with . HSML is a common language that enables seamless interoperability between Active Inference Intelligent Agents, humans, and the scores of devices that comprise the Internet of Things (IoT). By programming context into objects and spaces, HSML provides Active Inference Agents with real-time data about their environments, as well as the sensory inputs from IoT devices and robots. VERSES AI’s Hyperspace Modeling Language (HSML) Embedding Human Values into the Spatial Web The “Shared Protentions” paper lays the theoretical foundation for In other words, it provides a principled way to embed human values and objectives into the very fabric of the Spatial Web, ensuring that the collective intelligences that emerge from it are aligned with our goals and aspirations. translating high-level human directives, encoded in HSML, into the inherent alignments of expectations that guide the cooperative behaviors of Autonomous Intelligent Agents. The Far-Reaching Implications of VERSES AI's Research The implications of this research are staggering. As we move towards a future in which , from Smart Cities to healthcare to scientific research, the ability to create cooperative, context-aware intelligences that can seamlessly coordinate their actions in pursuit of common goals will be essential. The “Shared Protentions” paper, in conjunction with the — & , offers a roadmap for achieving this vision. Autonomous Intelligent Systems are increasingly integrated into every aspect of our lives Spatial Web Protocol HSTP HSML Challenges and Opportunities on the Path to Shared Intelligence Of course, there are still many challenges to be overcome. This robust, secure, and ethical framework, as well as the deployment of . However, the publication of this paper marks a crucial turning point in our understanding of what is possible. these technologies, will require ongoing collaboration between researchers, policymakers, and the broader public Natural Intelligence-A New Era in Artificial Intelligence As we stand at the threshold of a new era in artificial intelligence, the pioneering work of Friston, Albarracin, and their colleagues at and the offers a promising vision of future in which humans and machines can work together in previously unimaginable ways. By cracking the code of shared intelligence and providing the tools to embed the building blocks for it in the very infrastructure of our world, they are laying the foundation for a new paradigm in AI — one that operates with the same mechanics as biological intelligence, and one that could transform our society in ways we are only beginning to imagine. VERSES AI Spatial Web Foundation , and Visit my blog at https://deniseholt.us/ Subscribe to my Spatial Web AI Podcast on , YouTube , and more. Spotify Subscribe to my newsletter on . LinkedIn Also published . here

VERSES AI's Revolutionary Framework Lays Down The Path to Natural Shared Super-Intelligence

While everyone is focusing on the hot news about Microsoft's acqui-hiring Inflection AI in disguise and the shakeup at Stability AI, we'd like to concentrate on the exciting developments unfolding in the world of model architectures. For the hot news, check the Usual Suspects © section below. – a new architecture that rivals the famous Transformer-based models. Mamba's innovations address significant challenges in processing long sequences, a problem that has limited traditional models. Now, let's talk about Mamba Mamba leverages state-space models (SSMs)*, particularly excelling with its incorporation of Structured State Space (S4) models into a large language model (LLM) framework. This integration allows Mamba to achieve linear complexity scaling with sequence length, marking a significant advancement over the quadratic scaling seen in traditional Transformer-based models. So what is it? Its streamlined architecture incorporates selective SSM layers, enhancing both efficiency and flexibility. As a result, Additionally, Mamba efficiently processes extremely long sequences, surpassing earlier models in performance. it benefits from hardware-aware optimizations, maximizing the potential of contemporary GPU architectures. This means you can process much longer sequences without hitting memory or compute bottlenecks. Think about applications like genomic analysis, long-form content generation, and complex multi-modal data processing, all becoming more feasible with Mamba's power. *State-space models are mathematical frameworks that describe a system's dynamics in terms of its state variables and observations, capturing the evolution and uncertainty of processes over time.  SSMs are known for efficiency with long sequences. Mamba's ability to efficiently process long sequences while maintaining competitive performance has fueled research interest in adapting and extending the architecture for various domains. It seems Mamba’s architecture is getting more attention (Attention is all you need ;) – last week, three papers showcased exciting developments. The paper, , makes Mamba more suitable for deployment on resource-constrained devices by introducing an efficient 2D scanning method and a dual-pathway module for balanced global-local feature extraction. Results show a significant reduction in FLOPs while maintaining strong accuracy. EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba The paper, , extends Mamba to be a multi-modal large language model capable of jointly reasoning over vision and language. Experiments demonstrate competitive performance on vision-language tasks with faster inference speeds compared to Transformer-based models. Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference The paper, , presents a simplified Mamba-based architecture that addresses stability issues when scaling Mamba to larger sizes. The key innovation is EinFFT, a novel channel mixing technique that ensures stable optimization. SiMBA shows strong results on vision tasks and multivariate time series forecasting, closing the gap with state-of-the-art Transformers. SiMBA: Simplified Mamba-based Architecture for Vision and Multivariate Time series These three papers highlight the architectural flexibility and potential of the Mamba model, which is promising for future advancements in context window size and data type support. If you want to move beyond Transformers, that might be the way to go. You can find the Mamba repository here: https://github.com/state-spaces/mamba News From The Usual Suspects © https://x.com/emollick/status/1770289508589707625?s=20&embedable=true Microsoft Is Hungry Microsoft basically owns OpenAI ( — from Elon Musk’s against OpenAI). “Indeed, as the November 2023 drama was unfolding, Microsoft’s CEO boasted that it would not matter “[i]f OpenAI disappeared tomorrow.” He explained that “[w]e have all the IP rights and all the capability.” “We have the people, we have the compute, we have the data, we have everything.” “We are below them, above them, around them.” lawsuit Last February, Microsoft in Mistral and brought its newest AI model, Mistral Large, to Azure. invested Last week, Microsoft basically Inflection AI’s team, including two of its co-founders: Mustafa Suleyman and Karén Simonyan. Eric Newcomer at Microsoft’s creativity in non-acquisition strategies, such as partnering with and investing in companies like Inflection and OpenAI without formal acquisitions, sidestepping antitrust reviews. The Stratechery why many aspects of Inflection and its rapid ascent to Unicorn status were odd from the very beginning. Soma Somasegar from Madrona VC that Microsoft partners with Inflection.ai to innovate AI for consumers, aiming to transform its presence in the consumer market. acqui-hired points out explains thinks Stability AI Evokes Many Jokes About Instability This January, we published with the subtitle: “Investigating the Thin Line Between Genuine Innovation and Strategic Exaggeration in AI”. On March 23, after too many exaggerations Emad Mostaque from his position as CEO of Stability AI. According to him, he to concentrate on decentralized AI. What does it mean exactly — no one knows. Stability AI profile resigns wants OpenAI https://x.com/gdb/status/1772334028009652415?s=20&embedable=true Hugging Face announces the release of , the largest public domain dataset designed for training (LLMs. Encompassing 500 billion words across multiple languages including English, French, Dutch, Spanish, German, and Italian, it’s a multilingual collection sourced from diverse cultural heritage initiatives. This release aims to demonstrate the feasibility of developing open LLMs using copyright-free materials, supported by an international collaboration with organizations dedicated to open science in AI. Common Corpus Sam Altman Is Open-Sourcing His Orb The Worldcoin Foundation the Orb’s software (Orb is a device that scans irises to create unique digital IDs) open-source, aiming to enhance privacy and security in proving humanness online. The software, available under MIT/Apache 2.0 licenses on GitHub, supports World ID verification by processing images locally and ensuring data privacy through secure transfers. has made Enjoyed This Story? I write a weekly analysis of the AI world in the newsletter. We aim to equip you with comprehensive knowledge and historical insights so you can make informed decisions about AI and ML. Turing Post

Mamba Architecture: What Is It and Can It Beat Transformers?

In 2017, a got tired of its job and committed . Then, in 2022, a , after interacting with LaMDA, the company's chatbot, that technology had become conscious. Recently, it was reported that in 2021, a Tesla software engineer was attacked by an uncontrolled robot. This is without considering the fear that machines will steal jobs, or the classic fiction that machines will take over the world. robot suicide in a water fountain Google engineer declared Although we don't see machines as replicants in Blade Runner, people are predisposed to anthropomorphizing or attributing human qualities to non-humans [1]. It's a way of being able to analyze a scenario and bring it into our understanding: we give names to our boats and tornadoes, we talk to our pets, and say that the pleasant feeling they give us is love. It's easier to understand a system that declares in the first person "I'm sorry, I don't know how to help you" than to accept that the model we're talking to may be nothing more than a stochastic parrot. [2] The anthropomorphized interaction, therefore, relates to the phenomenon of "illusion of agency", a psychological concept that refers to the tendency to attribute autonomous action or control to entities that, in reality, do not possess such capabilities. In other words, it's the mistaken perception that an agent (be it a person, an animal, or a machine) is acting of its own free will, when in fact its actions are being determined by external factors (in the computational case, development rules). In situations where AI systems make flawed decisions or actions, there is a tendency to see these errors as "choices" of the entity rather than results of its programming and design by humans, not least because it has been reported that the selfish bias of a human being that leads someone to see the cause and responsibility for a negative outcome as not attributable to themselves can be observed even in human-machine interactions [3]. This change in perception is dangerously inclined to absolve human creators, operators, and regulators of their responsibility, not as a matter of judicial regulation (which remains a gap and has challenges in its realization not only because of the complexity of the subject but because Artificial Intelligence is often understood only as machine learning and the premise is not properly structured. Do we need stricter regulation? Do we need to take more risks?), but on a techno-ethical issue. Let's take a more extreme but true event from 2023: a user who was sentimentally attached to a chatbot committed suicide after sharing his thoughts with the bot and r to "turn your words into actions", as well as other messages. Would a court conviction of the developers of this product cause another user to behave in the same way with another chatbot, assuming this one has been deactivated, if the content of the messages and the affection are the same? It's not just a legal situation. It's a social, psychological, and technological education issue. eceiving the response The concept of humanizing AI is ambiguous, and a significant challenge lies in the absence of a universally accepted approach that dictates the best practices for designing and utilizing AI. While an interface that mimics human behavior can be more approachable, no clear boundaries are defining what should or shouldn't be done in a product. Ultimately, user rejection becomes the only limiting factor, although potential harm may manifest before the interface becomes too unfamiliar. A user-friendly interface is a reduction in the complexity of the system that operates behind it. But as long as there is no education about how systems work, the user will not be able to think critically about what they use. This doesn't mean that everyone should become a programmer, but at least understand that the output on their screen comes from a path of data collection, model development, and design. There is a set of rules taking place. Since humanization is an almost unconscious act on the part of us users, let's at least limit it with a little knowledge. And it's not easy to avoid anthropomorphism when communicating about AI, especially considering the standard language of the industry, the media, and everyday life itself: machine , computer , generative . Perceptions of Artificial Intelligence can be influenced by the specific use of language. learning vision modelling How AI is presented has "concrete impacts", particularly on the way people distribute responsibility and recognition for the work done. When described merely as a tool in the hands of humans, it tends to attribute greater responsibility and merit to certain individuals - such as the person operating the code. On the other hand, if AI is characterized with human traits - such as the ability to create - then it is seen as deserving of greater credit and responsibility, like an agent with independent thought and mental capacity. [4] When we read news stories about incidents or atypical events involving Artificial Intelligence, we often come across these terms. Also, the attribution of "intelligence" means that more blame or credit for the outcome of a task was attributed to a robot with autonomous behavior than to a non-autonomous robot, even if the autonomous behavior did not directly contribute to the task. [3] These studies therefore suggest that humans can assign responsibility to a computer or robot based on the machines' anthropomorphised mental capacities. The humanization of machines not only shifts the distinction between the responsibility of the device and that of its human creator, but the attribution of intentions or consciousness to a machine at the same time blurs the boundaries of what constitutes true autonomy and consciousness. However, the difficulty in imputing humanity and sentience to a machine lies not only in the fact that Artificial Intelligence is not capable of having it and when it says, for example, that it feels fear, it is actually emulating what it has learned, repeating that phrase without any kind of essence behind it. Even today, there is a heated debate about how to define consciousness. Our consciousness, as humans. Our understanding of how the brain works is still quite limited. We have considerable knowledge about fundamental chemistry: the way neurons activate, and the transmission of chemical signals. We also have a good understanding of the main functions of various brain areas. However, we have very little knowledge about how these functions orchestrate us. To some extent, theoretical speculation has supplanted detailed neurophysiological studies of what  happens in the brain. But what about beyond that? [5] Why do we have this magical notion of ourselves? Why does the same experience affect us differently? Is the same feeling felt in the same way by all of us? If being a human is something that, although we experience it, we still don't have a full understanding of what it is as a whole, how can we say that a machine also experiences this complexity? By elevating machines to human capabilities, we diminish the special character of people. For the 2023 Jabuti Prize, one of the highest honors in Brazilian literature, the Brazilian Book Chamber (CBL) decided to disqualify Frankenstein, an edition of the 1818 classic, from the Best Illustration category, since the artist reported having used AI solutions to develop the art. Ironically, one of the books recognized by the Prize in the Non-Fiction category deals with the impact of Artificial Intelligence on human beings (“Humanamente digital: Inteligência artificial centrada no humano”, something like "Humanly Digital: Human-Centered Artificial Intelligence" in English, by Cassio Pantaleone). On the one hand, we recognize the intertwining of the human experience with machines. On the other hand, we still haven't managed to validate whether an algorithm used as an art tool can be considered a valid method of creation or not, even though the artistic process, even if carried out by machine learning, requires the action (and appreciation of beauty and aesthetics) of a human being. Machines don't steal jobs if they aren't used indiscriminately to do so. Machines don't kill unless they are used as weapons. Nor do machines suffer or empathize, although their texts emulate this since they have been trained with data from us, loaded with feelings that only we can truly feel. They are almost the modern version of the Golem myth. How can humans relate to non-human intelligence? Anthropomorphism is a valid answer, but not the only one. And when it is used, it cannot absolve us of the real responsibility for its consequences, whether appropriate or not: us. Artificial Intelligence is, in the end, a mirror of ourselves. And if we're afraid of where it's going, it's actually because we're afraid of the path we're going to create. REFERENCES [1] Epley, N., Waytz, A., & Cacioppo, J. T. (2007). On seeing human: A three-factor theory of anthropomorphism. Psychological Review, 114(4), 864–886. https://doi.org/10.1037/0033-295X.114.4.864 [2] Shneiderman, B. & Muller, M. (2023) . On AI Antropomorphism https://medium.com/human-centered-ai/on-ai-anthropomorphism-abff4cecc5ae [3] Kawai, Y., Miyake, T., Park, J. et al. Anthropomorphism-based causal and responsibility attributions to robots. Sci Rep 13, 12234 (2023). https://doi.org/10.1038/s41598-023-39435-5 [4] Epstein, Z., Levine, S., Rand, D. G., & Rahwan, I. (2020). Who Gets Credit for AI-Generated Art? In iScience (Vol. 23, Issue 9, p. 101515). Elsevier BV. https://doi.org/10.1016/j.isci.2020.101515 [5] Goff, P. (2023). Understanding Consciousness Goes Beyond Exploring Brain Chemistry. Scientific American https://www.scientificamerican.com/article/understanding-consciousness-goes-beyond-exploring-brain-chemistry/ The following prompt generated the image used in this article: “Create a man looking in a mirror, but the mirror is shaped like a robot.” Sofia × DALL·E

No, You Are Not Afraid of Artificial Intelligence. You’re Afraid of Other People

না, আপনি কৃত্রিম বুদ্ধিমত্তাকে ভয় পান না। আপনি অন্য লোকেদের ভয় পান

Nein, Sie haben keine Angst vor künstlicher Intelligenz. Du hast Angst vor anderen Menschen

No, no le tienes miedo a la inteligencia artificial. Tienes miedo de otras personas

Non, vous n'avez pas peur de l'intelligence artificielle. Vous avez peur des autres

नहीं, आप आर्टिफिशियल इंटेलिजेंस से नहीं डरते। आप अन्य लोगों से डरते हैं

いいえ、人工知能を恐れることはありません。他人が怖い

아니요, 당신은 인공지능을 두려워하지 않습니다. 당신은 다른 사람을 두려워합니다

Não, você não tem medo da inteligência artificial. Você tem medo de outras pessoas

Нет, вы не боитесь искусственного интеллекта. Вы боитесь других людей

Hayır, Yapay Zekadan Korkmuyorsunuz. Başkalarından Korkuyorsunuz

Không, bạn không sợ trí tuệ nhân tạo. Bạn sợ người khác

不，你并不害怕人工智能。你害怕别人

Systems Interoperability Types: The Results of Our Tertiary Study and What It All Means

This paper is available on under CC 4.0 license. Arxiv Authors: (1) Md Masud Rana, Department of Mathematics, University of Kentucky; (2) Duc Duy Nguyen, Department of Mathematics, University of Kentucky & ducnguyen@uky.edu. Table of Links Abstract & Introduction Datasets and Results Methods Conclusion, Data and Software Availability, Competing interests, Acknowledgments & References 4 Conclusion The study of protein-protein interactions (PPIs) and the prediction of mutation-induced binding free energy changes are of great importance in understanding the molecular basis of biological processes. The application of geometric graph theory and atom-level graph coloring techniques provides a powerful framework for analyzing biomolecules and capturing their intricate relationships. By utilizing the concept of geometric subgraphs and constructing multi-scale weighted colored geometric subgraphs (MWCGS), we can effectively represent the structural and functional properties of PPIs. The site-specific MWCGS features allow us to extract meaningful patterns and characteristics, shedding light on the effects of mutations and the underlying molecular interactions. In this work, we developed a mutation-induced binding free energy change predictor, called GGL-PPI, by incorporating site-specific MWCGS features for PPIs and gradient-boosting trees. Our method demonstrates superior performance compared to existing methods. The model was validated on three datasets: AB-Bind S645, SKEMPI 1.0 S1131, and SKEMPI 2.0 S4169 and S8338, showcasing its robustness and effectiveness. Furthermore, GGL-PPI was evaluated on a blind test set, the Ssym dataset. To prevent data leakage between the test and training sets, the model was trained on a homology-reduced balanced training set Q3488. This approach ensures the reliability and fairness of the evaluation process. GGL-PPI exhibits the most unbiased and superior performance in predicting binding free energy changes for both direct and reverse mutations, outperforming other existing methods, particularly for reverse mutations. Overall, the results highlight the potential of the GGL-PPI approach in accurately predicting mutation-induced binding free energy changes in protein-protein interactions, providing valuable insights into the molecular mechanisms underlying protein-protein interactions and facilitating drug design and discovery efforts. 5 Data and Software Availability The source code is available at Github: https://github.com/NguyenLabUKY/GGL-Mutation. 6 Competing interests No competing interest is declared. 7 Acknowledgments This work is supported in part by funds from the National Science Foundation (NSF: # 2053284, # 2151802, and # 2245903), and the University of Kentucky Startup Fund. References [1] Dana Chuderland and Rony Seger. Protein-protein interactions in the regulation of the extracellular signal-regulated kinase. Mol. Biotechnol., 29:57–74, 2005. [2] Harry Jubb, Tom L Blundell, and David B Ascher. Flexibility and small pockets at protein–protein interfaces: new insights into druggability. Prog. Biophys. Mol., 119(1):2–9, 2015. [3] Mileidy W Gonzalez and Maricel G Kann. Chapter 4: Protein interactions and disease. PLoS Comput. Biol., 8(12):e1002819, 2012. [4] Cunliang Geng, Li C Xue, Jorge Roel-Touris, and Alexandre MJJ Bonvin. Finding the δδg spot: are predictors of binding affinity changes upon mutations in protein–protein interactions ready for it? Wiley Interdiscip. Rev. Comput. Mol. Sci., 9(5):e1410, 2019. [5] Rocco Moretti, Sarel J Fleishman, Rudi Agius, Mieczyslaw Torchala, Paul A Bates, Panagiotis L Kastritis, Joao PGLM Rodrigues, Mika¨el Trellet, Alexandre MJJ Bonvin, Meng Cui, et al. Community-wide evaluation of methods for predicting the effect of mutations on protein–protein interactions. Proteins: Struct. Funct. Genet., 81(11):1980–1987, 2013. [6] Jeffrey R Brender and Yang Zhang. Predicting the effect of mutations on protein-protein binding interactions through structure-based interface profiles. PLoS Comput. Biol., 11(10):e1004494, 2015. [7] Ning Zhang, Yuting Chen, Haoyu Lu, Feiyang Zhao, Roberto Vera Alvarez, Alexander Goncearenco, Anna R Panchenko, and Minghui Li. Mutabind2: predicting the impacts of single and multiple mutations on protein-protein interactions. Iscience, 23(3):100939, 2020. [8] Carlos HM Rodrigues, Yoochan Myung, Douglas EV Pires, and David B Ascher. mcsm-ppi2: predicting the effects of mutations on protein–protein interactions. Nucleic Acids Res., 47(W1):W338–W344, 2019. [9] Minghui Li, Marharyta Petukh, Emil Alexov, and Anna R Panchenko. Predicting the impact of missense mutations on protein–protein binding affinity. J. Chem. Theory Comput., 10(4):1770–1780, 2014. [10] Hafumi Nishi, Manoj Tyagi, Shaolei Teng, Benjamin A Shoemaker, Kosuke Hashimoto, Emil Alexov, Stefan Wuchty, and Anna R Panchenko. Cancer missense mutations alter binding properties of proteins and their interaction networks. PloS one, 8(6):e66273, 2013. [11] Mu Gao, Hongyi Zhou, and Jeffrey Skolnick. Insights into disease-associated mutations in the human proteome through protein structural analysis. Structure, 23(7):1362–1369, 2015. [12] Alessia David, Rozami Razali, Mark N Wass, and Michael JE Sternberg. Protein–protein interaction sites are hot spots for disease-associated nonsynonymous snps. Hum. Mutat., 33(2):359–363, 2012. [13] H Billur Engin, Jason F Kreisberg, and Hannah Carter. Structure-based analysis reveals cancer missense mutations target protein interaction interfaces. PLoS One, 11(4):e0152929, 2016. [14] Amelie Stein, Douglas M Fowler, Rasmus Hartmann-Petersen, and Kresten Lindorff-Larsen. Biophysical and mechanistic models for disease-causing protein variants. Trends Biochem. Sci., 44(7):575–588, 2019. [15] Stephanie Portelli, Jody E Phelan, David B Ascher, Taane G Clark, and Nicholas Furnham. Understanding molecular consequences of putative drug resistant mutations in mycobacterium tuberculosis. Sci. Rep., 8(1):1–12, 2018. [16] Sundeep Chaitanya Vedithi, Sony Malhotra, Madhusmita Das, Sheela Daniel, Nanda Kishore, Anuja George, Shantha Arumugam, Lakshmi Rajan, Mannam Ebenezer, David B Ascher, et al. Structural implications of mutations conferring rifampin resistance in mycobacterium leprae. Sci. Rep., 8(1):5016, 2018. [17] Vytautas Gapsys, Servaas Michielssens, Daniel Seeliger, and Bert L de Groot. Accurate and rigorous prediction of the changes in protein free energies in a large-scale mutation scan. Angew. Chem., Int. Ed., 55(26):7364–7368, 2016. [18] Elizabeth H Kellogg, Andrew Leaver-Fay, and David Baker. Role of conformational sampling in computing mutationinduced changes in protein structure and stability. Proteins: Structure, Function, and Bioinformatics, 79(3):830–838, 2011. [19] Brian J Bender, Alberto Cisneros III, Amanda M Duran, Jessica A Finn, Darwin Fu, Alyssa D Lokits, Benjamin K Mueller, Amandeep K Sangha, Marion F Sauer, Alexander M Sevy, et al. Protocols for molecular modeling with rosetta3 and rosettascripts. Biochemistry, 55(34):4748–4763, 2016. [20] Kurt S Thorn and Andrew A Bogan. Asedb: a database of alanine mutations and their effects on the free energy of binding in protein interactions. Bioinformatics, 17(3):284–285, 2001. [21] MD Shaji Kumar and M Michael Gromiha. Pint: protein–protein interactions thermodynamic database. Nucleic Acids Res., 34(suppl 1):D195–D198, 2006. [22] MD Shaji Kumar, K Abdulla Bava, M Michael Gromiha, Ponraj Prabakaran, Koji Kitajima, Hatsuho Uedaira, and Akinori Sarai. Protherm and pronit: thermodynamic databases for proteins and protein–nucleic acid interactions. Nucleic Acids Res., 34(suppl 1):D204–D206, 2006. [23] Iain H Moal and Juan Fern´andez-Recio. Skempi: a structural kinetic and energetic database of mutant protein interactions and its use in empirical models. Bioinformatics, 28(20):2600–2607, 2012. [24] Justina Jankauskait˙e, Brian Jim´enez-Garc´ıa, Justas Dapk¯unas, Juan Fern´andez-Recio, and Iain H Moal. Skempi 2.0: an updated benchmark of changes in protein–protein binding energy, kinetics and thermodynamics upon mutation. Bioinformatics, 35(3):462–469, 2019. [25] Sarah Sirin, James R Apgar, Eric M Bennett, and Amy E Keating. Ab-bind: antibody binding mutational database for computational affinity predictions. Protein Sci., 25(2):393–409, 2016. [26] Cunliang Geng, Anna Vangone, and Alexandre MJJ Bonvin. Exploring the interplay between experimental methods and the performance of predictors of binding affinity change upon mutations in protein complexes. Protein Eng. Des. Sel., 29(8):291–299, 2016. [27] Sherlyn Jemimah, K Yugandhar, and M Michael Gromiha. Proximate: a database of mutant protein–protein complex thermodynamics and kinetics. Bioinformatics, 33(17):2787–2788, 2017. [28] Douglas EV Pires, David B Ascher, and Tom L Blundell. mcsm: predicting the effects of mutations in proteins using graph-based signatures. Bioinformatics, 30(3):335–342, 2014. [29] Cunliang Geng, Anna Vangone, Gert E Folkers, Li C Xue, and Alexandre MJJ Bonvin. isee: Interface structure, evolution, and energy-based machine learning predictor of binding affinity changes upon mutations. Proteins: Structure, Function, and Bioinformatics, 87(2):110–119, 2019. [30] Sherlyn Jemimah, Masakazu Sekijima, and M Michael Gromiha. Proaffimuseq: sequence-based method to predict the binding free energy change of protein–protein complexes upon mutation using functional classification. Bioinformatics, 36(6):1725–1730, 2020. [31] Guangyu Zhou, Muhao Chen, Chelsea JT Ju, Zheng Wang, Jyun-Yu Jiang, and Wei Wang. Mutation effect estimation on protein–protein interactions using deep contextualized representation learning. NAR genom. bioinform., 2(2):lqaa015, 2020. [32] Alexey Strokach, Tian Yu Lu, and Philip M Kim. Elaspic2 (el2): combining contextualized language models and graph neural networks to predict effects of mutations. J. Mol. Biol., 433(11):166810, 2021. [33] Xianggen Liu, Yunan Luo, Pengyong Li, Sen Song, and Jian Peng. Deep geometric representations for modeling effects of mutations on protein-protein binding affinity. PLoS Comput. Biol., 17(8):e1009284, 2021. [34] Piero Fariselli, Pier Luigi Martelli, Castrense Savojardo, and Rita Casadio. Inps: predicting the impact of nonsynonymous variations on protein stability from sequence. Bioinformatics, 31(17):2816–2821, 2015. [35] Grant Thiltgen and Richard A Goldstein. Assessing predictors of changes in protein stability upon mutation using self-consistency. PLoS One, 7(10):e46084, 2012. [36] Fabrizio Pucci, Katrien V Bernaerts, Jean Marc Kwasigroch, and Marianne Rooman. Quantification of biases in predictions of protein stability changes upon mutations. Bioinformatics, 34(21):3659–3665, 2018. [37] Dinara R Usmanova, Natalya S Bogatyreva, Joan Ari˜no Bernad, Aleksandra A Eremina, Anastasiya A Gorshkova, German M Kanevskiy, Lyubov R Lonishin, Alexander V Meister, Alisa G Yakupova, Fyodor A Kondrashov, et al. Self-consistency test reveals systematic bias in programs for prediction change of stability upon mutation. Bioinformatics, 34(21):3653–3658, 2018. [38] Duc D Nguyen, Tian Xiao, Menglun Wang, and Guo-Wei Wei. Rigidity strengthening: A mechanism for protein– ligand binding. J. Chem. Inf. Model., 57(7):1715–1721, 2017. [39] Md Masud Rana and Duc Duy Nguyen. Geometric graph learning with extended atom-types features for proteinligand binding affinity prediction. Comput. Biol. Med., 164:107250, 2023. [40] Jian Jiang, Rui Wang, and Guo-Wei Wei. Ggl-tox: geometric graph learning for toxicity prediction. J. Chem. Inf. Model., 61(4):1691–1700, 2021. [41] Menglun Wang, Zixuan Cang, and Guo-Wei Wei. A topology-based network tree for the prediction of protein–protein binding affinity changes following mutation. Nat. Mach. Intell., 2(2):116–123, 2020. [42] Duan Chen, Zhan Chen, Changjun Chen, Weihua Geng, and Guo-Wei Wei. Mibpb: a software package for electrostatic analysis. J. Comput. Chem., 32(4):756–770, 2011. [43] Xiang Liu, Huitao Feng, Jie Wu, and Kelin Xia. Hom-complex-based machine learning (hcml) for the prediction of protein–protein binding affinity changes upon mutation. J. Chem. Inf. Model., 62(17):3961–3969, 2022. [44] Douglas EV Pires and David B Ascher. mcsm-ab: a web server for predicting antibody–antigen affinity changes upon mutation with graph-based signatures. Nucleic Acids Res., 44(W1):W469–W473, 2016. [45] Peng Xiong, Chengxin Zhang, Wei Zheng, and Yang Zhang. Bindprofx: assessing mutation-induced binding affinity change by protein interface profiles with pseudo-counts. J. Mol. Biol., 429(3):426–434, 2017. [46] Quanya Liu, Peng Chen, Bing Wang, Jun Zhang, and Jinyan Li. dbmpikt: a database of kinetic and thermodynamic mutant protein interactions. BMC bioinformatics, 19(1):1–7, 2018. [47] Bian Li, Yucheng T Yang, John A Capra, and Mark B Gerstein. Predicting changes in protein thermodynamic stability upon point mutation with deep 3d convolutional neural networks. PLoS Comp. Bio., 16(11):e1008291, 2020. [48] Lijun Quan, Qiang Lv, and Yang Zhang. Strum: structure-based prediction of protein stability changes upon single-point mutation. Bioinformatics, 32(19):2936–2946, 2016. [49] Leo Breiman. Random forests. Machine learning, 45:5–32, 2001. [50] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20:273–297, 1995. [51] Duc Duy Nguyen, Kaifu Gao, Menglun Wang, and Guo-Wei Wei. Mathdl: mathematical deep learning for d3r grand challenge 4. J. Comput. Aided. Mol. Des., 34(2):131–147, 2020. [52] Jerome H Friedman. Greedy function approximation: a gradient boosting machine. Annals of statistics, pages 1189–1232, 2001. This paper is available on under CC 4.0 license. Arxiv

PPIs and the Prediction of Mutation-Induced Binding Free Energy Changes: What It All Means

This paper is available on arxiv under CC 4.0 license. Authors: (1) Md Masud Rana, Department of Mathematics, University of Kentucky; (2) Duc Duy Nguyen, Department of Mathematics, University of Kentucky & ducnguyen@uky.edu. Table of Links Abstract & Introduction Datasets and Results Methods Conclusion, Data and Software Availability, Competing interests, Acknowledgments & References 2 Datasets and Results In this section, we perform validation and evaluation of our proposed models on several benchmark datasets. We develop two types of GGL-PPI models: GGL-PPI1 and GGL-PPI2. The first model, GGL-PPI1, is built solely on geometric graph features discussed in Section 3. On the other hand, GGL-PPI2 incorporates both geometric graph features and auxiliary features, as detailed by Wang et al. [41]. The electrostatic potential calculations for the auxiliary components are conducted using the MIBPB software [42]. 2.1 Validation To validate our models, we primarily consider the AB-Bind dataset [25], SKEMPI 1.0 dataset [23], and SKEMPI 2.0 dataset [24]. We employ a rigorous evaluation methodology by conducting a 10-times 10-fold cross-validation (CV) on each datasets. The mean Pearson correlation coefficient (R ) and root-mean-square error (RMSE) serve as our evaluation metrics. p In comparing the CV performance of our proposed models with other existing methods, we specifically assess TopNetTree [41], Hom-ML-V2 [43], and Hom-ML-V1 [43]. Both TopNetTree and Hom-ML-V2 incorporate auxiliary features in conjunction with their topology-based and Hom-complex-based features, respectively. On the other hand, Hom-ML-V1 solely relies on Hom-complex-based features without utilizing any auxiliary features. Validation on AB-Bind S645 Data Set The AB-Bind dataset contains 1,101 mutational data points for 32 antibody-antigen complexes, providing experimentally determined binding affinity changes upon mutations. Pires et al. curated a subset known as AB-Bind S645 [44], consisting of 645 single-point mutations observed in 29 antibody-antigen complexes. The dataset comprises a mix of stabilizing (20%) and destabilizing (80%) mutations. Additionally, the dataset includes 27 non-binders that do not show any binding within the assay’s sensitivity range. For these non-binders, the binding free energy changes have been uniformly set to a value of 8 kcal/mol. It is crucial to consider these non-binders as outliers during model development and evaluation to ensure model accuracy and robustness. Our GGL-PPI2 achieved an Rp of 0.58 on the AB-Bind S645 dataset, as shown in Figure 2a. The comparison results in Table 1 indicate that our model tied for second place with Hom-ML-V2 43, while TopNetTree [41] claimed the top position. However, when we exclude the 27 nonbinders from the dataset, our model outperforms all other existing models. Specifically, the Rp value increases to 0.74 from 0.58 after removing the nonbinders (Figure 2b). Furthermore, GGL-PI1, our purely geometric graph-based features model, demonstrated competitive performance with an Rp of 0.57 on the AB-Bind S645 dataset. Intriguingly, when excluding the nonbinders, GGL-PPI1 surpassed all other models with an improved Rp of 0.73. These performances reveal that our multiscale weighted colored geometric graphs can effectively characterize the wide range of interactions in biomolecular complexes. Validation on SKEMPI 1.0 S1131 Data Set The SKEMPI 1.0 dataset consists of a collection of 3,047 mutations of 158 complexes obtained from literature sources, where the complexes have experimentally determined structures [23]. The dataset includes both single-point mutations and multi-point mutations. Specifically, there are 2,317 entries in the dataset that represent single-point mutations, which are collectively known as the SKEMPI S2317 set. Additionally, a subset of 1,131 non-redundant interface single-point mutations has been selected from the SKEMPI S2317 set and labeled as the SKEMPI S1131 set [45]. This subset focuses on studying the impact of single-point mutations on protein-protein interactions. Table 1: Performance comparison of different methods in terms of Pearson correlation coefficients (Rp) for the AB-Bind (S645) dataset. Figure 2c shows that our model GGL-PPI2 achieves an Rp of 0.873 and an RMSE of 1.21 kcal/mol in 10-fold CV on the S1131 dataset. Table 2 presents the performance comparison of various methods on the S1131 dataset, including our proposed models, GGL-PPI1 and GGL-PPI2. Among them, our model, GGL-PPI2, achieved the highest performance, underscoring its superiority in predicting binding affinity changes due to mutation. Notably, even without auxiliary features, our GGL-PPI1 outperformed both TopNetTree and Hom-ML-V2 methods that do leverage auxiliary features. This again highlights the efficacy of our geometric graph-based molecular representation. Table 2: Performance comparison of different methods in terms of Pearson correlation coefficients (Rp) for the single-point mutations in the SKEMPI 1.0 (S1131) dataset. Validation on SKEMPI 2.0 S4169 and S8338 Data Sets The SKEMPI 2.0 dataset is an updated and expanded version of the original SKEMPI dataset, incorporating new mutations collected from various sources [24]. Released in 2018, it significantly increased in size, now containing a total of 7,085 entries, including both single-point and multi-point mutations. The data was obtained by merging several databases, including SKEMPI 1.0 [23], AB-Bind [25], PROXiMATE27, and dbMPIKT46. Additionally, new data from the literature were manually curated and added to the dataset. The mutations cover a wide range of protein complexes, such as protease-inhibitor, antibody-antigen, and TRCpMHC complexes. Among the mutations, approximately 3,000 are single-point alanine mutations, 2,000 are single-point non-alanine mutations, and another 2,000 involve multiple mutations. Notably, the authors of the mCSM-PPI2 [8] method filtered the single-point mutations, yielding S4169 set, comprising 4,169 variants in 139 different complexes/ The S8338 set, derived from S4169, represents hypothetical reverse mutation energy changes with negative values. This comprehensive dataset serves as a valuable resource for studying protein interactions and their thermodynamic properties. Perforamnce-wise, Our GGL-PPI2 model posts an Rp of 0.81 with an RMSE of 1.03 kcal/mol for the S4169 dataset as shown in Figure 2d, outstripping all existing models (Table 3). It is noteworthy that our GGL-PPI1 model, which solely relies on geometric graph-based features, demonstrated comparable performance to GGL-PPI2, outperforming TopNetTree and mCSM-PPI2 with an Rp of 0.80 and an RMSE of 1.06 kcal/mol. In the case of the S8338 dataset, we applied a stratified cross-validation approach similar to mCSM-PPI2. We ensured that hypothetical reverse mutations were consistently placed either in the training or test sets during the dataset splits, maintaining their relationship to the corresponding original mutations intact throughout the cross-validation process. GGL-PPI2 achieved an Rp of 0.85 with an RMSE of 1.07 kcal/mol as depicted in Figure 2e, and GGL-PPI1 closely followed, attaining an Rp of 0.84 with the same RMSE value. As Table 3 attests, our GGL-PPI2 is on par with TopNetTree and outperforms mCSM-PPI2 on the S8338 dataset. Table 3: Performance comparison of different methods in terms of Pearson correlation coefficients (Rp) for the single-point mutations in the SKEMPI 2.0 (S4169 and S8338) dataset. 2.2 Evaluation To evaluate our proposed model for predicting binding free energy (BFE) changes of protein-protein interactions, we consider two datasets sourced from the ProTherm database [22]. The first dataset, carefully selected by Pucci et al. [36], named S[sym] dataset. This data assembles 684 mutations from the ProTherm, comprising 342 direct mutations and their corresponding reverse mutations, resulting in a balanced dataset. The dataset specifically focuses on mutations in fifteen protein chains with solved 3D structures, ensuring highresolution data with a resolution of at least 2.5˚A. By providing experimentally measured ∆∆G values and a balanced representation of stabilizing and destabilizing mutations, the S[sym] dataset serves as a valuable resource for evaluating prediction biases in the context of predicting mutation-induced binding affinity changes. To address the issue of data leakage and enhance the generalization capability of our method, we employed the Q1744 dataset [47]. Quan et al. [48] compiled the Q3421 dataset from ProTherm, consisting of 3421 single-point mutations across 150 proteins with available PDB structures. However, the presence of homologous proteins in both the training and test set can lead to interdependent effects of mutations, compromising the model’s performance. To mitigate this, Li et al. [47] created the Q1744 dataset, derived by excluding overlapping data points and refining protein-level homology between Q3421 and S[sym] datasets, resulting in 1744 distinct mutations. Furthermore, the Q3488 dataset was created by augmenting reverse mutations in the Q1744 set. We utilized the Q3488 dataset as our training set, thereby enhancing our ∆∆G predictor’s capability to accurately predict BFE changes in PPIs. We conduct an evaluation of our model on the blind test set S[sym], with a distinct focus on both direct and reverse mutations. To assess the performance, we utilize the Pearson correlation coefficient and root-mean-square error as our primary metrics. Additionally, to discern any prediction bias, we incorporated two statistical measures: Rpdir−rev and δ. The former calculates the Pearson correlation between predictions for direct and reverse mutations, while the latter represents the sum of predicted ∆∆G values for both types of mutations. The hypothesis is that an unbiased predictor would yield Rpdir−rev = −1 and an average δ ( ¯δ) of 0 kcal/mol. Our main focus is to highlight the effectiveness of our model, GGL-PPI2, particularly emphasizing its robust geometric graph-based molecular featurization. GGL-PPI2 has demonstrated exceptional prediction accuracy, maintaining consistency for both direct and reverse mutations. As depicted in Figure 3a and 3b, our model achieves consistent Rp values of 0.57 and an RMSE of 1.28 kcal/mol, indicating its efficiency against overfitting to direct mutations. Additionally, the analysis reveals that a significant proportion of mutations fall within a prediction error of 0.5 kcal/mol and 1.0 kcal/mol, with 34.6% and 65.8% for direct mutations and 35.1% and 66.0% for reverse mutations, as depicted in Figure 3d and 3e. Furthermore, Figure 3c demonstrates that GGL-PPI2 effectively addresses prediction bias by achieving a nearly perfect Rpdir−rev value of -0.999 and an extremely low average ¯δ of 0.006 kcal/mol. Finally, the distribution plot in Figure 3f illustrates that 99.4% of mutations exhibit a prediction bias under 0.05 kcal/mol. In Table 4, we present the prediction results of our models and conduct a comprehensive comparison with other ∆∆G predictors. We observe that our GGL-PPI2 model outperforms ThermoNet [47], which was also trained on the homologyreduced set Q3488, across all evaluation measures. It outperforms ThermoNet by 21.3% for direct mutations and 18.7% for reverse mutations. Furthermore, the GGL-PPI1 model, which only uses geometric graph-based features, also performs better than ThermoNet in both direct and reverse prediction tasks. This further emphasizes the effectiveness of our geometric-graph approach. For a broader comparison against other ∆∆G predictors, we introduce the GGL-PPI2∗ model, trained on the Q6428 set constructed before the homology reduction of the set Q3421 [47]. As illustrated in Table 4, GGL-PPI2∗ excels over other methods in reverse mutation predictions. It is noteworthy that while some methods surpass GGL-PPI2∗ for direct mutations, they frequently exhibit significant bias towards reverse mutations. This paper is available on under CC 4.0 license. Arxiv

Our Datasets and Results From Our Study: GGL-PPI Models

আমাদের গবেষণা থেকে আমাদের ডেটাসেট এবং ফলাফল: GGL-PPI মডেল

Unsere Datensätze und Ergebnisse unserer Studie: GGL-PPI-Modelle

Nuestros conjuntos de datos y resultados de nuestro estudio: modelos GGL-PPI

Nos ensembles de données et résultats de notre étude : modèles GGL-PPI

हमारे डेटासेट और हमारे अध्ययन के परिणाम: जीजीएल-पीपीआई मॉडल

私たちのデータセットと研究結果: GGL-PPI モデル

연구 데이터 세트 및 결과: GGL-PPI 모델

Nossos conjuntos de dados e resultados de nosso estudo: modelos GGL-PPI

Наши наборы данных и результаты нашего исследования: модели GGL-PPI

Veri Setlerimiz ve Çalışmamızın Sonuçları: GGL-PPI Modelleri

Bộ dữ liệu và kết quả từ nghiên cứu của chúng tôi: Mô hình GGL-PPI

我们的数据集和研究结果：GGL-PPI 模型

Comparing Tools for Fitting Mutational Signatures: The Results of Our Work

This paper is available on arxiv under CC 4.0 license. Authors: (1) Jeffrey Ouyang-Zhang, UT Austin (2) Daniel J. Diaz, UT Austin (3) Adam R. Klivans, UT Austin (4) Philipp Krähenbühl, UT Austin Table of Links Abstract & Introduction Related Works Preliminary Methodology Results Limitations Conclusion, Acknowledgements, & References Experimental Setting & Additional Results 7 Conclusion We present a method that efficiently scales thermodynamic stability prediction from single mutations to higher-order mutations. Our key insight is that the effects of mutations on the same protein are correlated. Thus, for a target protein, it suffices to run a deep backbone once and decode the effect of all mutations simultaneously using a shallow decoder. With the AlphaFold model as our backbone, our method outperforms existing methods on a variety of single and multiple mutation benchmarks. Our method scales to millions of mutations with minimal computational overhead and runs in a fraction of the time it would take prior works. 8 Acknowledgements This work is supported by the NSF AI Institute for Foundations of Machine Learning (IFML). References [1] Randomization of genes by pcr mutagenesis. Genome research 2(1), 28–33 (1992) 2 [2] Adams, J.P., Brown, M.J., Diaz-Rodriguez, A., Lloyd, R.C., Roiban, G.D.: Biocatalysis: A Pharma Perspective. Advanced Synthesis and Catalysis 361(11), 2421–2432 (2019). https://doi.org/10.1002/adsc.201900424 1 [3] Ahdritz, G., Bouatta, N., Kadyan, S., Xia, Q., Gerecke, W., O’Donnell, T.J., Berenberg, D., Fisk, I., Zanichelli, N., Zhang, B., Nowaczynski, A., Wang, B., Stepniewska-Dziubinska, M.M., Zhang, S., Ojewole, A., Guney, M.E., Biderman, S., Watkins, A.M., Ra, S., Lorenzo, P.R., Nivon, L., Weitzner, B., Ban, Y.E.A., Sorger, P.K., Mostaque, E., Zhang, Z., Bonneau, R., AlQuraishi, M.: Openfold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization. bioRxiv (2022). https://doi.org/10.1101/2022.11.20.517210 6 [4] Arnold, F.H.: Design by directed evolution. Accounts of chemical research 31(3), 125–131 (1998) 1, 2, 3 [5] Bell, E.L., Finnigan, W., France, S.P., Green, A.P., Hayes, M.A., Hepworth, L.J., Lovelock, S.L., Niikura, H., Osuna, S., Romero, E., Ryan, K.S., Turner, N.J., Flitsch, S.L.: Biocatalysis. Nature Reviews Methods Primers 1(1), 1–21 (2021). https://doi.org/10.1038/s43586-021-00044-z 1 [6] Benevenuta, S., Pancotti, C., Fariselli, P., Birolo, G., Sanavia, T.: An antisymmetric neural network to predict free energy changes in protein variants. Journal of Physics D: Applied Physics 54(24), 245403 [7] Benevenuta, S., Pancotti, C., Fariselli, P., Birolo, G., Sanavia, T.: An antisymmetric neural network to predict free energy changes in protein variants. Journal of Physics D: Applied Physics 54(24), 245403 (2021) 8 [8] Benevenuta, S., Birolo, G., Sanavia, T., Capriotti, E., Fariselli, P.: Challenges in predicting stabilizing variations: An exploration. Frontiers in Molecular Biosciences 9, 1075570 (2023) 5, 6, 17 [9] Brückner, A., Polge, C., Lentze, N., Auerbach, D., Schlattner, U.: Yeast two-hybrid, a powerful tool for systems biology. International journal of molecular sciences 10(6), 2763–2788 (2009) 1 [10] Cadet, F., Saavedra, E., Syren, P.O., Gontero, B.: Machine learning, epistasis, and protein engineering: From sequence-structure-function relationships to regulation of metabolic pathways. Frontiers in Molecular Biosciences 9, 1098289 (2022) 1 [11] Capriotti, E., Fariselli, P., Casadio, R.: I-mutant2. 0: predicting stability changes upon mutation from the protein sequence or structure. Nucleic acids research 33(suppl_2), W306–W310 (2005) 3, 8 [12] Chen, K., Arnold, F.H.: Tuning the activity of an enzyme for unusual environments: sequential random mutagenesis of subtilisin e for catalysis in dimethylformamide. Proceedings of the National Academy of Sciences 90(12), 5618–5622 (1993) 1, 3 [13] Chen, T., Gong, C., Diaz, D.J., Chen, X., Wells, J.T., Wang, Z., Ellington, A., Dimakis, A., Klivans, A., et al.: Hotprotein: A novel framework for protein thermostability prediction and editing. In: The Eleventh International Conference on Learning Representations (2022) 3 [14] Chen, Y., Lu, H., Zhang, N., Zhu, Z., Wang, S., Li, M.: Premps: Predicting the impact of missense mutations on protein stability. PLoS computational biology 16(12), e1008543 (2020) 3, 8 [15] Cheng, J., Randall, A., Baldi, P.: Prediction of protein stability changes for single-site mutations using support vector machines. Proteins: Structure, Function, and Bioinformatics 62(4), 1125–1132 (2006) 3, 8 [16] Dehghanpoor, R., Ricks, E., Hursh, K., Gunderson, S., Farhoodi, R., Haspel, N., Hutchinson, B., Jagodzinski, F.: Predicting the effect of single and multiple mutations on protein structural stability. Molecules 23(2), 251 (2018) 4, 7 [17] Dehouck, Y., Kwasigroch, J.M., Gilis, D., Rooman, M.: Popmusic 2.1: a web server for the estimation of protein stability changes upon mutation and sequence optimality. BMC bioinformatics 12(1), 1–12 (2011) 3, 8 [18] Diaz, D.J., Gong, C., Ouyang-Zhang, J., Loy, J.M., Wells, J.T., Yang, D., Ellington, A.J., Dimakis, A., Klivans, A.R.: Stability oracle: A structure-based graph-transformer for identifying stabilizing mutations (2023). https://doi.org/10.1101/2023.05.15.540857 3, 4, 7, 8, 16, 17, 18, 19 [19] Diaz, D.J., Kulikova, A.V., Ellington, A.D., Wilke, C.O.: Using machine learning to predict the effects and consequences of mutations in proteins. Current Opinion in Structural Biology 78, 102518 (2023) 3 [20] d’Oelsnitz, S., Diaz, D.J., Acosta, D.J., Schechter, M.W., Minus, M.B., Howard, J.R., Do, H., Loy, J., Alper, H., Ellington, A.D.: Synthetic microbial sensing and biosynthesis of amaryllidaceae alkaloids. bioRxiv pp. 2023–04 (2023) 3 [21] d’Oelsnitz, S., Kim, W., Burkholder, N.T., Javanmardi, K., Thyer, R., Zhang, Y., Alper, H.S., Ellington, A.D.: Using fungible biosensors to evolve improved alkaloid biosyntheses. Nature Chemical Biology 18(9), 981–989 (2022) 1 [22] Ellefson, J.W., Gollihar, J., Shroff, R., Shivram, H., Iyer, V.R., Ellington, A.D.: Synthetic evolutionary origin of a proofreading reverse transcriptase. Science 352(6293), 1590–1593 (2016) 1 [23] Feng, X., Sanchis, J., Reetz, M.T., Rabitz, H.: Enhancing the efficiency of directed evolution in focused enzyme libraries by the adaptive substituent reordering algorithm. Chemistry–A European Journal 18(18), 5646–5654 (2012) 3 [24] Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J.K., Brock, K., Gal, Y., Marks, D.S.: Disease variant prediction with deep generative models of evolutionary data. Nature 599(7883), 91–95 (2021) 9 [25] Gebauer, M., Skerra, A.: Engineered protein scaffolds as next-generation therapeutics. Annual Review of Pharmacology and Toxicology 60, 391–415 (2020). https://doi.org/10.1146/annurev-pharmtox-010818- 021118 1 [26] Gerasimavicius, L., Livesey, B.J., Marsh, J.A.: Correspondence between functional scores from deep mutational scans and predicted effects on protein stability. bioRxiv pp. 2023–02 (2023) 9 [27] Giver, L., Gershenson, A., Freskgard, P.O., Arnold, F.H.: Directed evolution of a thermostable esterase. Proceedings of the National Academy of Sciences 95(22), 12809–12813 (1998) 2, 3 [28] Hie, B.L., Shanker, V.R., Xu, D., Bruun, T.U., Weidenbacher, P.A., Tang, S., Wu, W., Pak, J.E., Kim, P.S.: Efficient evolution of human antibodies from general protein language models. Nature Biotechnology (2023) 3 [29] Jaroszewicz, W., Morcinek-Orłowska, J., Pierzynowska, K., Gaffke, L., W˛egrzyn, G.: Phage display and other peptide display technologies. FEMS Microbiology Reviews 46(2), fuab052 (2022) 1 [30] Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS) 20(4), 422–446 (2002) 6, 17 [31] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al.: Highly accurate protein structure prediction with alphafold. Nature 596(7873), 583–589 (2021) 2, 3, 4, 6, 9, 10 [32] Kellogg, E.H., Leaver-Fay, A., Baker, D.: Role of conformational sampling in computing mutation-induced changes in protein structure and stability. Proteins: Structure, Function, and Bioinformatics 79(3), 830–838 (2011) 3 [33] Klabunde, T., Petrassi, H.M., Oza, V.B., Raman, P., Kelly, J.W., Sacchettini, J.C.: Rational design of potent human transthyretin amyloid disease inhibitors. Nature structural biology 7(4), 312–321 (2000) 1, 2 [34] Kouba, P., Kohout, P., Haddadi, F., Bushuiev, A., Samusevich, R., Sedlar, J., Damborsky, J., Pluskal, T., Sivic, J., Mazurenko, S.: Machine learning-guided protein engineering. ACS Catalysis 13, 13863–13895 (2023) 3 [35] Laimer, J., Hofer, H., Fritz, M., Wegenkittl, S., Lackner, P.: Maestro-multi agent stability prediction upon point mutations. BMC bioinformatics 16(1), 1–13 (2015) 3, 4, 7, 8 [36] Li, B., Yang, Y.T., Capra, J.A., Gerstein, M.B.: Predicting changes in protein thermodynamic stability upon point mutation with deep 3d convolutional neural networks. PLoS computational biology 16(11), e1008291 (2020) 3, 4, 8 [37] Li, G., Panday, S.K., Alexov, E.: Saafec-seq: a sequence-based method for predicting the effect of single point mutations on protein thermodynamic stability. International journal of molecular sciences 22(2), 606 (2021) 3 [38] Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S., et al.: Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv (2022) 3, 4, 6, 9, 10, 18 [39] Loell, K., Nanda, V.: Marginal protein stability drives subcellular proteome isoelectric point. Proceedings of the National Academy of Sciences 115(46), 11778–11783 (2018) 1 [40] Lu, H., Diaz, D.J., Czarnecki, N.J., Zhu, C., Kim, W., Shroff, R., Acosta, D.J., Alexander, B.R., Cole, H.O., Zhang, Y., et al.: Machine learning-aided engineering of hydrolases for pet depolymerization. Nature 604(7907), 662–667 (2022) 3 [41] Meghwanshi, G.K., Kaur, N., Verma, S., Dabi, N.K., Vashishtha, A., Charan, P.D., Purohit, P., Bhandari, H.S., Bhojak, N., Kumar, R.: Enzymes for pharmaceutical and therapeutic applications. Biotechnology and Applied Biochemistry 67(4), 586–601 (2020). https://doi.org/10.1002/bab.1919 1 [42] Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., Rives, A.: Language models enable zero-shot prediction of the effects of mutations on protein function. Advances in Neural Information Processing Systems 34, 29287–29303 (2021) 9, 18 [43] Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., Steinegger, M.: Colabfold: making protein folding accessible to all. Nature methods 19(6), 679–682 (2022) 6 [44] Montanucci, L., Capriotti, E., Frank, Y., Ben-Tal, N., Fariselli, P.: Ddgun: an untrained method for the prediction of protein stability changes upon single and multiple point variations. BMC bioinformatics 20, 1–10 (2019) 3, 4, 7, 8, 16, 18 [45] Montanucci, L., Savojardo, C., Martelli, P.L., Casadio, R., Fariselli, P.: On the biases in predictions of protein stability changes upon variations: the inps test case. Bioinformatics 35(14), 2525–2527 (2019) 16 [46] Musdal, Y., Govindarajan, S., Mannervik, B.: Exploring sequence-function space of a poplar glutathione transferase using designed information-rich gene variants. Protein Engineering, Design and Selection 30(8), 543–549 (2017) 3 [47] Nijkamp, E., Ruffolo, J., Weinstein, E.N., Naik, N., Madani, A.: Progen2: exploring the boundaries of protein language models. arXiv preprint arXiv:2206.13517 (2022) 9 [48] Nikam, R., Kulandaisamy, A., Harini, K., Sharma, D., Gromiha, M.M.: Prothermdb: thermodynamic database for proteins and mutants revisited after 15 years. Nucleic acids research 49(D1), D420–D424 (2021) 2, 16 [49] Notin, P., Dias, M., Frazer, J., Hurtado, J.M., Gomez, A.N., Marks, D., Gal, Y.: Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. In: International Conference on Machine Learning. pp. 16990–17017. PMLR (2022) 2, 3, 9, 16, 18 [50] Notin, P.M., Van Niekerk, L., Kollasch, A.W., Ritter, D., Gal, Y., Marks, D.: Trancepteve: Combining family-specific and family-agnostic models of protein sequences for improved fitness prediction. bioRxiv pp. 2022–12 (2022) 9 [51] Nutschel, C., Fulton, A., Zimmermann, O., Schwaneberg, U., Jaeger, K.E., Gohlke, H.: Systematically scrutinizing the impact of substitution sites on thermostability and detergent tolerance for bacillus subtilis lipase a. Journal of chemical information and modeling 60(3), 1568–1584 (2020) 9 [52] Paik, I., Ngo, P.H., Shroff, R., Diaz, D.J., Maranhao, A.C., Walker, D.J., Bhadra, S., Ellington, A.D.: Improved bst dna polymerase variants derived via a machine learning approach. Biochemistry (2021) 3 [53] Pak, M.A., Markhieva, K.A., Novikova, M.S., Petrov, D.S., Vorobyev, I.S., Maksimova, E.S., Kondrashov, F.A., Ivankov, D.N.: Using alphafold to predict the impact of single mutations on protein stability and function. Plos one 18(3), e0282689 (2023) 3 [54] Pancotti, C., Benevenuta, S., Birolo, G., Alberini, V., Repetto, V., Sanavia, T., Capriotti, E., Fariselli, P.: Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset. Briefings in Bioinformatics 23(2), bbab555 (2022) 2, 8, 16, 17 [55] Pancotti, C., Benevenuta, S., Repetto, V., Birolo, G., Capriotti, E., Sanavia, T., Fariselli, P.: A deep-learning sequence-based method to predict protein stability changes upon genetic variations. Genes 12(6), 911 (2021) 8 [56] Pires, D.E., Ascher, D.B., Blundell, T.L.: Duet: a server for predicting effects of mutations on protein stability using an integrated computational approach. Nucleic acids research 42(W1), W314–W319 (2014) 3, 8 [57] Pires, D.E., Ascher, D.B., Blundell, T.L.: mcsm: predicting the effects of mutations in proteins using graph-based signatures. Bioinformatics 30(3), 335–342 (2014) 8 [58] Pucci, F., Schwersensky, M., Rooman, M.: Artificial intelligence challenges for predicting the impact of mutations on protein stability. Current opinion in structural biology 72, 161–168 (2022) 17 [59] Qiu, Y., Wei, G.W.: Persistent spectral theory-guided protein engineering. Nature Computational Science pp. 1–15 (2023) 3, 6, 17 [60] Rao, R.M., Liu, J., Verkuil, R., Meier, J., Canny, J., Abbeel, P., Sercu, T., Rives, A.: Msa transformer. In: International Conference on Machine Learning. pp. 8844–8856. PMLR (2021) 3, 9, 10 [61] Riesselman, A.J., Ingraham, J.B., Marks, D.S.: Deep generative models of genetic variation capture the effects of mutations. Nature methods 15(10), 816–822 (2018) 9 [62] Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C.L., Ma, J., Fergus, R.: Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS (2019). https://doi.org/10.1101/622803 3, 7 [63] Rodrigues, C.H., Pires, D.E., Ascher, D.B.: Dynamut: predicting the impact of mutations on protein conformation, flexibility and stability. Nucleic acids research 46(W1), W350–W355 (2018) 3, 8 [64] Rodrigues, C.H., Pires, D.E., Ascher, D.B.: Dynamut2: Assessing changes in stability and flexibility upon single and multiple point missense mutations. Protein Science 30(1), 60–69 (2021) 3, 4, 7 [65] Sanavia, T., Birolo, G., Montanucci, L., Turina, P., Capriotti, E., Fariselli, P.: Limitations and challenges in protein stability prediction upon genome variations: towards future applications in precision medicine. Computational and Structural Biotechnology Journal 18, 1968–1979 (2020). https://doi.org/10.1016/j.csbj.2020.07.011, https://doi.org/10.1016/j.csbj.2020.07.011 6, 17 [66] Savojardo, C., Fariselli, P., Martelli, P.L., Casadio, R.: Inps-md: a web server to predict stability of protein variants from sequence and structure. Bioinformatics 32(16), 2542–2544 (2016) 3, 8 [67] Schymkowitz, J., Borg, J., Stricher, F., Nys, R., Rousseau, F., Serrano, L.: The foldx web server: an online force field. Nucleic acids research 33(suppl_2), W382–W388 (2005) 3, 7, 8, 18 [68] Shroff, R., Cole, A.W., Diaz, D.J., Morrow, B.R., Donnell, I., Annapareddy, A., Gollihar, J., Ellington, A.D., Thyer, R.: Discovery of novel gain-of-function mutations guided by structure-based deep learning. ACS synthetic biology 9(11), 2927–2935 (2020) 3 [69] Starr, T.N., Thornton, J.W.: Epistasis in protein evolution. Protein science 25(7), 1204–1218 (2016) 1 [70] Steinegger, M., Soding, J.: Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology 35(11), 1026–1028 (2017) 6 [71] Stemmer, W.P.: Rapid evolution of a protein in vitro by dna shuffling. Nature 370(6488), 389–391 (1994) 2, 3 [72] Tsuboyama, K., Dauparas, J., Chen, J., Laine, E., Mohseni Behbahani, Y., Weinstein, J.J., Mangan, N.M., Ovchinnikov, S., Rocklin, G.J.: Mega-scale experimental analysis of protein folding stability in biology and protein design. bioRxiv pp. 2022–12 (2022) 1, 2, 6, 16 [73] Tung, J.W., Heydari, K., Tirouvanziam, R., Sahaf, B., Parks, D.R., Herzenberg, L.A., Herzenberg, L.A.: Modern flow cytometry: a practical approach. Clinics in laboratory medicine 27(3), 453–468 (2007) 1 [74] Umerenkov, D., Shashkova, T.I., Strashnov, P.V., Nikolaev, F., Sindeeva, M., Ivanisenko, N.V., Kardymon, O.L.: Prostata: Protein stability assessment using transformers. bioRxiv pp. 2022–12 (2022) 3, 4, 6, 7, 8, 18, 19 [75] Wittmann, B.J., Johnston, K.E., Wu, Z., Arnold, F.H.: Advances in machine learning for directed evolution. Current opinion in structural biology 69, 11–18 (2021) 3 [76] Wittmann, B.J., Yue, Y., Arnold, F.H.: Informed training set design enables efficient machine learningassisted directed protein evolution. Cell Systems 12(11), 1026–1045 (2021) 3 [77] Worth, C.L., Preissner, R., Blundell, T.L.: Sdm—a server for predicting effects of mutations on protein stability and malfunction. Nucleic acids research 39(suppl_2), W215–W222 (2011) 3, 8 [78] Wu, S., Snajdrova, R., Moore, J.C., Baldenius, K., Bornscheuer, U.T.: Biocatalysis: Enzymatic Synthesis for Industrial Applications. Angewandte Chemie - International Edition 60(1), 88–119 (2021). https://doi.org/10.1002/anie.202006648 1 [79] Yeung, N., Lin, Y.W., Gao, Y.G., Zhao, X., Russell, B.S., Lei, L., Miner, K.D., Robinson, H., Lu, Y.: Rational design of a structural and functional nitric oxide reductase. Nature 462(7276), 1079–1082 (2009) 1, 2 [80] Zhou, B., Lv, O., Yi, K., Xiong, X., Tan, P., Hong, L., Wang, Y.G.: Accurate and definite mutational effect prediction with lightweight equivariant graph neural networks. arXiv preprint arXiv:2304.08299 (2023) 3 [81] Zhou, Y., Pan, Q., Pires, D.E., Rodrigues, C.H., Ascher, D.B.: Ddmut: predicting effects of mutations on protein stability using deep learning. Nucleic Acids Research p. gkad472 (2023) 3

Predicting a Protein’s Stability
under a Million Mutations: Conclusion, Acknowledgement & References

Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References ACKNOWLEDGMENTS The proof for the equidistant case was adapted from a proof by David J. Fleet and Allan D. Jepson from the University of Toronto [18]. REFERENCES [1] C. Haley, “Missing-data nonparametric coherency estimation,” IEEE Signal Processing Letters, vol. 28, pp. 1704–1708, 2021. [2] S. Dilmaghani, I. C. Henry, P. Soonthornnonda, E. R. Christensen, and R. C. Henry, “Harmonic analysis of environmental time series with missing data or irregular sample spacing,” Environmental science & technology, vol. 41, no. 20, pp. 7030–7038, 2007. [3] J. W. Cooley and J. W. Tukey, “An algorithm for the machine calculation of complex fourier series,” Mathematics of computation, vol. 19, no. 90, pp. 297–301, 1965. [4] F. Rasheed, P. Peng, R. Alhajj, and J. Rokne, “Fourier transform based spatial outlier mining,” in Intelligent Data Engineering and Automated Learning-IDEAL 2009: 10th International Conference, Burgos, Spain, September 23-26, 2009. Proceedings 10. Springer, 2009, pp. 317–324. [5] C. Van Loan, Computational frameworks for the fast Fourier transform. SIAM, 1992. [6] H. Carslaw, “A historical note on gibbs’ phenomenon in fourier’s series and integrals,” Bulletin of the American Mathematical Society, 1925. [7] J. Markel, “Fft pruning,” IEEE transactions on Audio and Electroacoustics, vol. 19, no. 4, pp. 305–311, 1971. [8] J. C. Bowman and Z. Ghoggali, “The partial fast fourier transform,” Journal of Scientific Computing, vol. 76, pp. 1578–1593, 2018. [9] M. Kircheis and D. Potts, “Direct inversion of the nonequispaced fast fourier transform,” Linear Algebra and its Applications, vol. 575, pp. 106–140, 2019. [10] S. Kunis and D. Potts, “Stability results for scattered data interpolation by trigonometric polynomials,” SIAM Journal on Scientific Computing, vol. 29, no. 4, pp. 1403–1419, 2007. [11] K. B. Petersen, M. S. Pedersen et al., “The matrix cookbook,” Technical University of Denmark, vol. 7, no. 15, p. 510, 2008. [12] C. M. Bishop, Neural networks for pattern recognition. Oxford university press, 1995. [13] H. Wickham, M. Averick, J. Bryan, W. Chang, L. D. McGowan, R. Franc¸ois, G. Grolemund, A. Hayes, L. Henry, J. Hester, M. Kuhn, T. L. Pedersen, E. Miller, S. M. Bache, K. Muller, J. Ooms, D. Robin- ¨ son, D. P. Seidel, V. Spinu, K. Takahashi, D. Vaughan, C. Wilke, K. Woo, and H. Yutani, “Welcome to the tidyverse,” Journal of Open Source Software, vol. 4, no. 43, p. 1686, 2019. [14] R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2023. [Online]. Available: https://www.R-project.org/ [15] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del R´ıo, M. Wiebe, P. Peterson, P. Gerard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, ´ H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sep. 2020. [Online]. Available: https://doi.org/10.1038/s41586-020-2649-2 [16] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors, “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,” Nature Methods, vol. 17, pp. 261– 272, 2020. [17] J. D. Hunter, “Matplotlib: A 2d graphics environment,” Computing in Science & Engineering, vol. 9, no. 3, pp. 90–95, 2007. [18] “Notes on fourier analysis,” http://www.cs.toronto.edu/ ∼jepson/csc320/notes/fourier.pdf, accessed: 2023-09-01. This paper is under CC 4.0 license. available on arxiv

Groundbreaking Research in Signal Theory

This paper is under CC 4.0 license. available on arxiv : Authors (1) Linda Pigureddu, University of Turin, Italy, 332079@edu.unito.it; (2) Cristina Gena, Dept. of Computer Science, University of Turin, Italy, cristina.gena@unito.it. Table of Links Introduction The Project Memes Conclusion and References CONCLUSION The analysis of the interactions and dialogues that took place during the therapeutic laboratory for autonomies endorsed with the inclusion of the Pepper robot provided an interesting perspective on the communication dynamics of children in the lab. The results obtained showed the effectiveness of the robot in promoting autonomy and functional acquisition, demonstrating that the therapeutic method based on assistive robotics is a valuable resource to support the rehabilitation needs of communication and social skills of autistic children. The analysis highlights new possibilities for their engagement and active participation as a co-designer, as we already experienced in the past with neuro-typical children [19], representing an innovative opportunity to promote the development and wellness of autistic people and opening to new perspectives for therapeutic intervention more aware of the needs of autistic minds. In conclusion, this paper highlights the need to provide the robot with the ability of adapt to the children’s peculiarity and features and sharing her/his vocabulary, especially by recognizing and using memes during interactions, to inspire greater trust in children and allowing the use of common slang, already in use with classmates, allowing them to consider Pepper a peer and effectively insert it in the role of mediator. In addition, it would allow children to use a simplified type of communication during meetings to compensate for the typical communication deficits associated with autism spectrum disorders. In the future we will work in this directions (co-design, user’s adaptations, using and recognizing memes during children-robot communication) having the robot able to communicate and express adapting to user tastes and preferences and and try again to field a real-world evaluation that takes into account the effectiveness of different levels of user adaptation [23]. REFERENCE [1] M. Biondi e M. Maj, DSM-5: diagnostic and statistical manual of mental disorders: text revision, 5. ed. Milano: Raffaello Cortina. [2] E. Hollander, R. Hagerman, e C. Ferretti, Textbook of Autism Spectrum Disorders, Second Edition. American Psychiatric Association Publishing, 2022. [3] ISS - Istituto superiore di sanità, «Osservatorio Nazionale Autismo», OssNA. https://osservatorionazionaleautismo.iss.it [4] P. Pennisi et al., «Autism and social robotics: A systematic review», Autism Research, vol. 9, fasc. 2, pp. 165–183, 2016, doi: 10.1002/aur.1527. [5] S. Shamsuddin, H. Yussof, S. Mohamed, e F. A. Hanapiah, «Design and Ethical Concerns in Robotic Adjunct Therapy Protocols for Children with Autism», Procedia Computer Science, vol. 42, pp. 9–16, gen. 2014, doi: 10.1016/j.procs.2014.11.027. [6] K. Dautenhahn e I. Werry, «Towards Interactive Robots in Autism Therapy: Background, Motivation and Challenges», Pragmatics & Cognition, vol. 12, pp. 1–35, giu. 2004, doi: 10.1075/pc.12.1.03dau. [7] C. Li, Q. Jia, e Y. Feng, «Human-Robot Interactoin Design for Robot-Assisted Intervention for Children with Autism Based on E-S Theory», in 2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, China: IEEE, ago. 2016, pp. 320–324. doi: 10.1109/IHMSC.2016.103. [8] A. Peca et al., «Robot Enhanced Therapy for Children with Autism Disorders: Measuring Ethical Acceptability», IEEE Technol. Soc. Mag., vol. 35, fasc. 2, pp. 54–66, giu. 2016, doi: 10.1109/MTS.2016.2554701. [9] B. Robins, K. Dautenhahn, e J. Dubowski, «Does appearance matter in the interaction of children with autism with a humanoid robot?», IS, vol. 7, fasc. 3, pp. 479–512, nov. 2006, doi:10.1075/is.7.3.16rob. [10] E. S. Kim, «Robots for social skills therapy in autism: evidence and designs toward clinical utility», 2013, doi: 10.13140/2.1.3038.2083. [11] «Autismo», Neuraxpharm Italia. https://www.neuraxpharm.com/it/disordini/autismo [12] L. Hull, K. V. Petrides, e W. Mandy, «The Female Autism Phenotype and Camouflaging: a Narrative Review», Rev J Autism Dev Disord, vol. 7, fasc. 4, pp. 306–317, dic. 2020, doi: 10.1007/s40489-020-00197-9. [13] D. Cohen et al., «NAO, a humanoid robot as a therapeutic mediator for young people with autism». [14] Cristina Gena, Claudio Mattutino, Andrea Maieli, Elisabetta Miraglio, Giulia Ricciardiello, Rossana Damiano, Alessandro Mazzei: Autistic Children's Mental Model of an Humanoid Robot. UMAP (Adjunct Publication) 2021: 128-129 [15] Rossana Damiano, Cristina Gena, Andrea Maieli, Claudio Mattutino, Alessandro Mazzei, Elisabetta Miraglio, Giulia Ricciardiello: UX Personas for Defining Robot's Character and Personality 213-216. IUI Workshops 2022: 213-216. [16] Cristina Gena, Claudio Mattutino, Stefania Brighenti, Andrea Meirone, Francesco Petriglia, Loredana Mazzotta, Federica Liscio, Matteo Nazzario, Valeria Ricci, Camilla Quarato, Cesare Pecone, Giuseppe Piccinni: et al., Sugar, Salt & Pepper - Humanoid robotics for autism, in Joint Proceedings of the ACM IUI 2021 Workshops co-located with 26th ACM Conference on Intelligent User Interfaces (ACM IUI 2021), College Station, United States, April 13-17, 2021, D. Glowacka e V. R. Krishnamurthy, A c. di, in CEUR Workshop Proceedings, vol. 2903. CEUR-WS.org, 2021. [17] Cristina Gena, Rossana Damiano, Claudio Mattutino, Alessandro Mazzei, Andrea Meirone, Loredana Mazzotta, Matteo Nazzario, Valeria Ricci, Stefania Brighenti, Federica Liscio, Francesco Petriglia: Preliminary results of a therapeutic lab for promoting autonomies in autistic children. arXiv, doi: 10.48550/arXiv.2305.02982. [18] B. C. Carbo, «The Use of Social Stories with Individuals with Autism Spectrum Disorders», University of Delaware ProQuest Dissertations Publishing, Newark, DE, 2005. [Online]. Disponibile su: https://udspace.udel.edu/server/api/core/bitstreams/cd5e8850-e57c-41d2-a7b4ac5e25a4b659/content [19] V. Cietto, C. Gena, I. Lombardi, C. Mattutino and C. Vaudano, "Co-designing with kids an educational robot," 2018 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), Genova, Italy, 2018, pp. 139-140, doi: 10.1109/ARSO.2018.8625810. [20] A. Mousa, «Sherry Turkle, Reclaiming Conversation. The Power of Talk in a Digital Age», Questions de communication, fasc. 34, Art. fasc. 34, dic. 2018, doi:10.4000/questionsdecommunication.16891. [21] H.-M. Chiang e M. Carter, «Spontaneity of Communication in Individuals with Autism», J Autism Dev Disord, vol. 38, fasc. 4, pp. 693–705, apr. 2008, doi: 10.1007/s10803-007-0436-7. [22] Gena, C., Cena, F., Vernero, F., Grillo, P. The evaluation of a social adaptive website for cultural events. User Model User-Adap Inter 23, 89–137 (2013) https://doi.org/10.1007/s11257-012-9129-9 [23] Gena, Cristina, and Ilaria Torre. "The importance of adaptivity to provide onboard services: A preliminary evaluation of an adaptive tourist information service onboard vehicles." Applied Artificial Intelligence 18.6 (2004): 549-580.

The Power of Memes: The Pepper Robot as a Communicative Aid for Autistic Children - Conclusion

This paper is under CC 4.0 license. available on arxiv Authors: (1) Shreyash Mishra has an equal contribution from IIIT Sri City, India; (2) S Suryavardan has an equal contribution from IIIT Sri City, India; (3) Megha Chakraborty, University of South Carolina, USA; (4) Parth Patwa, UCLA, USA; (5) Anku Rani, University of South Carolina, USA; (6) Aman Chadha, work does not relate to a position at Amazon from Stanford University, USA, or Amazon AI, USA; (7) Aishwarya Reganti, CMU, USA; (8) Amitava Das, University of South Carolina, USA; (9) Amit Sheth, University of South Carolina, USA; (10) Manoj Chinnakotla, Microsoft, USA; (11) Asif Ekbal, IIT Patna, India; (12) Srijan Kumar, Georgia Tech, USA. Table of Links Abstract & Introduction Related Work Task Details Participating systems Results Conclusion and Future Work and References 6. Conclusion and Future Work In this paper, we summarize the approaches used by the participants for the Memotion 3 task and analyze the results. Due to the multi-modal nature of the dataset, all teams use a pre-trained image and text embedding models. However, each team presents a novel model pipeline. The highest scores achieved in Task A, Task B and Task C of Memotion 3.0 are 34.41%, 79.77% and 59.82% respectively, which shows there is significant room for improvement. On analysis of the results and the mis-classified examples on the test set, we find that "Sarcasm" and "Humour" are difficult to identify, especially in code-mixed memes. While we address Hind-English code-mixed memes in this paper, future work could include exploring other languages/language pairs. A unified baseline model to analyze memes in multiple languages could also be an interesting possibility. References [1] R. Dawkins, The selfish gene, Granada Publishing Lim, 1979. [2] A. Marwick, Memes, Contexts 12 (2013) 12–13. [3] N. Akhther, Internet memes as form of cultural discourse: A rhetorical analysis on facebook, 2018. doi:10.31234/osf.io/sx6t7. [4] S. Suryawanshi, B. R. Chakravarthi, M. Arcan, P. Buitelaar, Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text, in: TRAC, 2020. [5] S. Mishra, S. Suryavardan, P. Patwa, M. Chakraborty, A. Rani, A. Reganti, A. Chadha, A. Das, A. Sheth, M. Chinnakotla, et al., Memotion 3: Dataset on sentiment and emotion analysis of codemixed hindi-english memes, arXiv preprint arXiv:2303.09892 (2023). [6] C. Sharma, D. Bhageria, W. Scott, S. PYKL, A. Das, T. Chakraborty, et al., SemEval-2020 task 8: Memotion analysis- the visuo-lingual metaphor!, in: SemEval, 2020. [7] P. Patwa, S. Ramamoorthy, N. Gunti, S. Mishra, S. Suryavardan, A. Reganti, A. Das, T. Chakraborty, A. Sheth, A. Ekbal, et al., Findings of memotion 2: Sentiment and emotion analysis of memes, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, ceur, 2022. [8] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, C. Potts, Recursive deep models for semantic compositionality over a sentiment treebank, in: EMNLP, 2013. [9] A. I. Saad, Opinion mining on us airline twitter data using machine learning techniques, in: 2020 16th international computer engineering conference (ICENCO), IEEE, 2020. [10] M. Alzyout, E. A. Bashabsheh, H. Najadat, A. Alaiad, Sentiment analysis of arabic tweets about violence against women using machine learning, in: 12th ICICS, 2021. [11] E. Prabhakar, M. Santhosh, A. H. Krishnan, T. Kumar, R. Sudhakar, Sentiment analysis of us airline twitter data using new adaboost approach, (IJERT) 7 (2019). [12] S. T. Kokab, S. Asghar, S. Naz, Transformer-based deep learning models for the sentiment analysis of social media data, Array 14 (2022) 100157. [13] S. G. Tesfagergish, J. Kapočiut¯ e-Dzikien ̇ e, R. Damaševičius, Zero-shot emotion detection ̇ for semi-supervised sentiment analysis using sentence transformers and ensemble learning, Applied Sciences (2022). [14] K. L. Tan, C. P. Lee, K. M. Lim, K. S. M. Anbananthen, Sentiment analysis with ensemble hybrid deep learning model, IEEE Access 10 (2022) 103694–103704. [15] L. Yue, W. Chen, X. Li, W. Zuo, M. Yin, A survey of sentiment analysis in social media, Knowledge and Information Systems 60 (2019). [16] K. Chakraborty, S. Bhattacharyya, R. Bag, A survey of sentiment analysis from social media data, IEEE Transactions on Computational Social Systems 7 (2020) 450–464. [17] L. Chiruzzo, S. Castro, M. Etcheverry, D. Garat, J. J. Prada, A. Rosá, Overview of haha at iberlef 2019: Humor analysis based on human annotation., in: IberLEF@ SEPLN, 2019. [18] E. Öhman, M. Pàmies, K. Kajava, J. Tiedemann, Xed: A multilingual dataset for sentiment analysis and emotion detection, 2020. arXiv:2011.01612. [19] F. A. Acheampong, C. Wenyu, H. Nunoo-Mensah, Text-based emotion detection: Advances, challenges, and opportunities, Engineering Reports (2020). [20] Z. Waseem, D. Hovy, Hateful symbols or hateful people? predictive features for hate speech detection on Twitter, in: NAACL, 2016. [21] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, et al., Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval), arXiv:1903.08983 (2019). [22] P. Patwa, M. Bhardwaj, V. Guptha, G. Kumari, S. Sharma, S. PYKL, A. Das, A. Ekbal, M. S. Akhtar, T. Chakraborty, Overview of constraint 2021 shared tasks: Detecting english covid-19 fake news and hindi hostile posts, in: Combating Online Hostile Posts in Regional Languages during Emergency Situation, Springer International Publishing, 2021, pp. 42–53. [23] R. Kumar, A. K. Ojha, S. Malmasi, M. Zampieri, Benchmarking aggression identification in social media, in: TRAC workshop, 2018. [24] R. Kumar, A. K. Ojha, S. Malmasi, M. Zampieri, Evaluating aggression identification in social media, in: TRAC workshop, 2020. [25] R. Kumar, S. Ratan, S. Singh, E. Nandi, L. N. Devi, et al., The ComMA dataset v0.2: Annotating aggression and bias in multilingual social media discourse, in: LREC, 2022. [26] B. Gambäck, U. K. Sikdar, Using convolutional neural networks to classify hate-speech, in: Proceedings of the first workshop on abusive language online, 2017, pp. 85–90. [27] A. Ribeiro, N. Silva, Inf-hateval at semeval-2019 task 5: Convolutional neural networks for hate speech detection against women and immigrants on twitter, in: SemEval, 2019. [28] K. Winter, R. Kern, Know-center at semeval-2019 task 5: multilingual hate speech detection on twitter using cnns, in: Semeval, 2019. [29] P. Patwa, S. Pykl, A. Das, P. Mukherjee, V. Pulabaigari, Hater-O-genius aggression classification using capsule networks, in: Proceedings of the 17th International Conference on Natural Language Processing (ICON), 2020. [30] A. C. Mazari, N. Boudoukhani, A. Djeffal, Bert-based ensemble learning for multi-aspect hate speech detection, Cluster Computing (2023) 1–15. [31] N. S. Samghabadi, P. Patwa, S. Pykl, P. Mukherjee, A. Das, T. Solorio, Aggression and misogyny detection using bert: A multi-task approach, in: Proceedings of the second workshop on trolling, aggression and cyberbullying, 2020. [32] J. Risch, R. Krestel, Bagging BERT models for robust aggression identification, in: Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, 2020. [33] S. Nagar, F. A. Barbhuiya, K. Dey, Towards more robust hate speech detection: using social context and user data, Social Network Analysis and Mining 13 (2023) 47. [34] A. Laddha, M. Hanoosh, D. Mukherjee, P. Patwa, A. Narang, Understanding chat messages for sticker recommendation in messaging apps, Proceedings of the AAAI Conference on Artificial Intelligence 34 (2020). doi:10.1609/aaai.v34i08.7019. [35] A. Laddha, M. Hanoosh, D. Mukherjee, P. Patwa, A. Narang, Large scale multilingual sticker recommendation in messaging apps, AI Magazine 42 (2022). doi:10.1609/aaai.12023. [36] P. Patwa, G. Aguilar, S. Kar, S. Pandey, S. PYKL, B. Gambäck, T. Chakraborty, T. Solorio, A. Das, SemEval-2020 task 9: Overview of sentiment analysis of code-mixed tweets, in: Proceedings of the Fourteenth Workshop on Semantic Evaluation, 2020. [37] B. R. Chakravarthi, et al., Findings of the shared task on offensive language identification in Tamil, Malayalam, and Kannada, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, 2021. [38] B. R. Chakravarthi, V. Muralidaran, R. Priyadharshini, J. P. McCrae, Corpus creation for sentiment analysis in code-mixed tamil-english text, arXiv:2006.00206 (2020). [39] B. R. Chakravarthi, N. Jose, S. Suryawanshi, E. Sherly, J. P. McCrae, A sentiment analysis dataset for code-mixed malayalam-english, arXiv preprint arXiv:2006.00210 (2020). [40] A. Hande, R. Priyadharshini, B. R. Chakravarthi, KanCMD: Kannada CodeMixed dataset for sentiment analysis and offensive language detection, in: Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media, 2020. [41] S. Dowlagar, R. Mamidi, Graph convolutional networks with multi-headed attention for code-mixed sentiment analysis, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, 2021, pp. 65–72. [42] J. Risch, A. Stoll, M. Ziegele, R. Krestel, hpidedis at germeval 2019: Offensive language identification using a german bert model., in: KONVENS, 2019. [43] D. Tula, P. Potluri, S. Ms, S. Doddapaneni, P. Sahu, R. Sukumaran, P. Patwa, Bitions@DravidianLangTech-EACL2021: Ensemble of multilingual language models with pseudo labeling for offence detection in Dravidian languages, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, 2021. [44] D. Tula, M. S. Shreyas, V. Reddy, P. Sahu, S. Doddapaneni, P. Potluri, R. Sukumaran, P. Patwa, Offence detection in dravidian languages using code-mixing index-based focal loss, SN Computer Science 3 (2022). doi:10.1007/s42979-022-01190-1. [45] Y. Ma, L. Zhao, J. Hao, XLP at SemEval-2020 task 9: Cross-lingual models with focal loss for sentiment analysis of code-mixing language, in: Semeval, 2020. [46] M. Ali, S. T. Kandukuri, S. Manduru, P. Patwa, A. Das, Pesto: Switching point based dynamic and relative positional encoding for code-mixed languages (student abstract), AAAI (2022). doi:10.1609/aaai.v36i11.21587. [47] A. Hu, S. Flaxman, Multimodal sentiment analysis to explore the structure of emotions, in: KDD, 2018. [48] R. Jha, V. Kaki, V. Kolla, S. Bhagat, P. Patwa, A. Das, S. Pal, Image2tweet: Datasets in Hindi and English for generating tweets from images, in: Proceedings of the 18th International Conference on Natural Language Processing (ICON), 2021. [49] R. Gomez, J. Gibert, L. Gomez, D. Karatzas, Exploring hate speech detection in multimodal publications, 2019. arXiv:1910.03814. [50] B. Zadeh, et al., Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph, in: ACL, 2018. [51] D. Kiela, H. Firooz, A. Mohan, V. Goswami, A. Singh, P. Ringshia, D. Testuggine, The hateful memes challenge: Detecting hate speech in multimodal memes, Neurips (2020). [52] S. Suryawanshi, B. R. Chakravarthi, Findings of the shared task on troll meme classification in Tamil, in: Speech and Language Technologies for Dravidian Languages, 2021. [53] E. Hossain, O. Sharif, M. M. Hoque, MUTE: A multimodal dataset for detecting hateful memes, in: Proceedings of the 2nd AACL Student Research Workshop, 2022. [54] Z. Xie, L. Liu, Y. Wu, L. Zhong, L. Li, Learning text-image joint embedding for efficient cross-modal retrieval with deep feature engineering, ACM Transactions on Information Systems 40 (2021) 1–27. URL: https://doi.org/10.1145%2F3490519. doi:10.1145/3490519. [55] V. Krishna, S. Suryavardan, S. Mishra, S. Ramamoorthy, P. Patwa, M. Chakraborty, A. Chadha, A. Das, A. Sheth, Imaginator: Pre-trained image+text joint embeddings using word-level grounding of images, 2023. arXiv:2305.10438. [56] N. Gunti, S. Ramamoorthy, P. Patwa, A. Das, Memotion analysis through the lens of joint embedding (student abstract), Proceedings of the AAAI Conference on Artificial Intelligence 36 (2022). doi:10.1609/aaai.v36i11.21616. [57] J. Lu, D. Batra, D. Parikh, S. Lee, Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks, 2019. arXiv:1908.02265. [58] L. H. Li, M. Yatskar, D. Yin, C.-J. Hsieh, K.-W. Chang, Visualbert: A simple and performant baseline for vision and language, 2019. arXiv:1908.03557. [59] S. Ramamoorthy, N. Gunti, S. Mishra, S. Suryavardan, A. Reganti, P. Patwa, A. Das, T. Chakraborty, A. Sheth, A. Ekbal, et al., Memotion 2: Dataset on sentiment and emotion analysis of memes, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, 2022. [60] K. N. Phan, G.-S. Lee, H.-J. Yang, S.-H. Kim, Little flower at memotion 2.0 2022: Ensemble of multi-modal model using attention mechanism in memotion analysis, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, 2022. [61] T. Morishita, G. Morio, S. Horiguchi, H. Ozaki, T. Miyoshi, Hitachi at SemEval-2020 task 8: Simple but effective modality ensemble for meme emotion recognition, in: SemEval, 2020. [62] Y. Guo, J. Huang, Y. Dong, M. Xu, Guoym at SemEval-2020 task 8: Ensemble-based classification of visuo-lingual metaphor in memes, in: SemEval, 2020. [63] G.-A. Vlad, G.-E. Zaharia, D.-C. Cercel, et al., Upb at semeval-2020 task 8: Joint textual and visual modeling in a multi-task learning architecture for memotion analysis, 2020. arXiv:2009.02779. [64] T. T. Nguyen, N. T. Pham, N. D. Nguyen, et al., Hcilab at memotion 2.0 2022: Analysis of sentiment, emotion and intensity of emotion classes from meme images using single and multi modalities, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, 2022. [65] G. G. Lee, M. Shen, Amazon pars at memotion 2.0 2022: Multi-modal multi-task learning for memotion 2.0 challenge, Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection (2020). [66] M. Bhange, N. Kasliwal, Hinglishnlp: Fine-tuned language models for hinglish sentiment detection, arXiv preprint arXiv:2008.09820 (2020). [67] A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv:2010.11929 (2020). [68] W. Yu, D. Kolossa, wentaorub at Memotion 3: Ensemble learning for multi-modal meme classification, in: Proceedings of De-Factify 2: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR, 2023. [69] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, et al., Learning transferable visual models from natural language supervision, in: ICML, 2021. [70] X. Li, X. Yin, C. Li, P. Zhang, et al., Oscar: Object-semantics aligned pre-training for vision-language tasks, 2020. arXiv:2004.06165. [71] Y.-C. Tang, K.-D. Wang, T.-Y. Ou, W.-C. Peng, NYCU_TWO at Memotion 3: Good foundation, good teacher, then you have good meme analysis, in: Proceedings of De-Factify 2: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR, 2023. [72] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, 2021. arXiv:2103.14030. [73] X. Guo, J. Ma, A. Zubiaga, NUAA-QMUL-AIIT at Memotion 3: Multi-modal fusion with squeeze-and-excitation for internet meme emotion analysis, in: Proceedings of De-Factify 2: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR, 2023. [74] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pretraining approach (2019). [75] X. Zhai, J. Puigcerver, A. Kolesnikov, et al., A large-scale study of representation learning with the visual task adaptation benchmark, 2020. arXiv:1910.04867. [76] G. Ke, Q. Meng, T. Finley, et al., Lightgbm: A highly efficient gradient boosting decision tree, in: Neurips, 2017. [77] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: CVPR, 2016. [78] F. Schroff, D. Kalenichenko, J. Philbin, FaceNet: A unified embedding for face recognition and clustering, in: CVPR, 2015.

Stories, News & Mentions about S4

S4’s Stories on HackerNoon