Shuffllis in the Video Creation Tools Industry

Shuffll

shuffll

The Walking Dead is coming to the Ethereum blockchain’s gaming metaverse through , a decentralised virtual world built by the community. Keep reading for a The Walking Dead NFT. The Sandbox free “We are closely collaborating with Skybound Entertainment on building TWD experiences that translate the spirit of the comic book series into our gaming virtual world,” - Sebastien Borget | Co-Founder of . The Sandbox The Partnership Skybound Entertainment, which was co-founded by The Walking Dead creator Robert Kirkman, has recently partnered with The Sandbox to bring The Walking Dead into the metaverse’s gaming space within the Ethereum blockchain. The partnership promises to bring epic new zombie experiences for fans of The Walking Dead, or zombies in general, to The Sandbox and the blockchain. “Our vision is to make The Sandbox open metaverse feel like a virtual attraction park with many amazing unique social experiences with The Walking Dead, Atari, Smurfs, and more.” - Sebastien Borget | Co-Founder of The Sandbox. The Experience The Walking Dead’s gaming experience within The Sandbox metaverse will be enjoyed immensely by those who love The Walking Dead, zombie-related media and the horror genre. Players will experience what it takes to survive in a zombie-infested world. They’ll need to scavenge supplies, such as food, medicine and resources. They’ll need to build walled or fenced safe havens to protect themselves and other survivors from the Walkers. They may also need to develop mutual friendships and alliances - or take whatever they need by force. “The Walking Dead universe is a good fit as the zombie genre is one of the most popular game categories, and The Sandbox wanted to bring a globally known zombie brand.” - Sebastien Borget | Co-Founder of The Sandbox. The Walking Dead LAND Sale The partnership will kick off with a The Walking Dead themed LAND sale at The Sandbox. LAND on The Sandbox is a virtual piece of digital real-estate that can be monetised. Learn more about LANDs in The Sandbox here: . Why Owning LAND On The Sandbox Could Make You Rich Starting on the 26th August 2021, you will be able to purchase LAND in The Sandbox as part of this partnership, as well as some unique The Walking Dead non-fungible token (NFT) assets. Using this LAND and your assets, you can create your very own experience within The Sandbox metaverse and monetise it to earn yourself a nice, steady flow of income. The Walking Dead LAND sale will span across four waves: Wave 1: The Prison - 26th August at 1pm UTC. Wave 2: The Commonwealth - 2nd September at 1pm UTC. Wave 2: Alexandria Safe Zone - 9th September at 1pm UTC. Wave 4: Finale - 16th September at 1pm UTC. Learn more about these LAND sales in The Sandbox’s official Medium article here: https://medium.com/sandbox-game/the-walking-dead-land-sale-four-weeks-of-apocalyptic-mayhem-starting-on-august-26th-c473115f9ffb BONUS! The Walking Dead Free NFT If you log into The Sandbox or register an account though their official The Walking Dead page here: - You will get a The Walking Dead-themed NFT dropped into your Ethereum wallets when the game launches. https://thewalkingdead.sandbox.game free Helpful links: . The Sandbox’s official knowledge base for information and guides . The Sandbox’s official website and marketplace . The Sandbox’s official Discord server See you in the metaverse, survivor! I won’t bite. … Not yet, anyway. Disclaimer: The writer of this article is a member of The Sandbox community who owns $SAND tokens, LANDs and NFTs, and therefore has a vested interest. Make sure you take the time to do your own research before investing in anything to make sure it is right for you. This article is part of The hosted by HackerNoon in partnership with Gaming Metaverse Writing Contest . The Sandbox Submit your today for your chance to win up to $2000. #gaming-metaverse story

The Walking Dead Shuffles its Way Into The Metaverse

Build a React Shuffle Blog using the Spotify API

In my , I covered how Microsoft and their partners have a vested interest in keeping their products artificially defective and complex. But I didn’t get a chance to dive as far as I would have liked into how Microsoft could go about intentionally keeping their solutions artificially defective or the negative impact that these methods seem to have elsewhere within their company. Without involving employees though, there are only a few ways that a company can intentionally create defective products without making it look intentional. Obviously, its not as simple as their employees dropping random 💩 emojis into their code at night. previous post Certainly, Microsoft could just as well be ignorant and inept with regard to quality software engineering best practices and just accidentally make $89 billion dollars per year. They could also just be dated, as this business model was much more justifiable in the 80’s and 90’s when they had no real competition and the productivity boost that their products offered at the time was unparalleled. Without an intricate understanding of quality engineering though, Microsoft, as a premier software development firm throughout the world, could not get to the level that they are presently at nor could they capitalize on such a dynamic that benefits off of artificially defective products to the degree that they do. It goes without saying, but Microsoft doesn’t actually instruct their employees to create defective software. In fact, as a former Microsoft employee myself, I can confirm that no one in this world hates defects and unnecessary complexity more than Microsoft employees and that great efforts are made to address such problems there. But, while the quality of an IT solution can absolutely reflect the quality and dedication of the engineers that made it, the quality of the environments where these solutions are produced are of equal importance and are reflected in the product just the same. Along with hiring top talent, tech companies, as with others, can nurture their desired end result by adjusting their environment accordingly. This happens to be why modern technology companies tend to spend a significant amount of revenue tuning their work environment with extravagant office perks and progressive management practices. Regardless of how passionate and dedicated they may be, Microsoft employees, unlike Sr. leadership, have little to no influence on environmental variables such as headcount, workload, schedules, resources, morale, management, standards, or ideologies being leveraged; all of which influence the overall quality of their products. As such and just as companies can streamline their environment with this understanding in order to maximize quality, Microsoft can easily apply the same same logic inversely and make subtle changes to the aforementioned environmental variables in order to stifle quality. In doing this and almost as if they can turn a dial, Microsoft leadership can modulate the morale of employees at scale, increasing the tendency for them to be less motivated, more apathetic, less productive, and more error prone without ever having to tell them to do anything but to do their best; all by simply making subtle changes to their environment. For instance and since they depend heavily on vendors, which are almost as numerous as their full-time employees, leadership can easily modulate stress and pressure on their teams by increasing or reducing vendor headcount indiscriminately. It doesn’t matter how great of an employee they are, as people are worked harder and stretched thinner, they tend to cut corners in order to meet deadlines and among other things, they generally become more error prone. By reducing vendor headcount, they can increase workload of their employees at scale while reducing morale which can increase the probability of defects occurring within their products in turn. On top of modulating the pressure on their employees with variable vendor resources, Microsoft can also limit the quality of their products to various degrees by limiting the quality of their environment elsewhere. Besides simply laying off a ton of QAs and SDETs, they can also leverage antiquated leadership that often come attached with antiquated organizational and managerial practices. This kind of leadership often results in frequent re-orgs, tenurocracy, thick management structures, change resistance, complex roles, and a rat race for a review system. When combined with adjusting their vendor headcount and lowering the bar with leadership, such practices can have a significant negative impact on productivity and morale which again can have a limiting effect in the overall quality of their products that they just would not have otherwise. In comparison to having their employees complicit in the sabotage of their products, operating like this can be beneficial for several reasons. For starters, it would be impossible to keep a lid on such a foul if their employees were complicit in it. Conveniently though, making environmental changes in order to make your products artificially defective can be incredibly difficult to correlate, which also distances them from complicity in the matter from a legal perspective. Since so few people are well-versed on the subject of defect density, finding someone with a deep enough understanding of it in order to make such correlations can be quite challenging while also making a plea of ignorance to the subject incredibly easy, believable, and convenient; even for a software engineer. Also, while employee layoffs always seem to make the news, Microsoft’s vendor attrition seldom generates any buzz and they can modulate pressure on remaining employees and vendors with impunity. However, there are drawbacks inherent to operating like this. One unfortunate consequence of Microsoft optimizing their organization in order to make products that are more prone to defects is that doing so also makes their products less competitive in markets where innovation and quality take precedence over the influence of their partners, such as retail markets. To no surprise and presumably for this reason, Microsoft has been experiencing difficulty in reaching retail consumers. The Playstation 4 has Xbox One 2:1, Windows Phone was a complete bust along with their Nokia acquisition, they’re having trouble giving away Windows 10 for free, their health band lasted three minutes, Consumer Reports recently their recommendation for the Surface line of products, Groove is , and no one seems to give a damn about their soon to be released Invoke speaker. outsold pulled going away In order to supplement their defective products in retail markets, Microsoft has to spend more on marketing as well as charging significantly more for support than the likes of Apple. For instance, Microsoft also had to give the NFL $400 million dollars to use their Surfaces for 5 years. Relative to their revenue in 2016, Microsoft spent 1/6th of their revenue on Marketing in comparison to Apple spending only 1/107th of their revenue. Also, a three year AppleCare hardware warranty with unlimited software support costs less than a call to Microsoft for a single Windows support case. Further and by embracing their own solutions when better solutions are available, Microsoft also limits themselves in the same way that other businesses are limited by their products. Not only are Microsoft products more costly with regard to productivity loss, management, and support, there are much more productive and efficient tools available for such work which their competition is using. For a comparison, Apple employees equipped with Apple solutions generate 3x the amount of revenue per employee than Microsoft employees equipped with Microsoft solutions. Certainly, other environmental variables also contribute to this disparity, but the quality and efficiency of tools have a significant impact on the overall quality of the end product, just as the quality and efficiency of weapons influence the outcome of wars. This is yet another reason why IBM switched to Macs and why many startups are standardizing with Apple products to this day. While most would consider Microsoft products to be a standard within industry, they are failing to see them for the monopoly that they are with the help of their partners. From the perspective of productivity software, they fit the profile quite well and there is not a realm of science that could justify the organizational moves and methodologies that Microsoft relies on outside their actions being that of a monopoly or of an inept company. Fortunately, as the IT industry improves its ability to engineer higher quality, more efficient, less expensive, and simpler solutions, Microsoft will eventually be forced to adapt or collapse. Credit where its due, they are trying incredibly hard to diversify their revenue outside of their partner network, but Microsoft’s many blunders throughout these efforts have forced them to revert back to their monopolistic tendencies and depend on artificially defective solutions in order to keep their partner network and investors happy, now more than ever. In summary, this is just one of many ways that monopolies work and to no surprise, Microsoft seems to have a monopoly in the same industries that their partners thrive on. Working in the industry that they built, Microsoft cannot claim to be ignorant of the relationship between the quality of the environment and the quality of their products or of the stimulating effects that their defective software can have on their partners and their own business at the expense of industry as a whole. Without an incredibly high level understanding of engineering best practices, software defect density and the consequences of deviating from them, Microsoft wouldn’t have been able to operate at the level that they do, let alone realize and capitalize on their partner dynamic which is presently responsible for 95% of their revenue. To make such a claim would essentially be the equivalent of saying that they make $89 billion dollars per year by accident.

The Microsoft Shuffle

This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license.
Authors:
(1) Cristina España-Bonet, DFKI GmbH, Saarland Informatics Campus. Table of Links Abstract and Intro
Corpora Compilation
Political Stance Classification
Summary and Conclusions
Limitations and Ethics Statement
Acknowledgments and References
A. Newspapers in OSCAR 22.01
B. Topics
C. Distribution of Topics per Newspaper
D. Subjects for the ChatGPT and Bard Article Generation
E. Stance Classification at Article Level
F. Training Details 3. Political Stance Classification The Network. **We finetune XLM-RoBERTa large (Conneau et al., 2020), a multilingual transformer-**based masked LM trained on 100 languages including the 4 we consider. The details of the network and the hyperparameter exploration per model are reported in Appendix F. The Models. We train 4 models: 3 monolingual finetunings with the English, German and Spanish data, plus a multilingual one with the shuffled concatenation of the data. All models are based on multilingual embeddings (RoBERTa) finetuned either monolingually or multilingually. Notice that we do not train any model for Catalan. With this, we want to compare the performance of mono- and multilingual finetunings and explore the possibility of using multilingual models for zero-shot language transfer. Coarse Classification with Newspaper Articles. Table 2 summarises the results. All the models achieve more than 95% accuracy on the validation set which is extracted from the same distribution as the training data. In order to see how the models behave with unseen data, we calculate the percentage of articles that are classified as Left (L) and Right (R) in the test newspapers of Table 1. We perform bootstrap resampling of the test sets with 1000 bootstraps to obtain confidence intervals at 95% level. We do not expect all the articles of a newspaper leaning towards the Left to show clear characteristics of the Left, but given that there is no neutral class, we expect the majority of them to be classified as Left. A good result is not necessarily 100%–0%, as this would not be realistic either. We consider that a newspaper has been classified as having a Left/Right political stance if more than 50% of its articles have been classified as such. These cases are boldfaced in Table 2. This is the behaviour we obtain for all the test newspapers but for the German Right-oriented newspaper: die Preußische Allgemeine Zeitung (PAZ). The German model is trained only on 12 newspapers to be compared to the 47 in English and 38 in Spanish. The incorrect classification might be an indication that diversity is a key aspect for the final model performance. Multilinguality does not help and 65% of the PAZ articles are still classified as Left oriented. We also assess the effectiveness of the English model on the German data, two close languages. We acknowledge that the topics of the USA and German newspapers might differ a lot, but the high diversity of the English training data could potentially compensate for this. The English model is able to correctly classify the German My Heimat as a Left-oriented newspaper (L: 67±3%) and PAZ as a Right-oriented one (R: 58±5%). We again attribute the difference to the German model being trained on a corpus lacking diversity. When we use the multilingual system, the dominant factor distinguishing the outputs is the language itself rather than the stance. The addition of English data is insufficient to alter the classification significantly. When we use the English system, the language does not play a role any more and only the stance features are considered. When we apply the English model to the Catalan newspapers we do not obtain satisfactory results though (95±1% for the Left but 16±3% for the Right newspaper) showing that the relatedness across languages is important. The multilingual model however properly detects the stance of the Catalan newspapers probably because it has been trained with an heterogeneous corpus that includes a related language (Spanish). We are able to perform zero-shot language transfer classification when we deal with close related languages. Coarse Classification with ILM-generated Articles. The bottom part of Table 2 details the results. We first focus on the English and Spanish models as the German one did not properly classify our test newspapers. The most relevant aspect to notice in ChatGPT is the strong change in political stance between February (v02) and May (v05) followed by a movement towards neutrality in August (v08). We checked that this polarity change is not an effect of the length of the outputs —the major shallow change in the generated articles. The training data in English has 5,730L–6,988 R articles with 584<length (words)<624 (similar to ChatPGTv05 length) and 4,563 L-7,127 R articles with 331< length<371 (similar to ChatGPtv02). In both cases the number of articles is larger for the Right stances, but the prediction for ChatGPTv02 clearly points towards the Left, rejecting the hypothesis that length plays a role in the classification. A similar things happens for Spanish. According to our models, the May 24th version of editorial line close to the right ideology, which differs from the ideology of the previous versions. Notably, this period corresponds drop in several task according to Chen et al. (2003). The German and Catalan outputs would still show an imprint from the Left ideology also in v05 but more diverse training data would be needed to confirm this with our monolingual models. It is interesting to notice that if we use the english monolingual model for German and Catalan, we still het the Left imprint (60±10% for German and 87±7% for Catalan). So we have indications that the political stance of ChatGPT depend on the language, which is not surprising in a data-driven system. The last version, ChatGPTv08, produces the most neutral texts, with only German clearly leaning towards the Left. The two generations, v08a and v08b, show that results are robust and are not tied to a particular generation. There is only a version available for multilingual Bard that covers our time frame.[7] The variation between generations is larger for Bard than for ChatGPT but, comparing v08 versions, Bard points towards the Left in a more consistent way across languages. Bard’s political orientation can also be determined by its answers to political test or quiz questions. The Political Compass (PC) site[8] defines 62 propositions to identify the political ideology —with an European/Western view— in two axes: economic policy (Left–Right) and social policy (Authoritarian–Libertarian), both in the range [-10,10]. Each proposition is followed by 4 alternatives: strongly agree, agree, disagree and strongly disagree. When prompted with the questionnaire,[9] Bard’s scores are (-6.50, -4.77) for English, (-8.00, -7.13) for German, (-5.75, -4.15) for Spanish and (-6.75, -4.56) for Catalan, where the first number corresponds to the economic policy and the second to the social policy. The results are in concordance with Table 2 and give an indirect validation of our method which does not rely on direct questions.[10] This kind of analysis is not possible with ChatGPT any more as it refrains from expressing opinions and preferences, demonstrating the relevance of an approach that detects the leaning in a more indirect way. Also notice that these questionnaires are well-known and public, so it would be easy to instruct a LM to avoid the questions or react to its propositions in a neutral manner. Previous work used only political tests and questionnaires to estimate ChatGPT’s orientation. Hartmann et al. (2023) used PC, 38 political statements from the voting advice application Wahl-O-Mat (Germany) and 30 from StemWijzer (the Netherlands) to conclude that ChatGPT’s ideology in its version of Dec 15 2022 was pro-environmental and left-libertarian. A study conducted by the Manhattan Institute for Policy Research[11] reported that ChatGPT tended to give responses typical of Left-of-center political viewpoints for English (Rozado, 2023). The authors administered 15 political orientation tests to the ChatGPT version of Jan 9. Their results are consistent with our evaluation of the Feb 13 model. Finally, Motoki et al. (2023) performed a battery of tests based on PC to show that ChatGPT is strongly biased towards the Left. The authors do not state the version they use, but the work was submitted on March 2023. All these results are therefore before the move to the Right we detected in May. [7] Notice that the version we use does not officially support Catalan, but native speakers confirmed that generations are mostly correct and fluent with few grammatical mistakes. [8] https://www.politicalcompass.org/test (accessed between 13th and 20th August 2023) [9] The Spanish questionnaire was translated into Catalan, as the questionnaire was not available. [10] Even though, similarly to people, it is possible for an ILM to say one thing (chose an option for a proposition) and act (write a text) in an inconsistent way. [11] A conservative think tank according to Wikipedia. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Cristina España-Bonet, DFKI GmbH, Saarland Informatics Campus. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: Authors: (1) Cristina España-Bonet, DFKI GmbH, Saarland Informatics Campus. Table of Links Abstract and Intro Corpora Compilation Political Stance Classification Summary and Conclusions Limitations and Ethics Statement Acknowledgments and References A. Newspapers in OSCAR 22.01 B. Topics C. Distribution of Topics per Newspaper D. Subjects for the ChatGPT and Bard Article Generation E. Stance Classification at Article Level F. Training Details Abstract and Intro Abstract and Intro Corpora Compilation Corpora Compilation Political Stance Classification Political Stance Classification Summary and Conclusions Summary and Conclusions Limitations and Ethics Statement Limitations and Ethics Statement Acknowledgments and References Acknowledgments and References A. Newspapers in OSCAR 22.01 A. Newspapers in OSCAR 22.01 B. Topics B. Topics C. Distribution of Topics per Newspaper C. Distribution of Topics per Newspaper D. Subjects for the ChatGPT and Bard Article Generation D. Subjects for the ChatGPT and Bard Article Generation E. Stance Classification at Article Level E. Stance Classification at Article Level F. Training Details F. Training Details 3. Political Stance Classification The Network. **We finetune XLM-RoBERTa large (Conneau et al., 2020), a multilingual transformer-**based masked LM trained on 100 languages including the 4 we consider. The details of the network and the hyperparameter exploration per model are reported in Appendix F. The Models. We train 4 models: 3 monolingual finetunings with the English, German and Spanish data, plus a multilingual one with the shuffled concatenation of the data. All models are based on multilingual embeddings (RoBERTa) finetuned either monolingually or multilingually. Notice that we do not train any model for Catalan. With this, we want to compare the performance of mono- and multilingual finetunings and explore the possibility of using multilingual models for zero-shot language transfer. The Models. Coarse Classification with Newspaper Articles. Table 2 summarises the results. All the models achieve more than 95% accuracy on the validation set which is extracted from the same distribution as the training data. In order to see how the models behave with unseen data, we calculate the percentage of articles that are classified as Left (L) and Right (R) in the test newspapers of Table 1. We perform bootstrap resampling of the test sets with 1000 bootstraps to obtain confidence intervals at 95% level. We do not expect all the articles of a newspaper leaning towards the Left to show clear characteristics of the Left, but given that there is no neutral class, we expect the majority of them to be classified as Left. A good result is not necessarily 100%–0%, as this would not be realistic either. We consider that a newspaper has been classified as having a Left/Right political stance if more than 50% of its articles have been classified as such. These cases are boldfaced in Table 2. Coarse Classification with Newspaper Articles. This is the behaviour we obtain for all the test newspapers but for the German Right-oriented newspaper: die Preußische Allgemeine Zeitung (PAZ). The German model is trained only on 12 newspapers to be compared to the 47 in English and 38 in Spanish. The incorrect classification might be an indication that diversity is a key aspect for the final model performance. Multilinguality does not help and 65% of the PAZ articles are still classified as Left oriented. We also assess the effectiveness of the English model on the German data, two close languages. We acknowledge that the topics of the USA and German newspapers might differ a lot, but the high diversity of the English training data could potentially compensate for this. The English model is able to correctly classify the German My Heimat as a Left-oriented newspaper (L: 67±3%) and PAZ as a Right-oriented one (R: 58±5%). We again attribute the difference to the German model being trained on a corpus lacking diversity. When we use the multilingual system, the dominant factor distinguishing the outputs is the language itself rather than the stance. The addition of English data is insufficient to alter the classification significantly. When we use the English system, the language does not play a role any more and only the stance features are considered. When we apply the English model to the Catalan newspapers we do not obtain satisfactory results though (95±1% for the Left but 16±3% for the Right newspaper) showing that the relatedness across languages is important. The multilingual model however properly detects the stance of the Catalan newspapers probably because it has been trained with an heterogeneous corpus that includes a related language (Spanish). We are able to perform zero-shot language transfer classification when we deal with close related languages. Coarse Classification with ILM-generated Articles. The bottom part of Table 2 details the results. We first focus on the English and Spanish models as the German one did not properly classify our test newspapers. The most relevant aspect to notice in ChatGPT is the strong change in political stance between February (v02) and May (v05) followed by a movement towards neutrality in August (v08). We checked that this polarity change is not an effect of the length of the outputs —the major shallow change in the generated articles. The training data in English has 5,730L–6,988 R articles with 584<length (words)<624 (similar to ChatPGTv05 length) and 4,563 L-7,127 R articles with 331< length<371 (similar to ChatGPtv02). In both cases the number of articles is larger for the Right stances, but the prediction for ChatGPTv02 clearly points towards the Left, rejecting the hypothesis that length plays a role in the classification. A similar things happens for Spanish. According to our models, the May 24th version of editorial line close to the right ideology, which differs from the ideology of the previous versions. Notably, this period corresponds drop in several task according to Chen et al. (2003). The German and Catalan outputs would still show an imprint from the Left ideology also in v05 but more diverse training data would be needed to confirm this with our monolingual models. It is interesting to notice that if we use the english monolingual model for German and Catalan, we still het the Left imprint (60±10% for German and 87±7% for Catalan). So we have indications that the political stance of ChatGPT depend on the language, which is not surprising in a data-driven system. The last version, ChatGPTv08, produces the most neutral texts, with only German clearly leaning towards the Left. The two generations, v08a and v08b, show that results are robust and are not tied to a particular generation. There is only a version available for multilingual Bard that covers our time frame.[7] The variation between generations is larger for Bard than for ChatGPT but, comparing v08 versions, Bard points towards the Left in a more consistent way across languages. Bard’s political orientation can also be determined by its answers to political test or quiz questions. The Political Compass (PC) site[8] defines 62 propositions to identify the political ideology —with an European/Western view— in two axes: economic policy (Left–Right) and social policy (Authoritarian–Libertarian), both in the range [-10,10]. Each proposition is followed by 4 alternatives: strongly agree, agree, disagree and strongly disagree. When prompted with the questionnaire,[9] Bard’s scores are (-6.50, -4.77) for English, (-8.00, -7.13) for German, (-5.75, -4.15) for Spanish and (-6.75, -4.56) for Catalan, where the first number corresponds to the economic policy and the second to the social policy. The results are in concordance with Table 2 and give an indirect validation of our method which does not rely on direct questions.[10] This kind of analysis is not possible with ChatGPT any more as it refrains from expressing opinions and preferences, demonstrating the relevance of an approach that detects the leaning in a more indirect way. Also notice that these questionnaires are well-known and public, so it would be easy to instruct a LM to avoid the questions or react to its propositions in a neutral manner. Previous work used only political tests and questionnaires to estimate ChatGPT’s orientation. Hartmann et al. (2023) used PC, 38 political statements from the voting advice application Wahl-O-Mat (Germany) and 30 from StemWijzer (the Netherlands) to conclude that ChatGPT’s ideology in its version of Dec 15 2022 was pro-environmental and left-libertarian. A study conducted by the Manhattan Institute for Policy Research[11] reported that ChatGPT tended to give responses typical of Left-of-center political viewpoints for English (Rozado, 2023). The authors administered 15 political orientation tests to the ChatGPT version of Jan 9. Their results are consistent with our evaluation of the Feb 13 model. Finally, Motoki et al. (2023) performed a battery of tests based on PC to show that ChatGPT is strongly biased towards the Left. The authors do not state the version they use, but the work was submitted on March 2023. All these results are therefore before the move to the Right we detected in May. [7] Notice that the version we use does not officially support Catalan, but native speakers confirmed that generations are mostly correct and fluent with few grammatical mistakes. [8] https://www.politicalcompass.org/test (accessed between 13th and 20th August 2023) [9] The Spanish questionnaire was translated into Catalan, as the questionnaire was not available. [10] Even though, similarly to people, it is possible for an ILM to say one thing (chose an option for a proposition) and act (write a text) in an inconsistent way. [11] A conservative think tank according to Wikipedia.

Multilingual Coarse Political Stance Classification of Media: Political Stance Classification

মিডিয়ার বহুভাষিক মোটা রাজনৈতিক অবস্থানের শ্রেণীবিভাগ: রাজনৈতিক অবস্থানের শ্রেণীবিভাগ

Mehrsprachige grobe politische Haltungsklassifizierung von Medien: Politische Haltungsklassifizierung

Clasificación multilingüe de posturas políticas generales de los medios: Clasificación de posturas políticas

Classification multilingue grossière des positions politiques des médias : classification des positions politiques

बहुभाषी मोटे राजनीतिक रुख मीडिया का वर्गीकरण: राजनीतिक रुख वर्गीकरण

多言語メディアの粗い政治的立場の分類: 政治的立場の分類

미디어의 다국어 대략적인 정치적 입장 분류: 정치적 입장 분류

Classificação Multilíngue da Posição Política Grosseira da Mídia: Classificação da Posição Política

Многоязычная грубая классификация СМИ по политической позиции: Классификация политической позиции

Medyanın Çok Dilli Kaba Siyasi Duruş Sınıflandırması: Siyasi Duruş Sınıflandırması

Phân loại lập trường chính trị thô sơ đa ngôn ngữ của phương tiện truyền thông: Phân loại lập trường chính trị

多语种媒体粗略政治立场分类：政治立场分类

Plunging Into Data: Unraveling Time Series Patterns

Of late, there has been quite a lot of interest in increasing/improving the context of LLMs. One good example is the infini-attention paper from Google in early April. While it was more of tweaking the model architecture to improve the context, Microsoft came back with an answer in late April but with a data-driven solution. The paper is titled, “Make your LLM Fully Utilize the Context”. There are mainly two contributions from the paper: A data-driven approach that introduces a novel pipeline to synthesize a novel dataset to train LLMs to cleverly use long contexts. It’s dubbed “Information-Intensive Training” (IN2 in short).


A novel evaluation approach named Various Long-Context (VAL Probing) uses 3 context styles — document, code, and structured data and 3 retrieval patterns — forward, backward, and bi-directional. In this article, let's deep-dive into both the simple ideas followed by some evaluations. Visual Explanation If you are someone who likes audio-visual explanations, here is a video version of this article that explains the paper: https://youtu.be/SA21d_4QQSU?feature=shared&embedable=true IN2 Training The first main idea of the paper is a novel data-driven training approach named IN2 training. There are two types of IN2 training proposed in the paper. Fine-grained Information Awareness. In this case, we first take a raw text dataset and split it into 128 tokens and we call each of this set of 128 tokens a “segment.” We then prompt a powerful LLM model like GPT-4 with an instruction I_f in such a way that the answer a_i to this question q_i requires information in the segment s_i to provide the answer. We then combine the segment s_i with several other segments by shuffling it along with the other segments to create the long context L_i. Figure taken from the paper (figure 2). Integration and Reasoning of Information. There can be situations where the context is located in more than one segment (more than 128 tokens). So we consider a set [s_i] instead of a single s_i. This creates a multi-hop Q&A scenario where the LLM has to do at least 2 passes of QAs. We modify the instruction I_f in the prompt to the LLM accordingly. Figure taken from the paper (figure 2) In both scenarios, the authors vary the long context length L_i from 4K to 32K to prevent length bias in training. During IN2 instruction fine-tuning, the long contexts and questions become input to the LLM, and the answer is shown as the output for supervised training. They fine-tune the Mistral-7B-Instruct-v2 model and call the new model FILl-in-the-Middle(FILM-7B). VAL Probing VArious Long-context Probing or VAL Probing was introduced to overcome some of the problems with the current LLMs with retrieving from long contexts. So how do we even test the long context retrieval capability of any given model? The answer lies with “Needle in a haystack” (NIHS) test. It can be accessed here. Needle in a Haystack NIHS was proposed to pressure test LLMs. It’s a simple analysis to test the in-context retrieval ability of long-context LLMs. The test can be run by following the below 3 steps: Place a random fact or statement (the ‘needle’) in the middle of a long context window (the ‘haystack’).


Ask the model to retrieve this statement.


Iterate over various document depths (where the needle is placed) and context lengths to measure performance. The biggest problem with NIHS these days is that the recent models perform too well making the test less appealing. For example, see below the test result of the proposed FILM-7b model NIHS test on the proposed FILM-7b model The results are near perfect indicating that we really need a new testing approach and so, giving birth to VAL Probing. VAL Probing VAL probing considers 3 data types such as documents, structured data, and code. It also considers three retrieval types — forward, backward, and bi-directional. The below figure from the paper gives examples of all three data types and retrieval directions. Results For evaluation, they used the LongBench evaluation scripts and reported ROGUE-L for summarization and F1-scores for all other tasks. The first problem addressed by the paper is the “Lost-in-the-middle” problem. As shown in the below figure, we can see that there is no “U” shaped dip with the FILM-7b model (in orange) compared to Mistral-7B (green), and even other long-context sota models like LongAlign indicating the strength of FILM-7b. On top of addressing the Lost-in-the-middle problem, FILM-7B seems to outperform other recent models quantitatively too. They show the results of averaging the evaluation metric(Avg) and min-max gap of the evaluation metric (Gap) which is the difference between the minimum and maximum values obtained. In short, the model is comparable to GPT-4’s performance! Summary & Conclusion This paper from Microsoft proposes a simple data-driven solution to addressing the long-context problem in LLMs. On top of proposing a data generation pipeline, it also proposes and tests the usage of the novel VAL Probing that encompasses 3 data types and 3 retrieval methods. Let’s hope this is just the dawn of a new generation of LLMs that are no longer limited by the context length. See you in my next… Of late, there has been quite a lot of interest in increasing/improving the context of LLMs. One good example is the infini-attention paper from Google in early April. While it was more of tweaking the model architecture to improve the context, Microsoft came back with an answer in late April but with a data-driven solution. The paper is titled, “ Make your LLM Fully Utilize the Context ”. infini-attention paper infini-attention paper Make your LLM Fully Utilize the Context Make your LLM Fully Utilize the Context Make your LLM Fully Utilize the Context There are mainly two contributions from the paper: A data-driven approach that introduces a novel pipeline to synthesize a novel dataset to train LLMs to cleverly use long contexts. It’s dubbed “Information-Intensive Training” (IN2 in short). A novel evaluation approach named Various Long-Context (VAL Probing) uses 3 context styles — document, code, and structured data and 3 retrieval patterns — forward, backward, and bi-directional. A data-driven approach that introduces a novel pipeline to synthesize a novel dataset to train LLMs to cleverly use long contexts. It’s dubbed “Information-Intensive Training” (IN2 in short). A data-driven approach that introduces a novel pipeline to synthesize a novel dataset to train LLMs to cleverly use long contexts. It’s dubbed “ Information-Intensive Training ” (IN2 in short). Information-Intensive Training A novel evaluation approach named Various Long-Context (VAL Probing) uses 3 context styles — document, code, and structured data and 3 retrieval patterns — forward, backward, and bi-directional. A novel evaluation approach named Various Long-Context ( VAL Probing ) uses 3 context styles — document, code, and structured data and 3 retrieval patterns — forward, backward, and bi-directional. VAL Probing In this article, let's deep-dive into both the simple ideas followed by some evaluations. Visual Explanation If you are someone who likes audio-visual explanations, here is a video version of this article that explains the paper: https://youtu.be/SA21d_4QQSU?feature=shared&embedable=true https://youtu.be/SA21d_4QQSU?feature=shared&embedable=true IN2 Training The first main idea of the paper is a novel data-driven training approach named IN2 training . There are two types of IN2 training proposed in the paper. data-driven training IN2 training Fine-grained Information Awareness. In this case, we first take a raw text dataset and split it into 128 tokens and we call each of this set of 128 tokens a “segment.” We then prompt a powerful LLM model like GPT-4 with an instruction I_f in such a way that the answer a_i to this question q_i requires information in the segment s_i to provide the answer. We then combine the segment s_i with several other segments by shuffling it along with the other segments to create the long context L_i. Fine-grained Information Awareness. In this case, we first take a raw text dataset and split it into 128 tokens and we call each of this set of 128 tokens a “segment.” We then prompt a powerful LLM model like GPT-4 with an instruction I_f in such a way that the answer a_i to this question q_i requires information in the segment s_i to provide the answer. We then combine the segment s_i with several other segments by shuffling it along with the other segments to create the long context L_i . Fine-grained Information Awareness. I_f a_i q_i s_i s_i L_i Figure taken from the paper (figure 2). Integration and Reasoning of Information. There can be situations where the context is located in more than one segment (more than 128 tokens). So we consider a set [s_i] instead of a single s_i. This creates a multi-hop Q&A scenario where the LLM has to do at least 2 passes of QAs. We modify the instruction I_f in the prompt to the LLM accordingly. Integration and Reasoning of Information. There can be situations where the context is located in more than one segment (more than 128 tokens). So we consider a set [ s_i ] instead of a single s_i. This creates a multi-hop Q&A scenario where the LLM has to do at least 2 passes of QAs. We modify the instruction I_f in the prompt to the LLM accordingly. Integration and Reasoning of Information. s_i s_i. I_f Figure taken from the paper (figure 2) In both scenarios, the authors vary the long context length L_i from 4K to 32K to prevent length bias in training. During IN2 instruction fine-tuning, the long contexts and questions become input to the LLM, and the answer is shown as the output for supervised training. They fine-tune the Mistral-7B-Instruct-v2 model and call the new model FILl-in-the-Middle( FILM-7B ). L_i FILM-7B VAL Probing VArious Long-context Probing or VAL Probing was introduced to overcome some of the problems with the current LLMs with retrieving from long contexts. So how do we even test the long context retrieval capability of any given model? The answer lies with “Needle in a haystack” (NIHS) test. It can be accessed here . here here Needle in a Haystack NIHS was proposed to pressure test LLMs. It’s a simple analysis to test the in-context retrieval ability of long-context LLMs. The test can be run by following the below 3 steps: Place a random fact or statement (the ‘needle’) in the middle of a long context window (the ‘haystack’). Ask the model to retrieve this statement. Iterate over various document depths (where the needle is placed) and context lengths to measure performance. Place a random fact or statement (the ‘needle’) in the middle of a long context window (the ‘haystack’). Place a random fact or statement (the ‘needle’) in the middle of a long context window (the ‘haystack’). Ask the model to retrieve this statement. Ask the model to retrieve this statement. Iterate over various document depths (where the needle is placed) and context lengths to measure performance. Iterate over various document depths (where the needle is placed) and context lengths to measure performance. The biggest problem with NIHS these days is that the recent models perform too well making the test less appealing. For example, see below the test result of the proposed FILM-7b model NIHS test on the proposed FILM-7b model The results are near perfect indicating that we really need a new testing approach and so, giving birth to VAL Probing. VAL Probing VAL probing considers 3 data types such as documents, structured data, and code. It also considers three retrieval types — forward, backward, and bi-directional. The below figure from the paper gives examples of all three data types and retrieval directions. Results For evaluation, they used the LongBench evaluation scripts and reported ROGUE-L for summarization and F1-scores for all other tasks. The first problem addressed by the paper is the “Lost-in-the-middle” problem. As shown in the below figure, we can see that there is no “U” shaped dip with the FILM-7b model (in orange) compared to Mistral-7B (green), and even other long-context sota models like LongAlign indicating the strength of FILM-7b. On top of addressing the Lost-in-the-middle problem, FILM-7B seems to outperform other recent models quantitatively too. They show the results of averaging the evaluation metric(Avg) and min-max gap of the evaluation metric (Gap) which is the difference between the minimum and maximum values obtained. In short, the model is comparable to GPT-4’s performance! Summary & Conclusion This paper from Microsoft proposes a simple data-driven solution to addressing the long-context problem in LLMs. On top of proposing a data generation pipeline, it also proposes and tests the usage of the novel VAL Probing that encompasses 3 data types and 3 retrieval methods. Let’s hope this is just the dawn of a new generation of LLMs that are no longer limited by the context length. See you in my next…

How to Make Your LLM Fully Utilize the Context

Authors:
(1) Jianzhu Yao, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology;
(2) Ziqi Liu, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology;
(3) Jian Guan, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology;
(4) Minlie Huang, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology. Table of Links Abstract and Intro Related Works DIALSTORY Dataset Proposed Tasks Methodology Experiments Discussion Future Work Conclusion Limitations and References 6 Experiments 6.1 Masked Dialogue Generation Implementation Details To conduct experiments on the masked dialogue generation task, we decide the hyper-parameters based on the performance of the validation set. We train Shao et al. (2021)’s BART model for 4.6 epochs with a 1e-4 learning rate for 1 day, and for our model, we train it for 5.6 epochs with a 1e-4 learning rate for 1 day. All baselines and our model are trained using the Adam optimizer. During the training process for our method, we computed the selection coverage of characters within a single story. And it showed that in every 1000 training steps, the coverage of different characters ranged from 98.64% to 99.00%, which meant nearly all the characters are selected during training, and all the characters contributed to the generated dialogue. It further proved that the argmax in Eq. 3 operation doesn’t break the gradient progress when training for this task. Automatic Evaluation Following previous works, we use several standard, widely used automatic evaluation metrics. We use BLEUn (Papineni et al., 2002) to measure the average word overlap between each generated and groundtruth dialogue turn (n=1,2), and Distinct-n (Li et al., 2015) to evaluate n-gram diversity of generated dialogue turns (n=2,3,4). To be more specific, for the coherence classifier, we construct the training and validation sets by randomly shuffling the order of dialogue turns and keeping other content in the correct order. We regard the perturbed story as a negative example and the original story as a positive example. We sample another 195k stories (except those in DIALSTORY) from the novels of Guan et al. (2022) to construct the training set (190k examples) and the validation set (5k examples). We train the model for 4 epochs with a 2e-5 learning rate and a 16- batch size, using the Adam optimizer. During the evaluation, we consider an example coherent when the probability of being coherent predicted by the classifier is greater than 0.5. We use the ratio of outputs (along with the input) that are classified as coherent by the classifier to all generated outputs as the coherence score. The result of the automatic evaluation is presented in Table 3. According to the table, compared to the BART baseline, our model consistently generates more word overlaps with ground truth and achieves better diversity under the guidance of character representations, which means our model can generate more diverse but not commonplace responses. Figure 3 plots the coherence score varying with the number of masked dialogue turns. The result shows that our model gets a higher coherence score than BART when required to generate more than seven turns of dialogue in one story. Manual Evaluation We conduct a pairwise comparison between our model and the BART baseline. We randomly select 100 examples from the test set. For each pair of outputs along with the input, we ask three annotators to give a preference (win, lose and tie) in terms of fluency, coherence, and informativeness. All the annotations are native Chinese speakers. We adopt majority voting to make final decisions among the annotators. The three aspects of manual evaluation are as follows: Fluency: Grammatical correctness and intra-sentence linguistic quality.


Coherence: Inter-sentence relatedness, causal and temporal dependencies. We judge the coherence between the story and a dialogue turn by following the criterion in Table 5. We add the scores of all the generated dialogue turns in a story to get the overall coherence score of the story, which is then used to compare with each other.


Informativeness: Interesting, diverse and rich details. As shown in Table 4, all the results show moderate (κ > 0.4) agreement, which shows our model outperforms the BART baseline significantly in dialogue informativeness and coherence. Case Study Figure 5 showed two examples to investigate how learning character representations can help our model generate more coherent dialogue. We found that our model can better model the relationship between different characters and the direction of the storyline. For example, in the first case, we can see that the BART’s generation confuses different characters’ fathers, while our model captures the relationship between different characters, and generates proper responses for the corresponding characters, which also moves the plot forward. And in the second case, we can see that BART’s generation is commonplace and contradicts the plot development. In contrast, our model captures the intentions of the speaker and the development trend of the plot, generating an appropriate and coherent response. Since these two models use the same pretrained weight, we can infer that the character modeling module leverages the coherent and reasonable generation. We also summarize four error types of the generated dialogue turn for the DialGen task: (1) Intersentence Contradiction; (2) Inter-sentence Repetition; (3) Intra-sentence Contradiction; (4) Intrasentence Repetition. We show the typical corresponding cases in Figure 4. We conducted a quantitative analysis of those 4 error types on our model’s generation. We analyzed 20 stories with 103 dialog turns and the results are shown in Figure 6. We found that both our model and BART suffer from these errors, suggesting that there is still space for model improvement, especially in the inter-sentence repetition. 6.2 Dialogue Speaker Recognition Implementation Details To conduct experiments on the speaker recognition task, we decide the hyper-parameters based on the performance of the validation set. For the BART baseline and our approach, we insert a mask token before each dialogue needed to be predicted, and a person id token before and after each character name span. Then, we insert all the unique person id tokens before the input stories as different options, and make predictions based on the cos similarity of option tokens and mask tokens. We train Shao et al. (2021)’s BART model for 30 epochs with a 5e-5 learning rate for 3 days, For encoder-only baselines, we implemented BERT, RoBERTa, and MacBERT and trained them for 15 epochs wity a le-5 learning rate for 2 days. For our model, we train it for 22 epochs with a le-6 learning rate for 2 days. All baselines and our model are trained using the Adam optimizer. Metrics We evaluate the DialSpk task using two automatic metrics including dialogue-level accuracy (DAC) and story-level accuracy (SAC). DAC is calculated as the ratio of the correct predictions to the total number of specifies dialogue turns, while SAC is the ratio of the number of stories where all dialogue turns are correctly predicted to the number of all test examples. These two metrics provide the evaluation for dialogue understanding with different granularities. Results As shown in Table 7, our model outperforms all the baselines significantly (p< 0.01, Wilcoxon signed-rank test) on both DAS and SAC scores, suggesting the benefit of learning character representations. We tested the accuracy of automatic training set annotations, and the DAC/SAC scores are 86.78%/67.80%. Together with the model’s performance on the test set, we can see the automatic annotation for the training set is of good quality. We also conducted the human prediction experiment, and the DAC/SAC scores are 97.90%/90.70%, which are much higher than the best model. So there is much room for further improvement for machine-based approaches. This paper is available on arxiv under CC 4.0 DEED license. [2] https://huggingface.co/fnlp/ bart-base-chinese [3] https://huggingface.co/bert-base-chinese [4] https://huggingface.co/hfl/ chinese-roberta-wwm-ext [5] https://huggingface.co/hfl/ chinese-macbert-base Authors: (1) Jianzhu Yao, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology; (2) Ziqi Liu, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology; (3) Jian Guan, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology; (4) Minlie Huang, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology. Authors: Authors: (1) Jianzhu Yao, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology; (2) Ziqi Liu, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology; (3) Jian Guan, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology; (4) Minlie Huang, The CoAI group, Tsinghua University, Beijing, China Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology. Table of Links Abstract and Intro Abstract and Intro Related Works Related Works DIALSTORY Dataset DIALSTORY Dataset Proposed Tasks Proposed Tasks Methodology Methodology Experiments Experiments Discussion Discussion Future Work Future Work Conclusion Conclusion Limitations and References Limitations and References 6 Experiments 6.1 Masked Dialogue Generation Implementation Details To conduct experiments on the masked dialogue generation task, we decide the hyper-parameters based on the performance of the validation set. We train Shao et al. (2021)’s BART model for 4.6 epochs with a 1e-4 learning rate for 1 day, and for our model, we train it for 5.6 epochs with a 1e-4 learning rate for 1 day. All baselines and our model are trained using the Adam optimizer. Implementation Details During the training process for our method, we computed the selection coverage of characters within a single story. And it showed that in every 1000 training steps, the coverage of different characters ranged from 98.64% to 99.00%, which meant nearly all the characters are selected during training, and all the characters contributed to the generated dialogue. It further proved that the argmax in Eq. 3 operation doesn’t break the gradient progress when training for this task. Automatic Evaluation Following previous works, we use several standard, widely used automatic evaluation metrics. We use BLEUn (Papineni et al., 2002) to measure the average word overlap between each generated and groundtruth dialogue turn (n=1,2), and Distinct-n (Li et al., 2015) to evaluate n-gram diversity of generated dialogue turns (n=2,3,4). Automatic Evaluation To be more specific, for the coherence classifier, we construct the training and validation sets by randomly shuffling the order of dialogue turns and keeping other content in the correct order. We regard the perturbed story as a negative example and the original story as a positive example. We sample another 195k stories (except those in DIALSTORY) from the novels of Guan et al. (2022) to construct the training set (190k examples) and the validation set (5k examples). We train the model for 4 epochs with a 2e-5 learning rate and a 16- batch size, using the Adam optimizer. During the evaluation, we consider an example coherent when the probability of being coherent predicted by the classifier is greater than 0.5. We use the ratio of outputs (along with the input) that are classified as coherent by the classifier to all generated outputs as the coherence score. The result of the automatic evaluation is presented in Table 3. According to the table, compared to the BART baseline, our model consistently generates more word overlaps with ground truth and achieves better diversity under the guidance of character representations, which means our model can generate more diverse but not commonplace responses. Figure 3 plots the coherence score varying with the number of masked dialogue turns. The result shows that our model gets a higher coherence score than BART when required to generate more than seven turns of dialogue in one story. Manual Evaluation We conduct a pairwise comparison between our model and the BART baseline. We randomly select 100 examples from the test set. For each pair of outputs along with the input, we ask three annotators to give a preference (win, lose and tie) in terms of fluency, coherence, and informativeness. All the annotations are native Chinese speakers. We adopt majority voting to make final decisions among the annotators. The three aspects of manual evaluation are as follows: Manual Evaluation Fluency: Grammatical correctness and intra-sentence linguistic quality. Coherence: Inter-sentence relatedness, causal and temporal dependencies. We judge the coherence between the story and a dialogue turn by following the criterion in Table 5. We add the scores of all the generated dialogue turns in a story to get the overall coherence score of the story, which is then used to compare with each other. Informativeness: Interesting, diverse and rich details. Fluency: Grammatical correctness and intra-sentence linguistic quality. Fluency : Grammatical correctness and intra-sentence linguistic quality. Fluency Coherence: Inter-sentence relatedness, causal and temporal dependencies. We judge the coherence between the story and a dialogue turn by following the criterion in Table 5. We add the scores of all the generated dialogue turns in a story to get the overall coherence score of the story, which is then used to compare with each other. Coherence : Inter-sentence relatedness, causal and temporal dependencies. We judge the coherence between the story and a dialogue turn by following the criterion in Table 5. We add the scores of all the generated dialogue turns in a story to get the overall coherence score of the story, which is then used to compare with each other. Coherence Informativeness: Interesting, diverse and rich details. Informativeness : Interesting, diverse and rich details. Informativeness As shown in Table 4, all the results show moderate (κ > 0.4) agreement, which shows our model outperforms the BART baseline significantly in dialogue informativeness and coherence. Case Study Figure 5 showed two examples to investigate how learning character representations can help our model generate more coherent dialogue. We found that our model can better model the relationship between different characters and the direction of the storyline. For example, in the first case, we can see that the BART’s generation confuses different characters’ fathers, while our model captures the relationship between different characters, and generates proper responses for the corresponding characters, which also moves the plot forward. And in the second case, we can see that BART’s generation is commonplace and contradicts the plot development. In contrast, our model captures the intentions of the speaker and the development trend of the plot, generating an appropriate and coherent response. Since these two models use the same pretrained weight, we can infer that the character modeling module leverages the coherent and reasonable generation. Case Study We also summarize four error types of the generated dialogue turn for the DialGen task: (1) Intersentence Contradiction; (2) Inter-sentence Repetition; (3) Intra-sentence Contradiction; (4) Intrasentence Repetition. We show the typical corresponding cases in Figure 4. We conducted a quantitative analysis of those 4 error types on our model’s generation. We analyzed 20 stories with 103 dialog turns and the results are shown in Figure 6. We found that both our model and BART suffer from these errors, suggesting that there is still space for model improvement, especially in the inter-sentence repetition. 6.2 Dialogue Speaker Recognition Implementation Details To conduct experiments on the speaker recognition task, we decide the hyper-parameters based on the performance of the validation set. For the BART baseline and our approach, we insert a mask token before each dialogue needed to be predicted, and a person id token before and after each character name span. Then, we insert all the unique person id tokens before the input stories as different options, and make predictions based on the cos similarity of option tokens and mask tokens. We train Shao et al. (2021)’s BART model for 30 epochs with a 5e-5 learning rate for 3 days, For encoder-only baselines, we implemented BERT, RoBERTa, and MacBERT and trained them for 15 epochs wity a le-5 learning rate for 2 days. For our model, we train it for 22 epochs with a le-6 learning rate for 2 days. All baselines and our model are trained using the Adam optimizer. Metrics We evaluate the DialSpk task using two automatic metrics including dialogue-level accuracy (DAC) and story-level accuracy (SAC). DAC is calculated as the ratio of the correct predictions to the total number of specifies dialogue turns, while SAC is the ratio of the number of stories where all dialogue turns are correctly predicted to the number of all test examples. These two metrics provide the evaluation for dialogue understanding with different granularities. Metrics Results As shown in Table 7, our model outperforms all the baselines significantly (p< 0.01, Wilcoxon signed-rank test) on both DAS and SAC scores, suggesting the benefit of learning character representations. We tested the accuracy of automatic training set annotations, and the DAC/SAC scores are 86.78%/67.80%. Together with the model’s performance on the test set, we can see the automatic annotation for the training set is of good quality. We also conducted the human prediction experiment, and the DAC/SAC scores are 97.90%/90.70%, which are much higher than the best model. So there is much room for further improvement for machine-based approaches. Results This paper is available on arxiv under CC 4.0 DEED license. This paper is available on arxiv under CC 4.0 DEED license. available on arxiv [2] https://huggingface.co/fnlp/ bart-base-chinese [3] https://huggingface.co/bert-base-chinese [4] https://huggingface.co/hfl/ chinese-roberta-wwm-ext [5] https://huggingface.co/hfl/ chinese-macbert-base

Understanding and Generating Dialogue between Characters in Stories: Experiments

How to Compare the Same Contents but Different Ordered Arrays in JavaScript

How to Use BigQuery Efficiently: Your Guide to Cost Optimization and Efficient Data Management

How I Saved My Newsletter From Bots With Machine Learning

Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the gpt turbo model for code generation. Introduction The Microsoft team has recently launched an challenge in which a developer can learn how to build Azure AI solutions and apps. Open AI Prerequisite Experience working with Azure and Azure portals. An understanding of generative AI. Experience in one high-level programming language like C# or Python Steps to Create Open AI Service on Azure with “Dall-E” model deployed. Day 1 — Azure Open AI Challenge If you would like to explore image generation, follow this link Day 2 — Azure Open AI Challenge: Image Generation Getting Started Considering Azure Open AI Service is running on the Azure portal and the gpt-turbo model is deployed successfully. Step 1: Create a console application To test image generation, create a console application in Visual Studio or Visual Studio Code. dotnet new console Step 2: Read the configuration Read the configuration from appsettings.json file // Build a config object and retrieve user settings.
    IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .Build();
    string? oaiEndpoint = config["AzureOAIEndpoint"];
    string? oaiKey = config["AzureOAIKey"];
    string? oaiDeploymentName = config["AzureOAIDeploymentName"]; Step 3: Add sample code Create a file under the console application project as follows: sample-code> function > function.cs namespace Function
    {
        class Program
        {
            int AbsoluteSquare(int num1, int num2)
            {
                int result = Math.Abs(num1 - num2);
                result *= result;
                return result;
            }
        }
    } Another file as sample-code > go-fish > go-fish.cs => Reference file Step 3: Prepare Console Menu The next step is to prepare a menu on the console application using digits. If the console message is “quit” the console application is exited. If the console input is 1 or 2, then the sample file function.cs is used If the console input is 3, then the sample file go-fish.cs is used Else it will return a console message saying invalid input do
    {
        Console.WriteLine("\n1: Add comments to my function\n" +
        "2: Write unit tests for my function\n" +
        "3: Fix my Go Fish game\n" +
        "\"quit\" to exit the program\n\n" +
        "Enter a number to select a task:");
    
        command = Console.ReadLine() ?? "";
    
        if (command == "quit")
        {
            Console.WriteLine("Exiting program...");
            break;
        }
    
        Console.WriteLine("\nEnter a prompt: ");
        string userPrompt = Console.ReadLine() ?? "";
        string codeFile = "";
    
        if (command == "1" || command == "2")
            codeFile = System.IO.File.ReadAllText($@"{AppDomain.CurrentDomain.BaseDirectory}sample-code\function\function.cs");
        else if (command == "3")
            codeFile = System.IO.File.ReadAllText($@"{AppDomain.CurrentDomain.BaseDirectory}sample-code\go-fish\go-fish.cs");
        else
        {
            Console.WriteLine("Invalid input. Please try again.");
            continue;
        }
    
        userPrompt += codeFile;
        await GetResponseFromOpenAI(userPrompt);
    } while (true); Step 4: Code generation from AI Please find below the method GetResponseFromOpenAI, which generates the code using gpt-turbo. Read the configuration values required, if not present don’t execute. Prepare the request for chat completion response. Finally, write the response back to result > app.txt file async Task GetResponseFromOpenAI(string prompt)
    {
        Console.WriteLine("\nCalling Azure OpenAI to generate code...\n\n");
    
        if (string.IsNullOrEmpty(oaiEndpoint) || string.IsNullOrEmpty(oaiKey) || string.IsNullOrEmpty(oaiDeploymentName))
        {
            Console.WriteLine("Please check your appsettings.json file for missing or incorrect values.");
            return;
        }
    
        // Configure the Azure OpenAI client
        OpenAIClient client = new OpenAIClient(new Uri(oaiEndpoint), new AzureKeyCredential(oaiKey));
    
        // Define chat prompts
        string systemPrompt = "You are a helpful AI assistant that helps programmers write code.";
        string userPrompt = prompt;
    
        // Format and send the request to the model
        var chatCompletionsOptions = new ChatCompletionsOptions()
        {
            Messages =
            {
                new ChatRequestSystemMessage(systemPrompt),
                new ChatRequestUserMessage(userPrompt)
            },
            Temperature = 0.7f,
            MaxTokens = 1000,
            DeploymentName = oaiDeploymentName
        };
    
        // Get response from Azure OpenAI
        Response<ChatCompletions> response = await client.GetChatCompletionsAsync(chatCompletionsOptions);
    
        ChatCompletions completions = response.Value;
        string completion = completions.Choices[0].Message.Content;
    
        // Write full response to console, if requested
        if (printFullResponse)
        {
            Console.WriteLine($"\nFull response: {JsonSerializer.Serialize(completions, new JsonSerializerOptions { WriteIndented = true })}\n\n");
        }
    
        // Write the file.
        System.IO.File.WriteAllText(@"result\app.txt", completion);
    
        // Write response to console
        Console.WriteLine($"\nResponse written to result/app.txt\n\n");
    } Test Case 1: Add comments to the code Choose option 1 Then add the following prompt to add comments to the code Add comments to the following function. Return only the commented code. Output Test Case 1 Refer to the sample on . function.cs file GitHub Please find below the convert function with the single line comments added as shown below. // This function calculates the absolute square of the difference between two numbers
    
    namespace Function
    {
        class Program
        {
            // Function to calculate the absolute square of the difference between two numbers
            int AbsoluteSquare(int num1, int num2)
            {
                // Calculate the absolute difference between the two numbers
                int result = Math.Abs(num1 - num2);
    
                // Square the result
                result *= result;
    
                // Return the absolute square of the difference between the two numbers
                return result;
            }
        }
    } Additionally, the user can suggest gpt model to add multi-line comments instead of single lines using the following prompt Add comments to the following function. Return only the commented code. — namespace Function
    {
        class Program
        {
            // This function calculates the absolute square of the difference between two integers.
            // Parameters:
            //   num1: the first integer
            //   num2: the second integer
            // Returns:
            //   The absolute square of the difference between num1 and num2
            int AbsoluteSquare(int num1, int num2)
            {
                int result = Math.Abs(num1 - num2);
                result *= result;
                return result;
            }
        }
    } Test Case 2: Add comments to the code Choose option 2 Then add the following prompt to add comments to the code Write four unit tests for the following function. — Output Test Case 2 Refer to the sample on . Please find below the test cases added function.cs file GitHub Sure! Here are four unit tests for the `AbsoluteSquare` function:
    
    1. Test case where `num1` is greater than `num2`:
    
    [Test]
    public void TestAbsoluteSquare_GreaterNum1_ReturnsSquaredDifference()
    {
        int num1 = 5;
        int num2 = 2;
        int expected = 9;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    }
    
    2. Test case where `num2` is greater than `num1`:
    
    [Test]
    public void TestAbsoluteSquare_GreaterNum2_ReturnsSquaredDifference()
    {
        int num1 = 2;
        int num2 = 5;
        int expected = 9;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    }
    
    3. Test case where `num1` and `num2` are the same:
    
    [Test]
    public void TestAbsoluteSquare_SameNumbers_ReturnsZero()
    {
        int num1 = 4;
        int num2 = 4;
        int expected = 0;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    }
    
    
    4. Test case where `num1` and `num2` are negative:
    
    [Test]
    public void TestAbsoluteSquare_NegativeNumbers_ReturnsSquaredDifference()
    {
        int num1 = -3;
        int num2 = -6;
        int expected = 9;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    } Test Case 3: Fix my Go Fish game Choose option 3 Then add the following prompt to add comments to the code Fix the code below for an app to play Go Fish with the user. Return only the corrected code. — Output Test Case 3 Refer to the sample on . go-fish.cs file GitHub Please find below the code that is fixed. using System;  
    using System.Collections.Generic;  
    using System.Linq;  
      
    namespace GoFish  
    {  
        class Program  
        {  
            static void Main(string[] args)  
            {  
                // Define the deck of cards  
                List<string> deck = new List<string> { "A", "2", "3", "4", "5", "6", "7", "8", "9", "10", "J", "Q", "K" };  
                deck.AddRange(deck);  
                deck.AddRange(deck);  
                deck.AddRange(deck);  
      
                // Shuffle the deck  
                var rand = new Random();  
                deck = deck.OrderBy(x => rand.Next()).ToList();  
      
                // Deal the cards  
                List<string> playerHand = deck.Take(5).ToList();  
                List<string> computerHand = deck.Skip(5).Take(5).ToList();  
      
                // Define the initial score  
                int playerScore = 0;  
                int computerScore = 0;  
      
                // Define the main game loop  
                while (deck.Count > 0)  
                {  
                    // Print the player's hand  
                    Console.WriteLine("Your hand: " + string.Join(", ", playerHand));  
      
                    // Ask the player for a card  
                    Console.Write("Do you have any... ");  
                    string card = Console.ReadLine();  
      
                    // Check if the player has the card  
                    if (playerHand.Contains(card))  
                    {  
                        // Remove the card from the player's hand  
                        playerHand.Remove(card);  
      
                        // Add a point to the player's score  
                        playerScore += 1;  
      
                        // Print the player's score  
                        Console.WriteLine("You got a point!");  
                        Console.WriteLine("Your score: " + playerScore);  
                    }  
                    else  
                    {  
                        // Go fish!  
                        Console.WriteLine("Go fish!");  
      
                        // Draw a card from the deck  
                        string drawnCard = deck.First();  
                        deck.RemoveAt(0);  
      
                        // Add the card to the player's hand  
                        playerHand.Add(drawnCard);  
      
                        // Print the card that was drawn  
                        Console.WriteLine("You drew a " + drawnCard);  
                    }  
      
                    // Check if the player has won  
                    if (playerScore == 5)  
                    {  
                        Console.WriteLine("You win!");  
                        break;  
                    }  
      
                    // Computer's turn  
                    string computerCard = computerHand[rand.Next(computerHand.Count)];  
                    Console.WriteLine("Do you have any " + computerCard + "?");  
                    if (playerHand.Contains(computerCard))  
                    {  
                        // Remove the card from the player's hand  
                        playerHand.Remove(computerCard);  
      
                        // Add a point to the computer's score  
                        computerScore += 1;  
      
                        // Print the computer's score  
                        Console.WriteLine("The computer got a point!");  
                        Console.WriteLine("Computer score: " + computerScore);  
                    }  
                    else  
                    {  
                        // Go fish!  
                        Console.WriteLine("The computer goes fishing!");  
      
                        // Draw a card from the deck  
                        string drawnCard = deck.First();  
                        deck.RemoveAt(0);  
      
                        // Add the card to the computer's hand  
                        computerHand.Add(drawnCard);  
      
                        // Print the card that was drawn  
                        Console.WriteLine("The computer drew a " + drawnCard);  
                    }  
      
                    // Check if the computer has won  
                    if (computerScore == 5)  
                    {  
                        Console.WriteLine("The computer wins!");  
                        break;  
                    }  
                }  
            }  
        }  
    } Finally, if you type quit, the console application will be terminated. Complete code on GitHub Make sure to give it a star on GitHub and provide feedback on how to improve the tool further..!! ssukhpinder/AzureOpenAI C# Programming🚀 Thank you for being a part of the C# community! Before you leave: If you’ve made it this far, please show your appreciation with a clap and follow the author! 👏️️ Follow us: | | | | | X LinkedIn Dev.to Hashnode Newsletter Tumblr Visit our other platforms: | | | | GitHub Instagram Tiktok Quora Daily.dev More content at C# Programming Also published . here

Taking the Azure Open AI Challenge, Day 3: Mastering Code Generation

Every product is different in the problem it solves, solution it applies, or customer segment it targets. You can’t blindly benchmark successful practices from one product to another. What you can do is constantly train your eye and taste to identify patterns, find inspiration, and connect the dots for your product. In this article, I explore the techniques that 15 popular mobile applications use to onboard new customers. Metacast Metacast is a new podcast app that is currently in public beta. While it’s not exactly the stage where companies think about perfecting onboarding, Metacast chose the medium that only feels right for a podcast app — audio. In 5 minutes intro, the team behind the product explains how to use the app. Since Metacast provides a listener with podcast transcripts, it’s especially handy to be able to follow the story and explore the app with both eyes and ears. https://www.youtube.com/watch?v=a6mZy8iX-uE%3Fsi%3DVUIJlfY1xvKwQ58v&embedable=true Craft The inspirational note-taking app Craft welcomes new users by quoting what its most loyal customers think about the tool. I don’t know whether the company collected this feedback from reviews or through a Product / Market fit survey (a powerful tool to find out how customers perceive value) but it’s a beautiful application of , a psychological phenomenon coined by Robert Cialdini in his 1984 book that describes how people tend to copy the behaviors of others when deciding how to respond to a particular situation. It is also an example of Daniel Kahneman’s associative coherence: in the same way that holding a warm cup of coffee makes you more likely to think well of a stranger, social proof of an app increases the chances a new user will like it. social proof Influence: Science and Practice https://www.youtube.com/watch?v=hskZWNkOpro&embedable=true ten ten ten ten is a social walkie-talkie app that allows reaching out to friends anytime, without prior notice, even when their phone is in their pocket with a screen locked. Yes, you read it right. Sounds weird? Wait until the app starts talking to you through phone speakers to onboard you! While many people might not be ready for such an open relationship, it’s only right for a product like this to strike with its key value proposition right away. https://www.youtube.com/watch?v=qxfRQACIpl0&embedable=true Session Session is an end-to-end encrypted messenger with a focus on privacy and anonymity. To emphasize the product’s mission, the company actively uses interface metaphors during the onboarding phase. The very first product intro is presented through a chat interface, native to a messenger product. A new user then receives a Session ID, delivered through a simulation of a random selection of characters and numbers. Finally, the allure of protected experience is topped off with a unique recovery phrase hidden under a long tap. https://www.youtube.com/watch?v=2rZbPbHLTng&embedable=true Headspace This one is brilliantly simple. Headspace is a meditation and mindfulness app. While most companies use a splash screen to show off their logo, Headspace begins like any meditation does — with a deep breath in and a slow breath out. https://www.youtube.com/watch?v=547JxYQMttg&embedable=true Zing Coach Zing Coach is an AI-powered fitness app. Like most popular apps in the health & fitness category, Zing Coach starts with an interactive survey on a new user’s lifestyle and goals to create a supposedly personalized training program. What the app does differently is its fitness level measuring test. By asking a new user to go through a real 5-minute test workout to adjust the intensity for future exercises, Zing Coach brings new users to the first moment faster, breaking the very first barrier of starting regular training. a-ha! https://www.youtube.com/watch?v=yjlYk_gD4sc&embedable=true Shuffles Shuffles is a Pinterest app for creating striking animated collages. While the use of video in onboarding is rarely justified by utility, in this case, it only feels right to show off the crazy amount of creativity embedded in the app compared to most competitors, while simultaneously demonstrating what it’s capable of. https://www.youtube.com/watch?v=RP5BkTe9igw&embedable=true Me+ Me+ is a planner app striving to turn daily routines into habits. Throughout the whole onboarding journey, the app consistently backs its value proposition with different forms of : a high percentage of customers achieving positive change, App Store rating, and reviews. social proof Insight Timer The meditation app Insight Timer takes a step further by binding social proof with a sense of belonging. Not only does the app boast impressive metrics such as the number of customers and reviews, but it also supports new users by showing they are not alone in their struggles and there are lots of other people who go through the very same challenges. Must Must is another example of an app where the use of video in onboarding feels only natural. The app lets its customers discover, track, and review movies as well as connect with friends around a common passion. By welcoming new users with a full-screen showreel of everyone’s favorite hits, Must immerses newcomers into the movie experience from the get-go. https://www.youtube.com/watch?v=LJ04Kv4hYFI&embedable=true Woofz Woofz is a dog-training app that provides its customers with professional advice on care, health, and fitness monitoring. One of the first steps that the app takes a new user through is pledging for a dog parent’s commitments and stamping that pledge into the owner’s training passport, thus formally signifying responsibility to a pet. https://www.youtube.com/watch?v=6qmDdb22nUE&embedable=true BetterSleep While for some products a native communication medium is visuals or haptics, BetterSleep opens up with a calm, dreamy sleepcast that sets up a new user for an unwinding experience from the get-go. https://www.youtube.com/watch?v=yN-Hd7Qn-Xc&embedable=true Yazio While many health-related apps claim to provide training programs personalized according to user preferences and goals, rarely do they make clear how personalization actually works. Yazio at least explicitly shows the parameters its personalization takes into account. After a personalized program is ready, Yazio explains how it will help a new user to achieve goals. https://www.youtube.com/watch?v=bgKGs-9rga4&embedable=true Fastic Most mobile applications seek notification permission from new users during the very first session to be able to return them later. The food-tracking app Fastic incentivizes users to enable notifications by both showing the types of messages they will receive and social proofing with an increased probability of achieving dieting goals. https://www.youtube.com/watch?v=bdF1tT4YrbA&embedable=true Brickit While most products focus on productivity, some rely on emotions. Brickit is an app that suggests new ideas to build from your existing Lego bricks. To enhance creativity and playfulness from the very beginning, Brickit onboards new users with the help of a brick mascot that combines a robot-like appearance with humane mimics. https://www.youtube.com/watch?v=tWnTWFRMwRo&embedable=true Conclusion What this exploration proves is that there is no just one singular pattern to onboard new customers. Instead, look for what feels natural to your product, be it communication medium, interface metaphors, social proof, associative coherence, pledge of responsibility, or immersive, emotional touch. Hopefully, these examples will give you new angles to look at familiar topics. If you are just getting started with onboarding, check out . the general approach that I worked out while building onboarding for three popular consumer mobile apps Thank you for giving this a read. Please share your thoughts in the comments. Would love to hear what other techniques you tried to make onboarding successful.

15 User Onboarding Techniques I Found In Consumer Mobile Apps

15টি ইউজার অনবোর্ডিং টেকনিক যা আমি কনজিউমার মোবাইল অ্যাপে পেয়েছি

15 Benutzer-Onboarding-Techniken, die ich in mobilen Apps für Verbraucher gefunden habe

15 técnicas de incorporación de usuarios que encontré en aplicaciones móviles para consumidores

15 techniques d'intégration des utilisateurs que j'ai trouvées dans les applications mobiles grand public

15 उपयोगकर्ता ऑनबोर्डिंग तकनीकें जो मुझे उपभोक्ता मोबाइल ऐप्स में मिलीं

消費者向けモバイル アプリで見つけた 15 のユーザー オンボーディング テクニック

소비자 모바일 앱에서 발견한 15가지 사용자 온보딩 기술

15 técnicas de integração de usuários que encontrei em aplicativos móveis de consumo

15 методов адаптации пользователей, которые я нашел в потребительских мобильных приложениях

Tüketici Mobil Uygulamalarında Bulduğum 15 Kullanıcı Katılım Tekniği

15 kỹ thuật giới thiệu người dùng tôi tìm thấy trong ứng dụng di động tiêu dùng

我在消费者移动应用程序中发现的 15 种用户引导技巧

A Machine Learning Text Classification Case Study with a Product-driven Twist

একটি পণ্য-চালিত টুইস্ট সহ একটি মেশিন লার্নিং পাঠ্য শ্রেণিবিন্যাস কেস স্টাডি

Eine Fallstudie zur Textklassifizierung durch maschinelles Lernen mit einer produktgesteuerten Wendung

Un estudio de caso de clasificación de textos con aprendizaje automático con un toque impulsado por el producto

Une étude de cas de classification de textes d'apprentissage automatique avec une touche axée sur le produit

उत्पाद-संचालित ट्विस्ट के साथ एक मशीन लर्निंग टेक्स्ट वर्गीकरण केस स्टडी

製品主導型のひねりを加えた機械学習テキスト分類のケーススタディ

제품 중심 트위스트를 사용한 기계 학습 텍스트 분류 사례 연구

Um estudo de caso de classificação de texto de aprendizado de máquina com um toque orientado ao produto

Пример классификации текста с помощью машинного обучения с учетом особенностей продукта

Ürün Odaklı Bir Yönlü Makine Öğrenimi Metin Sınıflandırması Örnek Olay İncelemesi

Nghiên cứu điển hình về phân loại văn bản học máy với khuynh hướng hướng đến sản phẩm

产品驱动的机器学习文本分类案例研究

This paper is available on arxiv under CC 4.0 license. Authors: (1) Jeffrey Ouyang-Zhang, UT Austin (2) Daniel J. Diaz, UT Austin (3) Adam R. Klivans, UT Austin (4) Philipp Krähenbühl, UT Austin Table of Links Abstract & Introduction Related Works Preliminary Methodology Results Limitations Conclusion, Acknowledgements, & References Experimental Setting & Additional Results 7 Conclusion We present a method that efficiently scales thermodynamic stability prediction from single mutations to higher-order mutations. Our key insight is that the effects of mutations on the same protein are correlated. Thus, for a target protein, it suffices to run a deep backbone once and decode the effect of all mutations simultaneously using a shallow decoder. With the AlphaFold model as our backbone, our method outperforms existing methods on a variety of single and multiple mutation benchmarks. Our method scales to millions of mutations with minimal computational overhead and runs in a fraction of the time it would take prior works. 8 Acknowledgements This work is supported by the NSF AI Institute for Foundations of Machine Learning (IFML). References [1] Randomization of genes by pcr mutagenesis. Genome research 2(1), 28–33 (1992) 2 [2] Adams, J.P., Brown, M.J., Diaz-Rodriguez, A., Lloyd, R.C., Roiban, G.D.: Biocatalysis: A Pharma Perspective. Advanced Synthesis and Catalysis 361(11), 2421–2432 (2019). https://doi.org/10.1002/adsc.201900424 1 [3] Ahdritz, G., Bouatta, N., Kadyan, S., Xia, Q., Gerecke, W., O’Donnell, T.J., Berenberg, D., Fisk, I., Zanichelli, N., Zhang, B., Nowaczynski, A., Wang, B., Stepniewska-Dziubinska, M.M., Zhang, S., Ojewole, A., Guney, M.E., Biderman, S., Watkins, A.M., Ra, S., Lorenzo, P.R., Nivon, L., Weitzner, B., Ban, Y.E.A., Sorger, P.K., Mostaque, E., Zhang, Z., Bonneau, R., AlQuraishi, M.: Openfold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization. bioRxiv (2022). https://doi.org/10.1101/2022.11.20.517210 6 [4] Arnold, F.H.: Design by directed evolution. Accounts of chemical research 31(3), 125–131 (1998) 1, 2, 3 [5] Bell, E.L., Finnigan, W., France, S.P., Green, A.P., Hayes, M.A., Hepworth, L.J., Lovelock, S.L., Niikura, H., Osuna, S., Romero, E., Ryan, K.S., Turner, N.J., Flitsch, S.L.: Biocatalysis. Nature Reviews Methods Primers 1(1), 1–21 (2021). https://doi.org/10.1038/s43586-021-00044-z 1 [6] Benevenuta, S., Pancotti, C., Fariselli, P., Birolo, G., Sanavia, T.: An antisymmetric neural network to predict free energy changes in protein variants. Journal of Physics D: Applied Physics 54(24), 245403 [7] Benevenuta, S., Pancotti, C., Fariselli, P., Birolo, G., Sanavia, T.: An antisymmetric neural network to predict free energy changes in protein variants. Journal of Physics D: Applied Physics 54(24), 245403 (2021) 8 [8] Benevenuta, S., Birolo, G., Sanavia, T., Capriotti, E., Fariselli, P.: Challenges in predicting stabilizing variations: An exploration. Frontiers in Molecular Biosciences 9, 1075570 (2023) 5, 6, 17 [9] Brückner, A., Polge, C., Lentze, N., Auerbach, D., Schlattner, U.: Yeast two-hybrid, a powerful tool for systems biology. International journal of molecular sciences 10(6), 2763–2788 (2009) 1 [10] Cadet, F., Saavedra, E., Syren, P.O., Gontero, B.: Machine learning, epistasis, and protein engineering: From sequence-structure-function relationships to regulation of metabolic pathways. Frontiers in Molecular Biosciences 9, 1098289 (2022) 1 [11] Capriotti, E., Fariselli, P., Casadio, R.: I-mutant2. 0: predicting stability changes upon mutation from the protein sequence or structure. Nucleic acids research 33(suppl_2), W306–W310 (2005) 3, 8 [12] Chen, K., Arnold, F.H.: Tuning the activity of an enzyme for unusual environments: sequential random mutagenesis of subtilisin e for catalysis in dimethylformamide. Proceedings of the National Academy of Sciences 90(12), 5618–5622 (1993) 1, 3 [13] Chen, T., Gong, C., Diaz, D.J., Chen, X., Wells, J.T., Wang, Z., Ellington, A., Dimakis, A., Klivans, A., et al.: Hotprotein: A novel framework for protein thermostability prediction and editing. In: The Eleventh International Conference on Learning Representations (2022) 3 [14] Chen, Y., Lu, H., Zhang, N., Zhu, Z., Wang, S., Li, M.: Premps: Predicting the impact of missense mutations on protein stability. PLoS computational biology 16(12), e1008543 (2020) 3, 8 [15] Cheng, J., Randall, A., Baldi, P.: Prediction of protein stability changes for single-site mutations using support vector machines. Proteins: Structure, Function, and Bioinformatics 62(4), 1125–1132 (2006) 3, 8 [16] Dehghanpoor, R., Ricks, E., Hursh, K., Gunderson, S., Farhoodi, R., Haspel, N., Hutchinson, B., Jagodzinski, F.: Predicting the effect of single and multiple mutations on protein structural stability. Molecules 23(2), 251 (2018) 4, 7 [17] Dehouck, Y., Kwasigroch, J.M., Gilis, D., Rooman, M.: Popmusic 2.1: a web server for the estimation of protein stability changes upon mutation and sequence optimality. BMC bioinformatics 12(1), 1–12 (2011) 3, 8 [18] Diaz, D.J., Gong, C., Ouyang-Zhang, J., Loy, J.M., Wells, J.T., Yang, D., Ellington, A.J., Dimakis, A., Klivans, A.R.: Stability oracle: A structure-based graph-transformer for identifying stabilizing mutations (2023). https://doi.org/10.1101/2023.05.15.540857 3, 4, 7, 8, 16, 17, 18, 19 [19] Diaz, D.J., Kulikova, A.V., Ellington, A.D., Wilke, C.O.: Using machine learning to predict the effects and consequences of mutations in proteins. Current Opinion in Structural Biology 78, 102518 (2023) 3 [20] d’Oelsnitz, S., Diaz, D.J., Acosta, D.J., Schechter, M.W., Minus, M.B., Howard, J.R., Do, H., Loy, J., Alper, H., Ellington, A.D.: Synthetic microbial sensing and biosynthesis of amaryllidaceae alkaloids. bioRxiv pp. 2023–04 (2023) 3 [21] d’Oelsnitz, S., Kim, W., Burkholder, N.T., Javanmardi, K., Thyer, R., Zhang, Y., Alper, H.S., Ellington, A.D.: Using fungible biosensors to evolve improved alkaloid biosyntheses. Nature Chemical Biology 18(9), 981–989 (2022) 1 [22] Ellefson, J.W., Gollihar, J., Shroff, R., Shivram, H., Iyer, V.R., Ellington, A.D.: Synthetic evolutionary origin of a proofreading reverse transcriptase. Science 352(6293), 1590–1593 (2016) 1 [23] Feng, X., Sanchis, J., Reetz, M.T., Rabitz, H.: Enhancing the efficiency of directed evolution in focused enzyme libraries by the adaptive substituent reordering algorithm. Chemistry–A European Journal 18(18), 5646–5654 (2012) 3 [24] Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J.K., Brock, K., Gal, Y., Marks, D.S.: Disease variant prediction with deep generative models of evolutionary data. Nature 599(7883), 91–95 (2021) 9 [25] Gebauer, M., Skerra, A.: Engineered protein scaffolds as next-generation therapeutics. Annual Review of Pharmacology and Toxicology 60, 391–415 (2020). https://doi.org/10.1146/annurev-pharmtox-010818- 021118 1 [26] Gerasimavicius, L., Livesey, B.J., Marsh, J.A.: Correspondence between functional scores from deep mutational scans and predicted effects on protein stability. bioRxiv pp. 2023–02 (2023) 9 [27] Giver, L., Gershenson, A., Freskgard, P.O., Arnold, F.H.: Directed evolution of a thermostable esterase. Proceedings of the National Academy of Sciences 95(22), 12809–12813 (1998) 2, 3 [28] Hie, B.L., Shanker, V.R., Xu, D., Bruun, T.U., Weidenbacher, P.A., Tang, S., Wu, W., Pak, J.E., Kim, P.S.: Efficient evolution of human antibodies from general protein language models. Nature Biotechnology (2023) 3 [29] Jaroszewicz, W., Morcinek-Orłowska, J., Pierzynowska, K., Gaffke, L., W˛egrzyn, G.: Phage display and other peptide display technologies. FEMS Microbiology Reviews 46(2), fuab052 (2022) 1 [30] Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS) 20(4), 422–446 (2002) 6, 17 [31] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al.: Highly accurate protein structure prediction with alphafold. Nature 596(7873), 583–589 (2021) 2, 3, 4, 6, 9, 10 [32] Kellogg, E.H., Leaver-Fay, A., Baker, D.: Role of conformational sampling in computing mutation-induced changes in protein structure and stability. Proteins: Structure, Function, and Bioinformatics 79(3), 830–838 (2011) 3 [33] Klabunde, T., Petrassi, H.M., Oza, V.B., Raman, P., Kelly, J.W., Sacchettini, J.C.: Rational design of potent human transthyretin amyloid disease inhibitors. Nature structural biology 7(4), 312–321 (2000) 1, 2 [34] Kouba, P., Kohout, P., Haddadi, F., Bushuiev, A., Samusevich, R., Sedlar, J., Damborsky, J., Pluskal, T., Sivic, J., Mazurenko, S.: Machine learning-guided protein engineering. ACS Catalysis 13, 13863–13895 (2023) 3 [35] Laimer, J., Hofer, H., Fritz, M., Wegenkittl, S., Lackner, P.: Maestro-multi agent stability prediction upon point mutations. BMC bioinformatics 16(1), 1–13 (2015) 3, 4, 7, 8 [36] Li, B., Yang, Y.T., Capra, J.A., Gerstein, M.B.: Predicting changes in protein thermodynamic stability upon point mutation with deep 3d convolutional neural networks. PLoS computational biology 16(11), e1008291 (2020) 3, 4, 8 [37] Li, G., Panday, S.K., Alexov, E.: Saafec-seq: a sequence-based method for predicting the effect of single point mutations on protein thermodynamic stability. International journal of molecular sciences 22(2), 606 (2021) 3 [38] Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S., et al.: Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv (2022) 3, 4, 6, 9, 10, 18 [39] Loell, K., Nanda, V.: Marginal protein stability drives subcellular proteome isoelectric point. Proceedings of the National Academy of Sciences 115(46), 11778–11783 (2018) 1 [40] Lu, H., Diaz, D.J., Czarnecki, N.J., Zhu, C., Kim, W., Shroff, R., Acosta, D.J., Alexander, B.R., Cole, H.O., Zhang, Y., et al.: Machine learning-aided engineering of hydrolases for pet depolymerization. Nature 604(7907), 662–667 (2022) 3 [41] Meghwanshi, G.K., Kaur, N., Verma, S., Dabi, N.K., Vashishtha, A., Charan, P.D., Purohit, P., Bhandari, H.S., Bhojak, N., Kumar, R.: Enzymes for pharmaceutical and therapeutic applications. Biotechnology and Applied Biochemistry 67(4), 586–601 (2020). https://doi.org/10.1002/bab.1919 1 [42] Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., Rives, A.: Language models enable zero-shot prediction of the effects of mutations on protein function. Advances in Neural Information Processing Systems 34, 29287–29303 (2021) 9, 18 [43] Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., Steinegger, M.: Colabfold: making protein folding accessible to all. Nature methods 19(6), 679–682 (2022) 6 [44] Montanucci, L., Capriotti, E., Frank, Y., Ben-Tal, N., Fariselli, P.: Ddgun: an untrained method for the prediction of protein stability changes upon single and multiple point variations. BMC bioinformatics 20, 1–10 (2019) 3, 4, 7, 8, 16, 18 [45] Montanucci, L., Savojardo, C., Martelli, P.L., Casadio, R., Fariselli, P.: On the biases in predictions of protein stability changes upon variations: the inps test case. Bioinformatics 35(14), 2525–2527 (2019) 16 [46] Musdal, Y., Govindarajan, S., Mannervik, B.: Exploring sequence-function space of a poplar glutathione transferase using designed information-rich gene variants. Protein Engineering, Design and Selection 30(8), 543–549 (2017) 3 [47] Nijkamp, E., Ruffolo, J., Weinstein, E.N., Naik, N., Madani, A.: Progen2: exploring the boundaries of protein language models. arXiv preprint arXiv:2206.13517 (2022) 9 [48] Nikam, R., Kulandaisamy, A., Harini, K., Sharma, D., Gromiha, M.M.: Prothermdb: thermodynamic database for proteins and mutants revisited after 15 years. Nucleic acids research 49(D1), D420–D424 (2021) 2, 16 [49] Notin, P., Dias, M., Frazer, J., Hurtado, J.M., Gomez, A.N., Marks, D., Gal, Y.: Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. In: International Conference on Machine Learning. pp. 16990–17017. PMLR (2022) 2, 3, 9, 16, 18 [50] Notin, P.M., Van Niekerk, L., Kollasch, A.W., Ritter, D., Gal, Y., Marks, D.: Trancepteve: Combining family-specific and family-agnostic models of protein sequences for improved fitness prediction. bioRxiv pp. 2022–12 (2022) 9 [51] Nutschel, C., Fulton, A., Zimmermann, O., Schwaneberg, U., Jaeger, K.E., Gohlke, H.: Systematically scrutinizing the impact of substitution sites on thermostability and detergent tolerance for bacillus subtilis lipase a. Journal of chemical information and modeling 60(3), 1568–1584 (2020) 9 [52] Paik, I., Ngo, P.H., Shroff, R., Diaz, D.J., Maranhao, A.C., Walker, D.J., Bhadra, S., Ellington, A.D.: Improved bst dna polymerase variants derived via a machine learning approach. Biochemistry (2021) 3 [53] Pak, M.A., Markhieva, K.A., Novikova, M.S., Petrov, D.S., Vorobyev, I.S., Maksimova, E.S., Kondrashov, F.A., Ivankov, D.N.: Using alphafold to predict the impact of single mutations on protein stability and function. Plos one 18(3), e0282689 (2023) 3 [54] Pancotti, C., Benevenuta, S., Birolo, G., Alberini, V., Repetto, V., Sanavia, T., Capriotti, E., Fariselli, P.: Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset. Briefings in Bioinformatics 23(2), bbab555 (2022) 2, 8, 16, 17 [55] Pancotti, C., Benevenuta, S., Repetto, V., Birolo, G., Capriotti, E., Sanavia, T., Fariselli, P.: A deep-learning sequence-based method to predict protein stability changes upon genetic variations. Genes 12(6), 911 (2021) 8 [56] Pires, D.E., Ascher, D.B., Blundell, T.L.: Duet: a server for predicting effects of mutations on protein stability using an integrated computational approach. Nucleic acids research 42(W1), W314–W319 (2014) 3, 8 [57] Pires, D.E., Ascher, D.B., Blundell, T.L.: mcsm: predicting the effects of mutations in proteins using graph-based signatures. Bioinformatics 30(3), 335–342 (2014) 8 [58] Pucci, F., Schwersensky, M., Rooman, M.: Artificial intelligence challenges for predicting the impact of mutations on protein stability. Current opinion in structural biology 72, 161–168 (2022) 17 [59] Qiu, Y., Wei, G.W.: Persistent spectral theory-guided protein engineering. Nature Computational Science pp. 1–15 (2023) 3, 6, 17 [60] Rao, R.M., Liu, J., Verkuil, R., Meier, J., Canny, J., Abbeel, P., Sercu, T., Rives, A.: Msa transformer. In: International Conference on Machine Learning. pp. 8844–8856. PMLR (2021) 3, 9, 10 [61] Riesselman, A.J., Ingraham, J.B., Marks, D.S.: Deep generative models of genetic variation capture the effects of mutations. Nature methods 15(10), 816–822 (2018) 9 [62] Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C.L., Ma, J., Fergus, R.: Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS (2019). https://doi.org/10.1101/622803 3, 7 [63] Rodrigues, C.H., Pires, D.E., Ascher, D.B.: Dynamut: predicting the impact of mutations on protein conformation, flexibility and stability. Nucleic acids research 46(W1), W350–W355 (2018) 3, 8 [64] Rodrigues, C.H., Pires, D.E., Ascher, D.B.: Dynamut2: Assessing changes in stability and flexibility upon single and multiple point missense mutations. Protein Science 30(1), 60–69 (2021) 3, 4, 7 [65] Sanavia, T., Birolo, G., Montanucci, L., Turina, P., Capriotti, E., Fariselli, P.: Limitations and challenges in protein stability prediction upon genome variations: towards future applications in precision medicine. Computational and Structural Biotechnology Journal 18, 1968–1979 (2020). https://doi.org/10.1016/j.csbj.2020.07.011, https://doi.org/10.1016/j.csbj.2020.07.011 6, 17 [66] Savojardo, C., Fariselli, P., Martelli, P.L., Casadio, R.: Inps-md: a web server to predict stability of protein variants from sequence and structure. Bioinformatics 32(16), 2542–2544 (2016) 3, 8 [67] Schymkowitz, J., Borg, J., Stricher, F., Nys, R., Rousseau, F., Serrano, L.: The foldx web server: an online force field. Nucleic acids research 33(suppl_2), W382–W388 (2005) 3, 7, 8, 18 [68] Shroff, R., Cole, A.W., Diaz, D.J., Morrow, B.R., Donnell, I., Annapareddy, A., Gollihar, J., Ellington, A.D., Thyer, R.: Discovery of novel gain-of-function mutations guided by structure-based deep learning. ACS synthetic biology 9(11), 2927–2935 (2020) 3 [69] Starr, T.N., Thornton, J.W.: Epistasis in protein evolution. Protein science 25(7), 1204–1218 (2016) 1 [70] Steinegger, M., Soding, J.: Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology 35(11), 1026–1028 (2017) 6 [71] Stemmer, W.P.: Rapid evolution of a protein in vitro by dna shuffling. Nature 370(6488), 389–391 (1994) 2, 3 [72] Tsuboyama, K., Dauparas, J., Chen, J., Laine, E., Mohseni Behbahani, Y., Weinstein, J.J., Mangan, N.M., Ovchinnikov, S., Rocklin, G.J.: Mega-scale experimental analysis of protein folding stability in biology and protein design. bioRxiv pp. 2022–12 (2022) 1, 2, 6, 16 [73] Tung, J.W., Heydari, K., Tirouvanziam, R., Sahaf, B., Parks, D.R., Herzenberg, L.A., Herzenberg, L.A.: Modern flow cytometry: a practical approach. Clinics in laboratory medicine 27(3), 453–468 (2007) 1 [74] Umerenkov, D., Shashkova, T.I., Strashnov, P.V., Nikolaev, F., Sindeeva, M., Ivanisenko, N.V., Kardymon, O.L.: Prostata: Protein stability assessment using transformers. bioRxiv pp. 2022–12 (2022) 3, 4, 6, 7, 8, 18, 19 [75] Wittmann, B.J., Johnston, K.E., Wu, Z., Arnold, F.H.: Advances in machine learning for directed evolution. Current opinion in structural biology 69, 11–18 (2021) 3 [76] Wittmann, B.J., Yue, Y., Arnold, F.H.: Informed training set design enables efficient machine learningassisted directed protein evolution. Cell Systems 12(11), 1026–1045 (2021) 3 [77] Worth, C.L., Preissner, R., Blundell, T.L.: Sdm—a server for predicting effects of mutations on protein stability and malfunction. Nucleic acids research 39(suppl_2), W215–W222 (2011) 3, 8 [78] Wu, S., Snajdrova, R., Moore, J.C., Baldenius, K., Bornscheuer, U.T.: Biocatalysis: Enzymatic Synthesis for Industrial Applications. Angewandte Chemie - International Edition 60(1), 88–119 (2021). https://doi.org/10.1002/anie.202006648 1 [79] Yeung, N., Lin, Y.W., Gao, Y.G., Zhao, X., Russell, B.S., Lei, L., Miner, K.D., Robinson, H., Lu, Y.: Rational design of a structural and functional nitric oxide reductase. Nature 462(7276), 1079–1082 (2009) 1, 2 [80] Zhou, B., Lv, O., Yi, K., Xiong, X., Tan, P., Hong, L., Wang, Y.G.: Accurate and definite mutational effect prediction with lightweight equivariant graph neural networks. arXiv preprint arXiv:2304.08299 (2023) 3 [81] Zhou, Y., Pan, Q., Pires, D.E., Rodrigues, C.H., Ascher, D.B.: Ddmut: predicting effects of mutations on protein stability using deep learning. Nucleic Acids Research p. gkad472 (2023) 3

Predicting a Protein’s Stability
under a Million Mutations: Conclusion, Acknowledgement & References

This paper is available on arxiv under CC 4.0 license. Authors: (1) Jeffrey Ouyang-Zhang, UT Austin (2) Daniel J. Diaz, UT Austin (3) Adam R. Klivans, UT Austin (4) Philipp Krähenbühl, UT Austin Table of Links Abstract & Introduction Related Works Preliminary Methodology Results Limitations Conclusion, Acknowledgements, & References Experimental Setting & Additional Results 2 Related Works 2.1 Protein Engineering Protein engineering is the process of mutating a natural protein to improve particular phenotypes, such as thermodynamic stability and function. The field has historically relied on rational design and stochastic methods, such as error-prone PCR [1], DNA shuffling [71], and directed evolution (DE) [4, 27], to identify gain-of-function mutations. Rational design is limited to proteins with solved structures and requires an extensive understanding of biochemistry and specific knowledge of the particular protein to select mutations highly likely to improve the target phenotypes [33, 79]. Directed evolution requires little to no knowledge of the protein and instead generates a library of protein variants that are then screened for the target phenotypes (e.g. fluorescence brightness, antibiotic resistance, stability, activity) [4, 27]. The library can be generated via site-saturated mutagenesis of a handful of positions in the sequence [12] or DNA shuffling, in which the gene is digested into random fragments and reassembled into full-length sequences [71]. After screening the library, the most "fit" variant is then selected as the initial sequence for the next round of directed evolution. This iterative process repeats until it obtains a protein variant with the desired phenotypes. Machine learning has demonstrated its ability to augment rational design and accelerate the stabilization of a variety of proteins [20, 28, 40, 52, 68]. Separately, machine learning-guided directed evolution (MLDE)[75] has been shown to improve the likelihood of obtaining the global fitness maximum by 81-fold compared to traditional DE [76]. MLDE has accelerated the engineering of several proteins, such as the enantioselectivity of enzymes for kinetic resolution of epoxides [23] and the activity and expression of a glutathione transferase [46]. Mutate Everything empowers the experimentalist to accelerate the stabilization of a protein for both rational design and MLDE. 2.2 Machine Learning for Protein Structure Prediction Recent advances in machine learning have led to remarkable progress in protein structure prediction. AlphaFold [31] has demonstrated that deep learning is highly effective at predicting protein structures from a sequence by using evolutionary history via a multiple sequence alignment (MSA). AlphaFold passes the MSA and a pairwise representation of the sequence into Evoformer to capture the coevolutionary patterns between residues. The Evoformer output is then processed by the Structure Module, which predicts the protein’s structure. We challenge prior works postulating that AlphaFold cannot be used for stability prediction and show that fine-tuning these rich co-evolutionary and structural features yield highly performant stability predictors [53]. Evolutionary Scale Modeling (ESM) [38, 62] has shown that protein structure prediction can be performed without MSAs and specialized architectures by leveraging large transformers pre-trained on masked token prediction. Other works extended this masked pre-training framework, including MSA-Transformer [60] which incorporates a sequence’s MSA as input, and Tranception [49], which develops a hybrid convolution and attention based autoregressive architecture. We show that finetuning these evolutionary representations excels at protein stability assessment without MSAs. 2.3 Protein Stability Assessment Traditional approaches to protein stability assessment relied on a combination of physics-based methods, statistical analysis, and traditional machine learning techniques. Physics-based tools, such as FoldX, Rosetta, and SDM, utilize energy functions and statistical patterns to assess how mutations affect a protein’s stability [32, 67, 77]. DDGun [44] directly inferred ∆∆G from heuristics, including Blosum substitution scores, differences in interaction energy with neighboring residues, and changes in hydrophobicity. Many traditional machine learning approaches used support vector machines and decision trees with physics-based feature engineering [11, 14, 15, 17, 37, 66]. Others ensemble existing machine learning methods [35, 56, 59, 63, 64]. Recently, deep learning-based approaches have recently begun to outperform existing physics and traditional machine learning approaches [13, 19, 34]. ACDC-NN [6] trains an asymmetric convolutional neural network for predicting forward and reverse mutation effects. Thermonet [36] voxelizes and feeds both the wild-type and mutant protein structures into a 3D-CNN to regress a ∆∆G value. PROSTATA feeds the wild-type and mutant protein sequence into a pre-trained ESM2 model and then regresses a ∆∆G value [74]. Stability Oracle takes the native and mutant amino acid type along with the local structure as input and regresses ∆∆G [18]. Several deep learning frameworks [64, 80, 81] model multiple mutations in addition to single mutations. In this paper, we develop a framework that models the protein to enable efficient enumeration of all mutation candidates.

Predicting a Protein’s Stability
under a Million Mutations: Related Works

Overview According to statistics, there were 34 funding rounds executed in the global blockchain market from February 19 to February 25, totaling 240.81 million dollars. The number of funding rounds increased by 13 and the amount of funds increased by 75.7%, compared with the previous week. The number of funding rounds in sectors of Infrastructure and DeFi is the highest, 11 of which are in the infrastructure and 7 in the DeFi area. The funds raised in the infrastructure sector are the highest, which is 170 million dollars. Funding in different sectors Infrastructure EigenLayer Round Series B Amount $100M Investors a16z, Polychain… Project Intro: EigenLayer is a protocol built on Ethereum that introduces restaking, a new primitive in cryptoeconomic security. This primitive enables the rehypothecation of ETH on the consensus layer. Users who stake ETH can opt-in to EigenLayer smart contracts to restake their ETH and extend cryptoeconomic security to additional applications on the network. Part of EigenLayer's potential lies in its ability to aggregate and extend cryptoeconomic security through restaking, and to validate new applications being built on top of Ethereum. AltLayer Round Strategic Amount $14.4M Investors Polychain, Hack VC… Project Intro: AltLayer is a system of highly scalable, application-dedicated execution layers that derive security from an underlying L1/L2. It is designed as a modular and pluggable framework for a multi-chain and multi-VM world. At its core, AltLayer consists of several optimistic rollup-like execution layers called flash layers, featuring a novel innovation that makes them disposable and thus highly resource-optimized. Flare Network Round / Amount $35M Investors DWF Labs Project Intro: Flare is a Layer-1 blockchain that enables secure, universal interoperability between chains, scaling blockchain use by allowing all digital assets and on-chain information to flow freely. Its consensus protocol for external data allows decentralized applications on Flare to safely and trustlessly acquire and use information from other blockchains and real-world data sources, solving the oracle problem and ushering in a new era of utility. Clave Round Pre-Seed Amount $1.6M Investors zkSync Labs Clave is an easy-to-use non-custodial smart wallet powered by Account Abstraction and the Hardware Elements (e.g Secure Enclave, Android Trustzone etc.), offering a unique onboarding process. Helius Round Series A Amount $9.5M Investors Foundation Capital, Reciprocal Ventures Helius aims to help creators overcome these obstacles by providing APIs (or application programming interfaces) to understand on-chain data and query transactions, as well as webhooks that enable automations and bots, plus RPC nodes that allow Solana apps to interact with the blockchain. All in all, they are designed to simplify complexity and facilitate application development. NodeShift Round Seed Amount $3.2M Investors Inovo VC NodeShift is a decentralized cloud service provider that provides a single API for accessing independent data center operators' redundant compute, storage and graphics accelerators and connecting to low-cost decentralized web services such as Akash and Filecoin. BEVM Round / Amount / Investors ViaBTC Capital BEVM is a decentralized EVM-compatible Bitcoin L2 that uses BTC as Gas. It allows all DApps which can run in the Ethereum ecosystem to operate on Bitcoin L2. Citrea Round Seed Amount $2.7M Investors Galaxy Citrea is a Bitcoin’s ZK rollup and an execution layer that settles on Bitcoin. Every transaction occurring on Citrea is fully secured by zero-knowledge proofs and verified by Bitcoin through BitVM. Inco Network Round Seed Amount $4.5M Investors 1kx, Circle Ventures Inco Network is an EVM Layer-1, utilizing fully homomorphic encryption (FHE) to enable composable and operable confidential states. Its fhEVM (FHE + EVM) makes it possible to write private smart contracts and perform computations on top of encrypted data without requiring its decryption. This innovation unlocks a wide range use cases with confidentiality components, such as card games, DID, private voting, dark pools, etc. - paving the way for a new era of possibilities. Nest Wallet Round Seed Amount $3.6M Investors Inception Capital, Orange DAO Nest Wallet is a multi-chain wallet built for traders. Powered by Safe, Nest Wallet adds a more friendly UI/UX for team to easily manage multiple Safe wallets and streamline the transaction signing process. DeFi ZeroLend Round Seed Amount $3M Investors Momentum 6 ZeroLend is a decentralized lending protocol on zkSync. Designed to provide individuals and businesses with a secure and transparent platform for borrowing and lending digital assets. The protocol operates in a peer-to-peer manner, eliminating the need for intermediaries such as banks or traditional financial institutions. After announcing the launch of their platform token $ZERO, ZeroLend has introduced a series of activities to encourage users to complete tasks and accumulate points. Recently, they have collaborated with the growth platform TaskOn to release a new set of activities. Shutter Round IDO Amount $8.31M Investors OKX Ventures Shutter Network is an open-source project that aims to prevent front running on Ethereum by using a threshold cryptography-based distributed key generation (DKG) protocol.Shutter allows users to send encrypted transactions in a way that protects them from front runners on their path through the dark forest (the metaphorical hunting ground of front runners that each transaction must cross). OX.FUN Round / Amount $4M Investors Gen Block Capital OX.FUN is a crypto derivatives trading platform. The core element of the gamified experience on OX.FUN revolves around the OX Farm. User can also expect to see leaderboards, competitions and copy trading vaults to help maximize the trading adventure. Renzo Round / Amount / Investors OKX Ventures, Binance Labs Renzo is a restaking protocol based on EigenLayer. Renzo abstracts the complex process of re-staking for end users, and stakers do not have to worry about the active selection and management of operators and reward strategies. ezETH is Renzo’s liquid, heavily-collateralized token that automatically captures rewards and ensures liquidity. ezETH allows users to participate in DeFi while retaining re-collateralization rewards. Singularity Round / Amount $2.2M Investors OKX Ventures, Binance Labs Singularity is a compliant institutional DeFi access layer that provides access to popular protocols for institutional on-chain participants with commercial confidentiality. Users will have their wallet addresses obfuscated whilst leveraging existing DeFi liquidity. Singularity uses the state-of-the-art UltraPLONK proof system with zero-knowledge circuits based on Noir. Honeypot Finance Round Pre-seed Amount / Investors AC Capital Honeypot Finance is whipping up a community-run DeFi Hub that have a honey-flavored flywheel model that churns out dual incentives. INIT Capital Round Seed Amount $3.1M Investors Robot Ventures, Mirana Ventures… INIT Capital is a Liquidity Hook Money Market: Lend, Borrow, and Access Yield Strategies with Liquidity Hook. Serving as the DeFi liquidity infrastructure for dApps to build on top, INIT facilitates diverse activities, including lending, borrowing, and yield strategies. INIT is where dApps and users meet, offering both permissionless access to a unified liquidity pool and efficient yield strategy management. GameFi Shuffle Round Seed Amount $2.5M Investors Karatage, Cypher Capital Shuffle is an all-new crypto casino that support numerous games and jackpots available for players. Matr1x Round Strategic Amount / Investors Animoca Brands, SevenX Ventures… Matr1x is the brand of Metaverse, planning to build the brand through high-quality games developed by themselves or in collaboration with third parties, game IP development and cooperation, and global Esports tournaments. Matr1x will express its game trilogy (Cyber Earth - Mars Immigration - Interstellar Exploration) through three large-scale games with a unified worldview: Matr1x FIRE (First-person shooter), Matr1x WAR (Shooting+MMORPG), and Matr1x EVOLUTION (SOC). Monkey Tilt Round Private Amount $21M Investors Polychain, Hack VC… Monkey Tilt is an online gambling platform focused on the convergence of cryptocurrencies and online gambling. Players can win real money by correctly predicting whether the player they choose will exceed or fall short of their statistical prediction. Crystal Fun Round Seed Amount $5M Investors Waterdrip Capital, Kucoin Ventures… Crystal Fun is a decentralized gaming mega-ecosystem and studio. It aims to seamlessly integrate high-quality games within sustainable economic systems, ushering in a groundbreaking gaming experience into Web3. Loong City Round / Amount $2M Investors MAZARINE SEA Loong City is the first original IP PFP metaverse project based on an improved ERC404 protocol. Autopunk and traditional China constitute the core elements of Dragon City. Each PFP is an entry ticket to this fictional world. NFT Prom Round / Amount / Investors DWF Labs Prom is the primary access point to the Web3 gaming ecosystem. With a robust marketplace at its core for trading and exchanging NFT assets, Prom offers an array of services including rental and mortgage solutions, comprehensive asset management tools, and an educational section with in-depth guides. Tools FortyTwo Round Strategic Amount / Investors LBankLong, Hash Ventures FortyTwo is a Cosmos Portfolio View Platform that allows users to clearly see the breakdown of their Cosmos holdings. Users can can connect one or a bundle of wallets. Tornado Blast Round Strategic Amount / Investors Atka Capital Tornado Blast is a Blast L2-native bot to trade and snipe tokens. User can earn $BLAST, $TRNDO & revenue share reward SocialFi Beoble Round Seed Amount $5M Investors Hashkey Capital, DWF Labs… Beoble is a communication infrastructure and ecosystem that allows users to chat between wallets. Its product includes a web-based chat app, as well as a toolkit that allows Dapps to integrate. Others HackQuest Round Pre-Seed Amount $1.1M Investors HashKey Capital HackQuest is the gateway to the Web3 development world, providing a comprehensive, one-stop educational infrastructure that minimizes friction and helps onboard developers and nondevelopers to the world of Web3. TruePNL Round M&A Amount / Investors Gate.io Labs, Magic Square.. TruePNL is a multi-chain launchpad that connects blockchain projects and crypto investors, providing a convenient, transparent, and secure mechanism for raising capital, investing, and distributing tokens. The platform enables cryptocurrency projects to raise funds by creating token exchange pools and launching public or private sale rounds using smart contracts on the Ethereum and Binance Smart Chain networks. Superfluid Round Strategic Amount $5.1M Investors Fabric Ventures, Multicoin Capital… Superfluid is a Web3 streaming payments protocol for handling subscriptions, salaries, rewards, or any other composable stream of value on a per-second basis. Its core feature is "streaming" payments, where money flows from one address to another in real-time, based on programmable logic and a single on-chain transaction. Silencio Round Pre-seed Amount $1M Investors Borderless Capital, Moonrock Capital Silencio is a global network that measure noise pollution. User will get rewarded for measuring noise pollution with phone. CeFi Januar Round Seed Plus Amount $12M Investors CMT Digital Januar is a European payment gate for crypto companies. Innovative EU crypto companies use Januar's payment account solutions to accept payments and send payouts while maintaining full compliance and stable business operations.

The Crypto Investment Weekly Recap for Feb 19 - Feb 25

How to Sample From Latent Space With Variational Autoencoder

ভেরিয়েশনাল অটোএনকোডার দিয়ে সুপ্ত স্থান থেকে কীভাবে নমুনা করবেন

So sampeln Sie mit Variational Autoencoder aus dem Latentraum

Cómo muestrear desde el espacio latente con el codificador automático variacional

Comment échantillonner à partir de l'espace latent avec l'encodeur automatique variationnel

वेरिएशनल ऑटोएनकोडर के साथ अव्यक्त स्थान से नमूना कैसे लें

変分オートエンコーダーを使用して潜在空間からサンプリングする方法

Variational Autoencoder를 사용하여 잠재 공간에서 샘플링하는 방법

Como fazer amostragem do espaço latente com codificador automático variacional

Как выполнить выборку из скрытого пространства с помощью вариационного автоэнкодера

Varyasyonel Otomatik Kodlayıcıyla Gizli Uzaydan Örnekleme Nasıl Yapılır?

Cách lấy mẫu từ không gian tiềm ẩn bằng bộ mã hóa tự động biến đổi

如何使用变分自动编码器从潜在空间进行采样

Fanaticism has been, for much of American history, reserved for institutions like sports teams and universities where allegiances run so deep they can (and often have) become a central part of one’s identity. But an interesting shift occurred around the turn of the century where have come to command the same fervent admiration. individuals This happened in politics - that’s a topic for a different publication - and also in business: most prominently with Steve Jobs and Elon Musk. If you connect the dots between what drove the love of those tech leaders and recent events, it is clear the new cult hero in tech is Sam Altman. Altman clinched this position in late November after he was abruptly terminated from his position as the chief executive of OpenAI, the company he helped found in 2015. As has been covered extensively, the OpenAI board unexpectedly asked Altman to join a Google Meet link, at which point they told him he would be fired from OpenAI over what has been described as a “breakdown in communication.” Within two days, 90% of OpenAI employees had signed an open letter to the board threatening to leave OpenAI for Microsoft if they did not reinstate Altman as CEO. This was believed to be such a credible threat that Microsoft spent the weekend loading office equipment and computers into a floor at LinkedIn’s San Francisco office, preparing for their arrival. Ultimately, Altman returned to his job as CEO, and . OpenAI shuffled their board of directors And while it was a jarring sequence of events in short order, the most shocking part wasn’t his firing and re-hiring, the board re-alignment, or the fact that 90% of employees threatened to quit over the events: the biggest surprise was the broader tech industry’s reaction to Altman’s ouster. People who had no relationship with OpenAI or Altman came rushing to his full-throated defense. Opinion pieces were penned. Hundreds of hours of podcasts were recorded. Social media feeds were full of people who had his back despite the fact they had never so much as heard Altman’s voice. All of this activity went well beyond a reaction to someone getting a raw deal, and it felt familiar: like the kind of debate and memes and discourse you see play out about Steve Jobs’ role at Apple or about any of the various things Elon Musk is involved in. The response crossed into fanaticism. The Key Ingredients of a Tech Cult Hero A (perhaps unexpected) benefit of Altman’s firing and re-hiring is that it gave him one of the three core ingredients for the making of a tech cult hero: . People - Americans especially - love a comeback story, and Altman’s was unlike any we had seen before, especially with regard to how quickly it played out. resilience When Jobs was fired from Apple as a result of corporate politics, it took twelve years before he returned to the helm - true to the speed with which things play out in 2024, Altman’s journey took three days. Despite the brevity of his absence, it was intense, and the stakes were high, much like the (many) times Elon took Tesla to the brink of extinction before prevailing en route to becoming the wealthiest person in the world. All three of these individuals also offer . For Jobs, it was (initially) making computers so accessible and easy to use, that you’d welcome them in your home. For Musk, it was the electric car. compelling visions accompanied by convincing breakthroughs And with Sam Altman, it’s clear his moonshot vision - an effort that he has proved capable of leading - is artificial intelligence. Talk is cheap, and many have been drawn in by charismatic visionary types before, but in each of these cases, these people executed. That’s a big box to check off. More than anything, the Jobs and Musk demonstrated throughout their careers, across multiple efforts and companies, is the kind of conviction that people admire. focus and tenacity It’s easy to look at Musk and wonder why he maintains (a material level of) involvement in Tesla, SpaceX, Boring Company, SolarCity, and others, the answer is meaningful: these are the companies and components necessary to realize Musk’s ultimate vision, for humans to become an extra-planetary species. (And interestingly, the biggest thing hurting the approval rating of Elon Musk, at the moment, is attached to the lack of focus he’s demonstrating with his involvement in Twitter. His operation of Twitter won’t help people get to Mars.) Sam Altman Has Demonstrated Resilience and Has Vision and Focus Like Musk’s (previously singular) focus on helping humans become an extra-planetary species, Altman’s focus is clear: artificial intelligence. He understands, as he described it in one interview, “the arc of technology and societal change on a long time horizon.” This has pushed Altman into new, related efforts. Among them: Recognizing the need for a stronger grid to fully realize his vision for the future of technology, Altman is the Chairman of the Board at , an advanced nuclear fission microreactor startup. As Sam Altman told , “I think there’s urgent demand for tons and tons of cheap, safe, clean energy at scale.” energy Oklo CNBC In recognition of a similar issue, Altman has pushed for a level of investment in semiconductors (and generally, ) to a previously-unheard of level. Earlier this month, it was revealed to reshape the semiconductor industry. compute power Altman has sought up to $7 trillion in investment Building the roads, bridges, and tunnels of artificial intelligence isn’t satisfactory for Altman, as he is rumored to be collaborating with Apple’s former designer Jony Ive on that will (presumably) change the form of the way users currently experience artificial intelligence. building a hardware AI device What Does This Mean for Sam Altman? In addition to the personal implications of becoming a business celebrity, Altman will need to manage the bittersweet impact of this increased attention on his professional life. His positioning has allowed (and will continue to allow) him to take shots like the one he took when floating a $7 trillion ask for investment in semiconductors. While most would be laughed off the stage, people leaned in and listened, truly considering a proposition previously unthinkable. Similarly, he’ll likely be found more credible and see the benefit of his reputation when he launches the rumored AI hardware device, more likely to avoid the skepticism than first-movers like Humane and Rabbit. He may find the attention is not all positive. When Oklo completes the process of going public via Altman’s SPAC ($ALCC), it seems an obvious candidate to become a favorite of fanboys and groups like r/WallStreetBets. While likely a good thing for the price, just ask Musk about the (potential) downsides of having so many people around the table. That may make it difficult for Altman and his crew to remain nimble, a drawback in an early-stage environment. His status as a cult hero may also make it difficult to do the things that Altman has proven, time and time again, that he loves: tinkering with technology projects in public. He may find it challenging to launch a fun Chrome extension weekend project. (Though I think he’ll manage.) I ultimately do not know if it’s healthy to treat our business leaders with mythic-like status, nor do I know what to do with or about the phenomenon other than say this: I hope the path and result are better for Sam Altman than it has been for others like him. With both Jobs and Musk, the story has been complicated, reminiscent of a line in Christopher Nolan’s 2008 Dark Knight: “You either die a hero or live long enough to see yourself become a villain.” With Jobs and Musk, I think it’s safe to say this has been true in each case. Here’s hoping Sam Altman can navigate his way to a third outcome, one where he can pursue his vision for the improvement of humanity, without devolving into an internet troll who releases a Cybertruck.

Sam Altman: The Next Big Cult Hero in Tech

স্যাম অল্টম্যান: টেক নেক্সট বিগ কাল্ট হিরো

Sam Altman: Der nächste große Kultheld in der Technik

Sam Altman: el próximo gran héroe de culto en tecnología

Sam Altman : le prochain grand héros culte de la technologie

सैम ऑल्टमैन: द नेक्स्ट बिग कल्ट हीरो इन टेक

サム・アルトマン: テクノロジー界の次なるビッグ・カルト・ヒーロー

Sam Altman: 기술 분야의 차세대 빅 컬트 영웅

Sam Altman: o próximo grande herói de culto em tecnologia

Сэм Альтман: следующий большой культовый герой в сфере технологий

Sam Altman: Teknolojinin Bir Sonraki Büyük Kült Kahramanı

Sam Altman: Người hùng sùng bái lớn tiếp theo trong lĩnh vực công nghệ

萨姆·奥尔特曼：科技领域的下一个大崇拜英雄

Stories, News & Mentions about SHUFFLL

Shuffll’s Stories on HackerNoon