Open Meta

openmeta

Last week, two remarkable events occurred: Meta announced its Llama 3, the most capable and highly anticipated open large language model (LLM) to date, and Boston Dynamics introduced its new fully electric Atlas robot platform — a stark departure from their hydraulic robots! At first glance, these developments may seem unrelated, but they are actually deeply interconnected, with the potential to synergistically drive each other forward and reshape how we work with and implement AI. At the heart of this connection is the transformative power of advanced AI. Breakthroughs in natural language processing and machine learning, exemplified by Llama 3, extend beyond language alone. These techniques, from deep neural networks to reinforcement learning, are also propelling significant advancements in computer vision, motion planning, and robot control. As language models like Llama 3 continue to expand the boundaries of AI’s ability to understand and interact with the world, they also lay the groundwork for more intelligent and capable robots. The implications, of course, extend beyond communication. The same techniques used to train LLMs on vast amounts of text data are also applicable to learning from massive datasets of sensor readings, images, and simulations for robots. Moreover, the open-source nature of models like Llama 3 democratizes access to state-of-the-art AI, enabling a broader spectrum of researchers and companies to integrate these capabilities into their robotic systems. At least start playing on that field! Every successful (and unsuccessful) attempt by Atlas to manipulate an object, navigate a cluttered factory floor, or assist a human worker becomes a valuable data point for the model, just as it does for Llama 3. Using this innovative model, Meta developed a standalone app, meta.ai, and embedded it into WhatsApp, Instagram, and Facebook, collecting vast and highly personalized data. This could lead us toward synthetic social networks or — which is more likely — an extremely personalized, embodied AI experience. As Llama 3 and the electric Atlas converge, they may accelerate each other’s development, blurring the lines between software and hardware, bits and atoms. As language models become more proficient at understanding and interacting with the world, robots will become more capable of applying that knowledge. This doesn’t necessarily mean evil. More capable and advanced robots, which can be fine-tuned and improved with your personal data and specifics, could take over mundane tasks at home. Imagine, instead of hundreds of text editors, finally having a robot that can load and unload the dishwasher and do the laundry. Currently, launches like Llama 3 are seen as enhancing AI’s understanding and processing capabilities, but in the long term, they will be one of the milestones in building and deploying machines that are finely attuned and aligned with us to assist in our daily lives. https://youtu.be/29ECwExc-_M?si=UyexFH1jOfd5TGY9&embedable=true Other Impressive Models: (I didn’t send you the FOD digest last Monday because I was at a conference dedicated to citizen diplomacy. Therefore, today, we have an extensive list of recently launched models that are worth checking out, along with other relevant research papers.) Mixtral-8x22B: Mistral AI introduced a scalable sparse mixture of expert models optimizing cost and latency by selectively using parameters during inference, offering a high capacity and efficiency model for further training and applications →read the paper
Rerank 3: Launched by Cohere, this model enhances enterprise search and Retrieval Augmented Generation systems, improving accuracy in document retrieval across multiple languages and data formats →read the paper
Idefics2: Hugging Face's model that excels in integrating text and image data, significantly improving on OCR and visual question-answering tasks →read the paper
Reka Core, Flash, and Edge: A series of multimodal language models from Reka that process text, images, video, and audio, demonstrating high performance across diverse tasks →read the paper
Ferret-UI: Developed by Apple, this model specializes in mobile UI interaction, enhancing user experience by accurately performing tasks tailored to the unique properties of UI screens →ead the paper
Zamba: Zyphra's compact and efficient SSM Hybrid model is designed for performance with reduced training data needs, optimized for consumer hardware →read the paper
YaART: Yandex's advanced text-to-image diffusion model that optimizes training efficiency and quality with smaller, high-quality datasets →read the paper
RHO-1: A novel approach by Xiamen University focusing on Selective Language Modeling to enhance efficiency by prioritizing useful tokens during training →read the paper
RecurrentGemma: Google DeepMind's model that moves past traditional transformers by incorporating recurrences for more efficient long-sequence processing →read the paper
JetMoE: An economical LLM from the MIT-IBM Watson AI Lab using a mixture-of-experts architecture to achieve high performance at reduced costs →read the paper News from The Usual Suspects © Google open-sourced Gemini Cookbook
and is merging its Research, DeepMind, and Responsible AI teams to accelerate its “capacity to deliver capable AI”
DeepMind published an article about the ethics of advanced AI assistants, stressing the significance of their integration into daily life Hugging Face https://x.com/QGallouedec/status/1782430246957994422?embedable=true Stanford published the 2024 AI Index report, reflecting the escalating impact of AI on society. It contains data with new estimates on AI training costs and detailed insights into the responsible AI landscape. It also introduces a new chapter on AI's influence on science and medicine. Highlights include the substantial cost of training state-of-the-art models, such as GPT-4 and Gemini Ultra; the dominance of the U.S. in producing top AI models; and significant investment growth in generative AI despite overall funding declines. Additionally, the report notes a major increase in AI-related regulations in the U.S. and heightened public awareness and nervousness about AI's future impact. Enjoyed This Story? I write a weekly analysis of the AI world in the Turing Post newsletter. We aim to equip you with comprehensive knowledge and historical insights so you can make informed decisions about AI and ML. Last week, two remarkable events occurred: Meta announced its Llama 3 , the most capable and highly anticipated open large language model (LLM) to date, and Boston Dynamics introduced its new fully electric Atlas robot platform — a stark departure from their hydraulic robots! At first glance, these developments may seem unrelated, but they are actually deeply interconnected, with the potential to synergistically drive each other forward and reshape how we work with and implement AI. Llama 3 Llama 3 electric Atlas robot electric Atlas robot At the heart of this connection is the transformative power of advanced AI. Breakthroughs in natural language processing and machine learning, exemplified by Llama 3, extend beyond language alone. These techniques, from deep neural networks to reinforcement learning, are also propelling significant advancements in computer vision, motion planning, and robot control. As language models like Llama 3 continue to expand the boundaries of AI’s ability to understand and interact with the world, they also lay the groundwork for more intelligent and capable robots. The implications, of course, extend beyond communication. The same techniques used to train LLMs on vast amounts of text data are also applicable to learning from massive datasets of sensor readings, images, and simulations for robots. Moreover, the open-source nature of models like Llama 3 democratizes access to state-of-the-art AI, enabling a broader spectrum of researchers and companies to integrate these capabilities into their robotic systems. At least start playing on that field! Every successful (and unsuccessful) attempt by Atlas to manipulate an object, navigate a cluttered factory floor, or assist a human worker becomes a valuable data point for the model, just as it does for Llama 3. Using this innovative model, Meta developed a standalone app, meta.ai , and embedded it into WhatsApp, Instagram, and Facebook, collecting vast and highly personalized data. This could lead us toward synthetic social networks or — which is more likely — an extremely personalized, embodied AI experience. As Llama 3 and the electric Atlas converge, they may accelerate each other’s development, blurring the lines between software and hardware, bits and atoms. As language models become more proficient at understanding and interacting with the world, robots will become more capable of applying that knowledge. meta.ai meta.ai synthetic social networks synthetic social networks This doesn’t necessarily mean evil. More capable and advanced robots, which can be fine-tuned and improved with your personal data and specifics, could take over mundane tasks at home. Imagine, instead of hundreds of text editors, finally having a robot that can load and unload the dishwasher and do the laundry. Currently, launches like Llama 3 are seen as enhancing AI’s understanding and processing capabilities, but in the long term, they will be one of the milestones in building and deploying machines that are finely attuned and aligned with us to assist in our daily lives. https://youtu.be/29ECwExc-_M?si=UyexFH1jOfd5TGY9&embedable=true https://youtu.be/29ECwExc-_M?si=UyexFH1jOfd5TGY9&embedable=true Other Impressive Models: (I didn’t send you the FOD digest last Monday because I was at a conference dedicated to citizen diplomacy. Therefore, today, we have an extensive list of recently launched models that are worth checking out, along with other relevant research papers.) (I didn’t send you the FOD digest last Monday because I was at a conference dedicated to citizen diplomacy. Therefore, today, we have an extensive list of recently launched models that are worth checking out, along with other relevant research papers.) Mixtral-8x22B: Mistral AI introduced a scalable sparse mixture of expert models optimizing cost and latency by selectively using parameters during inference, offering a high capacity and efficiency model for further training and applications →read the paper Rerank 3: Launched by Cohere, this model enhances enterprise search and Retrieval Augmented Generation systems, improving accuracy in document retrieval across multiple languages and data formats →read the paper Idefics2: Hugging Face's model that excels in integrating text and image data, significantly improving on OCR and visual question-answering tasks →read the paper Reka Core, Flash, and Edge: A series of multimodal language models from Reka that process text, images, video, and audio, demonstrating high performance across diverse tasks →read the paper Ferret-UI: Developed by Apple, this model specializes in mobile UI interaction, enhancing user experience by accurately performing tasks tailored to the unique properties of UI screens →ead the paper Zamba: Zyphra's compact and efficient SSM Hybrid model is designed for performance with reduced training data needs, optimized for consumer hardware →read the paper YaART: Yandex's advanced text-to-image diffusion model that optimizes training efficiency and quality with smaller, high-quality datasets →read the paper RHO-1: A novel approach by Xiamen University focusing on Selective Language Modeling to enhance efficiency by prioritizing useful tokens during training →read the paper RecurrentGemma: Google DeepMind's model that moves past traditional transformers by incorporating recurrences for more efficient long-sequence processing →read the paper JetMoE: An economical LLM from the MIT-IBM Watson AI Lab using a mixture-of-experts architecture to achieve high performance at reduced costs →read the paper Mixtral-8x22B: Mistral AI introduced a scalable sparse mixture of expert models optimizing cost and latency by selectively using parameters during inference, offering a high capacity and efficiency model for further training and applications →read the paper Mixtral-8x22B: →read the paper Rerank 3: Launched by Cohere, this model enhances enterprise search and Retrieval Augmented Generation systems, improving accuracy in document retrieval across multiple languages and data formats →read the paper Rerank 3: →read the paper Idefics2: Hugging Face's model that excels in integrating text and image data, significantly improving on OCR and visual question-answering tasks →read the paper Idefics2: →read the paper Reka Core, Flash, and Edge: A series of multimodal language models from Reka that process text, images, video, and audio, demonstrating high performance across diverse tasks →read the paper Reka Core, Flash, and Edge: →read the paper Ferret-UI: Developed by Apple, this model specializes in mobile UI interaction, enhancing user experience by accurately performing tasks tailored to the unique properties of UI screens →ead the paper Ferret-UI: →ead the paper Zamba: Zyphra's compact and efficient SSM Hybrid model is designed for performance with reduced training data needs, optimized for consumer hardware →read the paper Zamba: →read the paper YaART: Yandex's advanced text-to-image diffusion model that optimizes training efficiency and quality with smaller, high-quality datasets →read the paper YaART: →read the paper RHO-1: A novel approach by Xiamen University focusing on Selective Language Modeling to enhance efficiency by prioritizing useful tokens during training →read the paper RHO-1: →read the paper RecurrentGemma: Google DeepMind's model that moves past traditional transformers by incorporating recurrences for more efficient long-sequence processing →read the paper RecurrentGemma: →read the paper JetMoE: An economical LLM from the MIT-IBM Watson AI Lab using a mixture-of-experts architecture to achieve high performance at reduced costs →read the paper JetMoE: →read the paper News from The Usual Suspects © Google Google open-sourced Gemini Cookbook and is merging its Research, DeepMind, and Responsible AI teams to accelerate its “capacity to deliver capable AI” DeepMind published an article about the ethics of advanced AI assistants, stressing the significance of their integration into daily life open-sourced Gemini Cookbook Gemini Cookbook and is merging its Research, DeepMind, and Responsible AI teams to accelerate its “ capacity to deliver capable AI ” merging capacity to deliver capable AI DeepMind published an article about the ethics of advanced AI assistants, stressing the significance of their integration into daily life published Hugging Face Hugging Face https://x.com/QGallouedec/status/1782430246957994422?embedable=true https://x.com/QGallouedec/status/1782430246957994422?embedable=true Stanford Stanford published the 2024 AI Index report, reflecting the escalating impact of AI on society. It contains data with new estimates on AI training costs and detailed insights into the responsible AI landscape. It also introduces a new chapter on AI's influence on science and medicine. Highlights include the substantial cost of training state-of-the-art models, such as GPT-4 and Gemini Ultra; the dominance of the U.S. in producing top AI models; and significant investment growth in generative AI despite overall funding declines. Additionally, the report notes a major increase in AI-related regulations in the U.S. and heightened public awareness and nervousness about AI's future impact. published the 2024 AI Index report , reflecting the escalating impact of AI on society. It contains data with new estimates on AI training costs and detailed insights into the responsible AI landscape. It also introduces a new chapter on AI's influence on science and medicine. Highlights include the substantial cost of training state-of-the-art models, such as GPT-4 and Gemini Ultra; the dominance of the U.S. in producing top AI models; and significant investment growth in generative AI despite overall funding declines. Additionally, the report notes a major increase in AI-related regulations in the U.S. and heightened public awareness and nervousness about AI's future impact. the 2024 AI Index report Enjoyed This Story? I write a weekly analysis of the AI world in the Turing Post newsletter. We aim to equip you with comprehensive knowledge and historical insights so you can make informed decisions about AI and ML. Turing Post

AI Industries Converge: Llama 3 and Electric Atlas Have More In Common Than You Think

The United States v Meta Platforms Court Filing October 24, 2023 is part of . You can jump to any part in this filing . This is part 13 of 100. HackerNoon’s Legal PDF Series here 3. Meta designs and deploys features to capture young users’ attention and prolong their time on its Social Media Platforms. 86. Acquiring young users helps secure Meta’s profit stream over time. By capturing users’ attention and engagement when they are young, Meta ensures future engagement and monetization as those young users grow up. 87. Meta thus develops and implements features to attract young users and keep them engaged on its Social Media Platforms for as long as possible. These features include: engagement-based (as opposed to chronological) feeds; infinite scroll; push notifications; ephemeral content; and video-based content. 88. Meta had originally displayed content on a user’s “Feed” chronologically, i.e., in the order the content was posted by people the user elected to follow. Meta moved from chronological Feeds to engagement-based Feeds in 2009 (for Facebook) and 2016 (for Instagram). 89. The engagement-based Feed is different and alters the users’ experience. It algorithmically presents material to users based on several engagement components: posts with more “Likes,” comments, and other indicia of user engagement are displayed to users first. 90. This change was designed to prioritize material most likely to engage users for longer periods of time. 91. In the fall of 2016, Instagram debuted its infinite scroll system. 92. Infinite scroll is characterized by the partial display of additional content at the bottom of the user’s screen, such that the user is typically unable to look at a single post in isolation (without seeing the top portion of the next post in their Feed). 93. The “teasing” of yet-to-be-viewed content continues indefinitely; as the user scrolls down the Feed, new content is automatically loaded and “teased.” 94. This “teasing” feature is intended to keep young users of the Platform engaged and continuing to scroll to the new content. 95. In April 2015, Meta introduced a variety of “push notifications” to Instagram. Push notifications are auditory and visual cues to alert users when accounts they follow add new content. 96. Push notifications allowed Instagram to draw its users back to the Platform at any time of day. [Redacted] 97. Meta also sought to increase engagement through making certain content available to users only temporarily—with notifications and visual design cues indicating that the content would soon disappear forever (ephemeral content). 98. Ephemeral content leads young users to more frequently open Meta’s Social Media Platforms so they do not “miss out” on any new content. This phenomenon is called “Fear of Missing Out,” or “FOMO.” Meta designed ephemeral content features in its Social Media Platforms to induce this sense of FOMO in young users. 99. For example, on August 2, 2016, Meta introduced a feature to Instagram designed to show images and narratives for only a short amount of time before disappearing, known as the “Stories” feature. Meta released a similar feature to Facebook in 2017. 100-101. [Redacted] 102. Another example is “Live,” which gives users the ability to livestream videos to followers or the public. 103. Meta launched Facebook Live on a limited basis to celebrities and other highprofile users in August 2015, with the feature being available to all users by April 2016. Instagram soon followed in November 2016. 104. Live allows users to create video content in real time that their followers can watch and react to, often called “going Live.” 105. When an account goes Live, the Instagram Platform sends out a notification. 106. [Redacted] 107. In addition to video-streaming offered through the Live feature, Meta has also designed and implemented several video features, including “IGTV,” “Instagram Video,” and ultimately “Reels.”[7] As with prior features, Meta focused on teen engagement with these video features. 108. [Redacted] 109. In 2020, when Meta introduced its short-form video feature, “Reels,” to the U.S. market on Instagram, [Redacted] Reels were made available on Facebook in September 2021. 110. Reels are algorithmically presented to users based on a number of factors, including the user’s activity, the popularity of the content, and the viewer’s connection to the creator. 111. Reels display metrics such as Like counts, comments, and views in the video itself, which reduces the need for the user to navigate away from the video. 112-116. [Redacted] [7] IGTV was revamped in October 2021 (in a shift to Instagram Video), and ultimately removed completely from the Platform in March 2022. Reels was merged with and superseded “Instagram Video” in June 2022. Continue Reading . Here About HackerNoon Legal PDF Series: We bring you the most important technical and insightful public domain court case filings. This court case 4:23-cv-05448 retrieved on October 25, 2023, from is part of the public domain. The court-created documents are works of the federal government, and under copyright law, are automatically placed in the public domain and may be shared without legal restriction. Washingtonpost.com

Meta Intentionally Deploys Features To Keep Young Users Glued to Their Phones

The Release of Meta’s LLAMA-2 Marks A New Era in The History of Humanity

The gaming field is always one of the hottest lands in the metaverse. Some people think that on-chain games would become one of the highest volume market segments in the Web3 space, while others doubt the long-term economic viability of the most common Play To Earn (P2E) business model. Including Axie Infinity and STEPN, the reason why most blockchain gaming projects are unsustainable is that their P2E models lead to an inability to withstand huge sell-offs, and all players have to either actively or passively participate in the death spiral that eventually cannot avoid falling into the ponzinomics. However, the dilemma that most blockchain games are currently facing is not insurmountable. Since 2022, while the flaws of traditional P2E games are gradually being revealed, there have been some projects that have made innovations in business models to try to solve sustainability problems. Some promoting themselves as F2O (free to own), PAE (play and earn), etc. have emerged in the market, transforming and enhancing the traditional P2E model, thus extending the life cycle of the game. Such transformation cannot fundamentally solve the impact of the marginal diminishing effect of the Web3 decentralised profit-sharing model on post-entry players. So does this mean it’s impossible for blockchain games to get rid of this death spiral? The answer is not unqualified. Looking back at the traditional game industry, from the initial time-killing single-player games to the complete eSports industry today, it has gone through several iterations. Today a brand new web3 game operation model built on the esports ecosystem is about to bring a revolution to the whole on-chain game market – it’s time to get into Gamefi 2.0. Why should we focus on Web3 game development? The gaming industry, which is currently around $200 billion, is expected to see robust growth in the years ahead. Accenture’s 2021 estimate of the direct and indirect value of the gaming industry stood at $300 billion. As one group of the earliest inhabitants of the virtual world, game users are more familiar with creating their own virtual profiles than many others. Yat Siu, the co-founder and chairman of Animoca Brands, pointed to the 3.2 billion people who play video games on a regular basis, which he says are already operating in an “enclosed metaverses.” The people playing those games have created virtual identities and acquired digital goods that many players think they own, but in reality they don’t. The gaming industry is effectively renting out those virtual goods to its players. Web3 is essentially a decentralised version of the Internet. It shifts digital ownership from tech companies to users so that they can move freely across platforms in an open meta-world. For gaming users in general, the most immediate change that the Web3 reform in the gaming industry has brought about for them is the ownership of their digital assets. With the support of blockchain technology, all game data held by players' NFTs will be permanently recorded, this data will also increase the value of the NFT itself. As a result, the value of the digital assets of quality players will continue to increase as the number of plays increases. From another perspective, blockchain technology offers more possibilities for gamers to stretch their assets. In traditional online games, the use of players' in-game assets is limited to in-game scenarios only. These assets are difficult to transfer. Even if a few players trade their game accounts with others, such transactions have significant limitations. Most of the time the price of the asset during the P2P transactions is much lower than the cost invested by the player. In the metaverse, players' tokens and NFT assets have more liquidity. Gamefi's inherent financial properties can help investors sell, rent out their gaming assets or place their passes in the pool as liquidity providers (LPs), thus maximising capital efficiency. In the metaverse, people are free to travel through the gaming, social, and artistic worlds unhindered by their digital assets as independent individuals. As the use of NFT expands, the cross-disciplinary NFT will provide more possibilities for people's metaverse lives. So the significance of the construction of web3 games is not only on a technical level but also as a bridge between the future entertainment life and social interaction. How can Esport change the Gamefi ecosystem? Before we talk about Game 2.0, let's go back to the p2e economic model that is now most common in web3 games. The fundamental reason why the P2E economic model is not sustainable in the long term is that the single-structured economic model is plagued by "ponzinomics". P2E projects need a constant stream of new investors to consume the output of the previous investors. When the number of "new" entrants does not cover the number necessary to sustain the existing number, it will collapse instantly. As the shortcomings of P2E became apparent, attempts were made to reverse or ameliorate the difficulty of matching supply with demand in the long term by adding structural pillars to the ecosystem. Stepn, for example, introduces a non-monetary benefit output to the model, which is the health improvement. This makes the return on investment not the only consideration for users, but also an important incentive to improve their health after using Stepn consistently. This non-monetary benefit makes this GameFi ecosystem not a zero-sum game, it generates additional positive benefits within the game framework and makes people more willing to be part of the whole ecosystem. Stepn is just one example. There are many elements that can be added to the gamefi ecosystem beyond health, such as social attributes, merchandise, platform-based advertising revenue, etc. But what we must realise is that without a fundamental change in the economic model of P2E's infinite inflation, all attempts will only slow down the project's death spiral. From this perspective, building a new economic system that can be self-circulating becomes the key to gamefi's breakthrough. Esport, as a complete ecosystem, can meet the needs of its audience for competition, education, social interaction and entertainment. For people who like the esport culture, video games are just not a mode of entertainment but are a preferred choice of interaction. In terms of the economic model, esports satisfies the conditions required for a deflationary model, namely the winner-takes-all principle. When a gamefi team set up an e-sports system, their own role changes from that of project operator to that of event organiser, while their revenue changes from a single project fee income to a diversified revenue structure. What should projects be aware of when they want to get into gamefi 2.0? Esport does not apply to all projects Unfortunately, not all commercially available P2E projects can introduce esport; on the contrary, only a very small number of projects are suitable for esport. It is a well-known fact that most gamefi projects are currently not favoured by traditional gamers because the games are not interesting enough and not well made. To meet the requirements of esport, gamefi projects need to be competitive and require a certain level of skill in addition to the two points mentioned above. Even though, esport is not the only way forward for the gamefi project. For some well-known p2e projects, there are many transformative directions that can be taken when they have already accumulated a large amount of user base and high traffic. They can go ahead to build a launchpad or platform, or just add more social functions to the project to support more users’ needs. On another hand, the gamefi teams can also change the revenue model by adding revenue pillars, for example advertising revenue, title revenue, etc. A good tokenomic system is essential In order to support the proper functioning of the gaming system, gamefi teams may refer to the following recommendations when setting up their tokenomic. Regular destruction of tokens is necessary. Whether it is a single-coin model or a two-coin model, the treasury should regularly recycle and destroy tokens in different ways, thus making the supply of tokens inherently deflationary. Increasing the number of scenarios in which tokens are used will prevent the game economy from bubbling. Project owners should consider adding different scenarios when designing their token consumption system so that tokens are well circulated. When valuing in-game NFTs, play records are used as one of the evaluation criteria. This can have several positive effects, as follows. OG players are rewarded for their playing time, and good players can permanently increase the value of their NFT through their superior playing skills Players who buy directly on the secondary market save time and effort The team earn a percentage of royalties Reference: Why play-to-earn gaming is losing steam and where GameFi must go next, RIAZ LALANI https://forkast.news/why-is-p2e-axie-infinity-declining-gamefi/ Animoca Brands: Blockchain Gaming Giant Says Web3 Future Isn’t Far Off, Robert Olsen https://www.forbes.com/sites/robertolsen/2022/09/02/animoca-brands-blockchain-gaming-giant-says-web3-future-isnt-far-off/?sh=6689799d1e09 An Overview of the Gaming Industry and Ways to Invest in It, Prableen Bajpai https://www.nasdaq.com/articles/an-overview-of-the-gaming-industry-and-ways-to-invest-in-it azf.eth https://twitter.com/azfnft/status/1537172472495235080?ref_src=twsrc^tfw|twcamp^tweetembed|twterm^1537172472495235080|twgr^b0faa015f1854d0fe85cd6e91a334e645c03a532|twcon^s1_c10&ref_url=https%3A%2F%2Fabmedia.io%2F20220616-p2e-gaming-sustainable-business Originally posted . here

Gamefi 2.0: The Movement From Play To Earn To Blockchain Esports

We all know Reactjs is a UI library in which we can build user interfaces rapidly. However, Reactjs’ ecosystem is so rich that you can build dynamic interactive web applications just by using some external libraries. In this post, we will discuss 5 Reactjs libraries you can use in your next project. We will discuss its use cases and how it can help you to build your next project without any hiccups. Here is the list of libraries that we are going to discuss: React Flow React Markdown Code Mirror React Draft WYSIWYG React Tippy Let's get started. 1. React Flow Introduction React Flow is a library to build node-based applications and it is built by webkid.io. Use Case Let's say you want to build an application where you want to render node diagrams/flow that could be anything from static to dynamic. Here we can use react flow to render node diagrams/flow and end-user can customize it, can add a new node, can update the existing node, and delete the existing node. Why Use React Flow Over Other Libraries Customizable. Utils-rich. Types support. Fast-rendering. And many more. Live Example Here is a live example of OpenMetadata. 2. React Markdown Introduction React markdown is a markdown parser react component which can take string markdown and render it to the react element. Use Case There could be a variety of use cases for this library. Let's say you are building a blog application and content would be in markdown, so there you can use react markdown to parse the content. We all have used GitHub or a similar application at least once in our developer journey and know that most of the contents are in markdown format. So, we can also build a tool using react markdown to create a GitHub readme profile. Why Use React Markdown Over other Libraries Easy to use. Rich API Support. Custome Component Support. Extendible with plugins system. Live Example Here you can see I have built a using react markdown. You can write markdown and see the output side by side. Also, you can download it as a markdown file. markdown previewer 3. Code Mirror Introduction Code Mirror is a text editor with support for different languages and add-ons to implement the advance and extend the existing functionality. Use Case Let's say you are building an LMS (Learning Management System) and you want to provide an editor to end-users for editing code snippets that could be in any language like python, javascript, etc., or format likes JSON, Avro, SQL, etc. Here you can use code mirror to build an editor that can allow end-users to play with code snippets. Why Use Code Mirror Over other Libraries Support for more than 100 languages Code Folding Support Support for inline and block widget Configurable bindings Used by browsers like Firefox, Chrome, and Safari. Live Example Here you can see the live use case of code mirror by Carbon tool for creating and sharing beautiful images of your source code. 4. React Draft WYSIWYG Introduction React Draft WYSIWYG is a rich text editor built on top of reactjs and .. It’s a ready to use react component to provide rich text support in your application. draftjs Use Case Nowadays most of the communities/publications like , , , and provide support for rich text content and they have built their own rich text editor. But you don't have to reinvent the wheel, you can use React Draft WYSIWYG directly in your application. Hashnode dev.to medium showscase devdojo Why Use React Draft WYSIWYG Over Other Libraries You can configure toolbar options. Support for Mention and Hashtags. Support for emojis. Support for internationalization. Live Example Here you can see the of . live example React Draft WYSIWYG 5. React Tippy Introduction React Tippy is a lightweight tooltip component library for reactjs. It provides a tooltip element or HOC (Higher-order component) and it uses the React DOM to render tooltip. Use Case The use case for React Tippy is simple: if you need to show some content using a tooltip then you should use React Tippy library. Why Use React Tippy Over other Libraries Work's well with reactjs. Enhancement of Tippy.js Support multiple triggers. Customizable. Support for different themes. Support for different sizes. Live Example Here you can find the live example of react tippy And that’s it for this topic. Thank you for reading. Connect With Me | LinkedIn Twitter Also published . here

Discover Why These 5 Libraries are Perfect for Your Next ReactJS Project

What is OpenTelemetry and How It Can Improve Your Backend Quality? 

Wat is OpenTelemetry en hoe dit u agterkantkwaliteit kan verbeter? 

OpenTelemetry ምንድን ነው እና የእርስዎን የኋላ ጥራት እንዴት ማሻሻል ይችላል? 

ما هو OpenTelemetry وكيف يمكنه تحسين جودة الواجهة الخلفية لديك؟

¿Kunas OpenTelemetry ukat Kunjamas Backend ukan calidad ukar askinchaspa? 

OpenTelemetry nədir və O, Backend Keyfiyyətinizi Necə Təkmilləşdirə bilər? 

Што такое OpenTelemetry і як ён можа палепшыць якасць бэкэнда? 

Какво е OpenTelemetry и как може да подобри качеството на вашия бекенд? 

OpenTelemetry কি এবং কিভাবে এটি আপনার ব্যাকএন্ড গুণমান উন্নত করতে পারে? 

Šta je OpenTelemetry i kako može poboljšati vaš backend kvalitet? 

Què és OpenTelemetry i com pot millorar la qualitat del vostre backend? 

Co je OpenTelemetry a jak může zlepšit kvalitu vašeho backendu? 

Hvad er OpenTelemetry, og hvordan det kan forbedre din backend-kvalitet? 

Was ist OpenTelemetry und wie kann es Ihre Backend-Qualität verbessern?

Τι είναι το OpenTelemetry και πώς μπορεί να βελτιώσει την ποιότητα του Backend σας; 

¿Qué es OpenTelemetry y cómo puede mejorar la calidad de su backend? 

Zer da OpenTelemetry eta nola hobetu dezake zure backend kalitatea? 

OpenTelemetry چیست و چگونه می تواند کیفیت Backend شما را بهبود بخشد؟ 

Mikä OpenTelemetry on ja miten se voi parantaa taustasi laatua? 

Qu'est-ce qu'OpenTelemetry et comment peut-il améliorer la qualité de votre backend ? 

Que é OpenTelemetry e como pode mellorar a calidade do teu backend? 

מהי OpenTelemetry וכיצד היא יכולה לשפר את איכות הקצה האחורי שלך? 

ओपनटेलीमेट्री क्या है और यह आपकी बैकएंड गुणवत्ता को कैसे सुधार सकती है?

Što je OpenTelemetry i kako može poboljšati vašu pozadinsku kvalitetu? 

Ki sa ki OpenTelemetry ak ki jan li ka amelyore kalite backend ou a? 

Mi az az OpenTelemetry, és hogyan javíthatja a háttérrendszer minőségét? 

Ի՞նչ է OpenTelemetry-ն և ինչպես այն կարող է բարելավել ձեր Backend-ի որակը: 

Apa itu OpenTelemetry dan Bagaimana Itu Dapat Meningkatkan Kualitas Backend Anda?

Che cos'è OpenTelemetry e come può migliorare la qualità del tuo backend?

OpenTelemetry とは何ですか? また、バックエンドの品質をどのように向上できますか?

რა არის OpenTelemetry და როგორ შეუძლია გააუმჯობესოს თქვენი Backend-ის ხარისხი? 

OpenTelemetry дегеніміз не және ол сіздің сервер сапасын қалай жақсарта алады? 

តើអ្វីជា OpenTelemetry និងរបៀបដែលវាអាចធ្វើអោយគុណភាព Backend របស់អ្នកប្រសើរឡើង? 

OpenTelemetry란 무엇이며 어떻게 백엔드 품질을 향상시킬 수 있습니까? 

OpenTelemetry деген эмне жана ал сиздин Backend сапатын кантип жакшыртат? 

OpenTelemetry ezali nini mpe Ndenge nini ekoki kobongisa qualité ya Backend na yo? 

OpenTelemetry ແມ່ນຫຍັງ ແລະມັນສາມາດປັບປຸງຄຸນນະພາບ Backend ຂອງເຈົ້າໄດ້ແນວໃດ? 

Kas yra „OpenTelemetry“ ir kaip ji gali pagerinti jūsų fono kokybę? 

Kas ir OpenTelemetry un kā tas var uzlabot jūsu aizmugursistēmas kvalitāti? 

Inona no atao hoe OpenTelemetry ary ahoana no hanatsarany ny kalitaon'ny backend anao? 

Што е OpenTelemetry и како може да го подобри квалитетот на вашиот заднина? 

OpenTelemetry гэж юу вэ, энэ нь таны арын хэсгийн чанарыг хэрхэн сайжруулах вэ? 

Apakah OpenTelemetry dan Bagaimana Ia Boleh Meningkatkan Kualiti Bahagian Belakang Anda? 

OpenTelemetry के हो र यसले तपाइँको ब्याकएन्ड गुणस्तर कसरी सुधार गर्न सक्छ? 

Wat is OpenTelemetry en hoe kan het de kwaliteit van uw backend verbeteren?

OpenTelemetry ke eng le gore e ka kaonafatša bjang Boleng bja gago bja Backend? 

Czym jest OpenTelemetry i jak może poprawić jakość Twojego zaplecza?

OpenTelemetry څه شی دی او دا څنګه کولی شي ستاسو د شالید کیفیت ښه کړي؟ 

O que é OpenTelemetry e como ele pode melhorar a qualidade do back-end? 

¿Imataq OpenTelemetry chaymanta Imaynatataq Backend calidadniyki allinchayta atin? 

Ce este OpenTelemetry și cum vă poate îmbunătăți calitatea backend-ului? 

Что такое OpenTelemetry и как она может улучшить качество вашей серверной части? 

OpenTelemetry niki kandi nigute ishobora kuzamura ireme ryinyuma yawe? 

OpenTelemetry යනු කුමක්ද සහ එය ඔබගේ පසුපෙළ ගුණාත්මකභාවය වැඩි දියුණු කරන්නේ කෙසේද? 

Čo je OpenTelemetry a ako môže zlepšiť kvalitu vášho backendu? 

Chii chinonzi OpenTelemetry uye Sei Inogona Kuvandudza Yako Kumashure Hunhu? 

Waa maxay OpenTelemetry iyo sidee ayay u hagaajin kartaa tayadaada dambe? 

Çfarë është OpenTelemetry dhe si mund të përmirësojë cilësinë tuaj të backend? 

Шта је ОпенТелеметри и како може да побољша ваш позадински квалитет? 

Vad är OpenTelemetry och hur det kan förbättra din backend-kvalitet? 

OpenTelemetry ni nini na Jinsi Inaweza Kuboresha Ubora wako wa Nyuma? 

OpenTelemetry என்றால் என்ன, அது உங்கள் பின்தள தரத்தை எவ்வாறு மேம்படுத்தலாம்? 

OpenTelemetry чист ва он чӣ гуна метавонад сифати пуштибонии шуморо беҳтар кунад? 

OpenTelemetry คืออะไร และสามารถปรับปรุงคุณภาพแบ็กเอนด์ของคุณได้อย่างไร

OpenTelemetry እንታይ እዩ ብኸመይ ንጽሬት ባክኤንድካ ከማሓይሽ ይኽእል? 

“OpenTelemetry” näme we arkaňyzyň hilini nädip ýokarlandyryp biler? 

Ano ang OpenTelemetry at Paano Ito Mapapabuti ang Kalidad ng Iyong Backend? 

OpenTelemetry Nedir ve Arka Uç Kalitenizi Nasıl Artırabilir? 

Xana OpenTelemetry I Yini Naswona Yi Nga Antswisa Njhani Khwalithi Ya Wena Ya Backend? 

Що таке OpenTelemetry і як він може покращити якість серверної частини? 

OpenTelemetry کیا ہے اور یہ آپ کے بیک اینڈ کوالٹی کو کیسے بہتر بنا سکتا ہے؟ 

OpenTelemetry nima va u sizning orqa tomoningiz sifatini qanday yaxshilashi mumkin? 

OpenTelemetry là gì và nó có thể cải thiện chất lượng phụ trợ của bạn như thế nào? 

Yintoni i-OpenTelemetry kwaye inokuphucula njani umgangatho wakho wokubuyela umva? 

什么是 OpenTelemetry 以及它如何提高您的后端质量？

Iyini i-OpenTelemetry futhi Ingayithuthukisa Kanjani Ikhwalithi Yakho Yasemuva? 

is a distributed unit of infrastructure. It can provide information about FreeBSD and Linux operating systems. It also supports metrics transmission to Clickhouse, InfluxDB, and Prometheus. Alligator The main interface of alligator is . It connects with spreading Prometheus as a monitoring system in many infrastructures. However, the Prometheus ecosystem has some problems. OpenMetrics First of all - the specialized and simplified agents. Every service on the host needs a specialized exporter. This is the reason for having a lot of exporters per host (like a node_exporter, redis_exporter, sentinel_exporter, and so on). Secondly, simplified agents. A lot of agents can be used for different tasks (like an SNMP, blackbox, etc.), and it inflates infrastructure for such daemons to collect information. Sometimes, a pull model has problems with multi-worker applications and cron jobs - nothing can give metrics from hosts when Prometheus makes a request, or it is connected with problems related to synchronizing results between workers. It forces us to use statsd/pushgateway. Then the question comes up: how to remove outdated metrics? How to make it a high-availability service? The approach of the alligator is different - it is a smart agent for the Prometheus. Alligator consists of a system exporter and an application exporter. It also provides a push model for pushgateway, statsd, and graphite emulation protocol. Metrics Export Alligator supports such sets of metrics: FreeBSD and GNU/Linux system metrics. Filesystem metrics (file’s checksum, sizes, and access level). Certificate metrics (PEM and P12). Some popular applications (ElasticSearch, uwsgi, Druid, RabbitMQ, and others). Ephemeral metrics. It supports pushgateway, statsd, and graphite protocol, and it can collect metrics from file on FS. It also has a cluster mode for centralized installation of pushgateway. Simple JSON to metrics parser. Simple HTTP/TCP checker. Use Cases A part of supported technologies for collecting metrics are reviewed in the . The rest I am describing here. official documentation Application Server Metrics First of all, alligator can collect application server metrics. For instance, alligator can gather metrics from uwsgi instances with this configuration: aggregate {
    uwsgi tcp://localhost:1717;
} and rerun the uwsgi server with the : expose stats port --stats 127.0.0.1:1717 Another example, fastcgi-application: aggregate {
    php-fpm fastcgi://127.0.0.1:9000/stats?json&full;
} and write this option in : php-fpm config pm.status_path = /stats After this, you will have common metrics from the application server. Application Metrics I recommend using synchronization structures to collect metrics in the application internally. Alternatively, you can make only 1 worker per instance and use such an abstraction as orchestration (Kubernetes, opennebula, and others). And next, use alligator as a push server for metrics. Alligator has many collectors to collect it: pushgateway prom-aggregation-gateway statsd graphite file These approaches ensure that metrics will be collected from all workers of the application Applications have to send metrics to alligator. To remove influence on one metric by different workers, you can use two different approaches: mark metric with the worker ID from prometheus_client import Counter, push_to_gateway, CollectorRegistry
    registry = CollectorRegistry()
    c = Counter('my_requests_total', 'HTTP Failures', ['method', 'worker_id'], registry=registry)
    c.labels(method='get', worker_id=uwsgi.worker_id()).inc()
    c.labels(method='post', worker_id=uwsgi.worker_id()).inc()
    push_to_gateway('localhost:1111', job='myapp', registry=registry) use instead of metric type if you can. File collector cannot calculate increment, but others can increment if the counter is a type of metric. You can only send increment value to alligator, and alligator will calculate the sum. Counter Gauge More information about the difference between metric types can be found or on the . here official page What Metrics Do I Recommend to Measure in Applications? Firstly, it is any outgoing requests to external systems. In worker applications, it often is the reason to overflow workers or cause of low speed of query processing. My recommended list of metrics: count of requests to external services grouped by request types request time from external services (avg, or if possible, by quantiles/buckets of time) grouped by request types status of requests grouped by request type (success or errors) It would be nice if you could group not only by request type but more by network layer of requests: DNS resolve count, time, and status connect count, time, and status write/read count, time, and status It is also important to measure internal application metrics: ingoing requests grouped by types processing ingoing request time avg (quantile or buckets if possible) by type status of ingoing request by type cache status (hit/miss if it exists) I guess you can make another list of metrics, but these metrics are the most important to reveal problems and give main observability about application health. If your application is working in podman or docker environment, you can expose the container statistics. Alligator mechanics are similar to Cadvisor's when collecting container statistics. To enable, you have to use option cadvisor in the system context: Container checks. system {
    cadvisor;
} . This is a good way to monitor applications or external sources for reliability. Let’s check the resources on two ports every minute: Blackbox checks aggregate_period 60;
aggregate {
    blackbox http://127.0.0.1:80/check;
    blackbox https://127.0.0.1:443/check;
} Alligator produces network and application response metrics for this resource (response code, response time, and certificate expiration date). If the application has a page with JSON, alligator can also convert JSON to metrics: aggregate {
    jsonparse https://jsonplaceholder.typicode.com/todos;
} The next level of blackbox metrics - checking the process, service, and listening port: system {
	services uwsgi.service;
	process uwsgi;
}


query {
	expr 'count by (src_port, process) (socket_stat{process="uwsgi", src_port="80"})';
	make socket_match;
	datasource internal;
} And in the end: launching external programs and proactive actions (like watchdog): aggregate {
	process 'exec:///usr/local/bin/mycheckscript.sh';
} If the script can produce a metric that can determine the exact application downtime, this metric can be used for proactively restart: query {
    expr 'myapplication_status < 1';
    make app_is_down;
    datasource internal;
    action app_is_down;
}
action {
    name app_is_down;
    type shell;
    expr 'exec://systemctl restart uwsgi;
    datasource internal;
} System Metrics Any application metrics are also important. But without system metrics of servers, you won’t be able to clearly understand what happens in your system. System metrics are a set of metrics that help us gain insight into the health of our systems. This is the basis for making decisions if something goes wrong. An alligator with an empty configuration doesn’t collect any metrics at all. Any system metrics are classified into the following categories: base - CPU, memory, time, logical resources, and other common metrics about the server disk - disk usage and I/O network - interface statistics, network counters, sockets statistics smart - collecting S.M.A.R.T. statistics cpuavg - analog for load average when alligator make by CPU-only load firewall - counters from the system firewall process - check process (uptime, ulimits usage, status, CPU, memory, and disk consumption) packages - metrics with a list of installed packets in the system services - systemd services stats This is configured in a block called “system” in alligator.conf: system {
    base;
    disk;
    network;
    process;
    cpuavg period=5;
} What about cron jobs? How can it be connected to the Prometheus via alligator? There are several ways to do this. The simple way is to output metrics to a file: aggregate {
    prometheus_metrics file:///var/lib/stats/metrics.txt state=save;
} The parameter called ‘state’ enables autosaving at a point of the file when alligator is stopped. It makes us sure that we can stop the alligator and run again, and it rereads only that part of the file that has not been read yet. If we want to reread a file from the start (every 10 sec), we have to use . The third way - is to read only realtime data. To enable this mode, we have to remove any state from the configuration file. state=begin The other way is to use statsd or extended pushgateway protocol (called prom-aggregation-gateway): It only needs to have this section in the configuration: entrypoint {
	tcp 1111;
    metric_aggregation count;
} For instance, we have a code: #!/bin/sh
TASKS=0
do_something() {
        echo 0
        TASKS=$1
}

res=`do_something 1`
echo "amount_of_done_tasks {method=\"do_something\", status=\"done\"} $TASKS
task_exit_code {method=\"do_something\"} $res" | curl --data-binary @- http://localhost:1111/ This script will make two metrics in alligator: amount_of_done_tasks {status="done", method="do_something"} 0.000000
task_exit_code {method="do_something"} 0.000000 Sometimes, we might get a situation when the script is not working, but the metric exists (because alligator doesn’t know that the application doesn't work). To solve this problem, alligator has TTL mechanics. You can specify the lifetime in seconds for a request: curl -H "X-Expire-Time: 30" --data "living_thirty_seconds_metric 4" http://localhost:1111/ Or in the configuration file: entrypoint {
    ttl 30;
    tcp 1111;
    metric_aggregation count;
} The next way is using statsd protocol (udp or tcp). TCP server for statsd is enabled by default. If you want to use udp, specify the UDP port in config: entrypoint {
        ttl 30;
        tcp 1111;
        udp 1111;
} And your job script might have a section like this: from statsd import StatsClient, TCPStatsClient
from time import sleep

statsd = StatsClient(host='localhost',
                     port=1111,
                     prefix=None,
                     ipv6=False)

statsd.incr('baz')
statsd.incr('baz') Graphite protocol is also supported by alligator, and it works like graphite_exporter. Internet Web Server If I have an internet web server, what options do I have? First of all, https is the most popular protocol for web servers today. It forces us to check the certificate expiration time: To check directories with certificates: x509
{
	name nginx;
	path /etc/nginx/ssl/;
	match .crt;
}
x509
{
	name letsencrypt;
	path /etc/letsencrypt/live;
	match .pem;
} Secondly, to check the nginx listening ports: query {
	expr 'count by (src_port, process) (socket_stat{process="nginx", src_port="80"})';
	make socket_match;
	datasource internal;
}
query {
	expr 'count by (src_port, process) (socket_stat{process="nginx", src_port="443"})';
	make socket_match;
	datasource internal;
} To check nginx -t to exclude the situation, when nginx after server reboot cannot be started: aggregate {
	process 'exec:///sbin/nginx -t';
} To check system, web-server process, and firewall counters system {
    base;
    disk;
    network;
    cpuavg period=5;
    services nginx.service;
    process nginx;
    firewall ipset=on;
} We also make alligator port accessible only on local interfaces: entrypoint {
    handler prometheus;
    allow 127.0.0.1;
    allow 10.0.0.0/8;
    allow 172.16.0.0/12;
    allow 192.168.0.0/16;
    tcp 1111;
} or you can use the notation tcp in the tcp directive. <localIP>:<port> Statsd Metric Mapping Simple statsd doesn’t work with labels. To create a label from a statsd metric name, you must use mapping with metric name templates to change new names and create new labels. Due to this, I recommend using an extended format of Statsd, like DogStatsd (alligator also supports this extension), which supports labels in statsd protocol. But, some programs have already made pure statsd protocol without labels. One such example is Airflow. You can catch metrics from Airflow like this: airflow.ti.finish.upload_job.load_data.success:1|c
airflow.ti.finish.upload_job.other_data.success:1|c where the name is the following template: airflow.ti.finish.<job name>.<process name>:<increment by>|type The mapping for such metric name can be: entrypoint {
    tcp 1111;
    udp 1111;
    mapping {
        template   airflow.ti.finish.*.*.*;
        label dag_id "$1;
        label task_id "$2";
        label state "$3";
        name airflow_ti_finish;
        match glob;
    }
} This mapping will create such a metric: airflow_ti_finish {state="success", dag_id="upload_job", task_id="load_data"} 1.000000
airflow_ti_finish {state="success", dag_id="upload_job", task_id="other_data"} 1.000000 Statsd or Pushgateway Cluster Firstly, you can only make some amount of servers statsd or pushgateway and simply proxying to them. The first problem with using this: if you want to use gauges, you will have problems with calculating. If there is a distributed application sending a metric with the same name and labels, the gauge metric will also be replaced by the application. The first way to fix this - is to specify different labels for each application (such as hostname and/or worker_id/process_id). But that is not all. You also have a problem with the same gauge metric: this metric can be a balancer on different machines. Here is what to do with a metric that was sent at different times, but now they are interfering with each other: 10:12 http_request_time_avg {appid="43"} 12
10:14 http_request_time_avg {appid="43"} 22
10:55 http_request_time_avg {appid="43"} 122
11:58 http_request_time_avg {appid="43"} 1232 You can use nginx upstreams with a balancing method called hash: upstream alligators {
    server srv1.example.com:1112;
    server srv2.example.com:1112;
    server srv3.example.com:1112;
    server srv4.example.com:1112;
    server srv5.example.com:1112;
    hash $remote_addr consistent;
} Then one backend will always push all metrics to one server. On the other hand - cluster mode in alligator. Let's set it up! Firstly, we allocated 3 ports: for getting metrics 1111 for replication works 1112 for getting metrics from other applications 80 Imagine that our server is listening to the internet. Then, we need to secure our push metric ports. We can use the directive “return” with arg “empty” to remove any answers outside. We also know that internet browsers don't send POST queries to other domains if CORS doesn't allow this. Thirdly, make an authentication for POST requests. entrypoint {
   allow 127.0.0.1;
   allow 10.0.0.0/8;
   tcp 1111;
}
entrypoint {
   allow 10.0.0.0/8;
   tcp 1112;
   cluster replication;
   instance srv1.example.com:1112;
}
entrypoint {
   return empty;
   allow 0.0.0.0/0;
   ttl 300;
   tcp 80;
   metric_aggregation count;
   auth basic user:setme;
   auth_header Authorization;
   header access-control-allow-headers 'Authorization';
   header access-control-allow-methods POST;
   header access-control-allow-origin *;
} That's good. We annotate that we have a cluster called “replication” and the current instance name. Since this moment, alligator allows to answer collected oplog to anyone from network 10.0.0.0/8 on port 1112. How to command an alligator to join other instances of cluster? For this, we will use the “cluster” directive. We can specify oplog size (the limit of in-memory stored metrics that will wait to sync to other instances), and distribute metrics by its metric name (sharding key in our case, but you can use any other label). The replication factor hides replicated metrics while another primary server is running in the cluster. If the primary server goes down, hidden metrics will be exposed and made available to Prometheus from other instances. cluster {
   name replication;
   size 10000;
   sharding_key __name__;
   replica_factor 2;
   type oplog;
   servers srv1.example.com:1112 srv2.example.com:1112 srv3.example.com:1112 srv4.example.com:1112 srv5.example.com:1112;
} Check the External Sites Status Blackbox mechanics for collecting information about network services have already been mentioned. Nevertheless, that is not all that alligator can be used for. Alligator also has a puppeteer parser for loading external pages with a lot of site resources (which can load a page DOM). puppeteer {
    https://site1.example.com;
    https://site2.example.com;
    https://example.com;
} They might measure domain lookup name in metric: puppeteer_domainLookupEnd - puppeteer_domainLookupStart Connect time: puppeteer_connectEnd - puppeteer_connectStart Time to load DOM: puppeteer_DomContentLoaded Status of loading: puppeteer_eventSourceResponseStatus And even errors in the browser console: puppeteer_eventConsole This is not all that a puppeteer collector can do. However, this engine needs nodejs to be installed on a machine with puppeteer dependencies. Install puppeteer dependency: cd /var/lib/alligator/
npm i puppeteer@14.3.0
npm i ps-node-promise-es6 Conclusion Monitoring systems is a really big challenge despite the amount of tools to solve it. You can choose between a monitoring system or tools in one system of monitoring (such as VictoriaMetrics, Thanos, some amount of exporters, and so on). I guess that the approaches outlined in the article can be applicable to any other tools of monitoring. Alligator is a tool that can simplify it. The main goal of any monitoring is to make your system observable. You must have all measured information from your systems to ensure that it always operates correctly.

Alligator Is a Prometheus Monitoring Agent: Everything You Need to Know About It

If you are using Databricks serving endpoint, and you wish to export metrics to Datadog, you can face with some challenges in Datadog documentation. This article will help you to solve them, at least it will help you to make a PoC. Here are list of links which you can use: https://docs.databricks.com/en/machine-learning/model-serving/metrics-export-serving-endpoint.html https://docs.datadoghq.com/agent/basic_agent_usage/ubuntu/?tab=agentv6v7 Databricks serving endpoint is a service which allows you to deploy your model and serve it. It is a part of Databricks MLflow. You can find more information about it here: https://docs.databricks.com/applications/mlflow/model-serving.html Once you created an endpoint, out of the box you can see some metrics in Databricks UI. Also list of metrics can be received via REST API: curl --request GET "https://${DATABRICKS_HOST}/api/2.0/serving-endpoints/{endpoint_name}/metrics" --header "Authorization: Bearer ${DATABRICKS_TOKEN}" But if you want to export them to Datadog, you need to install and configure Datadog agent. Pre-requisites Created Databricks serving endpoint Bearer PAT token behalf of Databricks user/service principal, which has `CAN_QUERY` permission on the endpoint Datadog API key Enabled OpenMetrics integration in your Datadog account, link: https://app.datadoghq.com/integrations Install Datadog agent Datadog offers several ways to install agent, they are listed here: , however looks like some of them are not working, or at least I was not able to make them work. That is why I decided to use the most simple way — install agent on Docker. https://app.datadoghq.com/account/settings/agent/latest?platform=overview Before agent installation, you need to create a file `datadog.yaml`, which should be located ` `, with the following content: /etc/datadog-agent/datadog.yaml api_key: "__DD_API_KEY__"
site: "datadoghq.com"
hostname: "__YOUR_HOST_NAME__" Then you need to create `metric-config.yaml` file, which should be located ` `, with the following content: /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml.default instances:
 - openmetrics_endpoint: https://__DATABRICKS_HOST__/api/2.0/serving-endpoints/__ENDPOINT_NAME__/metrics

   metrics:
     - cpu_usage_percentage:
         name: cpu_usage_percentage
         type: gauge
     - mem_usage_percentage:
        name: mem_usage_percentage
        type: gauge
     - provisioned_concurrent_requests_total:
        name: provisioned_concurrent_requests_total
        type: gauge
     - request_4xx_count_total:
        name: request_4xx_count_total
        type: gauge
     - request_5xx_count_total:
        name: request_5xx_count_total
        type: gauge
     - request_count_total:
        name: request_count_total
        type: gauge
     - request_latency_ms:
        name: request_latency_ms
        type: histogram

   tag_by_endpoint: false

   send_distribution_buckets: true

   headers:
     Authorization: Bearer __DATABRICKS_TOKEN__
     Content-Type: application/openmetrics-text The full possible options for ` ` can be found in template file which is located in a container under ` ` conf.yaml.default /opt/datadog-agent/agent/conf.d/openmetrics.d/ I’ve created the next , because I have some difficulties with that one which is provided in Datadog documentation: Dockerfile FROM ubuntu:20.04

# Set environment variables
ENV DD_API_KEY=[SPECIFY_DD_API_KEY]
ENV DD_SITE=datadoghq.com
ENV DD_HOSTNAME=datadog-agent-ubuntu
ENV DATABRICKS_HOST=[SPECIFY_DATABRICKS_HOST]
ENV DATABRICKS_TOKEN=[SPECIFY_DATABRICKS_TOKEN]
ENV ENDPOINT_NAME=[SPECIFY_ENDPOINT_NAME]

# Install curl
RUN apt-get update && apt-get install -y curl

# Copy the datadog.yaml file into the Docker image
COPY datadog.yaml /etc/datadog-agent/datadog.yaml
RUN sed -i -e "s@__DD_API_KEY__@$DD_API_KEY@g" /etc/datadog-agent/datadog.yaml

# Install the Datadog agent
RUN bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_agent7.sh)"

# Copy the metrics-config.yaml file into the Docker image
COPY metrics-config.yaml /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml.default

RUN sed -i -e "s@__DATABRICKS_HOST__@$DATABRICKS_HOST@g; s@__ENDPOINT_NAME__@$ENDPOINT_NAME@g; s@__DATABRICKS_TOKEN__@$DATABRICKS_TOKEN@g; " /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml.default

RUN service datadog-agent restart

# Run a process that doesn't exit
CMD ["tail", "-f", "/dev/null"] Usage Run serving endpoint Define environment variables in `Dockerfile’ Build docker image Run docker container # Step 1: Build the Docker image
# Navigate to the directory containing the Dockerfile
cd /path/to/directory/with/dockerfile

# Build the Docker image
# Replace 'my-image' with the name you want to give to your Docker image
docker build -t datadoc-agent-image .

# Step 2: Run a container from the built image
# Replace 'my-container' with the name you want to give to your Docker container
docker run --name datadog-agent -d datadoc-agent-image

# Step 3: Connect to a container and restart agent
docker exec -it datadog-agent  bash
service datadog-agent restart Some usefully commands for DataDog agent: datadog-agent status
datadog-agent health

service datadog-agent start
service datadog-agent restart Also appears . here

How to Export Metrics from Databricks Serving Endpoint to Datadog

With the adoption of AI, usage attribution, and chargeback use cases are on the rise. Modern businesses are also eager to gather and assign usage data to their customers and internal departments, to drive billing, sales, product development, and cloud cost analysis. The FinOps Foundation also recently unveiled the initial draft of its (Open Cost & Usage Specification). Why usage data can be complex, and what differentiates event metering from time series metrics? FOCUS Usage Data Use Cases Before delving into the intricacies of billing, analytics, and monitoring use cases, let's define what we mean by usage data. Usage describes someone consuming a good in a time period. For example, between 1 PM and 2 PM, Alice sent 100 SMS via Twillio API. Usage is usually described for a time period instead of a single date because computers are fast, but humans are slow. Let’s look at some of the common use cases requiring usage data: This necessitates accurate usage data as customers are charged based on legally binding contract terms. While the data dimensions are often limited, the cardinality is high as usage data must be tracked for every customer. Billing: Real-time data is optional, but prompt notifications are required when a user reaches a billing threshold. Data retention is crucial for validating bills, although this becomes less important once the invoice is settled. This requires real-time usage data for alerting purposes. Accuracy is important but is more flexible than billing. Monitoring systems are often limited around cardinality. Monitoring: Data retention is usually short due to the costs of storing large volumes of monitoring data, which is rarely utilized after a few weeks. Typical use cases such as cloud cost, margin analysis, and pricing necessitate accurate historical data from the past three to five years to train models and identify trends effectively. Analytics is rarely real-time. Analytics: Summarized as a table: Use-Case Accuracy Cardinality Real-Time Retention Billing High High Moderate 1-2 Years Monitoring Moderate Low High Weeks Analytics High Moderate Low 3+ Years As you can see, each use case has different needs, which can be confusing when discussing usage data. Understanding Auditable and Operational Data The concept of classifying data as auditable or operational was first brought to my attention in 2018 through a by Charity Majors, cofounder of . tweet Honeycomb.io is classified as such when the loss of any data record is intolerable, and full retention of records is necessary. When utilizing an auditable dataset, it is expected to be comprehensive and complete. Auditable data Examples of auditable data include transaction logs, replication logs, and billing/finance events. , conversely, doesn't require strict completeness. To maintain manageable costs, sampling is often employed, and some degree of data loss is acceptable. Operational data Tools designed to manage operational data often prioritize effort efficiency, bypassing retries and costly guarantees of exactly one delivery. Examples of operational data include telemetry, metrics, and contextual data that describe each request and system component. Before deciding on the methodology for collecting, processing, and storing your usage data, it is important to determine whether your data needs to be auditable or operational. In the following section, we will compare two data collection strategies: event-driven metering, generally better suited to auditable use cases, and time-series monitoring, the preferred method for collecting operational usage data. Event-driven Metering vs. Time-Series Metrics There are two main ways to collect usage data: and event-driven metering . time-series monitoring systems Here's how they compare: Usage-based billing companies favor this approach as it is auditable due to its inherent consistency in handling unique events. Events can be double-delivered in distributed systems and deduplicated using unique identifiers to prevent over or under-billing. Event-driven metering: Metering deals well with high cardinality, which is necessary to track every customer's usage. The challenge, however, lies in data collection. The industry has robust infrastructure collectors for monitoring, but these were designed with something other than events in mind. Most vendors provide a POST API for event submission, leaving the collection process up to the user. Monitoring systems like Prometheus scrape counters and histograms to store and supply metrics as time-series operational data. Time-series monitoring: Keeping cardinality low is advised, making it difficult to track individual user resource consumption on a large scale. Metrics collection is a well-paved path in the industry, with out-of-the-box metrics extractors available for most infrastructure components. APM vendors have invested significantly in standards like OpenTelemetry to streamline data collection. The challenge lies in the metrics collector's limited guarantees around delivery and deduplication since they were designed with operational data use cases in mind. Prometheus contributors share some thoughts about accuracy . If you want to dig deeper, you can also find some debate about adjusting scraping to increase counter accuracy . here here Summarized as a table: Collecting Usage Auditable Consistency Collectors and Standards Event Metering Yes High Low Time-Series Metrics No Moderate High The Future of Usage: Consolidation The current challenge lies in collecting and integrating usage data. These tasks are complex because usage collection must balance accuracy, cardinality, and real-time aspects differently per use case (as illustrated in the events vs. metrics comparison), while integration is time-consuming due to the need for a usage specification standard. Just think about all the custom vendor APIs or the generic PromQL interface. This lack of consolidation creates difficulties in integrating usage data into billing, chargeback, and cost analysis use cases, often resulting in separate systems for usage data collection rather than sharing amongst each other. FinOps' FOCUS: A New Specification for Usage Integration FOCUS (Open Cost & Usage Specification) by FinOps aims to address the integration challenges of usage data. FOCUS outlines a specification for producing and consuming normalized usage, and billing data by Cloud Providers and SaaS vendors. FOCUS will allow you to seamlessly integrate usage data between vendors, for billing, and cloud cost analysis use cases. FOCUS specification is currently under development; the 0.5 preview version was just released at the end of June 2023, and the specification is currently focused more on billing than usage data. You can follow or join the FOCUS working group . here Reconciling Accuracy, Cardinality, and Timeliness I don't anticipate a convergence of event metering and metrics systems as they each balance distinct business and engineering trade-offs to cater to their use cases. Just think about the differences between auditable and operational data. But I do expect convergence on standards around integrating usage data between vendors like FinOps’s FOCUS. We need your input. Should OpenMeter ingest metrics and integrate with Prometheus to streamline billing and chargeback use cases? Let us know in our open-source repository: https://github.com/openmeterio/openmeter

The Difference Between Auditable and Operational Usage Data

La diferencia entre datos de uso auditables y operativos

La différence entre les données d'utilisation vérifiables et opérationnelles

ऑडिटेबल और ऑपरेशनल उपयोग डेटा के बीच अंतर

監査可能な使用状況データと運用上の使用状況データの違い

A diferença entre dados de uso auditáveis e operacionais

Sự khác biệt giữa dữ liệu sử dụng có thể kiểm tra và hoạt động

可审计和运营使用数据之间的区别

Wayra, Telefónica's open innovation initiative, invests in . The video game studio in Barcelona has created Metasoccer, the first in which you can manage your own club and generate income while playing. The game is aimed at both play-and-earn players and soccer fans interested in getting into the metaverse and web3 gaming. Champion Games soccer metaverse It consists of a form of online games that allow players to earn rewards, with value in the real world, by playing against other players and progressing through the game. These rewards come as game assets, such as cryptocurrencies or NFTs, such as virtual stadiums and sports equipment, among others. All of these actions are recorded in the blockchain, a public database on the Internet. “It is a privilege for us to have Wayra as an investor in our company. We hope that all the knowledge and experience in the field of telecommunications and innovation can help us consolidate our business,” said Marc Cercós, CEO and co-founder of Champion Games, creators of MetaSoccer. In 2021, Champion Gamers raised a in which investors such as Play Ventures, ParaFi Capital, Defiance Capital and DAO Maker, among others, stood out. The objective of the round was to lay the foundations for creating the first sports metaverse with a new generation of digital native players and soccer clubs in MetaSoccer, which in the coming years will achieve ratings similar to those of professional soccer leagues. funding round of 2 million euros To this end, it is investing in hiring profiles specialized in the development of Golang, React technologies and onchain programming in Solidity and, in addition, they are reinforcing marketing actions to provide more visibility to the project. So far they have won the support of great players such as Ronaldinho or Dani Carvajal and clubs like Vasco da Gama. “Play-and-Earn is causing a new paradigm in the gaming industry. The use of decentralized technologies opens up a new world of opportunities that will revolutionize the industry. Champion Games is taking advantage of this wave and create a web3 game that can reach the mainstream audience. We are delighted to support this team of great professionals and we are sure that opportunities for collaboration with great potential can be opened up through Telefónica and Movistar,” says Marta Antunez, director of Wayra Barcelona. A few months ago, Wayra launched the global Open2Metaverse call, to search for companies that are developing technology for the metaverse, which brought together 135 startups and analyzed 1625 startups from all over the world. Telefónica also organized Metaverse Day on September 29th, where Champion Games was presented as one of the Web3 startups that the company is promoting.

Telefonica Bets on Web3 Gaming by Investing in MetaSoccer

Stories, News & Mentions about OPENMETA

Openmeta’s Stories on HackerNoon