MindPortis in the Virtual Reality Industry

MindPort

mindport

This paper is available on arxiv under CC 4.0 license. Authors: (1) Jeffrey Ouyang-Zhang, UT Austin (2) Daniel J. Diaz, UT Austin (3) Adam R. Klivans, UT Austin (4) Philipp Krähenbühl, UT Austin Table of Links Abstract & Introduction Related Works Preliminary Methodology Results Limitations Conclusion, Acknowledgements, & References Experimental Setting & Additional Results 5 Results 5.1 Implementation Details Our primary feature extractor is AlphaFold, implemented in OpenFold [3, 31]. The multiple sequence alignment (MSA) is computed using Colabfold [43]. Sequence features from the Evoformer and Structure Module are aggregated as input to the decoder. An adapter maps the backbone hidden dimension to D = 128. Our amino acid projections and single mutation decoders are implemented with a linear layer. The higher-order mutation decoder transforms the previous embedding with a 2-layer MLP. These representations are aggregated and fed into a 3-layer MLP to predict ∆∆G. We train on the cDNA display proteolysis dataset [72], which leverages a high throughput proteolysis screen enabled by next generation sequencing (NGS) to extract noisy ∆∆G values for 100 miniproteins totaling over 100,000 single and double mutations. To evaluate generalization on unseen proteins, we remove proteins from our training set that have high sequence similarity using MMSeqs2 [70] to any protein in our evaluation benchmarks. The short protein lengths (< 100 amino acids) keep the memory requirement small for the higher-order decoder. See Section A.1 for more information about cDNA and other datasets. We fine-tune a pre-trained backbone on single mutations for 20 epochs. Then, we finetune the model on both single and double mutations for 100 epochs using a cosine learning rate schedule with 10 warmup epochs. We use a batch size of 3 proteins due to the high memory requirements of AlphaFold. We use a learning rate of 3×10−4 and weight decay of 0.5. Training takes 6 hours on 3 A100 GPUs. 5.2 Finding Stabilizing Double Mutations We evaluate our model’s ability to find the stabilizing mutations on cDNA2. cDNA2 is our validation split of the cDNA double mutations dataset, consisting of 18 mini-proteins totaling 22,000 double mutations. The proteins in the validation set have at most 36% homology with those in the cDNA training set. Of these mutations, only 198 or 0.8% are stabilizing with ∆∆G < −0.5 kcal/mol [8]. Like protein engineering, this scenario closely resembles the challenge of identifying a small number of stabilizing mutations amidst a much larger pool of destabilizing or neutral mutations. We evaluate different methods on stabilizing mutations in Table 1. Stabilizing rs is the Spearman coefficient on the experimentally stabilizing subset introduced in [65]. Normalized discounted cumulative gain (nDCG) [30] measures the quality of the ranked mutation list by taking into account its experimentally validated ∆∆G and position in the list [59]. Detection Precision (DetPr) is the proportion of experimentally validated stabilizing mutations among the top K = 30 predictions. We adapt previous state-of-the-art single mutation predictors by naively adding both single mutation predictions. A detailed description of the metrics is found in Section A.2. Our model demonstrates exceptional performance in prioritizing stabilizing double mutations over destabilizing ones, achieving a significantly higher normalized discounted cumulative gain of 0.43 compared to 0.25, as well as a superior detection precision of 0.16 compared to 0.10. Our model additionally improves classification metrics Matthews Correlation Coefficient (MCC) and Area under Precision-Recall Curve (AUC) by 0.02 and 0.03, respectively. Table 2: Multiple Mutation Results on PTMul. Our additive baseline naively adds predicted single mutation ∆∆Gs. Our model presents a strong mutation assessor. The proteins in PTMul have at most 35% sequence similarity to those in the training set. Parenthesis is standard error across 7 runs. To the best of our knowledge, Mutate Everything is the first work that models all double mutations in a computationally tractable manner. Figure 4 shows the runtime of several methods on a protein of 317 amino acids on an A100 GPU. The dashed line indicates the transition from evaluating single mutants to double mutants. Mutate Everything predicts ∆∆G for all single and double mutations in one pass of the model. It runs in 0.6 seconds using an ESM2 backbone, and 12.1 seconds on an AlphaFold backbone. PROSTATA [74] also uses the ESM2 backbone but takes a protein sequence and its mutated sequence as input and outputs the change in stability. PROSTATA takes 306 hours to evaluate all double mutations with a batch size of 768. On an 8 GPU node, this will take 1.5 days. Figure 4: Runtime analysis. Our model’s runtime is constant. 5.3 Higher-order Mutation Results We evaluate our model for predicting changes in thermodynamic stability for higher-order mutations in ProTherm. ProTherm is a database of thermodynamic (∆∆G) and thermostability (∆Tm) experimental characterization of protein mutations curated from the literature. We consider a subset of this database that contains ∆∆G values for higher-order mutations, named ProTherm Multiple (PTMul). PTMul contains 858 mutations [44]. We keep 846 experimentally validated ∆∆G for higher-order mutants, including 536 doubles and 183 triples, after removing 4 duplicate and 8 ambiguous mutations. PTMul proteins have at most 35% homology to the proteins used in training. Table 2 shows our results on PTMul. A description of baselines is provided in Section A.3. Our additive baselines naively add the sum of single mutation ∆∆Gs. This baseline rigorously evaluates our model’s ability to learn epistasis, or the non-additive interactions when combining two mutations. Mean always predicts the global test dataset mean statistic. ESM and MSA perform poorly when comparing mutations across proteins. On ProTherm, Mutate Everything achieves the state-of-the-art performance of 0.53 Spearman correlation rs, compared to 0.50 of our additive baseline. A breakdown of PTMul performance on double mutations and more (> 2) mutations is found in Table 6. While other methods also handle multiple mutations, they usually adopt a mutation- or protein-level train and test split. This results in data leakage since the same protein or a homolog will have mutations in both the training and testing sets, leading to inflated reported metrics [16, 18, 35, 64]. In our study, we use sequence similarity-based splitting where our training proteins have at most 35% homology to those in PTMul. To fairly evaluate generalization to new proteins, we exclude these inflated comparisons from our study. Table 3: Comparisons on Single Mutation Thermodynamic Stability prediction in S669. We report regression and classification metrics on the forward and reverse datasets for structure-based approaches (top block) and sequence-based approaches (bottom block). r refers to Spearman correlation coefficient. Our training proteins have at most 30% sequence similarity with those in S669. Mutate Everything is state-of-the-art on S669 forward. 5.4 Single Mutation Results We additionally validate our model’s ability to predict changes in thermodynamic stability under single mutations. We compare Mutate Everything to prior works on the newly introduced S669 dataset, which was established to address the data leakage issue and enable fair comparisons between existing methods [54]. We also evaluate the commonly studied reverse dataset setting, in which the model takes the mutant sequence as input and predicts ∆∆G for mutating back to the wild-type sequence. The experimental value is obtained by negating the original ∆∆G value going from the wild-type sequence to the mutant sequence. S669 is well curated and extensive, containing 94 proteins totaling 669 mutations [54]. S669 proteins have at most 30% sequence similarity with those in our training set to ensure separation between the training and validation sets. Table 3 shows our comparisons with prior works. Mutate Everything is state-of-the-art on S669, achieving a Spearman correlation of 0.56, where the prior art obtained 0.53. Mutate Everything outperforms the existing sequence model by a large margin, with a 6-point gain on Spearman correlation and a 3-point improvement on AUC. Our method is even as powerful as the latest structure-based methods which have access to atom coordinates. Mutate Everything is competitive on the reverse dataset evaluation, in which the original and mutated sequences are flipped and ∆∆G negated. Our performance drops on reverse mutations because reverse mutations are out of distribution for our model. We did not perform data augmentation to train on reversed mutations as commonly done in the literature [18, 36, 74]. We found that it was beneficial to bias our reverse predictions using ∆∆G to the amino acid found in the original sequence. We found the performance to be similar across 5 runs (< 0.01 standard error). Table 4: Comparisons on mutation fitness prediction in ProteinGym. We report ranking and classification metrics on both a subset of ProteinGym with stability-like phenotypes and the entire ProteinGym dataset. Our stability model can generalize to all phenotypes, outperforming existing methods on stability-like phenotypes and performing similarly to existing methods otherwise. The maximum homology between our training set and ProteinGym proteins is 43%. 5.5 Generalization to other Phenotypes We show that stability predictors generalize to other phenotypes as well. ProteinGym substitutions is a dataset of 87 proteins with 1.6 million mutation sets labeled with a generic fitness score, such as activity, growth, or expression. The mutation sets have up to 20 substitutions and 14 proteins contain higher-order mutations. The ProteinGym-Stability split contains 7 proteins and 26,000 mutations with fitness labels that are known to correlate with thermodynamic stability (thermostability, abundance, expression). ProteinGym proteins have at most 45% sequence similarity with those in our training set. Due to memory constraints, we truncate proteins to at most 2000 amino acids. Table 4 compares our work with previous works. Our MSA baseline uses the empirical amino acid distribution in the MSA at a given position to compute a likelihood ratio. The MSA retrieval prior weights the scores using the per-position first-order statistics in a multiple sequence alignment. On ProteinGym-Stability, Mutate Everything obtains a Spearman correlation of 0.52, compared to 0.50 of the next best method, TranceptEVE. On the one protein with labeled stability, Mutate Everything obtains 0.51 vs 0.33 of Tranception with MSA retrieval [51]. On the full ProteinGym benchmark, we outperform the strongest methods by ensembling our predictions with Tranception [49]. Ensembling our method with evolutionary models by averaging scores gains 0.11 points on ProteinGym and 0.01 points on ProteinGym-Stability, suggesting that our model learns complementary information to the evolutionary models. For comparison, TransEVE ensembles an evolutionary model (Tranception) and a family-based model (EVE) for a 0.03 point gain [50]. We found the performance to be similar across 5 runs (< 0.01 standard error). An in-depth study on how protein stability related to function can be found in [26]. 5.6 Ablation Studies We ablate several architectural design choices in Table 5. Core architectural designs are evaluated on S669. Architectural designs for only multiple mutations are evaluated on cDNA2. We analyze the effect of different feature extractors on single mutation stability assessment in Table 5a. ESM2 [38] takes only sequence as input whereas MSA-Transformer [60] and AlphaFold [31] take the sequence and its MSA as input. Both ESM2 and MSA-Transformer are trained on masking residue modeling while AlphaFold is also trained on structure prediction. MSA input improves the stability prediction performance and AlphaFold’s architecture and training further improve stability prediction. Backbone. (c) Backbone Optimization on S669. Finetuning the AlphaFold backbone improves performance. (d) Higher-order Modeling on cDNA2. We model the ∆∆G of the double mutation as a residual combined with the ∆∆G of its single mutation. Table 5: Default Settings are marked in grey. Mutate Everything ablation experiments. We try fine-tuning and freezing the AlphaFold backbone during stability finetuning in Table 5c. We find that fine-tuning the backbone improves stability prediction performance. Backbone Optimization The technique to aggregate mutation representations for higher-order mutations is ablated in Table 5b. To control for feature dimension size, we reduce the head dimension when aggregating with outer product and flattening. Aggregating with product and summation performs similarly on double mutations. We add the features for a natural extension to more mutations. Aggregation In the direct prediction model, the multi-mutant head directly predicts ∆∆G. In the multiply and add models, the multi-mutant head learns how interactions among single mutations affect the higher-order mutation’s ∆∆G. In these models, the multi-mutant head output learns an additive bias or multiplicative scaling to the sum of single mutation ∆∆Gs. In Table 5d, we show that learning a bias performs the strongest among these three options. Higher Order Modeling

Predicting a Protein’s Stability
under a Million Mutations: Results

Dmitry Pyanov has been creating and driving products for some of the leading tech companies, startups, and cultural projects for over a decade. In 2017, he directed the launch of Replika (YC’16 San Francisco), an AI chatbot designed to help individuals improve their emotional well-being, which gained over 10 million users. Later on, he moved to work for The Pattern app, which helps people connect on a deeper level through astrology. In 2020, Dmitry and his co-founder Abdul Algharbi developed Mindpet, a game designed to help young people improve their habits by learning meditation and journaling - all through a cute gaming world. Most recently, Dmitry joined Prog.AI to bring his creative talent and build an AI recruitment platform that analyzes programming code and helps engineers get discovered by top employers and recruiters. Dmitry has also recently worked with brands such as LogRock.com and Nodle.com. What do you remember as your first creative success with consumer apps? It happened with , a conversational partner that becomes a friend and can help people feel less lonely or provide tools to improve well-being. Replika AI When Replika was first launched, the world was a different place, and AI wasn't such a hot topic. Replika started with a personal story about our friend Roman Mazurenko, who was a creative person and a huge fan of the positive ideas for AI. After Roman's tragic death in 2015, the team collected his text messages around his best friends and created a chatbot that could talk like him. The story went viral. People started using it and sent positive feedback to the team: "this person was amazing, I got so much empathy and support from this app." "Roman made me feel better about myself, and I didn't even know this person," - they said. After this, the team came up with the idea to launch an app called Replika, an AI app to train your personal AI that learns from you. As a creative director, I came up with the concept of a logo with a cracked egg, inspired by the Tamagotchi craze of the 90s. I also directed a campaign to build the first community of beta testers and worked with the development team to iterate on the product with consumer feedback. The app quickly started gaining millions of new users every quarter, but the purpose of the app was not clear. The community reported amazing mental health benefits, such as helping people learn more about their personalities through personality tests, self-reflection, basic CBT exercises that were implemented in the AI model, and feeling less lonely and more confident. Mental health apps were not as popular in 2016, with Headspace being relatively new, and Woebot, another AI therapy chatbot, launching a year later in 2017. It was challenging to navigate the creative sense for product positioning. At one point, Replika crossed the mark of 10 million users and still has a very strong community today. Let’s pause on Replika for a bit. After the launch of Chat GPT everyone is talking about human - AI interactions, and then the movie M3gan about how AI friends can replace a real one came out. How do you feel those concerns are justified? What do you think about human attachment to their virtual friends or romantic partners? I first became interested in this subject after reading , a 2005 non-fiction book about artificial intelligence and the future of humanity by inventor and futurist Ray Kurzweil. The Singularity Is Near: When Humans Transcend Biology AI ethics is extremely important these days. There's this amazing novel by a Nobel prize winning writer, Klara and the Sun, about an emotional-support robot, and I highly recommend reading it. I think creating a consumer AI product takes a ton of responsibility. AI is a blessing and a curse. I believe that we need more scholar research, digital mental health studies to equip us for a better future. We won’t be able to stop AI’s progress, but we must learn how it affects the economy, mental health, education and other fields to balance its impact. Personally, I’m excited about the future of digital tools for mental health. The biggest problem these days is accessibility. Finding therapy or support is extremely expensive in the US, so apps are often the only solution available. One of your accomplishments with Replika is based around gaining the first early adopters. Can you lead us through the early stages of the app and how did it evolve? After the Replika brand, logo, and creative direction were established, the team began building a waitlist of early adopters who were interested in the idea of training their own AI friends. The concept quickly gained traction, and people eagerly shared invitations to join the beta with their friends. In a short time, the waitlist grew to over 100,000, then 300,000, and eventually 1 million people. It was similar to what happened with the Clubhouse app, which also employed a well-designed invitation system and closed beta rollout where friends could invite friends. When launching a consumer app, it is crucial to test out a variety of different creative ideas for the brand and not to commit to any until receiving positive feedback and gaining traction. For example, if you don't have a marketing budget, you can still kickstart the process by finding your first users through Reddit, Twitter, Facebook Groups, and by reaching out to people directly to bring the first 100 users on board. Moving to the Pattern, which brings people together around astrology, it’s safe to say that your work continues evolving around people’s need to curtain loneliness and gain like-minded friends. Can you talk a bit about how your career developed after Replika and what brought you to the Pattern? How do you personally feel about solving that issue by tech mechanics? I had the privilege of working with The Pattern app team and it was an incredible experience. Their brand is exceptional and they have a vast community of highly engaged fans. The Pattern initially gained fame from a viral video featuring Channing Tatum, who was left amazed and wanting to know more about the app, saying "You need to DM me right now and tell me how you know this stuff!" Astrology is an intriguing subject that serves as a source of inspiration for many. Sometimes when you hear something that resonates with your personality or feelings, it can be incredibly comforting and encourage exploration of yourself and your place in the universe. During my time at The Pattern, I was able to utilize my creative expertise to launch and grow large, popular consumer apps, working with communities, product design, and developer teams to come up with innovative solutions, strengthen the brand, and introduce new features. I am proud to have played a role in launching the Shared Experiences feature, which allowed people to comment on each other's experiences and share self-care advice. It is important to me that the products I work on are beloved by the community and bring real, meaningful benefits. What are you working on these days? What product problems are you going to solve in the near future? I joined in 2022 to bring my experience in product and creative development to build a recruitment platform that leverages programming code analysis to help engineers get discovered by top employers and recruiters. The brand I’m creating is addressing the needs of both engineers and employers, and we're dedicated to showcasing the talent of the best and brightest minds through their open-source contributions. Prog.AI By using code analysis, we provide talented individuals a spotlight they deserve, and the opportunity to work for some of the greatest companies in the United States and worldwide. Our goal is to create a win-win solution for both engineers and employers, offering engineers the opportunity to showcase their skills while providing employers with access to a pool of talented candidates. I enjoy working on a brand that is centered around engineering culture. It presents a unique challenge that requires a deep understanding of the nuances and intricacies of the engineering community. I'm excited to be a part of this project and contribute my skills and expertise to help shape a brand that resonates with developers. What are three main implementations you see for AI these days? How are they going to impact society? The developments around AI, particularly with Open AI and Microsoft, are fascinating. With the GPT technology, people now have access to knowledge that they had never imagined possible, empowering them to write and learn in ways they never thought were achievable. Google will no doubt catch up in the near future. However, it's important that companies approach AI with caution, considering potential safety concerns. They should slowly gather feedback and data on how AI affects society before diving in headfirst. One of the concerns related to the rise of the smartest AI systems being only accessible to the wealthy 1% can potentially further increase inequality. It's important to recognize this downside and make sure AI is accessible to everyone and doesn’t affect the economy in a negative way. On a brighter note, I'm excited to see how AI is impacting creative industries. The technology makes high-quality art creation easier, allowing people to create their own AI paintings, design video games, and even co-create movies with AI. I recently attended the first AI film festival in San Francisco, organized by The Culture DAO, and was blown away by dozens of DIY short-movies co-created with the help of AI technologies. Those films were diverse, ranging from visual poems and autobiographical works to abstract pieces, all created by indie artists and technologists. In this respect, AI can help create a thriving independent producer community in the arts scene. Imagine being able to use AI to create your own animated show, without having to go to Hollywood or hire an expensive team. You can already do it - to focus on your story and let AI assist you with the more costly and technical aspects of the production. It's an exciting time to be alive!

How to Find Your Product Market Fit Using Creative Brand Voice – A Guide By Creative Director

In recent years, cryptocurrencies and blockchain technologies have been gaining momentum, with more and more companies and individuals turning to blockchain for various purposes. As a result, there is a growing demand for web development services that specialize in building platforms based on cryptocurrency and blockchain. As we remember, 2017-2018, the active boom of cryptocurrencies, as well as high-profile falls like FTX and Luna, taught many investors to better immerse themselves in the study of the activities of leading blockchain teams. What factors should be considered when choosing cryptocurrencies as an investment instrument? In this part of the article, I propose to consider development activity as a key factor that helps to determine where the hype is and where work is being done on the blockchain and application tools for it. And so we drove! GitHub as a project activity vector As we all know that GitHub is a powerful tool for developers to collaborate and share code with each other. In the context of blockchain, it serves as a key indicator of a project's activity and development progress. As well as to show development and code transparency. By publicly sharing code and updates on a project's development, the community can see what is being worked on and hold the development team accountable for their progress. This level of transparency helps to build trust and credibility among the community and new investors. GitHub also provides a way for developers to easily collaborate and contribute to a project. This can lead to a more active and engaged community, which can help to drive the development and adoption of a blockchain project. Additionally, GitHub allows for easy integration with other tools, such as automated testing and continuous integration, which can help to improve the quality and reliability of a project. Another important aspect of GitHub is that it can be used to track the number of contributors and the amount of code being added to a project. This can provide insight into the health and growth of a project and can be used to compare the development progress of different blockchain projects. In summary, GitHub is an important project activity vector in blockchain because it provides transparency, and accountability, and facilitates collaboration and contributions. Tracking developer activity on GitHub can help to gauge the progress and health of a blockchain project, which can be an important factor when assessing the potential success of a project. What are the best GitHubs to follow in 2023-2024? As I described above, when choosing a project as an investment, you need to look not only at the team, but also at the activity of projects there. For example, I put together a list of Github projects that are most worth paying attention to (my own opinion): Ethereum 👉 https://github.com/ethereum Binance Smart Chain 👉 https://github.com/bnb-chain Everscale 👉 https://github.com/tonlabs Near Protocol 👉 https://github.com/near Avalanche 👉 https://github.com/ava-labs Cardano 👉 https://github.com/cardano-foundation Algorand 👉 https://github.com/algorand Сosmos 👉 https://github.com/cosmos Elrond 👉 https://github.com/ElrondNetwork Fantom 👉 https://github.com/Fantom-Foundation Dfinity 👉 https://github.com/dfinity Kusamana 👉 https://github.com/kusamanetwork Polkadot 👉 https://github.com/polkadot NEO 👉 https://github.com/neo-project Polygon 👉 https://github.com/maticnetwork TRON 👉 https://github.com/tronprotocol Zilliqa 👉 https://github.com/Zilliqa Aleph Zero 👉 https://github.com/aleph-zero-foundation VeChain 👉 https://github.com/vechain Mina Protocol 👉 https://github.com/MinaProtocol Smart Contract 👉 https://github.com/smartcontractkit Stellar 👉 https://github.com/stellar Flow Blockchain  👉 https://github.com/onflow EOS 👉 https://github.com/EOSIO Hedera 👉 https://github.com/hashgraph Waves 👉 https://github.com/wavesplatform Harmony 👉 https://github.com/harmony-one Qtum 👉 https://github.com/qtumproject Kadena 👉 https://github.com/kadena-io And so, if you track activity on GitHub among famous blockchains, you will always be aware of the latest developments and improvements made to the technology. This can help in making investment decisions, as well as give an idea of the direction in which a particular blockchain project is heading (whether it is worth investing in at all). In addition, tracking activity on GitHub can also give developers the opportunity to contribute to the project by submitting pull requests or reporting bugs (or avoid cyber victim attacks, that is really important for DeFi sector). In general, tracking activity on GitHub helps you stay as informed and involved as possible in the blockchain ecosystem. Remember, even if you don’t understand absolutely anything about it, you can always follow a relevant blogger on Twitter / YouTube and watch his analysis (only facts, without crypto soothsayers😂).

Using GitHub Activity to Gauge Blockchain & Crypto Projects

Will Layer-1 Public Blockchains Rise or Fall in the Next Bull Market?

¿Subirán o caerán las cadenas de bloques públicas de capa 1 en el próximo mercado alcista?

Ethereum is turning from PoW (Proof of Work) to PoS (Proof of Stake). However, PoS launch time is postponed again and again. Ethereum uses to switch the network to PoS. After the difficulty bomb takes effect, mining on the PoW network will be exponentially difficult. The block time increases dramatically. After nobody can mint new blocks on the PoW network, all participants can only switch to PoS network. The following graph shows the average block time. difficulty bomb Source: Etherscan Every wave stands for the impact of the difficulty bomb. The block time increases a lot. The wave disappears because the difficulty bomb is postponed during network upgrades. The launch of PoS is essential for Ethereum. In this article, we will explain the history of PoS and why it is so hard for Ethereum to switch to PoS. How to Learn a Consensus PoW and PoS are two common consensus approaches. We can ask ourselves these two questions when learning a consensus: Who has the right to propose new blocks If there is a fork, which will be the canonical chain PoW Overview With the framework in the last section, we can analyze PoW shortly. The node which first gets the answer to the computation puzzle right can propose the new block. If there is a fork, the longest fork will become the canonical chain. If anyone would like to manipulate the chain, the attacker needs at least 51% computation power. Compared to PoS, PoW is relatively simple. PoW does not tap into problems like random numbers. PoW networks are protected by the cost of the work. In the long term, nodes will get rewards proportional to their computation power. PoS Deep Dive Let’s answer the question at the beginning, why it is so hard for Ethereum to switch to PoS. Part of the reason is that Ethereum is pushing decentralization and encourages more nodes to join the network. This requires a more complex protocol to support the P2P communication of thousands of nodes. Most networks employ PoS because of , , higher TPS, and fast finality. If any nodes cheat, the network can immediately slash their staking. sustainability security History of PoS PoS comes through two-phase development. PoS 1.0 is blockchain-based PoS consensus. PoS 2.0 is based on BFT(Byzantine Fault Tolerance). Examples of PoS 1.0 are PeerCoin, NextCoin, BlackCoin, and Ethereum Serenity. PoS 2.0 is based on BFT and is widely adopted by projects like Ethereum 2.0, Tendermint, and Cosmos. PoS 1.0 During phase 1.0, most PoS looks like this: Nodes stake tokens Randomly select one node to produce new blocks. The probability is proportional to the number of staked tokens The biggest problem of this workflow is the generation of random numbers. If the random number generation is based on timestamp and block hash, the block proposer would generate the new block in a way that increases the probability of being elected as the block proposer again. Attack Vector Common attack vectors are listed below: : The block proposer tries to influence the random number generator to get more chances to produce blocks Staking grinding : In PoS networks, producing new blocks does not cost lots of computation resources. When there is a fork, nodes will produce new blocks on every fork. In this way, nodes can maximize their yields. However, this is harmful to the network stability Nothing at stake : Early validators can roll back the state in the future. This will easily cause forks Long-range attack You can find more attack vectors on . page 58 of this paper PoS 2.0 PoS 2.0 employs BFT. The goal of BFT is to reach a consensus between different nodes. The leader will make a proposal. Among all nodes, there are Byzantine nodes. Byzantine nodes cheat in the following ways: Failure to return a result Respond with an incorrect result Respond with a deliberately misleading result Respond with a different result to different parts of the system Except for Byzantine nodes, the rest of the nodes are all honest nodes. The most difficult scenario is that the leader node is Byzantine. Besides these settings, there are some other limitations: Nodes can only communicate peer to peer Nodes keep the log of sent-out messages P2P communication causes different nodes to have different views. For example, node A finds 3 new blocks, but node B only sees 1 new block. The main goal of BFT is that after the leader makes proposals, every node except Byzantine nodes can reach an agreement. The side goal is to find the Byzantine nodes. The current algorithm can achieve the main goal in the following condition: Among 4 nodes, 1 is Byzantine node Among 25 nodes, 8 are Byzantine nodes Among 100 nodes, 33 are Byzantine nodes Network operation status can be categorized in the following conditions. f is the number of Byzantine nodes. There is a total of 3k+1 nodes in the network. f ≤ k, the network is stable k < f < 2k+1, some nodes may fail to reach consensus. The blockchain pauses f ≥ 2k+1, there are security risks The popular BFT algorithms are and . Tendermint can keep the network stable in the condition that among 3k+1 nodes, there are k byzantine nodes. PBFT Tendermint BFT Modern BFT algorithms employ multi-round votes to ensure every node knows that more than 2/3 of nodes approve the proposal. After the block is finalized, the honest node will never revert it. Every round of communication has four steps: The client sends requests to the leader The leader pass request to all nodes Nodes respond to leader with execution results After getting f+1 same results, the leader responds to the client (f is the number of Byzantine nodes) The below diagrams show 5 stages during execution. C is the client, 0 is the leader and 3 is the Byzantine node. Prepare and commit stages are used to ensure the synchronization between different nodes. For Tendermint, it has several limitations: P2P communications cannot support more than 100 nodes in the network Complex P2P communications design Execution delay. Nodes first decide the new blocks. Then nodes execute all transactions in the new block. Although individual transactions work fine, executing all transactions once may cause problems, like the double-spending problem If you would like to learn more about PBFT, check out this . blog Ouroboros Compared to BFT algorithms, Ouroboros can support more and more nodes and it also supports nodes to come in and out at anytime. Ouroboros is more decentralized and flexible than BFT but has a longer finality time. Cardano and Mina uses Ouroboros. Ouroboros divides time into epochs. Epochs can be divided into slots. Every slot uses VRF to generate random numbers and use this random number to decide the block proposer. Nodes staking more tokens are more possible to be a block proposer. When there is a fork, Ouroboros will choose the correct chain with predefined selection rules. For example, it may choose the most “dense” chain. Future Blockchains are balancing decentralization and performance. Switching from PoW to PoS shows that blockchains are chasing We will see more innovations around these four characteristics. TPS, decentralization, tokenomics, and fast finality. Some blockchains use sampling techniques to lower the communication frequency between nodes. Byzantine nodes need to send proofs with their messages. This can prevent Byzantine nodes from cheating. Through verifying the proof, honest nodes can also find byzantine nodes. ZK can be applied to it. The tokenomics innovations in GameFi and DeFi applications can be applied in consensus. For example, we can . may also be an interesting direction. Separating the rewards and sharing tokens to catch more value. use Ve tokens to improve the staking ratio Using dual token design Application iterates much faster than low-level protocol, like consensus. It is hard to apply innovations on applications to low-level protocols. Modular blockchains may solve this problem. However, it is still risky for blockchains to hot-swap essential modules. Reference https://www.youtube.com/watch?v=J34dCMIMxd4 https://www.geeksforgeeks.org/practical-byzantine-fault-tolerancepbft/ http://muratbuffalo.blogspot.com/2020/01/practical-byzantine-fault-tolerance.html https://medium.com/@carloslopezdelara/whats-ouroboros-the-cardano-proof-of-stake-protocol-ad4b958e152e https://minaprotocol.com/blog/how-ouroboros-samasika-upholds-minas-goals-of-decentralization https://iohk.io/en/research/library/papers/ouroboros-a-provably-secure-proof-of-stake-blockchain-protocol/ https://arxiv.org/pdf/1807.04938.pdf http://www.cs.cmu.edu/\~dga/15-712/F13/papers//castro99.pdf http://muratbuffalo.blogspot.com/2020/01/practical-byzantine-fault-tolerance.html This article was first published here.

What Ethereum's Move To Proof Of Stake Means

Deploy Your Application with Kafka and Docker in 20 Minutes 

Creating Microservices in Nest.js

Introduction Angular is a development platform for building WEB, mobile, and desktop applications using HTML, CSS, and TypeScript (JavaScript). Currently, Angular is at version 14, and Google is the main maintainer of the project. is a rich and interactive charts library based on SVG. highcharts Prerequisites Before you start, you need to install and configure the tools: git Node.js and npm Angular CLI IDE (e.g. ) Visual Studio Code Getting started Create the Angular Application Let’s create the application with the Angular base structure using the with the route file and the SCSS style format. 1. @angular/cli ng new angular-highcharts
? Would you like to add Angular routing? Yes
? Which stylesheet format would you like to use? SCSS   [ https://sass-lang.com/documentation/syntax#scss                ]
CREATE angular-highcharts/README.md (1063 bytes)
CREATE angular-highcharts/.editorconfig (274 bytes)
CREATE angular-highcharts/.gitignore (604 bytes)
CREATE angular-highcharts/angular.json (3279 bytes)
CREATE angular-highcharts/package.json (1080 bytes)
CREATE angular-highcharts/tsconfig.json (783 bytes)
CREATE angular-highcharts/.browserslistrc (703 bytes)
CREATE angular-highcharts/karma.conf.js (1435 bytes)
CREATE angular-highcharts/tsconfig.app.json (287 bytes)
CREATE angular-highcharts/tsconfig.spec.json (333 bytes)
CREATE angular-highcharts/src/favicon.ico (948 bytes)
CREATE angular-highcharts/src/index.html (303 bytes)
CREATE angular-highcharts/src/main.ts (372 bytes)
CREATE angular-highcharts/src/polyfills.ts (2820 bytes)
CREATE angular-highcharts/src/styles.scss (80 bytes)
CREATE angular-highcharts/src/test.ts (788 bytes)
CREATE angular-highcharts/src/assets/.gitkeep (0 bytes)
CREATE angular-highcharts/src/environments/environment.prod.ts (51 bytes)
CREATE angular-highcharts/src/environments/environment.ts (658 bytes)
CREATE angular-highcharts/src/app/app-routing.module.ts (245 bytes)
CREATE angular-highcharts/src/app/app.module.ts (393 bytes)
CREATE angular-highcharts/src/app/app.component.scss (0 bytes)
CREATE angular-highcharts/src/app/app.component.html (24617 bytes)
CREATE angular-highcharts/src/app/app.component.spec.ts (1109 bytes)
CREATE angular-highcharts/src/app/app.component.ts (223 bytes)
✔ Packages installed successfully. Install and configure the Bootstrap CSS framework. Then, do steps 2 and 3 of the post, . 2. Adding the Bootstrap CSS framework to an Angular application Install the library. 3. highcharts npm install highcharts Remove the contents of the class from the file. Import the , and services and create the , , and methods as below. 4. AppComponent src/app/app.component.ts Highcharts HighchartsMore HighchartsSolidGauge createChartGauge createChartPie createChartColumn createChartLine import { AfterViewInit, Component } from '@angular/core';
import * as Highcharts from 'highcharts';
import HighchartsMore from 'highcharts/highcharts-more';
import HighchartsSolidGauge from 'highcharts/modules/solid-gauge';

HighchartsMore(Highcharts);
HighchartsSolidGauge(Highcharts);

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.scss'],
})
export class AppComponent implements AfterViewInit {

  public ngAfterViewInit(): void {
    this.createChartGauge();
    this.createChartPie();
    this.createChartColumn();
    this.createChartLine();
  }

  private getRandomNumber(min: number, max: number): number {
    return Math.floor(Math.random() * (max - min + 1) + min)
  }

  private createChartGauge(): void {
    const chart = Highcharts.chart('chart-gauge', {
      chart: {
        type: 'solidgauge',
      },
      title: {
        text: 'Gauge Chart',
      },
      credits: {
        enabled: false,
      },
      pane: {
        startAngle: -90,
        endAngle: 90,
        center: ['50%', '85%'],
        size: '160%',
        background: {
            innerRadius: '60%',
            outerRadius: '100%',
            shape: 'arc',
        },
      },
      yAxis: {
        min: 0,
        max: 100,
        stops: [
          [0.1, '#55BF3B'], // green
          [0.5, '#DDDF0D'], // yellow
          [0.9, '#DF5353'], // red
        ],
        minorTickInterval: null,
        tickAmount: 2,
        labels: {
          y: 16,
        },
      },
      plotOptions: {
        solidgauge: {
          dataLabels: {
            y: -25,
            borderWidth: 0,
            useHTML: true,
          },
        },
      },
      tooltip: {
        enabled: false,
      },
      series: [{
        name: null,
        data: [this.getRandomNumber(0, 100)],
        dataLabels: {
          format: '<div style="text-align: center"><span style="font-size: 1.25rem">{y}</span></div>',
        },
      }],
    } as any);

    setInterval(() => {
      chart.series[0].points[0].update(this.getRandomNumber(0, 100));
    }, 1000);
  }

  private createChartPie(): void {
    let date = new Date();
    const data: any[] = [];

    for (let i = 0; i < 5; i++) {
      date.setDate(new Date().getDate() + i);
      data.push({
        name: `${date.getDate()}/${date.getMonth() + 1}`,
        y: this.getRandomNumber(0, 1000),
      });
    }

    const chart = Highcharts.chart('chart-pie', {
      chart: {
        type: 'pie',
      },
      title: {
        text: 'Pie Chart',
      },
      credits: {
        enabled: false,
      },
      tooltip: {
        headerFormat: `<span class="mb-2">Date: {point.key}</span><br>`,
        pointFormat: '<span>Amount: {point.y}</span>',
        useHTML: true,
      },
      series: [{
        name: null,
        innerSize: '50%',
        data,
      }],
    } as any);

    setInterval(() => {
      date.setDate(date.getDate() + 1);
      chart.series[0].addPoint({
        name: `${date.getDate()}/${date.getMonth() + 1}`,
        y: this.getRandomNumber(0, 1000),
      }, true, true);
    }, 1500);
  }

  private createChartColumn(): void {
    let date = new Date();
    const data: any[] = [];

    for (let i = 0; i < 10; i++) {
      date.setDate(new Date().getDate() + i);
      data.push({
        name: `${date.getDate()}/${date.getMonth() + 1}`,
        y: this.getRandomNumber(0, 1000),
      });
    }

    const chart = Highcharts.chart('chart-column' as any, {
      chart: {
        type: 'column',
      },
      title: {
        text: 'Column Chart',
      },
      credits: {
        enabled: false,
      },
      legend: {
        enabled: false,
      },
      yAxis: {
        min: 0,
        title: undefined,
      },
      xAxis: {
        type: 'category',
      },
      tooltip: {
        headerFormat: `<div>Date: {point.key}</div>`,
        pointFormat: `<div>{series.name}: {point.y}</div>`,
        shared: true,
        useHTML: true,
      },
      plotOptions: {
        bar: {
          dataLabels: {
            enabled: true,
          },
        },
      },
      series: [{
        name: 'Amount',
        data,
      }],
    } as any);

    setInterval(() => {
      date.setDate(date.getDate() + 1);
      chart.series[0].addPoint({
        name: `${date.getDate()}/${date.getMonth() + 1}`,
        y: this.getRandomNumber(0, 1000),
      }, true, true);
    }, 1500);
  }

  private createChartLine(): void {
    let date = new Date();
    const data: any[] = [];

    for (let i = 0; i < 10; i++) {
      date.setDate(new Date().getDate() + i);
      data.push([`${date.getDate()}/${date.getMonth() + 1}`, this.getRandomNumber(0, 1000)]);
    }

    const chart = Highcharts.chart('chart-line', {
      chart: {
        type: 'line',
      },
      title: {
        text: 'Line Chart',
      },
      credits: {
        enabled: false,
      },
      legend: {
        enabled: false,
      },
      yAxis: {
        title: {
          text: null,
        }
      },
      xAxis: {
        type: 'category',
      },
      tooltip: {
        headerFormat: `<div>Date: {point.key}</div>`,
        pointFormat: `<div>{series.name}: {point.y}</div>`,
        shared: true,
        useHTML: true,
      },
      series: [{
        name: 'Amount',
        data,
      }],
    } as any);

    setInterval(() => {
      date.setDate(date.getDate() + 1);
      chart.series[0].addPoint([`${date.getDate()}/${date.getMonth() + 1}`, this.getRandomNumber(0, 1000)], true, true);
    }, 1500);
  }

} Remove the contents of the file. Add the tags for each chart as below. 5. src/app/app.component.html div <div class="container-fluid py-3">
  <h1>Angular Highcharts</h1>

  <div class="row mt-5">
    <div class="col-6">
      <div id="chart-gauge"></div>
    </div>
    <div class="col-6">
      <div id="chart-pie"></div>
    </div>
  </div>
  <div class="row mt-5">
    <div class="col-6">
      <div id="chart-column"></div>
    </div>
    <div class="col-6">
      <div id="chart-line"></div>
    </div>
  </div>
</div> Add the style in the file as below. 6. src/app/app.component.scss #chart-gauge, #chart-pie, #chart-column, #chart-line {
  display: block;
  min-width: 150px;
  max-width: 400px;
  height: 200px;
  margin: 0 auto;
} Run the application with the command below. 7. npm start

> angular-highcharts@1.0.0 start
> ng serve

✔ Browser application bundle generation complete.

Initial Chunk Files | Names         |      Size
vendor.js           | vendor        |   2.79 MB
styles.css          | styles        | 266.63 kB
polyfills.js        | polyfills     | 128.52 kB
scripts.js          | scripts       |  77.06 kB
main.js             | main          |  18.40 kB
runtime.js          | runtime       |   6.64 kB

                    | Initial Total |   3.28 MB

Build at: 2021-09-09T01:25:33.291Z - Hash: 2495029c373d88bd55c1 - Time: 27953ms

** Angular Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200/ **


✔ Compiled successfully. Ready! Access the URL and check if the application is working. See the application working on and . 8. http://localhost:4200/ GitHub Pages Stackblitz The application repository is available at . https://github.com/rodrigokamada/angular-highcharts This tutorial was posted on my in Portuguese. blog

Adding charts using the Highcharts library to an Angular application

RT-Thread Studio IDE v2.1.0 is releasing! Back to the , you may get to know the features of RT-Thread Studio and may have downloaded it for development. So let's see what's NEW in . last article v2.1.0 Make Your Own Board Supported Packages! RT-Thread Studio V2.1.0 offers a tool associated with tutorials, which helps developers create the BSP visually. Developers can now easily make a board support package (BSP) and upload it online via SDK Manager. The BSP tool supports configuring graphically the information of dev boards, documentation, and projects. The prompt of every configuration item is shown in front of the interface to help you understand. Also, this time, the Studio team gives a sweet thought. They make the configuration information is available for preview! Check out this tutorial to make a BSP by yourself. 40+ Fresh BSPs Online More than 40 new board support packages are supported in RT-Thread Studio V2.1.0, so we now have a total of 70 BSPs, covering eight vendors such as Allwinner, AlphaScale, ArteryTek, Bluetrum, GigaDevice, MicroChip, MindMotion, NXP, ST, TI, Synwit. In particular, the RT-Thread V4.0.3 source resource pack has been added on RT-Thread Studio V2.1.0. Support MDK Development RT-Thread Studio v2.1.0 supports bilateral synchronous co-development with the MDK project. You can import an existing RT-Thread MDK project directly into RT-Thread Studio, and the configuration of the MDK will be automatically synchronized with the RT-Thread Studio project. RT-Thread Studio provides a bilateral synchronization mechanism that allows the project to switch between MDK and RT-Thread Studio at any time. The MDK configuration feature is also supported to perform configuration items such as C/C++, ASM, Linker and automatically synchronized with MDK projects when the configuration is completed. If you modify some configurations on the MDK, you can manually trigger synchronization at RT-Thread Studio to sync the modified configurations with RT-Thread studio projects. Support CubeMX Development RT-Thread Studio v2.1.0 is also in collaboration with STM32CubeMX, where you can open CubeMX Settings directly in RT-Thread Studio. After configuration, click the button GENERATE CODE, and the code generated in CubeMX will automatically be copied into the RT-Thread Studio project directory, no further modifications are required. Then it is automatically added to the compilation. We only need to compile, download and debug programs as usual. Check out this tutorial for more information: Perfect and Add A New QEMU Simulator QEMU in RT-Thread Studio v2.1.0 has added two simulators for the stm32f401 and the stm32f410 series respectively. You can download the latest version of QEMU in SDK Manager. When configuring QEMU, select the emulator in the pull-down box of the Emulator configuration bar. The configuration interface has also made some updates: First, the configuration of the serial port was added in this version. When a different serial is selected, the standard IO device is relocated to the corresponding serial port. Second, SD Card Memory is now optional and compatible with situations where an SD card is not required. More importantly, the commands such as -show-cursor are moved to Extra Command, where you can customize the parameters of these commands to make QEMU more flexible to use. Download RT-Thread Studio V2.1.0 Ideas for RT-Thread Studio, talk with them. Questions when using RT-Thread Studio, create a post to ask in RT-Thread Club! Also published at: https://club.rt-thread.io/ask/question/56.html

RT-Thread Studio IoT IDE v2.1.0 Update: Fresh Boards, NXP, and MicroChip

Alan Kay said: “Simple things should be simple, complex things should be possible”. This sentence was the inspiration for the founder of RT-Thread to stick to his beliefs. Background: RT-Thread was born in 2006, it is an open source, neutral, and community-based real-time operating system (RTOS). RT-Thread has Standard version and Nano version: For resource-constrained microcontroller (MCU) systems, the NANO kernel version that requires only 3KB Flash and 1.2KB RAM memory resources can be tailored with easy-to-use tools; For resource-rich IoT devices, RT-Thread can use the on-line software package management tool, together with system configuration tools, to achieve intuitive and rapid modular cutting, seamlessly import rich software packages, thus achieving complex functions like Android's graphical interface and touch sliding effects, smart voice interaction effects, and so on. RT-Thread has not only a real-time kernel, but also rich components. Its architecture is as follows: RT-Thread Architecture: It includes: Kernel Layer: RT-Thread kernel, the core part of RT-Thread, includes the implementation of objects in the kernel system, such as multi-threading and its scheduling, semaphore, mailbox, message queue, memory management, timer, etc.; libcpu/BSP (Chip Migration Related Files/Board Support Package) is closely related to hardware and consists of peripheral drivers and CPU porting. Components and Service Layer: Components are based on upper-level software on top of the RT-Thread kernel, such as virtual file systems, FinSH command-line interfaces, network frameworks, device frameworks, and more. Its modular design allows for high internal cohesion inside the components and low coupling between components. RT-Thread Software Package: A general-purpose software component running on the RT-Thread IoT operating system platform for different application areas, consisting of description information, source code or library files. RT-Thread provides an open package platform with officially available or developer-supplied packages that provide developers with a choice of reusable packages that are an important part of the RT-Thread ecosystem. The package ecosystem is critical to the choice of an operating system because these packages are highly reusable and modular, making it easy for application developers to build the system they want in the shortest amount of time. RT-Thread supports more than 200 software packages. RT-Thread Features: Supports resource-constrained devices, the minimum kernel requires only 1.2KB of RAM and 3 KB of Flash. It has rich components and a prosperous and fast growing package ecosystem. Elegant code style, easy to use, read and master. High Scalability. RT-Thread has high-quality scalable software architecture, loose coupling, modularity, is easy to tailor and expand. Supports high-performance applications. Supports cross-platform and a wide range of chips. RT-Thread supports many architectures, and has covered the major architectures in current applications. Architecture and chip manufacturer involved: Supported Architectures: ：manufacturers like ST. ARM Cortex-M0/M0+ ：manufacturers like ST、Winner Micro、MindMotion, etc. ARM Cortex-M3 ：manufacturers like ST、Nuvton、NXP、GigaDevice、Realtek、Ambiq Micro, etc. ARM Cortex-M4 ：manufacturers like ST、NXP ARM Cortex-M7 ：manufacturers like GigaDevice ARM Cortex-M23 ：manufacturers like NXP ARM Cortex-R4ARM Cortex-A8/A9 ：manufacturers like Samsung ARM7 ：manufacturers like Allwinner、Xilinx 、GOKE ARM9 ：manufacturers like Fullhan ARM11 ：manufacturers like loongson、Ingenic MIPS32 ：manufacturers like Hifive、Kendryte RISC-V ：manufacturers like SYNOPSYS ARC ：manufacturers like TI DSP C-Sky X 86 The main IDE/compilers supported by RT-Thread are: Supported IDE and Compiler: MDK KEIL IAR GCC RT-Thread Studio(RT-Thread studio is a one-stop development tool built by RT-Thread) Use Python-based for command-line builds. RT-Thread has received many many supports from the community developers when it starts. Now, RT-Thread has gathered 200+ software packages which are created by the community developers also RT-Thread had 9357 commits and gained 4.2K stars on Github. Real-time system (RTOS) is increasingly widely used in the high-end Internet of Things (like AIoT) because of its low cost, high real-time, and fast start-up characteristics, soon or later, more and more RTOSs would support multi-kernel SMP, AI, audio & video and this is inevitable. In the near future, RT-Thread Studio IDE, next-generation microkernel architecture, AI frameworks and more will all be released step by step. This is a new world of RTOS. scons Community: RT-Thread Contact Info： Website: https://www.rt-thread.io/ Github: https://github.com/RT-Thread Twitter: https://twitter.com/rt_thread Facebook： https://www.facebook.com/RT-Thread-IoT-OS-110395723808463/?modal=admin_todo_tour Youtube: https://www.youtube.com/channel/UCdDHtIfSYPq4002r27ffqPw?view_as=subscriber

An Introduction to Open-source Real-time Operating System

By , Founder and CEO of . Edited by Bing He and Alex C-G Han Xiao Jina AI In early 2020, I left Tencent AI and founded my startup. Not long after, on April 28th, we released our core product, Jina, as open source. in short, Jina aims to search everything: image-to-image, video-to-video, tweet-to-tweet, audio-to-audio, code-to-code, etc. To understand our ambition, I often explain with two analogies: TensorFlow, PyTorch, MXNet, Mindspore, are universal frameworks for deep learning. You can use them to tell cats from dogs, or play Go and DOTA. They’re powerful and versatile but not optimized for a specific domain. In Jina, we’re focusing on one domain only: Search. We’re building on top of the universal deep learning frameworks and providing infrastructure for any AI-powered search application. “TensorFlow” for search: There are design patterns for every era: from functional programming to object-oriented programming. The same goes for search systems. Thirty years ago, it all started with a simple textbox. Many design patterns have been proposed for implementing search systems behind this, some of which are incredibly successful commercially. In the era of neural search, queries go beyond a few keywords: They can be an image, video, code snippet, or audio file. When traditional symbolic search systems can’t handle these data formats effectively, people need a new design pattern for building neural search systems. That’s what Jina is: a new design pattern for a new era.Who set me on this path? A design pattern: I’ve been working in AI, especially in open-source AI, for quite some time. You may have heard of or used my previous work on and . From 2018 to early 2020, I was an Engineering Lead at Tencent’s AI Lab, where I led a team to build the search infrastructure of China’s ubiquitous app, Fashion-MNIST bert-as-service WeChat. In 2019, I represented Tencent as a board member of the . That was the year I learned how professional open-source initiatives work. LF AI Foundation I picked up two things at the LF AI Foundation: Open source = Open source code + Open governance. Community is the key. Open source AI infrastructure is the future, and I need to act now. I’m sure many share my vision. But my belief is so strong that it drove me to jump out of the tech giant and build my company from scratch as an open-source startup. Challenging as it is, this is the opportunity I just can’t miss, and this is the future I believe in. All of my team share this belief as strongly as I do. At Jina AI, we only do what we believe in. I always tell my team: people who actually make change are the ones who believe that change is possible in the first place. Challenges of an open-source software company Running an open-source software (OSS) company needs courage, an open mindset, and a strong belief. As an OSS company, when you first show your codebase to the world, you need courage. The code quality now symbolizes the company. “Are you following best practices? Are you building technical debt here and there?” Open source is an excellent touchstone to help you understand and improve the quality of your software engineering and development procedures. Embracing community is vital for every OSS company. It needs an . Doing open source is not the same as making a press release or giving a speech in the spotlight: it is not one-way communication. You need to walk into the community, talk to them, solve their issues, answer their questions, and accept their criticisms. You have to manage your ego and do trivial things like maintenance and housekeeping. open mindset The best time for AI engineering For engineers who want to do open source AI, the time is now. Thanks to deep learning frameworks and off-the-shelf pre-trained models, there are so many opportunities in the end-to-end application market for anyone to make a significant contribution. Ask your colleagues or friends “which AI package do you use for daily tasks like machine translation/image enhancement/data compression/code completion?” — You’ll get different answers every time. That’s often a sign that the market is still uncontested, and there is ample opportunity for growth. One thing I like to remind AI open-source developers about is the of a project. With new AI algorithms popping up every day, how do you keep up the pace? What is the scope of your project? How do you maintain the project when facing community requests? When I was developing , I got so many requests to extend it to , , etc. I prioritized those that fit my roadmap. Sometimes this means hard feelings for some people. But let’s be honest: You can’t solve every issue, not on your own. That’s not how open source works and certainly not how you work. The biggest risk of open-source software is that the core developers behind it burn out. The best open source may not be the shiniest, but the one that survives the longest. So stay enthusiastic and play the long game! sustainability bert-as-service AlBERT DistilBERT BioBERT Doing open-source is doing a startup In the end, doing open source projects is itself like running a startup: Technical advantage is only part of the story. Uploading code to GitHub is just a starting point, and you need to consider tasks like operations, branding, and community management as well. Like entrepreneurship, you need to draw a “pie” that captures the passions and dreams of the community. You need to have determination and precise targets so you don’t get sidetracked by community issues. As someone with a Machine Learning Ph.D., I’ve never believed that some black-magic algorithm would be the competitive advantage of an open-source project. Instead, I’m more convinced that sound engineering, attention to detail, slick user-experience, and community-driven governance ultimately drive user retention. Often, the most important thing is your understanding and belief in open source. If you’re an idealist, then you’ll inspire idealists to march with you. If you’re detail-oriented, every little feature in your project will be worshiped by those who sweat the small stuff. If you’re warm-hearted, then the community you build up will appreciate your selfless giving. Whatever kind of person you are, it’s always your belief in open source that makes open source what it is. Jina is currently hiring AI engineers and full-stack developers. Find out more! Han Xiao is the founder of Jina AI, and creator of bert-as-service and Fashion-MNIST

Here’s Why I’m Gambling It All Away On My Open Source Startup

This short post is written for recent graduates and current students who aim to find a job as a Software Engineer in the Tech industry and contains a list of resources to help them. This list is by no means comprehensive and is only supposed to provide a starting point to freshers who are beginning their job search and preparation. : I will keep adding resources to the list. Please contact me, if you have some. Note Competitive Coding Every major tech company these days, judges their candidate’s problem solving skills through competitive coding based assessments and interviews. All the websites mentioned below are popular among coders and you can find almost all the coding questions asked by major tech companies here: Online Judges : This website covers all the aspects of tech job preparation. It’s famous for its vast pile of questions but also has separate sections for discussing design questions, offer details etc. LeetCode : Good for beginners, this portal has a clean interface and is not very daunting for freshers. Hackerrank : This is the most popular online judge for competitive coding and regularly holds contests. TopCoder CodeSignal Books : Cracking the Coding Interviews provides a very good base for freshers. This books contains questions from all the Top tech companies and is a must have. CTCI : Elements of Programming Interviews is a good book for coders experienced with competitive coding. I’ll recommend using this after completing CTCI. EPI Some of the best online MOOCs available on Data Structures and Algorithms: edX MIT Stanford Educative System Design Though this topic is more expected to come up during experienced developer interviews, a basic knowledge of System Design not only helps in tech interviews but is a requirement for becoming a good Software Developer. These resources are some of the best available in the market as of now. These are helpful to beginners and experienced alike: : A comprehensive post focussing on all areas of System Design. System Design Primer : Different system design questions and posts with answers and explanations. Educative Grokking System Design : System Architectures of large companies. High Scalability : A place to discuss different System Design Interview questions. LeetCode System Design Object Oriented Design OOP concepts are asked from both freshers and experienced developers. Good knowledge of OOP helps in designing large systems and hence most of the Top tech companies include one round of interviews focussing on OOP. Below mentioned resources are tried by me and proved to be very helpful. : Different types of OOP questions and answers with complete architecture explained. Educative Grokking OOP Concepts (Book) OOP Object Oriented Design Job Search There are tons of websites online for finding jobs in Tech, so there’s no point in repeating them. Instead, I’m going to list the platforms which are a bit different but helps tremendously in landing you your dream job. : Tech-Twitter is the best Twitter. Engineers, Designers, Entrepreneurs share knowledge as well as have meaningful conversations about tech, startups, frameworks etc. But it is also great for job search! Check out and example of someone asking for a job and being helped by the Twitter community. Tech Twitter this this 3rd party sources: There has been a surge of startups who are helping Engineers to get hired at the Top tech firms. I’m not talking consultancies but companies which are known to do this job well. Most famous are and . Triplebyte Pathrise SDE Openings: This and this are some good resources to find tech jobs in 2020. Also Y Combinator’s own . Github list website job portal Women in Tech: Numerous platforms are helping women in technology to get great offers. Not only are there specific communities on Twitter, Reddit etc there are some amazing platforms out there too. Most notable are by and Hire Tech Ladies Allison Esposito Medina Women In Technology Communities Job Search can become very stressful especially in Tech where you face a lot of competition. These communities are meant to support each other in job search and preparation. These are discord groups where you can find sub-groups focussed on different aspects of job search such as coding, interview preparation, resume review, ML focussed jobs etc. : Focussed only on 2020 New grads. Taleo : For both freshers and experienced devs. CS Career Hackers : Another great community of developers helping each other out and sharing knowledge. DEV and : An anonymous network of professionals that helps in getting referrals, information of teams, career advice etc. Though the anonymity makes it a little bit toxic sometimes, overall it’s a great help for finding and preparing for jobs. Blind Rooftop Slushie There are tons of resources available online for the fields mentioned below. The resources added below are focussed towards a fresher entering any of these fields: Android Application Development : Google’s original documentation is by far the best way to learn about Android application development. Google Android Developers : Android publication containing tutorials and blogs. Mindorks Android Weekly : This tutorial website is very popular for it’s Android and IOS tutorials. They are very well explained but are paid. Highly recommend! Ray WenderLich Android Full Stack Web Development : Many MOOCS available for learning web development. Coursera : A free online library providing certificates on completion of courses. Very famous and very useful for people who want to start a career in Software Development. FreeCodeCamp Data Science / Machine Learning : One of the best free resources available to learn about Deep Learning. Fast AI : Platform for showing your DS and ML skills a.k.a LeetCode for Machine Learning. Kaggle Berkeley AI Course : Another great (and free!) resource for learning about AI and ML. NPTL AI Course Hope these resources help you in landing your dream job. All the very best! Previously published at https://adityarohilla.com/2020/01/08/top-resources-for-software-engineer-new-grad-job-search/

Top Resources for Software Engineers: How A New Grad Can Find A Job

Android Security [Disclaimer: Only for educational purposes] Hola, In this post we are going to talk about an interesting vulnerability that exists on present day android devices and can be exploited by hackers. The vulnerability is called Strandhogg and was released by in December first week. What all a hacker can do using this vulnerability? Promon Be ready to be amazed, here’s the list: They can listen to the user through the microphone Take photos through the camera Read and send SMS messages Make and/or record phone conversations Phish login credentials/ hijack social media accounts Get access to all private photos and files on the device Get location and GPS information Get access to the contacts list Access phone logs To top this information, this vulnerability exist on Android versions, including Android 10, the latest one. all So what is this vulnerability exactly and how is it being exploited? To answer this question, let’s dig into the Android system details a bit. Any typical Android app is made up of multiple . In layman terms, whenever you click on any app icon, the screen that appears on device is one activity usually. There can be more activities, depending on the functionality of app, but for simplicity sake, let’s say every new UI of the app is a new activity (it’s not always the case though, Android supports other things like Fragments etc). Every activity has set of defined attributes which one can find defined in an XML file called . Activities AndroidManifest Here’s what it looks like: You see, every <Activity> tag has multiple attributes such as name, label etc. The one which is of interest in this case is . Now how is that tag being used? Before understanding that, let’s go one level deeper and understand a bit about Android Tasks. taskAffinity A task is a collection of activities that users interact with when performing a certain job. The activities are arranged in a stack — the ) — in the order in which each activity is opened. For example, an email app might have one activity to show a list of new messages. When the user selects a message, a new activity opens to view that message. This new activity is added to the back stack. If the user presses the button, that new activity is finished and popped off the stack. back stack Back So, usually all the activities of one app are associated with one task. As they are opened, they are being pushed on the back-stack pertaining to this task. Whenever you launch an app by clicking on icon or notification or pressing back, this stack for the task is referred and accordingly the right view in brought in foreground by the Android system. By default, all activities in an application have the same affinity which implies they belong together and hence placed in the same stack. However, you can set the “taskAffinity” attribute to group them differently, and even place activities defined in different applications within the same task. Using taskAffinity one can change the app the activity has affinity for. So if for my calculator app, for one of the activities I set the taskAffinity to Facebook app rather than my app, this particular activity can possibly sit in the stack for Facebook app and not the calculator app. Do you see the problem here? If you see it, that’s the catch. So the hacker codes an app with different activities. Say one of them has taskAffinity for Facebook app. Now when this malicious app is launched, the activity with taskAffinity for Facebook app already starts a stack for the Facebook App, now if you click on Facebook app icon from the launcher, Android system checks this stack and shows the activity by malicious app instead of actual Facebook app. Combining taskAffinity with other flags like ends up giving even more control to the hacker in the flow. FLAG_ACTIVITY_CLEAR_TASK, FLAG_ACTIVITY_NEW_TASK, FLAG_ACTIVITY_MULTIPLE_TASK The exploits one can design on top of this behaviour is limited only by imagination. So does this actually work? Yes, I tried testing this out by creating an app which has affinity for another legit app and it works on most devices including Pixel. However, for system apps or particular Google apps, I found that the behaviour was not very reliable. How will it look like to you? Why do we have it still on Android? As Android OS is designed to support multi-tasking, this attribute comes in significance. It provides users with a seamless experience and interaction within the apps. It’s just a guess, but getting rid of this, might demand major redesigns on OS level. So, that’s all for this post. Hope the post helped bring clarity on the existing vulnerability on Android device. If you liked it please leave comments/claps. For more such interesting posts, hit that follow button! I’ll see you in next post soon, till then be safe and keep an eye out on odd behaviours on your phone. Stay aware! References: https://promon.co/security-news/strandhogg/ https://blog.mindorks.com/android-task-and-back-stack-review-5017f2c18196 https://developer.android.com/guide/topics/manifest/activity-element.html

Would You Notice If You Were Hacked?

TL;DR In January 2019 we released a tool that significantly raises the bar for detecting security vulnerabilities in Python code. We built a fully automated system that couples data flow analysis algorithms with a novel ML component and detects many more new security issues than traditional approaches. After running it on several Open Source repositories, we found and reported 35 critical security vulnerabilities from the list. Here’s a motivating screenshot that shows one of them: OWASP Top Ten One of the XSS vulnerabilities that we found and reported The interest towards Information Security and, in particular Web Applications security, is steadily growing over the past years. Unfortunately so does the threat level posed by security vulnerabilities that are silently awaiting to be exploited by attackers. This fact obviously puts an additional burden on developers’ shoulders, which a code review tool relieve. That is why we decided to make a big push to improve detection of security issues. This blog post covers how we did it. must When it comes to detecting all these scary vulnerabilities like Cross-Site Scripting (XSS), SQL Injection (SQLi), Path Traversal (PT), Remote Code Execution (RCE), etc. probably the most used code analysis technique is called . I guess its popularity comes from the simplicity of the core idea, which very simply put sounds like: “check that all user input is properly escaped before it reaches a critical (from the security point of view) piece of code ”. Taint Analysis So, if I ask you to exploit this super-complicated snippet of code and you think “hmm, user input from the HTTP request gets into the variable and then into the call, so if I set it to , then will get executed in victim’s browser”, then you already know how it works. If not, here’s an informal description of the technique in less than a hundred words. name django.http.HttpResponse <script>evil_code</script> evil_code Taint Analysis emphasizes three categories of objects (variables, function calls, etc.): — contain user input (e.g. ) Sources request.GET.get(“name”) — transform user input from unsafe to safe (e.g. ) Sanitizers django.utils.html.escape() — trigger security vulnerabilities if an attacker’s input reaches them (e.g. ) Sinks django.http.HttpResponse() The analysis validates that all data flows from a source to a sink pass through a sanitizer. If not, then each unsanitized flow is reported as a potential vulnerability. One of the main limitations of Taint Analysis is its dependence on the specification of sources, sinks and sanitizers. Such a specification has to be manually compiled by a “security expert”, who must be very familiar with both the project code base and all the used third party frameworks. Ideally, the expert would then write down all the sources, sinks and sanitizers present in the code and let the taint tool do its job. It is not hard to see that this approach has some obvious downsides. First, imagine how much work it is to manually assemble this security specification. One would have to go through tons of documentation and/or code of both the project and the frameworks. Now think of this famous quote by Larry Wall, which says that “laziness and impatience are two true virtues of a great programmer”, and feel some pity for the poor fellow assigned to the task. Second, each missed entity in the list is a potentially neglected vulnerability. Humans are notorious for making mistakes, especially when it comes to routine jobs like this. Can you trust the completeness of the obtained list? Third, this is not a one-time effort. This process has to be incrementally redone every time a framework gets updated or the project starts using a different one or its own code changes significantly. This considerably increases the cost of maintaining the project. Fourth, this approach does not scale. Each project and each framework are unique, which means that the security specifications can’t be 100% successfully reused. These four basically mean that the traditional approach will not work efficiently for a tool like our , that is run daily on thousands of different projects. We simply do not have an army of security consultants that would work on specifications for each project. Code Review We solved this problem the DeepCode way — we learned the answer from Open Source code. We built a system that analyzes publicly available code, collects stats about all encountered APIs, neatly combines them in a model and infers new sources, sinks and sanitizers in a fully automated way. It also predicts the vulnerability types for sinks and provides suggestions on which sanitizer to use to fix the issue. Clearly our new approach helps against limitations 1, 3 and 4 listed above. But what about the second (and main) one? To check the quality of our inferred specification, we exposed it to the ultimate challenge — a bug hunt. We ran the on multiple Open Source repositories hosted on GitHub. We specifically targeted web applications that are meant to be (or already are) used by multiple (honest and malicious) users. publicly available version of our analyzer This experiment resulted in the following list of vulnerabilities that were discovered and fixed. It also proved that our tool is capable of finding interesting, non-trivial and sometimes unexpected bugs. A more detailed description with cool examples deserves its own blog post, which will hopefully come out soon. So, for now let us just conclude with presenting the GitHub pull requests/issues we filed. (XSS = Cross-Site Scripting, SQLi = SQL Injection, PT = Path Traversal, RCE = Remote Code Execution) Multiple XSS in mozilla/pontoon Mutliple SQLi in earthgecko/skyline XSS in earthgecko/skyline XSS in DataViva/dataviva-site SQLi in DataViva/dataviva-site SQLi in lmfdb/LMFDB Multiple XSS in lmfdb/LMFDB Multiple XSS in gestorpsi/gestorpsi Multiple SQLi in sharadbhat/VideoHub RCE in UDST/urbansim Mutliple XSS in viaict/viaduct XSS in MLTSHP/mltshp Multiple XSS in kylewm/woodwind XSS in kylewm/silo.pub Multiple XSS in anyaudio/anyaudio-server SQLi in MinnPost/election-night-api PT in mitre/multiscanner SQLi in PadamSethia/shorty Curious to see how many vulnerabilities we can find in your project? Looking for CVEs in Open Source? Or simply not yet convinced that it works? Well, then see you at . P.S. https://www.deepcode.ai/

Meet the tool that automatically infers security vulnerabilities in Python code

Six months ago we boarded the plane on a one-way ticket from New York City to Tunis, to pursue a lifelong dream of connecting brilliant minds with opportunity. What started as an ambitious idea has now become a mission-driven venture and a growing movement of the most selective yet inclusive force of talented minds in North Africa and the Sahel. We’re transforming the region into a hub of mind power and technology excellence. Each of us still vividly remembers the day when we took the decision. Amal and Mehemed were sitting at the Watson Library at Columbia Business School in New York while Joscha joined on the screen sitting by his desk in Santiago de Chile. Once again, the Skype connection failed. After months of research, brainstorming and interviews, on that special day, we decided to abandon jobs and graduate school in order to pursue our passion by launching Think.iT. We knew we were in a unique position. We had formed exceptionally strong networks, built our knowledge in new technologies at leading companies such as Microsoft and Amazon, and acquired access to the world’s major tech ecosystems. For the past half a year, we leveraged those resources to shape the brightest software engineers into technology changemakers and integrate them as distributed team members in international engineering teams. In 2017, Think.iT has hence grown from a concept and idea into a real organisation and is operating on three different continents. Here are five highlights of our work as we look back on a very eventful 2017: Next month, our team will grow to 30 people, including 22 young engineers with good salaries who are collectively supporting more than 100 family members; We formed multiple partnerships with EU/US startups and SMEs, who highlighted their strong satisfaction with our engineers and asked for more Think.iTeers to join their teams; We were joined by international profiles and graduates from top schools such as Harvard, Columbia, Berkeley, ESADE Barcelona and Leiden University on almost no salary, simply as a result of their belief in our vision, people and management capability; Our founders were invited to share our approach at various conferences and panels and joined the ranks of global organizations supporting aspiring talent, such as and BMW Foundation’s Responsible Leaders Network; Atlas Corps We were awarded the , a $125k innovation grant by the German government (EXIST) and selected as a portfolio company by Columbia University’s ($25k award money); UN SDSN Special Youth Prize 2017 Tamer Center for Social Enterprise We do believe in a movement and revolution; a revolution through rethinking education and access to labor. We will continue to focus our mindpower, networks and resources around education and self-empowerment in order to build an entire generation of technology change makers in North Africa. We will provide youth with access to reclaim education and opportunity by exhibiting discipline, curiosity and real passion, all the ingredients needed for success and a good life. We signed up for something that is much bigger than ourselves, and we need more exceptional minds who share this belief. If you want to get involved, we invite you to reach out and collaborate. Let’s make 2018 count! Amel, Mehemed & Joscha Become an investor Become a hiring partner Become a team member Learn more

Transforming North Africa into a hub of mind power and tech excellence

Generating random numbers in is not simple. For starters, Solidity doesn’t come with a native random function because the deterministic nature of the system. Solidity In mathematics and physics, a is a in which no randomness is involved in the development of future states of the . A model will thus always produce the same output from a given starting condition or initial state. — deterministic system system system deterministic https://en.wikipedia.org/wiki/Deterministic_system There are a number of services and APIs that allow us to generate random numbers, but the problem is that they are not decentralized and you have to trust someone else that the generated number is truly random. Additionally, you can’t trust seemingly random values available to the contract, such as blockhash, timestamp, or other miner-defined value. Miners can manipulate these values to such extent, and event choose not to publish a block. For a casino app you can’t just trust miners, but there are a lot of other use cases where you could rely on for generating a random outcome, as long as the total amount of value resting on the quality of that randomness is lower than what a miner earns by mining a single block. blockhash More info on this: https://ethereum.stackexchange.com/questions/419/when-can-blockhash-be-safely-used-for-a-random-number-when-would-it-be-unsafe So, how can we generate a random number securely inside a Solidity smart contract? There are a few solutions out there that can help us achieve this, such as Randao (Randao: .) and Oraclize ( ). In this article I’m going to focus on Oraclize, mainly because it’s extremely easy to get started with it and secondly, because it also can be used to do many more things other than generating a random number, such as accessing any other API from inside a Solidity smart contract. https://github.com/randao/randao http://www.oraclize.it/#services Building the Raffle contract In this tutorial we’ll build a very simple raffle smart contract to show how to use Oraclize to generate a random number. The contract will work as follows:We’ll deploy the contract and set a minimum and maximum number of participants the raffle will accept. Once there are enough participants, the raffle organizer (or anyone for that matter) can execute the function that will have Oraclize generate a random number. With that random number we’ll just select the winner out of the participants’ array.*For this first version we won’t include all the payments logic, so anyone is free to participate but they won’t receive any ETH prize, of course. Project Setup For this tutorial, we’ll just deploy everything on Remix, using Metamask to connect to Ropsten, as Oraclize won’t work locally or on the JavaScript VM. I’m going to assume you have some basic understanding on how to work with Remix and Metamask. Also, make sure you have at least 3 or 4 accounts with some ether balance as we’ll need it to pay for the gas cost of joining the raffle as well as paying Oraclize to generate the random number (more on this later). Coding the contract In Remix, create a new file called Raffle.sol, this will be our contract containing all the logic for the raffle. pragma solidity ^0.4.4; contract Raffle { uint private chosenNumber;address private winnerParticipant;uint8 maxParticipants;uint8 minParticipants;uint8 joinedParticipants;address organizer; bool raffleFinished = false; address[] participants;mapping (address => bool) participantsMapping; event ChooseWinner(uint _chosenNumber,address winner);event RandomNumberGenerated(uint); function Raffle(){address _org = msg.sender;uint8 _min = 2;uint8 _max = 10; require(\_min < \_max && \_min >=2 && \_max <=50);

organizer = \_org;  
chosenNumber = 999;  
maxParticipants = \_max;  
minParticipants = \_min; } function() payable {} function joinraffle(){require(!raffleFinished);require(msg.sender != organizer);require(joinedParticipants + 1 < maxParticipants);require(!participantsMapping[msg.sender]); participants.push(msg.sender);  
participantsMapping\[msg.sender\] = true;

joinedParticipants ++; } function chooseWinner(uint _chosenNum) internal{chosenNumber = _chosenNum;winnerParticipant = participants[chosenNumber];ChooseWinner(chosenNumber,participants[chosenNumber]);} function generateRandomNum(){require(!raffleFinished);require(joinedParticipants >=minParticipants && joinedParticipants<=maxParticipants); raffleFinished=true;  
  
chooseWinner(0); //We'll replace this with a call to Oraclize service later on. } function getChosenNumber() constant returns (uint) {return chosenNumber;} function getWinnerAddress() constant returns (address) {return winnerParticipant;} function getParticipants() constant returns (address[]) {return participants;} } I’ll go over the most important parts of the code: The constructor of the contract sets the configuration state variables of the contract, such as the required minimum and maximum participants and who the organizer of the raffle is (for now, the creator of the contract). Notice we are hardcoding these variables, just to make the example simpler, but you could have the constructor receive these parameters. Also, we require there are at least 2 participants and no more than 50. This maximum could also be changed if you wanted. function Raffle(): I added the fallback function as we’ll need to send Ether to the contract so it can pay for Oraclize’s services. Did you think they would provide their services for free? :) — More on this later. fallback function: This function allows the caller to join the raffle. First, we make a few checks to make sure we are not adding a new participant to a raffle that has already finished or that has reached its maximum amount of participants. We also check that the organizer is not the one trying to join and that this person hasn’t already joined. Then we add the new participant to the participants array (so we can later select one of them at random) and we also add it to a participants mapping (which allows us to easy check if that participant had already joined). Having 2 variables that hold the same information feels a bit hacky to me, but given the example is rather simple, I felt we could get away with it. function joinRaffle(): This function will be called to have Oraclize generate the random number. Notice we made it public and doesn’t require the organizer to be the one that calls it. This is on purpose, so that if the organizer, for some reason, disappears, the participants can select a winner anyways. This could be changed as well, for example, to allow this mechanism but only after some time has passed. For now, all this function does is set the raffleFinished flag to true (so this function can only be called once) and call the chooseWinner with a 0 as parameter. When we implement Oraclize, instead of sending a hardcoded 0 (which automatically makes the first person who joined the raffle the winner) we’ll generate the random number. function generateRandomNumber(): This is an internal function that receives an uint which indicates the position of the winner in the participants array. With this number, we also set the address of the winner and fire an event that logs who the winner was so we can later retrieve it and display it. function chooseWinner(): The remaining functions are just used later in the front-end to display the winners. Go ahead and try the contract in . Just copy and paste the code above and deploy the contract. You should be able to join the raffle with different accounts and then, after there are enough participant, you can call generateRandomNumber() to select a winner (It will always be the address at position 0 in the participants array). Remix Now that we have made sure the logic of the raffle is ok, it’s time to use Oraclize to generate a random number for us. Using Oraclize to generate a random number Integrating Oraclize in our smart contract is pretty easy and straight-forward. The fist step is to import the Oraclize API in our contract. Add the following line before the Raffle contract declaration. import "github.com/oraclize/ethereum-api/oraclizeAPI.sol"; contract Raffle is usingOraclize{...} Notice that we also changed Raffle to inherit from usingOraclize. Now that we imported Oraclize, we have access to its functions. One thing you should be aware of is that importing Oraclize this way will fail, for example, if you are using Truffle to compile your contracts. If you are using Truffle, you will have to make a local copy of the Oraclize file and rename it to “usingOraclize.sol” as Truffle needs the file have the same name as the contract. The first thing we will do is add the Oraclize callback function. The way Oraclize works is by first making a query — we could be querying an external API, Wolfram’s Mathematica, or, like in our case, the built-in random number generator — which will fire a transaction to an Oraclize contract, and after that gets processed, it fires the callback function we define inside our contract with the corresponding result. This is how the callback function will look like: // the callback function is called by Oraclize when the result is ready// the oraclize_randomDS_proofVerify modifier prevents an invalid proof to execute this function code:// the proof validity is fully verified on-chain function __callback(bytes32 _queryId, string _result, bytes _proof){// If we already generated a random number, we can't generate a new one.require(!raffleFinished);// if we reach this point successfully, it means that the attached authenticity proof has passed!require (msg.sender == oraclize_cbAddress()); if (oraclize_randomDS_proofVerify__returnCode(_queryId, _result, _proof) != 0) {// the proof verification has failed, do we need to take any action here? (depends on the use case)} else {// the proof verification has passedraffleFinished=true; // for simplicity of use, let's also convert the random bytes to uint if we need  
uint maxRange = joinedParticipants; // this is the highest uint we want to get. It should never be greater than 2^(8\*N), where N is the number of random bytes we had asked the datasource to return  
uint randomNumber = uint(sha3(\_result)) % maxRange; // this is an efficient way to get the uint out in the \[0, maxRange\] range  
  
chooseWinner(randomNumber);

RandomNumberGenerated(randomNumber); // this is the resulting random number (uint) }} First, we will make sure that raffleFinished is false so we don’t allow this function to be called more than once. Then we are checking that the data we got has not been tampered with while being delivered to the smart contract. You can read more about that here: https://blog.oraclize.it/the-random-datasource-chapter-2-779946e54f49 If the verification has passed, meaning the data has not been tampered with, then we proceed to use the random number. We will be calling the chooseWinner() function that we previously used, but instead of passing a hardcoded 0 as parameter, we’ll pass the random number we generated.As you can see, the random number that gets generated uses the joinedPaticipants state variable to determine its max range. We are also firing an event to log the random number we generated. The last thing we have to do is modify our generateRandomNum() function so when someone executes it, it makes the query to Oraclize. function generateRandomNum(){require(!raffleFinished);require(joinedParticipants >=minParticipants && joinedParticipants<=maxParticipants); oraclize\_setProof(proofType\_Ledger); // sets the Ledger authenticity proof  
uint N = 4; // number of random bytes we want the datasource to return  
uint delay = 0; // number of seconds to wait before the execution takes place  
uint callbackGas = 200000; // amount of gas we want Oraclize to set for the callback function  
bytes32 queryId = oraclize\_newRandomDSQuery(delay, N, callbackGas); // this function internally generates the correct oraclize\_query and returns its queryId } By calling oraclize_newRandomDSQuery(delay, N, callbackGas) we’ll have Oraclize generate a random number for us, and when it’s ready, the callback function we defined earlier will be fired. oraclize_newRandomDSQuery receives 3 parameters. delay: which is the number of seconds to wait before the execution takes place. We set it to 0 so it executes the callback as soon as we have results. N: which is how many bytes we want the datasource to return. In our case, since we made the raffle to accept up to 50 participants if the organizer wants so, 4 bytes is more than enough. callbackGas: How much gas we will forward Oraclize so it can execute the callback function. We are setting it up to 200.000 which is enough to execute this function. (In my tests, it has consumed around 125.000 gas). And that’s it! You can give it a try in Remix and see how it works. A few considerations and words of advice: As I mentioned before, Oraclize isn’t free. You can check their pricing here: . Notice that the first query your contract makes is free, then you will have to pay for the transaction. How? The contract executing the query pays for it from it’s balance. So, before you call the generateRandomNum() function, make sure you send some ether to the contract. — That’s why we included the fallback function. Oraclize’s pricing model: http://docs.oraclize.it/#pricing Testing your contract can be quite tedious. First, since we are using Oraclize, you can’t test this locally, we need to connect to a testnet (I’m using Ropsten). You should remember that when you call generateRandomNum() it will take 30–50 seconds to get mined and then, it will take another minute to actually process the callback function. Also, switching accounts in Metamask + Remix takes some work. Arm yourself with patience. Testing in Remix: I’m still playing around with the necessary gas required to execute this contract. On Remix, I had to up the limit to 5.000.000 when deploying the contract. Make sure you are also sending enough gas when generating the random number, or else the callback function will fail. How much gas is needed to execute this? The contract we built today allows us to run a simple Raffle that people can join and then a winner is selected by using Oraclize to generate a random number we can be certain hasn’t been tampered with. There’s a lot of room for improvements and new features, such as allowing people to enter the Raffle with ether and win a prize. Let me know in the comments section below if you have any suggestions or if you encounter any problems with the code.

Building a Raffle Smart Contract Using Oraclize

All hail tea pot This post serves as my first introduction into the “blogosphere” as well, hopefully, the first of many write-ups about computing and technology in general. So with that out of the way lets get on with it. Today I will be discussing the Scan Line Polygon Fill (SLPF) algorithm, and then showing my implementation of the algorithm in C++. The purpose of the SLPF algorithm is to fill (color) the interior pixels of a polygon given only the vertices of the figure. Basic Idea: The basic idea is to collect all of the edges (except horizontal edges) that compose the polygon, fill in the figure scan line by scan line using the edges as starting and stopping points. Main Components: To successfully fill in a polygon three main components will be used: , an and an . These components will contain an edge’s information, hold all of the edges that compose the figure and maintain the current edges being used to fill in the polygon, respectively. Edge Buckets Edge Table Active List Edge Buckets The edge bucket is a structure that holds information about the polygon’s edges. The edge bucket looks different pending on the implementation of the algorithm, in my case it looks like this: Edge Bucket representation Here is a breakdown of what each member represent in an edge bucket: Maximum Y position of the edge yMax: Minimum Y position of the edge yMin: The current x position along the scan line, initially starting at the same point as the of the edge x: yMin The sign of the edge’s slope ( either -1 or 1) sign: The absolute delta x (difference) between the edge’s vertex points dX: The absolute delta y (difference) between the edge’s vertex points dY: Initiated to zero. Used as the scan lines are being filled to x to the next position sum: Edge Table (ET) The ET is a list that contains all of the edges that make up the figure. Important ET notes: When creating edges, the vertices of the edge need to be ordered from left to right The edges are maintained in increasing yMin order The edges are removed from the ET once the Active List is done processing them The algorithm is done filling the polygon once all of the edges are removed from the ET Active List (AL) The AL contains the edges that are being processed/used to fill the polygon. Every edge in the AL has a pairing buddy edge, because when filling a scan line, pixels are filled starting from one edge until the buddy edge is encountered. Important AL notes: Edges are pushed into the AL from the Edge Table once an edge’s yMin is equal to the current scan line being processed Edges will always be in the AL in pairs Edges in the AL are maintained in increasing x order. The AL will be re-sorted after every pass Steps: Now that we have the main components covered, lets go over the SLPF algorithm in more detailed steps. ~General assumptions~ The user has access to a method that can set the color of individual pixels. You will see that I simply call the setPixel() method whenever I fill in a pixel. The vertices of the given shape are listed in order around the circumference of the polygon This one is important, otherwise the polygons will not be drawn properly Here are the steps: 1. Create ET1. Process the vertices list in pairs, start with [numOfVertices-1] and [0].2. For each vertex pair, create an edge bucket2. Sort ET by yMin3. Process the ET1. Start on the scan line equal to theyMin of the first edge in the ET2. While the ET contains edges1. Check if any edges in the AL need to be removes (when yMax == current scan line)1. If an edge is removed from the AL, remove the associated the Edge Bucket from the Edge Table.2. If any edges have a yMin == current scan line, add them to the AL3. Sort the edges in AL by X4. Fill in the scan line between pairs of edges in AL5. Increment current scan line6. Increment all the X's in the AL edges based on their slope1. If the edge's slope is vertical, the bucket's x member is incremented. NOT (Why doesn’t Medium support nested lists??) Final Important Notes: When creating the edge buckets if the vertices of the edge are not in left to right order you might get the incorrect sign for the bucket. If the edge buckets have the wrong sign this will happen when you try to fill them: Something is not right here… To maintain only integers in the Edge Bucket, the slope is not actually calculated but is instead maintained as three distinct members: sign, dX, dY. This allows the Edge Buckets’ x member to be incremented discreetly without worrying about what happens when encountering a real number in a point If the slope is not interpreted correctly, this may happen: No x increment == pixelated image? Implementation: Now that the details are covered, here is the implementation of the SLPF in C++. Due to a request from my professor the C++ implementation of the code has been taken down and replaced with pseudo code (There is a strong concern that students will copy the code for projects). If you are interested in my C++ implementation feel free to message me for it. Main Program (Pseudo code) /*Creates edge buckets from the given edges [@param](http://twitter.com/param "Twitter profile for @param") n    Number of vertices  
[@param](http://twitter.com/param "Twitter profile for @param") x\[\]  array of x points  
[@param](http://twitter.com/param "Twitter profile for @param") y\[\]  array of y points  

[@return](http://twitter.com/return "Twitter profile for @return")     List of edge buckets */createEdges(n, x[], y[]) {instantiate a new edge table loop through x[] & y[] pairs {if the edge's slope is NOT undefined (verticle) {create bucket with edgeadd bucket to edge table}}} /*Given the edge table of the polygon, fill the polygons [@param](http://twitter.com/param "Twitter profile for @param") edgeTable The polygon's edge table representation */processEdgeTable (edgeTable) {while (edge table is NOT empty) { // Remove edges from the active list if y == ymaxif (active list is NOT empty) {for (iterate through all buckets in the active list) {if (current bucket's ymax == current scanline) {remove bucket from active listremove bucket from edge table}}} // Add edge from edge table to active list if y == yminfor (iterate through the bucket in the edge table) {if (bucket's ymin == scanline) {add bucket to active list}} // Sort active list by x position and slopesortTheActiveList(); // Fill the polygon pixelfor (iterate through the active list) {for (from vertex1.x to vertex2.x of the bucket) {setPixelColor()}} // Increment X variables of buckets based on the slopefor (all buckets in the active list) {if (bucketsdX != 0) {bucket's sum += bucket's dX while (bucket's sum >= bucket's dY) {  
        increment or decrement bucket's X depending on sign of   bucket's slope  
        edge's sum -= dY  
            }  
        }  
    }  
} } ///// Draw a filled polygon in the Canvas C.//// The polygon has n distinct vertices.  The coordinates of the vertices// making up the polygon are stored in the x and y arrays.  The ith// vertex will have coordinate (x[i],y[i]).//// n - number of vertices// x - x coordinates// y - y coordinates///drawPolygon(n, x[], y[]) {// Create edge tablefinalEdgeTable = createEdges() @param @param @param // Sort edges by minYsort(finalEdgeTable) processEdgeTable(finalEdgeTable)} Header file with structs /*Bucket struct to hold edge information*/struct Bucket {int yMax;int yMin;int x;int sign;int dX;int dY;double sum;}; /*Vertex struct to hold x and y values*/struct Vertex {int x;int y;}; That all it takes to implement the SLPF algorithm! I hope this post is constructive and that you enjoyed reading through it. If there are any mistakes and you want to leave comment, feel free to do so or send me a message. Cheers!

Computer Graphics: Scan Line Polygon Fill Algorithm

I have always wondered what set of code to run in the background to make my app powerful and responsive, but I don’t really know how. Some time ago, I got to know about isolates and tried implementing them. And I should tell you, it was painful. But I recently discovered how easy it has become. So here it is. You might have heard about isolates but never really understood them. Or maybe you might have implemented isolates, but the code was always messy and tedious to write. In any case, this blog will guide you through the ups and downs of the Isolate history and the current and better implementation that you should follow. You might want to use the latest method, or you might want to use the old method after all, it’s all up to you. The basics This is how the defines Isolates Flutter Documentation An isolated Dart execution context. Did you understand anything? At least I did not. So let’s begin understanding Isolates and then we’ll write our own definition. So what do we need for Isolates Recipe? What are Isolates? Why do we need them? What is Event Handling? How to implement Isolates? And finally, what are Isolate Groups? We’ll start with the bigger picture of Isolates and see what it really means and go deep down and piece all the parts together to see how each part of it works together so that we can understand what Isolates really do and why we need them. What are Isolates? To really understand isolates, first we need to go further back and make sure we know the answer to these two questions: What is the difference between Processor Cores and Threads? Core is a physical hardware component whereas thread is the virtual component that manages the tasks of the core. Cores enable completion of more work at a time, while threads enhance computational speed and throughput. Cores use content switching but threads use multiple processors for executing different processes. What is the difference between concurrent and parallel processing? Concurrency is when two or more tasks can start, run, and complete in overlapping time periods. It doesn’t necessarily mean they’ll ever both be running at the same instant. For example, multitasking on a single-core machine. Parallelism is when tasks literally run at the same time, e.g., on a multicore processor. Let’s get back to Isolates. Dart uses the Isolate model for concurrency. Isolate is nothing but a wrapper around the thread. But threads, by definition, can share memory which might be easy for the developer but makes code prone to race conditions and locks. Isolates, on the other hand, cannot share memory and instead rely on message-passing mechanisms to talk with each other. If anything is difficult to comprehend, keep reading. I am sure, you’ll get it. Using isolates, Dart code can perform multiple independent tasks at once, using additional cores if they’re available. Each Isolate has its own memory and a single thread running an event loop. We’ll get to the event loop in a minute. Why do we need Isolates? Before we get into more detail, we first need to understand how async-await really works. void main() async {
  // Read some data.
  final fileData = await _readFileAsync();
  final jsonData = jsonDecode(fileData);

  // Use that data.
  print('Number of JSON keys: ${jsonData.length}');
}

Future<String> _readFileAsync() async {
  final file = File(filename);
  final contents = await file.readAsString();
  return contents.trim();
} We want to read some data from a file, decode that JSON, and print the JSON Keys length. We don’t need to go into the implementation details here but can take the help of the image below to understand how it works. When we click on this button , it sends a request to _readFileAsync, all of which is dart code that we wrote. But this function _readFileAsync, executes code using Dart Virtual Machine/OS to perform the I/O operation which in itself is a different thread, the I/O thread. This means, the code in the main function runs inside the main isolate. When the code reaches the _readFileAsync, it transfers the code execution to I/O thread and the Main Isolate waits until the code is completely executed or an error occurs. This is what await keyword does. Place Bid Now, once the contents of the files are read, the control returns back to the main isolate and we start parsing the String data as JSON and print the number of keys. This is pretty straight forward. But let’s suppose, the JSON parsing was a very big operation, considering a very huge JSON and we start manipulating the data to conform to our needs. Then this work is happening on the Main Isolate. At this point of time, the UI could hang, making our users fustrated. What is Event Handling? As we discussed, Isolate is a wrapper around thread and each Isolate has an event loop executing events. These events are nothing but what happens when we use the application. These events are added in a queue which then the Event loop takes in and processes. These events are processed in the first-in-first-out fashion. The image below is just an example. Let’s use this code again for understanding event handlers. We already know what is happening in this block. void main() async {
  // Read some data.
  final fileData = await _readFileAsync();
  final jsonData = jsonDecode(fileData);

  // Use that data.
  print('Number of JSON keys: ${jsonData.length}');
}

Future<String> _readFileAsync() async {
  final file = File(filename);
  final contents = await file.readAsString();
  return contents.trim();
} Our apps start and it draws the UI (Paint Event) is pushed on to the queue. We click on the button and and file handling code starts. So the Tap Event is pushed in the queue. After it is complete, let’s suppose the UI is updated, so again the Paint Event is pushed in the queue. Place Bid Now because the our logic for handling the file and JSON was very small, the UI doesn’t stutter or jank. But let’s, for a while, imagine again that out code for handling file was huge and it takes a lot of time. Now the event queue and the event loop looks similar to this image below. Now that the main isolate takes a lot of time to actually process that event, our animation or UI might hang and irritate your users, causing huge dropoffs. This is where spawning a new isolate or a worker Isolate comes in. How to implement Isolates? All our dart code in the flutter app runs in isolate. Whether it is a main isolate or a worker isolate is up to you. The main isolate is created for you and you don’t have to do anything else here. The main function starts on the Main Isolate. Once we have our main function running, we can start spawning new isolates. So there’s 2 ways of implementing Isolates, the short and new method or the long and old method. We can use either, depending on the use case. Let’s start with the already existing method. As we had already discussed, Isolates, unlike threads, don’t share memory. This is done so as to prevent race conditions and locks. But the communication between Isolates is done using message passing. These messages are primitives and you can check out the whole list of objects that can be passed between isolates . here To pass messages, Dart provides us with Ports. SendPort and ReceivePort. Since we’re discussing the old method for spawning Isolates, we need to know that isolate methods need to be top-level or static functions. Here is the to the code if you want to follow along. link Future<String> startDownloadUsingOldIsolateMethod() async {
  const String imageDownloadLink = 'this is a link';
  // create the port to receive data from
  final resultPort = ReceivePort();
  // spawn a new isolate and pass down a function that will be used in a new isolate
  // and pass down the result port that will send back the result.
  // you can send any number of arguments.
  await Isolate.spawn(
    _readAndParseJson,
    [resultPort.sendPort, imageDownloadLink],
  );
  return await (resultPort.first) as String;
} What does this code do: Here we create an instance of RecievePort to receive data. Remember this is the old method for spawning Isolates. It can be a little long but it is necessary to know the details. We create a Worker Isolate on the Main Isolate using Isolate.spawn and pass down a top-level function that runs the blocking code. We also pass down a list of arguments, the first one, SendPort which will be used to send the data from the worker Isolate, and the second is the download link. We wait till the new Isolate has been spawned. We then wait for the result, which is some form of a String and use it however we want. This data can be anything from list of object. this ResultPort.first uses a stream subscription behind the screen and waits for the data from the worker isolate to be pushed onto it. As soon as the first item arrives, we return the result. This is the _readAndParseJson function that receives the argument and runs the worker isolate code. This is a dummy function that does nothing but delays the control for 2 seconds and then exits. The exit function terminates the current isolate synchronously. Certain checks are performed before sending the data back to the calling isolate and the data is sent back using the SendPort. // we create a top-level function that specifically uses the args
// which contain the send port. This send port will actually be used to
// communicate the result back to the main isolate

// This function should have been isolate-agnostic
Future<void> _readAndParseJson(List<dynamic> args) async {
  SendPort resultPort = args[0];
  String fileLink = args[1];

  String newImageData = fileLink;

  await Future.delayed(const Duration(seconds: 2));

  Isolate.exit(resultPort, newImageData);
} Although this functions correctly but we have not handled any errors which can be thrown from the worker isolate or any error which can occur while spawning a new isolate. // Error Handling
Future<String> startDownloadUsingOldIsolateMethodWithErrorHandling() async {
  const String imageDownloadLink = 'this is a link';
  // create the port to receive data from
  final resultPort = ReceivePort();
  // Adding errorsAreFatal makes sure that the main isolates receives a message
  // that something has gone wrong
  try {
    await Isolate.spawn(
      _readAndParseJson,
      [resultPort.sendPort, imageDownloadLink],
      errorsAreFatal: true,
      onExit: resultPort.sendPort,
      onError: resultPort.sendPort,
    );
  } on Object {
    // check if sending the entrypoint to the new isolate failed.
    // If it did, the result port won’t get any message, and needs to be closed
    resultPort.close();
  }

  final response = await resultPort.first;

  if (response == null) {
    // this means the isolate exited without sending any results
    // TODO throw error
    return 'No message';
  } else if (response is List) {
    // if the response is a list, this means an uncaught error occurred
    final errorAsString = response[0];
    final stackTraceAsString = response[1];
    // TODO throw error
    return 'Uncaught Error';
  } else {
    return response as String;
  }
} Everything is pretty same here, we just have added error handling here. What this code does is: We add errorsAreFatal to true while spawning a new isolate to make sure that the Main Isolate is aware of any errors. We assign the SendPort for onExit and onError handlers to make sure that if any errors that occur during exiting or spawning. We also add a try-catch block while spawning a new isolate to make sure that if any error occurs during spawning, we catch that and stop this operation, altogether. If the spawning is successful and some data do come from the worker isolate, we need to check if it’s error or not. If the message sent back is null, this means the isolate exited without any message, and an error has occurred. If the response is a list, this means the worker Isolate has sent back an error and a stacktrace. Else this is a successful transaction. This does seem like an overkill if we wanted to do just one-off message passing. One message and close the Isolate. Everytime you wanted to spawn a new isolate you’ll have to write the same code again. Since the Isolate logic is pretty custom. Every time you might want to pass in some different arguments and it would be very tedious. This is why a new method was devised for one-off transactions. The new method: Isolate.run // Isolates with run function
Future<String> startDownloadUsingRunMethod() async {
  final imageData = await Isolate.run(_readAndParseJsonWithoutIsolateLogic);
  return imageData;
}

Future<String> _readAndParseJsonWithoutIsolateLogic() async {
  await Future.delayed(const Duration(seconds: 2));
  return 'this is downloaded data';
} This is all there is for the new method. We spawn a new Isolate using the run method which abstracts out all the granular details and the error handling and saving you a lot of time. This helps in spawning, error handling, message passing and terminating the Isolate all using these few little lines of code. One thing to note here is that the function _readAndParseJsonWithoutIsolateLogic, does not contain any custom logic for the Isolate. No ports, no arguments. When to use the new Run method and when to use the old spawn method? These examples above shows message passing that happens only 1 time. So run method should be used. It greatly reduces the code lines and test cases. But if you want to create something that needs multiple messages to be passed between the Isolates, we need to use the old method. An example of this could be when you start downloading a file on a worker isolate and want to show the progress of the download on the UI. This means the progress count needs to be passed again and again. Isolate.spawn() With this, we need to implement the whole SendPort and ReceivePort for message passing and the custom logic for receiving the arguments and sending the progress back to the main Isolate. What are Isolate Groups? So, we already know how Isolates passes messages to each other. But let’s assume, the message we are passing is a huge JSON. Before Dart 2.15, this huge object passing, could involve stutter in UI. This is because, we already know that Isolate has some memory, and when one Isolate passes an object to the other, that object had to be deep copied. This meant, a lot of time for copying the object to the main Isolate which can cause a jank. To avoid this circumstance, Isolates were reworked and Isolate Groups were invented. Isolate Groups, meaning a group of isolates, which share some common internal data structures representing the running application. This means each time a new Isolate is spawned, new internal data structures don’t need to be constructed again. Because they share them together. Don’t confuse these internal data structures with the mutable objects. The Isolates still can’t share this memory with each other. Message passing is still needed. But, because Isolates in the same Isolate group, share the same heap, this means spawning a new Isolate is 100 times faster and consume 10–100 times less memory. An example is a worker isolate that makes a network call to get data, parses that data into a large JSON object graph, and then returns that JSON graph to the main isolate. Before Dart 2.15, that result needed to be deep-copied, which could itself cause UI jank if the copy took longer than the frame budget. This means that the main Isolate can receive this JSON in almost constant time. And sending messages is now approximately 8 times faster. The good news is that, if you’re using Flutter version greater than 2.8, you don’t need to do anything to use these advancements. Hope you liked the understanding of Isolates. If you have any doubts, please comment. Reference for this code: https://github.com/DhruvamSharma/NFT-Material3?embedable=true Also published . here

Stories, News & Mentions about MINDPORT

Mindport’s Stories on HackerNoon