This blog post is about how retired its last pieces of infrastructure on DigitalOcean, marking its migration to AWS as complete. ChartMogul The journey was not your regular AWS migration as it involved moving our infrastructure from classic VMs to containers orchestrated by Kubernetes. In a series of articles, we will share our experiences about: Our journey to AWS EKS (Kubernetes managed service). Some of the most critical roadblocks we encountered. Our current stack and tooling. Our infrastructure plans going forward, hoping they can be helpful to the entire community. Life With DigitalOcean Since our inception in 2014 and up to mid-2021, our entire infrastructure has run on DigitalOcean droplets (self-managed cloud virtual machines). We needed a cloud provider to get us off the ground quickly, reliably, and cost-effectively. DigitalOcean made a lot of sense and was a great choice. We are where we are because of them. That choice gave us the freedom to focus on product building without worrying about scalability and infrastructure complexity – aspects that typically kick in at a later stage. Every aspect of our infrastructure was provisioned, configured, and managed in-house. We used configuration management and Infrastructure as Code tools (Saltstack and Terraform) to manage things. We kept growing over the years, and by 2019 we found ourselves looking at a fleet of somewhere around 50 machines in constant need of management, software updates, security patches, and so on. And with new projects in our pipeline, we expected our compute power needs to double by the end of 2020. Why Move and Why Now? As great of a choice DigitalOcean was, our organic growth was pushing the boundaries of our setup over the years. We faced challenges with multiple areas, some fixable and preventable, some not. Various Failures Ad-hoc unannounced maintenance windows that suddenly broke production. Hardware failures on several occasions, affecting our primary – replica database setup (e.g., droplets entering a ”live migration state to other hardware machines” without notice, meaning 1-2 hours of downtime for that droplet.) Unexplained networking issues with latency between our machines that DigitalOcean’s support team never cleared out (this was critical for our Postgres read-replicas lag, Redis instances, and HA in general). AMS2 Region Deprecation Our DigitalOcean region (AMS2) was announced as “soon to be retired”, meaning limited support. We could not secure additional resources on-demand, and executing simple tasks usually meant long planning and wasted resources. Simple things such as upgrading a Postgres version and provisioning a new machine to perform a task were becoming impossible to do. Limited Hardware Choices Being in the subscription analytics space means data-intensive operations, large volumes, and the ability to often scale accordingly. Modern machines with more extensive hardware resources were only available in other regions. Network performance degradation was a frequent occurrence, and we soon realized that migrating to a different region was our best bet. Lack of Modern Cloud Features and Managed Services The volume of operational work to maintain our infrastructure to keep up with the growth rate (and deal with tech debt simultaneously) increased. We had to take a hard look at our setup and understand whether moving into a different DigitalOcean region or a new cloud provider was the best choice. Should We Stay or Should We Go? We started looking into the benefits of staying with DigitalOcean and simply moving to a new region – a more leisurely, quicker, cheaper, less painful option. But at the same time, we treated this move as an opportunity to modernize parts of our stack in service of expected user growth and an increased rate of progress. By the end of our assessment, we realized that specific must-have requirements would be hard to achieve by staying and simply switching regions. The most important ones were: Flexibility in auto-scaling compute resources. Managed databases. Provisioning of resources based on temporary usage. Low(er) latency. Service interoperability. Container-based infrastructure with Kubernetes orchestration. This list of requirements along with the challenges listed in the previous section tipped the scale in favor of switching providers. Why AWS? Choosing a new cloud provider to power ChartMogul infrastructure was a long journey. We researched the market and discovered many tradeoffs and advantages a new provider could bring to the table. Our options were Amazon Web Services (AWS), Google Cloud (GCP), and Azure. Ultimately, we decided to go with AWS. We list some of the main reasons below. Team Expertise We were already using some AWS services in production (e.g., S3 for storing incremental Postgres backups). More importantly, a few of our engineers had prior professional experience using various AWS services extensively in production systems. Scalability We can ramp AWS instances up or down at the push of a button. We can instantly provision resources like RDS databases and compute resources temporarily. We can iterate through experiments and proof of concepts quickly. The flexibility and scalability of Kubernetes node pools backed up by EC2 auto-scaling are hard to beat. Data Security and Compliance Data security has always been top of mind. Over the years, AWS security capabilities have grown substantially. The number of new services AWS developed around data security covers most of our needs in the container/Kubernetes space. They play nicely with well-established services such as private VPC isolation, fine-grain control of policies, and IAM roles. Compliance-wise, we plan to become SOC II certified asap, and we found AWS to be an advantage that can help fast-track that journey. compliance programs Managed Services Postgres is at the heart of what we do at ChartMogul, and we’ve typically spent a lot of time actively managing our database fleet of machines to support our growth. High availability and reliability of databases were becoming growing concerns, so we decided to evaluate multiple offers from major cloud providers with managed PostgreSQL. AWS RDS was the clear winner. Managed Kubernetes was another major factor to consider, and this was head to head with Google Cloud (GCP). Google’s managed Kubernetes (GKE) felt better than what AWS had at the time, but comparing RDS to CloudSQL wasn’t close feature-wise. Nowadays it seems that AWS is catching up with EKS however; We benefit from great RDS features such as snapshots flexibility, backup durability (with SLA), read replicas for Postgres, painless upgrades, dedicated IOPS, Cloudwatch metrics, Performance Insights, and the list goes on. The Insane Number of AWS Services At the time of writing, AWS offers over 200 services. Most of them give you the ability to get instant access to managed services from so many areas like compute, databases, data analytics, data warehousing, serverless, and storage. Our engineering teams can now leverage top-notch integrations to solve core problems quickly and prioritize buy vs. build where it makes sense. Disaster Recovery AWS cloud is an essential part of our Disaster Recovery plan. That’s because instances are easy to spin up, we can promote RDS read-replicas to primary at the click of a button, snapshots are a breeze, we can host in multiple regions, and we have a top-notch integration with our IaC tool of choice. AWS Credits We secured $100k worth of credits through the AWS Startup program. We were able to plan, test, and complete our migration without considerable expenses. Migration to AWS Our migration from DigitalOcean to AWS was a ten-month-long journey. The entire effort was backed up by volunteers from all of our engineering teams and driven by a DevOps engineer, a backend engineer, and our head of engineering. Some things involved trial & error. We tried multiple ways of: Moving data from Postgres to RDS with near-zero downtime. Moving our app and services from VM-based architecture to containerized ones in Kubernetes. Fundamentally changing the way we deploy. A perfect plan was in place, and everything looked good to go on paper, but we learned the hard way that things will not always go to plan. At times, our near-zero downtime migration goal was at serious risk, and back to the drawing board we went. Perseverance, drive, and fantastic team effort helped us overcome the challenges we faced. Careful planning did wonders too; Given our capacity, we established early on that breaking down the actual migration into three stages (or days) would work best. Week Prior D-Day Start Postgres replication from DigitalOcean to RDS instances. Review our AWS future production infrastructure. Configuration of secrets (AWS Parameter Store). Ensure CI/CD pipelines are ready to deploy to our new Kubernetes clusters. The Day Before D-Day Prepare our AWS temp webhook recorder infrastructure (losing events during our migration was not an option). Move some data in advance (e.g., DigitalOcean Spaces to S3). Update all Parameter Store secrets to production values. Prepare DNS changes. Set all Kubernetes deployments to zero pods to prevent services from accessing production data during migration. D-day: Flicking the switch Redirect all webhooks to AWS temporary recorder. Stop all the services on DigitalOcean. Wait for Postgres replication to catch up on the latest updates. Compare DigitalOcean and RDS Postgres data (to ensure integrity and replication catch up). Drop the subscription from RDS to Postgres running in DigitalOcean. Create RDS read replicas. Update our Parameter Store secrets with new RDS endpoints and secrets. Deploy to Kubernetes and restart PgBouncer to load new configurations. Switch DNS records for to AWS. app.chartmogul.com At this point, we were running our production workload on the shiny new infrastructure! We finished the whole thing in 10 hours (we initially estimated 8 hours – not too bad). Challenges With AWS The biggest struggle was with the DMS service (AWS managed service to move databases into RDS). It was not as easy to use as advertised. In our case with Postgres, it was not helpful. Eventually, we developed a custom way of moving data into AWS. We also came to the hard realization that moving databases with zero downtime to AWS with webhook support is complicated. We developed a custom approach to support this setup. More on these custom approaches in future articles. Future Articles in the Series Look out for future articles documenting our migration journey from DigitalOcean to AWS. We will touch on topics such as: Why we chose Kubernetes to power ChartMogul. How we migrated PostgreSQL to RDS. How we migrated our Rails app to Kubernetes. How we set up an IPSEC tunnel to AWS VPC.

How to Move Infrastructure From Classic VMs to Containers Orchestrated By Kubernetes

Cómo mover la infraestructura de máquinas virtuales clásicas a contenedores orquestados por Kubernetes

Comment déplacer l'infrastructure des machines virtuelles classiques vers des conteneurs orchestrés par Kubernetes

कुबेरनेट्स द्वारा ऑर्केस्ट्रेटेड कंटेनरों में बुनियादी ढाँचे को क्लासिक वीएम से कैसे स्थानांतरित करें

クラシック VM から Kubernetes によってオーケストレーションされたコンテナにインフラストラクチャを移行する方法

Como mover a infraestrutura de VMs clássicas para contêineres orquestrados pelo Kubernetes

Cách di chuyển cơ sở hạ tầng từ máy ảo cổ điển sang vùng chứa do Kubernetes tổ chức

如何将基础架构从经典 VM 迁移到由 Kubernetes 编排的容器

DEVELOP AND RUN APPLICATIONS WITHOUT THINKING ABOUT SERVERS If you’re navigating the landscape of questions about the best way to build, deploy, and manage applications, you’ll be able to find some answers below. We’ll analyse the three most popular cloud technologies – containers, virtual machines, and serverless computing. When used the right way, cloud technologies can assist you in the running applications without thinking about servers. WHAT IS A VIRTUAL MACHINE (VM)? is a piece of software that mimics the functions of a computer. In a sense, a VM is a computer created by another computer. A virtual machine (VM) VM can execute applications and programs without the need to use physical hardware. It is isolated from the rest of the machine that hosts it and behaves as if it is the only operating system on it. Virtual machines have the following characteristics, which offer several benefits: Partitioning. Isolation. Encapsulation. Hardware independence. Disadvantages: Low performance. HOW DO VIRTUAL MACHINES OPERATE? (operating system), thus enabling a group of VMs sitting alongside each other, each one with its unique OS. Multiple operating systems run side-by-side with a piece of software called a hypervisor to manage them, while desktop computers typically employ one OS to run the other operating systems within its program windows. Every VM runs its own unique guest OS For instance, a UNIX VM can sit beside a Linux VM, and every VM has a virtual hardware stack of its own comprised of libraries, binaries, applications, storage, network, and a unique CPU. Having numerous VMs with operating systems of their own adds significant overhead with regards to storage footprints and RAM. . The assignment of resources is absolute. For example, if the virtual machine is assigned 1 GB of memory and uses 50% at a time, the remaining 50% will be useless, and it would be a waste of resources. There is no way to utilize that 50% in the case of virtualization. The same applies to each resource. The concept is to partially use the host’s or the primary resource to provide an isolated environment for the new system WHAT ARE CONTAINERS? Containers are a lightweight virtualization architecture that allows you to deploy individual applications inside portable, isolated environments. A single container might be used to run anything from a small microservice or software process to a larger application. Inside a container, there are all the necessary executables, binary code, libraries, and configuration files, in common ways so that they can be run anywhere, whether it be on the desktop or the cloud. Just as a computer can be virtualized by a VM, containers can virtualize whole operating systems so that numerous workloads can operate on one OS. Benefits of containers: Increased portability. Reduce complexity through abstractions. Less overhead. Better application development. Better security and governance. Container disadvantages: Higher cost. Not all applications benefit from containers. HOW DO CONTAINERS OPERATE? For all intent and purposes, containers look like a VM. They have private space for processing, can execute commands as root, have a private network interface and IP address, allow custom routes and IPtable rules, can mount file systems, etc. Containers offer hardware virtualization at an OS level (typically Linux) so each individual container contains only the application, its libraries, and dependencies.The absence of the guest OS is why containers are so lightweight and fast. There are key use cases where containers are especially relevant: The combination of microservices as architecture and containers as a platform helps to build a well-designed container deployment strategy, orchestrate container infrastructure, and find the optimal ways to make deployment faster, secure, seamless. Containers are an ideal underlying architecture for hybrid cloud and multi-cloud scenarios because they can run consistently across laptops, on-premises and cloud environments. By containerizing, applications can be easily migrated to the cloud. WHAT IS SERVERLESS? There is no server to maintain; the only work necessary to run the code is to upload it and trigger it when it should execute. Serverless computing eliminates infrastructure management tasks such as server or cluster provisioning, patching, operating system maintenance, and capacity provisioning. Serverless is a cloud computing execution architecture where the cloud provider manages the allocation and provisioning of servers. The cost model of serverless is execution-based: A company that gets backend services from a serverless vendor is charged based on usage, not a fixed amount of the number of servers. you’re charged for the number of executions rather than pre-purchased compute capacity. The downside is that serverless functions are accessed only as private APIs. This doesn’t have an impact on your pricing or process, but it means . you cannot directly access them through the usual IP Businesses can leverage cloud-based and external servers, which are maintained and run by companies such as Amazon AWS. Example of a serverless platform is: AWS Lambda Azure Functions Google Cloud Functions IBM OpenWhisk Benefits of Serverless Architecture: Lower costs. Simplified scalability. Agility. Simplified backend code. Drawbacks: Reduced liability. Reduced overall control. Cost is unpredictable because the number of executions is not predefined. SERVERLESS COMPUTING: HOW DOES IT WORK? One instance of a FaaS is Lambda, which allows you to call functions using applications that run on AWS services, such as EC2 or S3. After a function is called, Lambda deploys it to a container that runs until a function is fully executed. Lambda handles the provisioning, deployment, and management of the code. You simply need to upload it. SERVERLESS OR CONTAINERS? . The code still needs to be written and loaded to Lambda. A completely serverless project will be hard to handle. Although AWS Lambda is an invaluable resource, it’s not an overall replacement for container deployment and management using applications like Kubernetes and Docker. Serverless computing doesn’t eliminate the necessity for developers to handle containers directly There are some stringent limitations when it comes to serverless computing. When Lambda is used, for instance, you will be subjected to memory usage limits, pre-established size restrictions, and a designated time for a function to be run. In addition to the narrow list of programming languages that are natively supported, the constraints of running the system using serverless architecture are quite apparent. For serverless architecture to be run successfully, . functions need to be kept small to stop them from overloading or hogging a system An application can be built as complex and large as you desire, which will give you total control over the container system (as well as individual containers) through software dedicated to container management. Docker ecosystems don’t restrict you to a pre-established memory, size, or time for a function to be run. Applications that are Docker-based are optimal for complex and large-scale applications. Serverless architecture is better suited to small tasks that can be accessed through external services to run behind the scenes. Ultimately, containers and serverless architecture work optimally when they are leveraged with one another. If a program that is container-based needs to be run, certain functions can be outsourced to serverless platforms for the sake of freeing up the primary program’s resources. FINDING THE BALANCE IN THE ENTERPRISE Serverless computing, containers, virtual machines – each of them have their own respective strengths and differ in several important ways. Choosing the right cloud deployment model for your application is vital to your company’s success. Try to understand all the advantages and disadvantages of various options. Rather than thinking in terms of containers vs. serverless vs. virtual machines, think of them as of technologies that complement each other. If you have questions on how to optimize the cost and performance of your cloud infrastructure, feel free to get in touch with DevCom cloud architects . By Halyna Vilchynska, Lead Marketing Lead at DevCom .

Get Better VMs, Containers and Serverless Computing Results By Following These Steps

Ever wanted to browse the files in your Google Cloud Platform’s Virtual Machine through your local file explorer? Here’s one way to do that! Setting up SSH Authentication First of all, follow the steps in this article mentioned below to setup keys in your local machine & GCP VM: How to create and connect to Google Cloud Virtual Machine with SSH Ensure that you’re able to connect to your GCP VM via SSH as directed in that article before proceeding into this article. Note for users: Windows Export the private key from as an OpenSSH key by selecting “ ” and save it as “id_rsa” somewhere. PuTTYGen Conversion->Export OpenSSH Key Copy your public key from PuTTYGen and save it as “ id_rsa.pub” along with the above file. (This will be helpful if you want the public key again to use it in a new VM) Remember that you can use the same keys for any number of VMs instead of regenerating different keys. Mounting the GCP VM on Windows First, download and install the tools and . (Recommended to install the 32-bit versions (x86)) WinFSP SSHFS-Win Then copy paste the following batch script in a text editor and save it as ‘ ’ GCP_Mount.bat @ off

:: OpenSSH Private Key File KEY_PATH=C:\\Users\\GokulNC\\Desktop\\Keys\\GCP_VM\\id_rsa USERNAME=GokulNC_GCP REMOTE_DIR=/ /p IP=Enter the IP: /p DriveLetter=Enter the mount drive letter: CYGFUSE=WinFsp =C:\Program Files (x86)\SSHFS-Win\bin; sshfs.exe -ofstypename=SSHFS -o idmap=user -o umask= -o volname= -o Compression=no -o reconnect,ServerAliveInterval= -o IdentityFile= @ : :

:: Wait a few secs it to become stable
timeout > . . :\ ( Connected to @ successfully and mounted :\ . Minimize this window and continue browsing your remote host's . To terminate the connection, close this window.
) ( Connection FAILED! Close this window.
) . . > > Connection terminated. Close this window. echo set set set set set set set PATH %PATH% 000 %IP% 120 %KEY_PATH% %USERNAME% %IP% %REMOTE_DIR% %DriveLetter% for 3 NUL echo echo IF EXIST %DriveLetter% echo %USERNAME% %IP% %REMOTE_DIR% at %DriveLetter% echo echo FS echo ELSE echo echo echo pause NUL pause NUL echo Edit the ‘KEY_PATH’ variable in the script to include the path of your ‘id_rsa’, and set the ‘USERNAME’ variable to the username you created along with the keys. Double-click the script to run it and enter the details that it asks for. You should get outputs similar to below: That's it! Now you can browse your VM's files from Windows Explorer itself. Mounting the GCP VM on Linux It’s just a matter of seconds in Linux! sudo apt install sshfs
sudo mkdir /mnt/gcpVM To mount your VM to the above created mount-point, run the following one-line command with your username (ZZ) and VM’s external IP: sshfs ZZ@X.X.X.X:/ /mnt/gcpVM/ -o IdentityFile=~/.ssh/id_rsa -o reconnect,ServerAliveInterval=120 That’s it, the VM's file system should have been mounted at the given path! (I believe similar steps should be sufficient for Mac Users, let me know in comments if it doesn’t work) To unmount, use the following command: fusermount -u /mnt/gcpVM References Specifying key in SSHFS Win-SSHFS using private key SSHFS on Windows Generating SSH keys ( ) SSHFS that survives disconnects GCP Timeout — 10mins Unmounting SSHFS Linux Update: Alternate way to use SSH without needing to setup keys

A Step by Step Guide to Mounting GCP VM's File System on your Local Machine

What makes more sense for an enterprise? In my previous , I talked about how enterprises can achieve continuous delivery of applications using microservices and containers. Here, I delve deeper to compare containers and VMs from a microservices architecture viewpoint. blog In this software era of constant evolution, we hear a lot of talk about using containers for microservices and the need to modernize monolithic applications. But, there is always an impending question for an enterprise that arises next and is rarely addressed — Why not use VMs instead of containers? offer virtualization of hardware as well as the OS and create an efficient, isolated duplicate of a real machine. In the case of , only the OS is virtualized and not the hardware, creating a lightweight environment that houses the application and the dependent assets. Containers have continued to gain popularity since 2014 and Virtual machines have existed for a very long time, the between the two is known to all. As the world keeps comparing the two, it is imperative to remember both are here to serve different purposes. Virtual Machines containers difference I like to think virtual Machines are to containers just as trains are to aeroplanes. While aeroplanes offer much faster mobility, we are always going to have trains around for some journeys that aeroplanes are not equipped to handle. Likewise, in certain business situations, containerization may be the prefered choice and in some, virtualization. But when microservices come into the picture, the question — “containerize or virtualize?” seems to be frequent. Benefits of using containers over VMs for microservices Lowered Cost VMs make it easy to partition execution environments among microservices by virtualizing hardware as well as operating systems. When you are executing every microservice through a separate VM, that requires replication of the OS as well. This will increase licensing costs. It is possible to execute multiple microservices in a single VM, but that would negate the single biggest advantage of breaking down a monolithic application into small easily executable microservices. The issue of conflicting libraries and application components will remain unsolved. Containers offer isolation at the OS level. Thus, a single operating system can support multiple containers, each running within its own, separate execution environment. By running multiple components on a single operating system you reduce overhead licensing costs. Higher Efficiency It is common knowledge that VMs often impose a large by using up server processing cycles which can be used to run applications instead. Every virtual machine runs its own execution environment on a separate copy of the operating system exhausting server processing cycles that you otherwise could have used to run the applications. performance penalty On the other hand, containers perform environment isolation at the operating system level. This way, a single operating system can support multiple containers, each running within its own, separate isolated environment. By running multiple components on a single operating system, containers reduce the overhead server processing cycles thereby freeing up processing power for other application components. Flexible Storage VMs have multiple options for storage. They can either have a local or a network-based storage type. Whichever option you choose, there will always be a physical disk space allocated to every VM which is isolated from the VM itself. So, at all times, the operating system, program files and data of a VM occupy storage space on the dedicated storage disk — . stateful storage In contrast, Containers, offer the choice of being stateful or stateless. Storage space is created or occupied when the container is created and discarded along with its deletion. A sandbox layer is created when a container image is edited and that stores all the data. This layer is active only as part of the container. Thus, each service of your microservices app can have its own storage, that can be managed differently from the storage of other services providing more flexibility and control. Better isolation As mentioned above, containers offer independent execution environments to microservices while enabling cohabitation on a single operating system. The databases, as well as the environments of microservices, are completely independent, despite being deployed on a single OS. If one tries to run multiple microservices on a single VM, there can be overlapping environments which may end up clogging up the server. With containers, one can make the most out of a given piece of hardware by smart application processing. For example, some microservices require a lot of processing power, while others may generate a lot of network traffic. With smart workload allocation within containers, app developers can efficiently utilize server resources, ensuring network capacity is not left unused. Reduced Size VMs, as we know, take up a lot more storage space than containers. A container is typically as small as 10MB whereas a VM occupies at least a few GBs of storage space. While segregating a complex application using the microservices architecture, where each functionality is a separate microservice, the isolation will require a large number of VMs or containers. VMs are bound to require a lot of space while containers will not. The same physical server can hold many more containers than VMs. So, given the need to ensure the enterprise app does not become too bulky, containerization is often the preferred choice. Faster execution VMs are created by a hypervisor that needs a lot of configurational decisions at the start — the guest OS for running the application, amount of storage space needed, network preferences, and many such settings. Though VMWare offers preset defaults in its creation wizard of VMs, the process still remains pretty complex and takes a couple of minutes to be executed on a platform. On the other hand, containers are created faster than VMs due to the absence of a hypervisor. Container images are stored in a repository from where they can be pulled as required through a few quick commands. Thus, the startup time for Docker containers ranges from a few milliseconds to a couple of seconds which makes it much swifter than a VM. Executing containerized microservices at scale can be further simplified by using like Kubernetes and Docker Swarm. Docker containers can be quickly launched, retired, and replaced at scale with such orchestrators. To further enhance and containers, one can use app delivery platforms designed exclusively to overcome modern delivery challenges for an enterprise. An app delivery platform can the complete process of containerization and subsequent delivery to orchestrators. container orchestrators working with microservices help accelerate, standardize, and sustain In reality, every enterprise comes with its own needs, requirements, and legacy systems. In the choice between VMs and containers, the latter is better suited for packaging microservices. So, if one is looking at a scalable & flexible modern architecture for their teams, containers are their best bet to ensure continuous delivery.

VMs vs. Containers for Microservices

How to Build and Deploy a Decentralized Crowdfunding Smart Contract on the Linea Blockchain

British Virgin Islands, November 7th 2024 — Cytonic, the world’s first multi-virtual-machine blockchain, today announced it has raised $8.3 million in a seed funding round co-led by Lattice and Lemniscap, with participation from IOBC, Nomura, Lyrik, Public Works, Arthur Hayes and other notable investors. Developed by MultiVM Labs, Cytonic is a decentralized execution layer that tackles one of the blockchain industry’s most pressing challenges: compatibility. As fragmentation across multiple chains continues to grow, developers and users face increasing barriers that limit the full potential of blockchain technology. Cytonic introduces the first Layer 1 blockchain capable of running multiple virtual machines within the same network layer while using shared storage. This breakthrough allows transactions involving different state transition functions (STFs) to be processed together within a single state transition. Currently, Cytonic is designed to be identical to both Ethereum and Solana, enabling all existing tooling, wallets, and decentralized applications (dApps) that operate on EVM and SVM chains to function seamlessly on Cytonic. Cytonic minimizes transaction fees and reduces waiting times for moving assets between chains, making decentralized applications more accessible and user-friendly. On the Cytonic blockchain, users can swap assets across different VMs atomically without rollback issues or cross-chain vulnerabilities. For example, users can swap Base tokens (ERC20) for Solana tokens (SPL) within the same liquidity pool on a single decentralized exchange (DEX), radically increasing capital efficiency. This approach fundamentally addresses interoperability challenges and vulnerabilities inherited from hundreds of different blockchain architectures, all while maintaining full compatibility with them. Having experienced firsthand how compatibility issues hinder growth across the crypto industry, Cytonic Founder Ivan Miskovic recognized the critical need for a solution that simplifies horizontal scaling for applications and addresses liquidity management challenges for users. Ivan, former Executive at Spin Labs, along with his co-executives—Chelsea, former Partner at Foresight Ventures, and Badconfig, former Senior Engineer at Faraway—have years of experience building and investing in decentralized technologies across Bitcoin, Ethereum, Solana, Near, and other notable projects. United by a shared vision to resolve blockchain’s compatibility challenges, the Cytonic team is dedicated to making on-chain interactions easier and more efficient for all participants. Thanks to its EVM and SVM compatibility, projects from any EVM or SVM chain can seamlessly migrate to Cytonic and tap into liquidity from different networks without the usual complexities of cross-chain operations or chain abstraction. This breakthrough in core infrastructure compatibility enables developers to deploy existing applications on Cytonic without rewriting any code, streamlining both development and growth processes. Projects can harness the unique strengths of different virtual machines while maintaining universal compatibility on the network. This not only simplifies the user experience but also reduces fragmentation, enhances ecosystem diversity, and makes blockchain technology more accessible and practical for widespread adoption. Ivan Miskovic, Founder of Cytonic, said: “Sincere thanks to our investors for backing our vision to bridge the gaps between blockchains and unlock new opportunities for developers and users alike. We are enabling any wallet from any chain to store any type of asset on Cytonic, greatly simplifying user experiences and broadening accessibility at a critical juncture in the Web3 development cycle. We are just getting started.” Regan Bozman, GP at Lattice, said: “We are proud to have co-led this $8.3 million round for Cytonic. As the blockchain ecosystem continues to grow, many innovative projects struggle with liquidity and compatibility due to the diverse range of blockchains available. Cytonic is tackling these compatibility issues head-on, breaking down the silos between blockchains while promoting greater collaboration, innovation, and adoption across the Web3 space.” About Cytonic Cytonic is pioneering the first multi-virtual-machine blockchain, with a mission to seamlessly integrate diverse networks like Bitcoin, Ethereum, and Solana into a unified Layer 1 blockchain solution. Operating on a global scale, Cytonic ensures connectivity and interoperability across multiple decentralized networks. Website X [Discord](https://: <https://discord.gg/K3RgfYS5](https://discord.gg/K3RgfYS5)) Media contact: Lorcan@forewordventure.com This story was distributed as a release by Btcwire under HackerNoon’s Business Blogging Program. Learn more about the program here British Virgin Islands, November 7th 2024 — Cytonic, the world’s first multi-virtual-machine blockchain, today announced it has raised $8.3 million in a seed funding round co-led by Lattice and Lemniscap, with participation from IOBC, Nomura, Lyrik, Public Works, Arthur Hayes and other notable investors. British Virgin Islands, November 7th 2024 Developed by MultiVM Labs , Cytonic is a decentralized execution layer that tackles one of the blockchain industry’s most pressing challenges: compatibility. As fragmentation across multiple chains continues to grow, developers and users face increasing barriers that limit the full potential of blockchain technology. MultiVM Labs MultiVM Labs Cytonic introduces the first Layer 1 blockchain capable of running multiple virtual machines within the same network layer while using shared storage. This breakthrough allows transactions involving different state transition functions (STFs) to be processed together within a single state transition. Currently, Cytonic is designed to be identical to both Ethereum and Solana, enabling all existing tooling, wallets, and decentralized applications (dApps) that operate on EVM and SVM chains to function seamlessly on Cytonic. Cytonic minimizes transaction fees and reduces waiting times for moving assets between chains, making decentralized applications more accessible and user-friendly. On the Cytonic blockchain, users can swap assets across different VMs atomically without rollback issues or cross-chain vulnerabilities. For example, users can swap Base tokens (ERC20) for Solana tokens (SPL) within the same liquidity pool on a single decentralized exchange (DEX), radically increasing capital efficiency. This approach fundamentally addresses interoperability challenges and vulnerabilities inherited from hundreds of different blockchain architectures, all while maintaining full compatibility with them. Having experienced firsthand how compatibility issues hinder growth across the crypto industry, Cytonic Founder Ivan Miskovic recognized the critical need for a solution that simplifies horizontal scaling for applications and addresses liquidity management challenges for users. Ivan Miskovic Ivan Miskovic Ivan, former Executive at Spin Labs, along with his co-executives— Chelsea , former Partner at Foresight Ventures, and Badconfig , former Senior Engineer at Faraway—have years of experience building and investing in decentralized technologies across Bitcoin, Ethereum, Solana, Near, and other notable projects. United by a shared vision to resolve blockchain’s compatibility challenges, the Cytonic team is dedicated to making on-chain interactions easier and more efficient for all participants. Chelsea Chelsea Badconfig Badconfig Thanks to its EVM and SVM compatibility, projects from any EVM or SVM chain can seamlessly migrate to Cytonic and tap into liquidity from different networks without the usual complexities of cross-chain operations or chain abstraction. This breakthrough in core infrastructure compatibility enables developers to deploy existing applications on Cytonic without rewriting any code, streamlining both development and growth processes. Projects can harness the unique strengths of different virtual machines while maintaining universal compatibility on the network. This not only simplifies the user experience but also reduces fragmentation, enhances ecosystem diversity, and makes blockchain technology more accessible and practical for widespread adoption. Ivan Miskovic, Founder of Cytonic, said: “Sincere thanks to our investors for backing our vision to bridge the gaps between blockchains and unlock new opportunities for developers and users alike. We are enabling any wallet from any chain to store any type of asset on Cytonic, greatly simplifying user experiences and broadening accessibility at a critical juncture in the Web3 development cycle. We are just getting started.” Ivan Miskovic, Founder of Cytonic, said: “Sincere thanks to our investors for backing our vision to bridge the gaps between blockchains and unlock new opportunities for developers and users alike. We are enabling any wallet from any chain to store any type of asset on Cytonic, greatly simplifying user experiences and broadening accessibility at a critical juncture in the Web3 development cycle. We are just getting started.” Regan Bozman, GP at Lattice, said: “We are proud to have co-led this $8.3 million round for Cytonic. As the blockchain ecosystem continues to grow, many innovative projects struggle with liquidity and compatibility due to the diverse range of blockchains available. Cytonic is tackling these compatibility issues head-on, breaking down the silos between blockchains while promoting greater collaboration, innovation, and adoption across the Web3 space.” Regan Bozman, GP at Lattice, said: “We are proud to have co-led this $8.3 million round for Cytonic. As the blockchain ecosystem continues to grow, many innovative projects struggle with liquidity and compatibility due to the diverse range of blockchains available. Cytonic is tackling these compatibility issues head-on, breaking down the silos between blockchains while promoting greater collaboration, innovation, and adoption across the Web3 space.” About Cytonic Cytonic is pioneering the first multi-virtual-machine blockchain, with a mission to seamlessly integrate diverse networks like Bitcoin, Ethereum, and Solana into a unified Layer 1 blockchain solution. Operating on a global scale, Cytonic ensures connectivity and interoperability across multiple decentralized networks. Website Website X X [Discord](https://: <https://discord.gg/K3RgfYS5](https://discord.gg/K3RgfYS5)) Media contact: Lorcan@forewordventure.com This story was distributed as a release by Btcwire under HackerNoon’s Business Blogging Program. Learn more about the program here This story was distributed as a release by Btcwire under HackerNoon’s Business Blogging Program. Learn more about the program here here here

Cytonic Secures $8.3 Million Seed Funding To Solve Blockchain Compatibility

Cytonic verseker $8,3 miljoen saadbefondsing om Blockchain-versoenbaarheid op te los

የብሎክቼይን ተኳሃኝነትን ለመፍታት ሳይቶኒክ የ8.3 ሚሊዮን ዶላር ዘር ድጋፍን ያረጋግጣል

Cytonic, Blockchain uyğunluğunun həlli üçün 8,3 milyon dollarlıq toxum maliyyəsini təmin edir

Cytonic забяспечвае пачатковае фінансаванне ў памеры 8,3 мільёна долараў для вырашэння сумяшчальнасці блокчейна

Cytonic осигурява първоначално финансиране от $8,3 милиона за разрешаване на съвместимостта на блокчейн

Cytonic assegura un finançament inicial de 8,3 milions de dòlars per resoldre la compatibilitat de la cadena de blocs

Cytonic obtiene 8,3 millones de dólares de financiación inicial para resolver la compatibilidad de blockchain

Cytonic מאבטחת מימון סיד של 8.3 מיליון דולר כדי לפתור תאימות לבלוקצ'יין

Cytonic osigurava početno financiranje od 8,3 milijuna dolara za rješavanje kompatibilnosti blokova

Cytonic Sekirize $8.3 Milyon Finansman Semans Pou Rezoud Konpatibilite Blockchain

Cytonic、ブロックチェーンの互換性問題を解決するために830万ドルのシード資金を確保

Cytonic verzekert zich van $8,3 miljoen startkapitaal om blockchain-compatibiliteit op te lossen

Introduction Building generative AI requires working with a variety of developing machine learning and AI technologies. It also requires buckets of patience. These two requirements are not unrelated. Through working alongside extremely talented machine learning experts from different firms building generative AI applications, it’s clear that there are a number of ‘yikes’ aspects to developing fully realized generative AI tools. In this article, I’ll briefly summarise five pitfalls to avoid when developing a generative AI application. Don’t Assume the Outputs of LLMs Will Be Accurate This may seem obvious, but I was surprised at how unreliable the output of commercial LLMs could be. When testing major LLMs (in particular, ChatGPT, Anthropic’s Claude AI, and Amazon Bedrock) on complex financial datasets, I noted that they can generate errors or hallucinations at a rate of one per page. LLMs are guilty of often generating information that sounds plausible but is false. For example, when testing financial data, I found that LLMs produced financial ratios like ‘Non-operating income and expense’. These sound legitimate but don’t exist in the document and are completely hollow financial terms. Though effective prompt engineering can mitigate some of these hallucinations, many of them occur for initially unexplainable reasons (hence why every LLM has a recommendation for manually reviewing its output somewhere in its user interface and documentation). OCR (Optical Character Recognition)—the technology that converts images to machine-readable text—also generates a high rate of errors. It is challenging to develop a mechanism to combat OCR infidelity for each model. AI-based adaptive learning for user-flagged errors can be a helpful strategy but requires huge amounts of manual intervention. Overall, the lack of a calibrated confidence score in commercial LLMs is a huge limitation. Application developers must develop their own methods for calculating confidence and validating errors. 2. Don’t Be Complacent With Your Production Credits Any generative AI dependent on LLM production credits (such as Google’s Tensor Processing Units (TPUs)) needed to be technologically regulated to avoid spending more than accounted for - potentially bankrupting the company or developer. Bankruptcy through overenthusiastic spending of production credits might seem like an urban myth - certainly, there are no noted cases of production credits running amok - but better safe than sorry, right? There are several ways to stretch out your production credits to their fullest potential: Take advantage of short-lived, discounted, preemptible VMs from your cloud application provider (but only if your application is designed to handle interruptions and then resume from checkpoints). Leverage spot instances for non-critical workloads to significantly reduce costs while automatically shutting down idle instances. Cloud provider tools can be very helpful for this. Where appropriate, you could experiment with open-source frameworks that don’t require production credits. I’m very pro the democratic approach of open source models and their potential for collaborative innovation. And, of course, open-source models can be accessed for free. 3. Don’t Forget to Test User Experience (UE) Again, it may seem obvious, but when stuck looking at a profusion of code, it’s easy to forget that a real human being will be using the product. In a production environment, consider the pathways the user will take, even in complex and code-heavy environments. Don’t be like Google Bard, whose generative AI model couldn’t answer simple user questions about space upon release or, later, what to put on pizza (hint: it’s not glue). Many traditional UX testing tools can be adapted for generative AI products - such as a card sorting tool. Another effective way to test the technology is with a human-in-the-loop system, like beta testers or reviewers. At the moment, the company I’m with is using human beta testers to test their new generative AI technology. The testers use the tool for free - and they can harvest the usage patterns. 4. Don’t Forget the Importance of Training Data Clean, normalize, and potentially enrich your data to improve the training process. Techniques like tokenization and feature engineering might be helpful. Generally, the more data, the better (the AI algorithms I used were from a document store of 25 million documents, for example), but too much data may result in overfitting or computational bottlenecks. Slightly off-topic, but in the future, it’s possible that wrangling with training data may not be an issue. Promising advancements like AutoML automatically cleanse training data and assess the model’s performance using a number of techniques like meta-learning, Bayesian inference, and neural architecture search. For SMEs with limited coding resources, AutoML might be a promising innovation. 5. Don’t Choose the Wrong Scale Modeling Strategy There are many ways to scale a model. For example, you might split the model across multiple machines (model parallelism) or replicate the model across multiple machines (data parallelism). Complex models will likely benefit from the model’s parallelism’s device distribution; larger datasets or models with smaller architectures may benefit from increased throughput. Another consideration is whether to upgrade compute resources on a single machine, such as by increasing GPU memory (vertical scaling) or adding more resources (e.g. GPUs) - also known as horizontal scaling. Don’t forget to containerize your generative AI application to ensure consistent behaviors across different environments. Of course, rigorous testing and validation after model scaling is a must. To ensure it can handle the increased load, consider trying a variety of by-the-book testing plus load and stress testing. Summary It’s unrealistic to expect building a generative AI product to work without encountering at least one of these challenges. Ultimately, each of these challenges presents a fork in decision-making. At each juncture, the choice you make gets you one step closer to building a mature, production-ready model. Working with the latest advances in academic and industrial machine learning will help counter some of the typical frustrations, as machine learning’s competitive landscape is constantly pushing out new innovations. I organize the London Machine Learning Meetup (the largest community of AI experts in Europe), which is a free community that hosts events that unpack the latest technical advances in machine learning. Above all, accuracy and cost-effectiveness don’t have to be mutually exclusive. With controlled and strategic experimentation, building a generative AI product can be more rewarding and less frustrating than you might think. Good luck! Introduction Introduction Building generative AI requires working with a variety of developing machine learning and AI technologies. It also requires buckets of patience. These two requirements are not unrelated. Through working alongside extremely talented machine learning experts from different firms building generative AI applications, it’s clear that there are a number of ‘yikes’ aspects to developing fully realized generative AI tools. In this article, I’ll briefly summarise five pitfalls to avoid when developing a generative AI application. Don’t Assume the Outputs of LLMs Will Be Accurate This may seem obvious, but I was surprised at how unreliable the output of commercial LLMs could be. When testing major LLMs (in particular, ChatGPT, Anthropic’s Claude AI, and Amazon Bedrock) on complex financial datasets, I noted that they can generate errors or hallucinations at a rate of one per page. LLMs are guilty of often generating information that sounds plausible but is false. sounds For example, when testing financial data , I found that LLMs produced financial ratios like ‘Non-operating income and expense’. These sound legitimate but don’t exist in the document and are completely hollow financial terms. testing financial data testing financial data Though effective prompt engineering can mitigate some of these hallucinations, many of them occur for initially unexplainable reasons (hence why every LLM has a recommendation for manually reviewing its output somewhere in its user interface and documentation). effective prompt engineering effective prompt engineering OCR (Optical Character Recognition)—the technology that converts images to machine-readable text—also generates a high rate of errors. It is challenging to develop a mechanism to combat OCR infidelity for each model. AI-based adaptive learning for user-flagged errors can be a helpful strategy but requires huge amounts of manual intervention. OCR OCR Overall, the lack of a calibrated confidence score in commercial LLMs is a huge limitation. Application developers must develop their own methods for calculating confidence and validating errors. 2. Don’t Be Complacent With Your Production Credits 2. Don’t Be Complacent With Your Production Credits Any generative AI dependent on LLM production credits (such as Google’s Tensor Processing Units (TPUs)) needed to be technologically regulated to avoid spending more than accounted for - potentially bankrupting the company or developer. Bankruptcy through overenthusiastic spending of production credits might seem like an urban myth - certainly, there are no noted cases of production credits running amok - but better safe than sorry, right? There are several ways to stretch out your production credits to their fullest potential: Take advantage of short-lived, discounted, preemptible VMs from your cloud application provider (but only if your application is designed to handle interruptions and then resume from checkpoints). Take advantage of short-lived, discounted, preemptible VMs from your cloud application provider (but only if your application is designed to handle interruptions and then resume from checkpoints). Leverage spot instances for non-critical workloads to significantly reduce costs while automatically shutting down idle instances. Cloud provider tools can be very helpful for this. Leverage spot instances for non-critical workloads to significantly reduce costs while automatically shutting down idle instances. Cloud provider tools can be very helpful for this. Leverage spot instances Where appropriate, you could experiment with open-source frameworks that don’t require production credits. I’m very pro the democratic approach of open source models and their potential for collaborative innovation. And, of course, open-source models can be accessed for free. Where appropriate, you could experiment with open-source frameworks that don’t require production credits. I’m very pro the democratic approach of open source models and their potential for collaborative innovation. And, of course, open-source models can be accessed for free. open-source frameworks 3. Don’t Forget to Test User Experience (UE) Again, it may seem obvious, but when stuck looking at a profusion of code, it’s easy to forget that a real human being will be using the product. In a production environment, consider the pathways the user will take, even in complex and code-heavy environments. Don’t be like Google Bard, whose generative AI model couldn’t answer simple user questions about space upon release or, later, what to put on pizza (hint: it’s not glue ). simple user questions about space simple user questions about space it’s not glue it’s not glue Many traditional UX testing tools can be adapted for generative AI products - such as a card sorting tool. Another effective way to test the technology is with a human-in-the-loop system, like beta testers or reviewers. At the moment, the company I’m with is using human beta testers to test their new generative AI technology. The testers use the tool for free - and they can harvest the usage patterns. card sorting tool. card sorting tool. 4. Don’t Forget the Importance of Training Data Clean, normalize, and potentially enrich your data to improve the training process. Techniques like tokenization and feature engineering might be helpful. tokenization tokenization Generally, the more data, the better (the AI algorithms I used were from a document store of 25 million documents, for example), but too much data may result in overfitting or computational bottlenecks. Slightly off-topic, but in the future, it’s possible that wrangling with training data may not be an issue. Promising advancements like AutoML automatically cleanse training data and assess the model’s performance using a number of techniques like meta-learning, Bayesian inference, and neural architecture search. For SMEs with limited coding resources, AutoML might be a promising innovation. AutoML AutoML 5. Don’t Choose the Wrong Scale Modeling Strategy There are many ways to scale a model. For example, you might split the model across multiple machines (model parallelism) or replicate the model across multiple machines (data parallelism). Complex models will likely benefit from the model’s parallelism’s device distribution; larger datasets or models with smaller architectures may benefit from increased throughput. Another consideration is whether to upgrade compute resources on a single machine, such as by increasing GPU memory ( vertical scaling ) or adding more resources (e.g. GPUs) - also known as horizontal scaling. Don’t forget to containerize your generative AI application to ensure consistent behaviors across different environments. vertical scaling vertical scaling Of course, rigorous testing and validation after model scaling is a must. To ensure it can handle the increased load, consider trying a variety of by-the-book testing plus load and stress testing. Summary It’s unrealistic to expect building a generative AI product to work without encountering at least one of these challenges. Ultimately, each of these challenges presents a fork in decision-making. At each juncture, the choice you make gets you one step closer to building a mature, production-ready model. Working with the latest advances in academic and industrial machine learning will help counter some of the typical frustrations, as machine learning’s competitive landscape is constantly pushing out new innovations. I organize the London Machine Learning Meetup (the largest community of AI experts in Europe), which is a free community that hosts events that unpack the latest technical advances in machine learning. London Machine Learning Meetup London Machine Learning Meetup Above all, accuracy and cost-effectiveness don’t have to be mutually exclusive. With controlled and strategic experimentation, building a generative AI product can be more rewarding and less frustrating than you might think. Good luck!

Building Production-Ready Generative AI: Five Things Not to Do

Everything You Need to Know About Zero-Knowledge Proof (ZKP) Technology

5 LLM-based Apps for Developers

One of my favorite interview questions is “What do such words as async and await tell you?” because it opens up an opportunity to have an interesting discussion with an interviewee… Or it doesn’t because they float on this topic. In my opinion, it is drastically important to understand why we use this technique. I feel like many developers prefer to rely on the “it is the best practice” statement and use asynchronous methods blindly. This article shows the difference between asynchronous and synchronous methods in practice. Tools .NET Web API application (test target) 2 Azure SQL Databases 2 Azure App Service on Windows (hosts the application) Azure App Insights (to gather metrics) locust framework (to simulate user load). Configuration I will run a benchmark in the following way. Two independent locust instances are running on two machines. Locust instances simulate a user that does the following: The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. Under the hood, each App Service connects to its own database and executes a SELECT query that takes five seconds and returns a few rows of data. See the controller’s code below for references. I’ll use Dapper to make a call to the database. I’d like to draw your attention to the fact that the asynchronous endpoint calls the database asynchronously as well (QueryAsync<T>). It's worth adding that I deploy the same code to both app services. During the test, the number of users grows evenly to the target number (Number of Users). The speed of growth is controlled by a Spawn Rate parameter (number of unique users to join per second) — the higher the number is, the quicker the users are being added. The spawn rate is set to 10 users/s for all experiments. All experiments are limited to 15 minutes. You may find machine configuration details in the Technical Details section of the article. Metrics requests per minute — shows the number of requests that the application actually processed and returned a status code.
thread count — shows the number of threads the app service consumes.
median response time, ms The red lines refer to asynchronous, and the blue lines — to the synchronous endpoint, respectively. That’s it about theory. Let’s start. Experiment #1 number of users: 75 (per service) We can see that both endpoints perform similarly — handling around 750 requests per minute with a median response time of 5200 ms. The most fascinating graph in this experiment is a thread trend. You can see significantly higher numbers for the synchronous endpoint (a blue graph) — more than 100 threads! That’s expected, though, and matches the theory — when a request comes in, and the application makes a call to the database, the thread is blocked because it has to wait for a roundtrip to complete. Therefore, when another request comes in, the application has to produce a new thread to handle it. The red graph — the asynchronous endpoint thread count— proves different behavior. When a request comes in and the application makes a call to the database, the thread returns to a thread pool instead of being blocked. Therefore, when another request comes in, this free thread is reused. Despite incoming requests growing, the application doesn’t require any new threads, so their count remains the same. It’s worth mentioning the 3rd metric — median response time. Both endpoints showed the same result — 5200 ms. So, there is no difference in terms of performance. Now, it is time to pull up the stakes. Experiment #2 number of users: 150 We doubled the load. The asynchronous endpoint handles this task successfully—its request per minute rate floats around 1500. The synchronous brother eventually reached a comparable number of 1410. But if you look at the graph below, you will see that it took 10 minutes! The reason is that the synchronous endpoint reacts to a new user's arrival by creating another thread, but users are being added to the system (just to remind you that the Spawn Rate is 10 users/s) quicker than the web server can adapt. That’s why it queued so many requests at the very beginning. Unsurprisingly, the thread count metric is still around 34 for the asynchronous endpoint, while it increased from 102 to 155 for the synchronous one. The median response time degraded similarly to the request per minute rate — the synchronous response time was much higher at the beginning of the experiment. If I had kept the test for 24 hours, the median numbers would become even. Experiment #3 number of users: 200 The third experiment is intended to prove trends revealed during the second one —we can see further degradation of the synchronous endpoint. Conclusion Using asynchronous instead of synchronous operations doesn’t directly improve performance or user experience. First, it enhances stability and predictability under pressure. In other words, it raises the load threshold so the system can process more before it degrades. Appendix #1. Technical Details Azure App Service: B1, 100 ACU, 1,75 Gb Memory, A-Series compute equivalent.
Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage.
SQL Connection settings: Max Pool Size=200. Appendix #2. Notes To achieve the cleanest test result, I should have run tests from 2 VMs located in the same network where the target App Services sit. However, I assumed that a network lag would impact both apps in a more or less similar way. Therefore, it cannot jeopardize the main goal — comparing how asynchronous and synchronous methods behave. Appendix #3. Bonus Experiment What did I hack to force the synchronous endpoint to perform almost as asynchronous and plot the graph below (experiment conditions are the same as in the third one — 200 users)? One of my favorite interview questions is “What do such words as async and await tell you?” because it opens up an opportunity to have an interesting discussion with an interviewee… Or it doesn’t because they float on this topic. In my opinion, it is drastically important to understand why we use this technique. async await I feel like many developers prefer to rely on the “it is the best practice” statement and use asynchronous methods blindly. This article shows the difference between asynchronous and synchronous methods in practice. Tools .NET Web API application (test target) .NET Web API application (test target) 2 Azure SQL Databases 2 Azure SQL Databases 2 Azure App Service on Windows (hosts the application) 2 Azure App Service on Windows (hosts the application) Azure App Insights (to gather metrics) Azure App Insights (to gather metrics) locust framework (to simulate user load). locust framework (to simulate user load). locust Configuration I will run a benchmark in the following way. Two independent locust instances are running on two machines. Locust instances simulate a user that does the following: The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. synchronous App Service 1, The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. asynchronous App Service 2. Under the hood, each App Service connects to its own database and executes a SELECT query that takes five seconds and returns a few rows of data. See the controller’s code below for references. I’ll use Dapper to make a call to the database. I’d like to draw your attention to the fact that the asynchronous endpoint calls the database asynchronously as well ( QueryAsync<T> ). QueryAsync<T> It's worth adding that I deploy the same code to both app services. During the test, the number of users grows evenly to the target number ( Number of Users ). The speed of growth is controlled by a Spawn Rate parameter (number of unique users to join per second) — the higher the number is, the quicker the users are being added. The spawn rate is set to 10 users/s for all experiments. Number of Users Spawn Rate spawn rate All experiments are limited to 15 minutes. You may find machine configuration details in the Technical Details section of the article. Metrics requests per minute — shows the number of requests that the application actually processed and returned a status code. thread count — shows the number of threads the app service consumes. median response time, ms requests per minute — shows the number of requests that the application actually processed and returned a status code. requests per minute thread count — shows the number of threads the app service consumes. thread count median response time, ms median response time, ms The red lines refer to asynchronous, and the blue lines — to the synchronous endpoint, respectively. That’s it about theory. Let’s start. Experiment #1 Experiment #1 number of users: 75 (per service) number of users : 75 (per service) number of users We can see that both endpoints perform similarly — handling around 750 requests per minute with a median response time of 5200 ms. The most fascinating graph in this experiment is a thread trend. You can see significantly higher numbers for the synchronous endpoint (a blue graph) — more than 100 threads! That’s expected, though, and matches the theory — when a request comes in, and the application makes a call to the database, the thread is blocked because it has to wait for a roundtrip to complete. Therefore, when another request comes in, the application has to produce a new thread to handle it. The red graph — the asynchronous endpoint thread count— proves different behavior. When a request comes in and the application makes a call to the database, the thread returns to a thread pool instead of being blocked. Therefore, when another request comes in, this free thread is reused. Despite incoming requests growing, the application doesn’t require any new threads, so their count remains the same. It’s worth mentioning the 3rd metric — median response time . Both endpoints showed the same result — 5200 ms. So, there is no difference in terms of performance. median response time Now, it is time to pull up the stakes. Experiment #2 number of users: 150 number of users : 150 number of users We doubled the load. The asynchronous endpoint handles this task successfully—its request per minute rate floats around 1500. The synchronous brother eventually reached a comparable number of 1410. But if you look at the graph below, you will see that it took 10 minutes! request per minute The reason is that the synchronous endpoint reacts to a new user's arrival by creating another thread, but users are being added to the system (just to remind you that the Spawn Rate is 10 users/s) quicker than the web server can adapt. That’s why it queued so many requests at the very beginning. Spawn Rate Unsurprisingly, the thread count metric is still around 34 for the asynchronous endpoint, while it increased from 102 to 155 for the synchronous one. The median response time degraded similarly to the request per minute rate — the synchronous response time was much higher at the beginning of the experiment. If I had kept the test for 24 hours, the median numbers would become even. thread count median response time request per minute Experiment #3 number of users: 200 number of users : 200 number of users The third experiment is intended to prove trends revealed during the second one —we can see further degradation of the synchronous endpoint. Conclusion Using asynchronous instead of synchronous operations doesn’t directly improve performance or user experience. First, it enhances stability and predictability under pressure. In other words, it raises the load threshold so the system can process more before it degrades. Appendix #1. Technical Details Azure App Service: B1, 100 ACU, 1,75 Gb Memory, A-Series compute equivalent. Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage. SQL Connection settings: Max Pool Size=200. Azure App Service: B1, 100 ACU , 1,75 Gb Memory, A-Series compute equivalent. ACU Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage. SQL Connection settings: Max Pool Size=200. Appendix #2. Notes To achieve the cleanest test result, I should have run tests from 2 VMs located in the same network where the target App Services sit. However, I assumed that a network lag would impact both apps in a more or less similar way. Therefore, it cannot jeopardize the main goal — comparing how asynchronous and synchronous methods behave. Appendix #3. Bonus Experiment Appendix #3. Bonus Experiment What did I hack to force the synchronous endpoint to perform almost as asynchronous and plot the graph below (experiment conditions are the same as in the third one — 200 users)?

Async vs Sync Benchmark (.NET): The Difference Between Asynchronous and Synchronous Methods

مقارنة بين Async وSync (.NET): الفرق بين الطرق المتزامنة وغير المتزامنة

Async vs Sync Benchmark (.NET): Asinxron və Sinxron Metodlar Arasındakı Fərq

Async vs Sync Benchmark (.NET): la diferencia entre métodos asincrónicos y sincrónicos

Async vs Sync Benchmark (.NET): a diferenza entre os métodos asíncronos e síncronos

Async vs Sync Benchmark (.NET): Az aszinkron és a szinkron módszerek közötti különbség

Async vs Sync Benchmark (.NET): la differenza tra metodi asincroni e sincroni

非同期 vs 同期ベンチマーク (.NET): 非同期メソッドと同期メソッドの違い

Async vs Sync Benchmark (.NET)៖ ភាពខុសគ្នារវាងវិធីសាស្ត្រ Asynchronous និង Synchronous Methods

Async vs Sync Benchmark (.NET): Асинхрондуу жана синхрондуу методдордун ортосундагы айырма

Async vs Sync Benchmark (.NET): różnica między metodami asynchronicznymi i synchronicznymi

Async vs Sync Benchmark (.NET): Rozdiel medzi asynchrónnymi a synchrónnymi metódami

Async vs Sync Benchmark (.NET): Musiyano uripakati peAsynchronous uye Synchronous Methods

Stories, News & Mentions about VMS

Vms’s Stories on HackerNoon