CloudBlue

cloudblue

The 40 Under 40 Fueling Spain's Startup Economy

40 অনূর্ধ্ব 40 স্পেনের স্টার্টআপ অর্থনীতিতে জ্বালানি দিচ্ছে

Die 40 unter 40 treiben Spaniens Startup-Wirtschaft voran

Los 40 menores de 40 que impulsan la economía de las startups en España

Les 40 de moins de 40 ans alimentent l'économie des startups espagnoles

40 अंडर 40 स्पेन की स्टार्टअप अर्थव्यवस्था को बढ़ावा दे रहा है

スペインのスタートアップ経済を支える40歳未満の40人

스페인의 스타트업 경제를 촉진하는 40세 미만 40세

Os 40 abaixo dos 40 impulsionam a economia das startups na Espanha

40 Yaş Altı 40, İspanya'nın Startup Ekonomisini Güçlendiriyor

Nhóm 40 Under 40 thúc đẩy nền kinh tế khởi nghiệp của Tây Ban Nha

40 位 40 岁以下的人士推动了西班牙的创业经济

Symfony is without doubts one of the most PHP frameworks. It has amazing flexibility and is written in an applaudable modular fashion. Some Symfony components even other prominent frameworks like Laravel. Running Symfony is a bit tricky at times, especially the of the cache and logs folder, this blog post demystifies running the Symfony in a docker container and hosting it serverless on . popular power permissions Demo App Google Cloud Run Prerequisites You have done at least one project with Symfony (of course you are comfortable with PHP and composer) You have a good working knowledge of git and Github Know a fair bit of , maybe even multi-stage build docker Have a working Google cloud account (they give you free for 1 yr, no reasons not to have an account) $300 credit This post is not going to be a detailed one like the last one about . If you are fairly new Git or Docker, please do read the above-mentioned post thoroughly. Laravel on Google Cloud Run Steps Before going to the steps, the repo is hosted on my and it is open-source of course. It might look insanely easy but we will go through some details after the steps. So let’s get started now: Github 1. Go to the repo Navigate to the open-source repo. That is a very easy step, isn’t it? :) Github Repo with Dockerized Symfony Demo App 2. Click on the Google Cloud button Then click on the Run on Google Cloud blue button, as shown below: Click on the button 3. Choose correct settings on Google cloud CLI After that, given you are logged into your Google account and have Google cloud setup with 1 or more projects, click Proceed. You might need to wait a bit, after that Choose the project — Choose a project to deploy this application Choose the region — Choose a region to deploy this application, I usually choose us-central-1 Then wait for the container to be built and deployed, you can see my process below: Deploying Symfony Demo app with Google Cloud shell script If everything goes fine on your Google Cloud Shell, you will see HTTPs URL you can go to in order to see your Symfony demo app running like below: Symfony Demo app running on Google Cloud Run One issue with this setup is that data will not be saved if you click around and add new posts. This happens because the data is saved in the SQLite file, as it is not persisted new containers get the old file and you get the same data pushed in the repo. So what just happened? There are some things you need to consider for this automagic deployment of the Symfony App. Let’s look at what just happened above: The Symfony Demo app has been dockerized, have a look at this and the too. The Dockerfile uses multistage build, on stage to install dependencies with composer and another one with Apache to run the application. Dockerfile Apache config If you want to run it locally, just run docker-compose up then go to http://localhost:8080. is the required port needed for Google Cloud Run. 8080 Thanks to the and the Cloud Shell script that does the automagic deployment. Google Cloud Run button To make the most out of this tutorial, you can fork the , add your own button then explore more with it. repo Conclusion Running Symfony entails some tricky configuration :) so the Dockerized version is a breeze to run. I really believe that Google Cloud Run is a superior product if you know how to dockerize your web app and run it on port 8080 you can get amazing scalability with unparallel ease to make it run. I would recommend Cloud Run for any pet project or even production application. Read more at https://geshan.com.np

How To Run Symfony On Google Cloud Run With The Demo App [Step-by-Step Guide]

There’s no doubt: DevOps and the sheer scale of the software it enables have truly revolutionized the world of software development. Evolving from humble single-server beginnings two decades ago, it’s finally reached the point where we can build elastic software at scale, thanks to cloud and orchestration layers such as IaaS, containers, Kubernetes, and serverless (FaaS). New developments create new challenges. As developers try to keep pace with the ever-increasing complexity, speed, and scale of their software, they are realizing that are the new bottleneck, still stuck working with the same development and debug tools from a simpler age. they A torrent of new challenges The most beloved characteristics of modern software, and those which make it so powerful — scale, speed, extreme modularity, and high distribution — are also the qualities that make developing and maintaining it so difficult. These new challenges can be roughly divided into three categories or levels: , , and . complexity connectivity development These categories are closely related; often challenges in one category are solved by translating the problem to the next level up. This ‘challenge flow’ isn’t a straightforward vertical waterfall but is rather best envisioned as a cascading spiral. Just as complexity and connectivity spur development, development challenges lead to greater degrees of complexity, and so forth. Complexity challenges Complexity challenges are a direct result of software solutions tackling more complex problems, making them the most obvious set of challenges. They include: Developing distributed modular systems (aka microservices), which requires engineers to take time to plan how to break down, deploy and connect software. Developing scalable software. Engineers must consider what scale the solution will need to support. (What ever happened to “this server supports up to X users?”) Handling data complexity in transit and at rest (usually coupled with “big data”). Modern software requires engineers to plan for extreme cases of throughput (high traffic, workloads), fast processing (real-time), storage, and search. What used to be a unique skill-set just a few years ago is now expected of every full-stack engineer (“You know Hadoop / Redis / Mongo / Kafka… right?”), raising the bar of most software projects. Saving your data as a .csv file or even a local SQL database is rarely an option anymore. The skillset developers require to address these challenges is constantly growing, along with the time and energy that they need just to get started — to understand the problem space and assess how the available tools can be optimally applied . Often when complexity issues become too great to manage, the problem escalates to become a next-level challenge, with complexity encapsulated in a new solution or method. Adding encapsulated software solutions such as databases, memcache, message-queues and ML engines in effect “delegates” the problem to be solved by the way we weave and orchestrate the overall solution. Once a specific pattern of complexity-to-connectivity escalation starts to be repeated frequently, it usually translates into a new standard development method or solution. connectivity Connectivity challenges Connectivity challenges result from the way modern software is woven together. There are at least three parallel connectivity chains: — connectivity within the software solution, such as the connections between microservices or modules Interconnectivity — connectivity with other software solutions including 3rd party servers, SDKs and SaaS External connectivity the configuration and orchestration layer used to build, deploy, and manipulate the software solution. Meta-connectivity — In even the simplest of layouts, each connectivity layer comprises dozens of elements, as well as multiple connections the layers. Just keeping track of all the connections and data flows is a huge — even sisyphean — management and architectural effort. between Now, consider that connections constantly change due to changing needs or as part of the architecture itself, as for load-balancing. Systems quickly reach a point where developers must invest many hours — even weeks in some cases — to understand what connects to what in the software solution they are working on. The days of are long gone. “That box is connected to that box, and that’s the wire connecting them” We’ve already listed plenty of ‘frosting’ on the connectivity ‘cake’, but let’s not forget the cherry on top: Engineers are expected not only to understand all aspects of connectivity but to design, build, monitor and maintain them so that the overall solution is secure and meets all required standards. This is mind-bogglingly difficult when even the smallest defect in the smallest element can bring the entire house down. It’s like saying to the engineers, ” security and compliance. “You know that huge haystack you’re trying to pile up? Make sure all those pieces of hay connect just right, and while you’re at it, check for needles, too. As time progresses (both within projects and in general) connectivity layouts grows beyond the ability of the human mind to keep track of, developers turn to the meta-connectivity layer, and try to automate and orchestrate connectivity (e.g. Puppet/Chef/Ansible, Kubernetes, Istio, Helm…). “You like code, right? So, here’s more code to orchestrate your code while you are writing code”. When this modus operandi, often described as ‘configuration as code’ approaches maturity it transfers to the next level of challenges — And elevates the resolution of each challenge to a software solution in its own right. development. Development challenges Development challenges are the classic pains involved in producing functional software within a framework. These emerge from the growing gap between the power of modern software and developers’ ability (or lack thereof) to keep pace with it. Moreover, since all the challenges from the previous levels gradually propagate to this one, the challenges here are at the bleeding-edge of the dev/devops experience. These include: Dev environment challenges Dev environment challenges require developers to assemble, learn, and manage all tools and workflows chosen for the software solution being developed: The sheer number of tools and methodologies is overwhelming and still rising, even after the huge growth of the past decade, to name just a few: IDEs, source-controls, compilers, transpilers, DBA tools, cloud consoles, development servers, debuggers, orchestrators, monitoring agents, task / ticket management, alerting solutions. Context switching makes matters worse. This ungodly pile of tools doesn’t hit developers just once, but continues to bombard them. Developers must always be ready to deal with a different situation. ( “It was raining Git commits yesterday but today I’ve been waiting for hours for it to start snowing containers… Oh wait, it’s actually going to be a typhoon of tickets with a chance of tracing.”) Today, for developers, “getting into the zone” is no longer a matter of efficiency; it’s essential for basic productivity. And that’s before we complicate the stack with AI, quantum-computing, bio-computing, and other miracles. (“If you think multithreaded programming is hard, I have some bad news for you…”) Replicating bugs Often the first step toward understanding a bug is observing it. The most basic approach to observation is replication of the bug in a controlled (usually local) environment. Sadly, with modern software, replication has become a herculean feat. The many challenges and factors mentioned above interact to create cases so complex and interconnected that any attempt to simulate them is doomed to fail, or break the budget trying. ( “We think the issue was a result of rain in Spain, and a butterfly beating its wings… Alright, so let’s start by building a life-size model of Spain, and I’ll check Amazon for bulk butterfly shipments.”) As in vitro bug replication becomes less of an option, in vivo observability becomes a must. Unfortunately, developers often discover that their ancient toolbox is ill-equipped for the task. The old tools simply don’t work in modern software environments; the reason usually comes down to loss of access. You can’t access the ephemeral: multiple layers of encapsulation, the result of translating complexity challenges into connectivity, have made access an issue, as layers like containers, Kubernetes, and serverless take ownership of the software environment. The issue is clearest for serverless / FaaS. Not only do developers lack access to the server, but as far as they are concerned there no server. Even if by some trick or hack one gains access to an underlying server through all the layers, it’s practically impossible to pinpoint a specific server/app since these fluctuate, controlled by orchestration layers, which constantly remove and add servers on the fly. is Bringing an SSH knife to a K8S gunfight: with access lost, existing tools have become obsolete. SSH, which provided an all powerful direct command-line console, has become unusable in most cases and is at best, a huge stability risk. Traditional debuggers (in contrast to Rapid Production Debuggers) have nothing to attach More significantly, the idea of breaking (setting a breakpoint) on a server in production is unacceptably risky, since it is likely to literally BREAK production or at best, knock microservices and functions out of sync, removing any chance of replicating or debugging substantial issues in a distributed system. to. IDEs used to represent a developer’s full view of the system from code through compiling to building and running. They enabled developers to sync their views and approaches to software they share. IDEs were never very close to live systems, but the gap between them has grown so great that they can no longer create shared views. While integration to modern source control (e.g. Git) and CI/CD solutions helps, the gap is still large. The CI/CD bottleneck With access lost and few alternatives, if any, developers have turned to deploying code as the main way to achieve access. Any need to interact with software translates to writing more code, merging it, testing it, approving and of course, finally deploying it. This approach is leveraged for adding/removing features, getting observability (monitoring and investigating), and applying fixes and patches. This overwhelming dependency on code deployment creates a bottleneck as all these deployments fight for space in the pipeline and the attention of those who maintain it. As the flow of challenges continues spiraling onwards, it gathers momentum. Today’s torrent is but a taste of the challenges the future holds. Once these challenges of modern software development are brought into focus, a clear picture emerges of a huge gap between the power of modern software and developers’ ability to keep pace with it. We see this most often in development challenges but in the other challenge levels as well. Bridging that gap is a key capability we require from modern dev and devops tools. observability gap A bridge over troubled waters Currently, only a handful of solutions is available to developers who face these challenges. Most developer tools have remained stagnant for years. . That said, the modern developer toolbox have some new solutions. These include solutions designed to enable greater focus, solutions that attempt to do more with the limited available data, and solutions that try to bridge the observability gap by improving on the deployment cycle. An engineer from 20 years ago would be completely baffled by modern servers in the cloud, but would recognize a current IDE or debugger in seconds does “Like a glove” — Tailored observability views With many modern software challenges starting to repeat and consolidate into known forms and formats (such as Docker, K8S, Serverless functions) a new breed of tools has emerged that identifies these patterns, and leverages their knowability to tailor specific solutions. Within this category are next-generation APM solutions such as , which provide views built specifically for on top of existing APM offering. In fact, you’ll find that most APMs adjust and provide capabilities for the microservices world, although not always as first-class citizens. DataDog Containers and Kubernetes In an even more modern tailored approach, we find solutions doubling down on structured data and specifically , such as (alongside open source projects such as and ). These identify the specific pain of reconstructing a distributed system behaviour, much as a detective reconstructs a crime scene, and arise from the very nature of microservice architectures. Tracing HoneyComb.io Zipkin Jager For Serverless, we find solutions like and , which are tailored specifically to the FaaS use case and target specific pain points such as discovery, management, and pricing. These issues, of course, were present before, but became more acute with Serverless. Epsagon Lumigo “Deus Ex Machina” — Advanced analysis With software grown big, big data is not only a challenge, but also a means of tackling the problem. Multiple solutions harness the strengths of machine-learning to attain observability, and some use it as a primary approach. Examples include solutions like and . Coralogix Anodot “Developers of the world, unite!” — Advanced workflows Harnessing machines to solve machine problems is a start, but it isn’t a silver bullet. As a result, many solutions focus on building a better developer workflow on top of the views and alerts provided by the automated part of the solution. is one great example and specifically with their recent release of Sentry 9, they improve the cooperative workflow on top of the existing exception management platform. Sentry.io “Getting a taste of the real stuff” — Canary deployments Most solutions reviewed here so far, and most solutions in the devops space in general, work by deriving value from data that exists in the system, or that can be collected in advance by focusing on a specific pattern (such as tracing). While a significant step forward, these still leave us highly dependent on the CI/CD channel to iterate by delivering new data from production by updating log lines or SDK calls that feed the systems upstream. One way to reduce pressure on the pipeline is by using , a variant on green-blue deployments. They enable developers to expose new code (in this case, the necessary observability changes) to a limited percentage of production, without affecting all of production. It also allows faster rollbacks. canary deployments Recipes for canary deployments can be found for most leading CI/CD tools such as and . GitLab CodeFresh Yet a huge gap still remains: What if the needed data isn’t within the limited canary percentage? While canary deployments reduce the friction and risk of observing production, they still leave most of it on the table, making it an expensive and risky effort that most can’t afford. “Bridging the gap with agility” — Rapid data collection Despite the many solutions above, a gap remains between the observability data developers need from live systems and the pace at which they can iterate to collect and deliver that data to their various solutions (including some listed here). A new cohort of solutions attempts to bridge this gap by completely decoupling data collection from the CI/CD pipeline and thus eliminating the friction, risks, and bottleneck that create the gap. These rapid solutions collect and drive data on demand without prior planning, creating evanescent bridges as needed, instead of trying to anticipate specific bridges for predictable cases. This ephemeral access approach, aligns with the ephemeral nature of modern software. Rapid Production Debugging solution is an example which leverages instrumentation and opcode manipulation capabilities to enable developers to collect data from live environments and pipeline it to whichever platform they need. (Full disclosure: author is a co-founder of Rookout) Rookout A glimpse into the future We’ve come a long way. Developers from the past would definitely envy the amazing accomplishments that enable modern software architectures. But when it comes to the tools we provide developers, we are not there yet. Devops has launched the world of software forward at tremendous speeds, but the from keeping pace with their own software. complexity, connectivity, and development challenges left in its wake are preventing developers Now is the time to take another leap forward by embracing the new solutions quickly and actively working to bridge the observability gap. and more. In the not-too-distant future we can reach a point where the pace of software evolution is matched by software observability, creating a feedback loop that endlessly increases the speed of creation. A future where no one is left behind. By more effectively connecting tools to one another, creating better workflows on top of them, and finding true agility in data collection and observability, we can bridge the gap About Or Weis: Or Weis is CEO and co-founder of , a Rapid debugging and observability solution for live systems. Rookout Rookout sets non-breaking breakpoints that collect any data you want and deliver it on the fly wherever you need it, when you need it. All without writing more code, re-deploying or restarting.

Is DevOps leaving developers behind?

for whom the bell tolls. I originally wrote this 2016 but given the WSJ discussion on then I thought it was worth repeating but with the snark dial turned all the way up to 11. Remember the old adage of “go away or I’ll replace you with a small Perl script” … well, maybe we might just get to enact our threat on the . post in April “World’s Largest Hedge Fund Is Building an Algorithmic Model From its Employees’ Brains” “robots” On machine intelligence I frequently read articles about how machine intelligence and robotics will replace basic roles in society, they will become the new drivers, the new waiters, the new hospital porters and nurses of this world. But is this likely? And if they are going to replace jobs then whose job are they most suited to? With questions like this I tend to look through three lenses — ability, cost and acceptability. Does it have the ability to do a better job at a reasonable cost and would we accept it? For example, let us take the idea of a waiter. Certainly there’s some mileage in terms of a marketing gimmick to begin with but humans are surprisingly good at taking orders, delivering food and cleaning up tables. Humans are also relatively cheap compared to the current cost of machine intelligence and robotics to replace them and in general though we might be able to trust a robot to deliver our food, would the banter be the same and would it be as socially acceptable to customers? Beyond the gimmick, I doubt this change will happen for quite a long time. However, there are other groups where ability, cost and acceptability are far more positive for replacement. One possible candidate is that of the CEO. The CEO? Of a company? Surely not? Well, I think so. Lets go through the reasons. Ability From my experience, most CEOs tend to demonstrate poor situational awareness and an inability to decipher doctrine from context specific play. My own work in 2012, which examined situational awareness versus action for 160+ Silicon Valley companies showed that those with high levels of situational awareness performed better. But this was Silicon Valley, it was supposed to be at the cutting edge and even here there were large companies operating with poor situational awareness of their environment and suffering for it. Going beyond Silicon Valley and boardrooms started to feel more akin to alchemy, gut feel and whatever is popular in the HBR than to chess playing masters. Situational awareness appeared to be the exception and not the rule. So rather than being a Darwinian competition of survival of the fittest where all conquering CEOs battle each other in a game of wits, there’s another less charitable way of looking at this — survival of the least hazardous. It’s governed by the same rules but rather than a positive chess like game of survival, it’s more random with the occasional hero characters. A bit like the games like world of warcraft where most run around shouting “I’m lost” and one player saves the team. An awful lot boils down to luck. Surely, this can’t be the case? Alas various studies have questioned the impacts of CEOs e.g. Markus Fitza’s study demonstrated that the measurable . brawl of new players in fantasy role playing CEO effect on firm performance varies little from chance Given that situational awareness can be improved, that doctrine and context specific play can be learned and we’ve seen first hand the experience of companies failing to understand basic lessons and (e.g. cloud) then we have to ask whether this is just inertia, blindness or caused by some other motivation? CEOs are not daft people but then neither are fund managers and the of financial indices. From my experience, this is an area where machine intelligence might vastly improve on the performance of the incumbent executives along with the industry of management consultants that often “ ” them. being disrupted by highly predictable change vast majority were outperformed by simple trackers supports Cost The other upside of replacing the CEO with machine intelligence is cost. Unlike your average waiter, CEOs aren’t cheap — they tend to be very expensive. Now, whilst Marissa Mayer’s performance at CEO of Yahoo has not been bad (despite what many think, the company has at least maintained its revenue), the question becomes whether this has been worth the speculated . It doesn’t actually matter how much the figure is, we simply know it’ll be a lot more than a waiter expects. In general, there’s a lot more money on the table replacing the CEO than there is your average worker. $365 million for 5 years of work Acceptability. But CEOs lead us! You wouldn’t trust a robot! Well, that’s probably the idea in the hallowed walls of executive thinking and business schools. But on the shop-floor we’re already becoming used to machines taking life and death decisions for us in areas such as driving cars. Most people tend to be motivated by their own autonomy, the ability to gain mastery, purpose (as in direction) and financial rewards. Using (e.g. ) then it should be possible for most companies to create such an environment. I’d argue a machine intelligence could manage this just as easily as a human. cell based structures, maps, fitness functions and other tools the pioneer, settler and town planner structure which AirBnB discussed A machine could direct a cell of human pioneers to go do what they are good at by exploring an uncertain area to see what is found. Equally, at the other extreme a machine is just as capable of telling a cell of human town planners to look at industrialising something and it is even more likely to take rational decisions without inertia and sentimental motivation or concerns over political capital due to past investments. Also, a machine is perfectly capable of learning the basic economic patterns that apply to an environment. The failures of past technology giants in cloud computing were executive failures and human error against highly predictable changes. It wasn’t lack of skilled engineers on the shop-floor. Back in 2008, at Canonical (Ubuntu) we exploited the predictable changes of cloud computing in order to take the company from a . Ubuntu has maintained that position ever since. Had we been up against machines rather than human executives then I’m not sure we could have got away with it and we certainly couldn’t have used inertia against others. I somehow doubt people will have a problem with a machine intelligence being the CEO if it leads to more stability and more security along with enhancements to their own autonomy and mastery. Given these conditions I suspect many on the shop-floor would happily cheer for our robot overlords — small fraction of the market to 70% of all cloud in around 18 months “hey, are you’re still running with that old Cyberdyne systems CEO? We’ve just installed the new HAL 9000, it comes with better stock performance and added EQ with moral imperative subroutines. Everyone is really happy.” Except of course, there’s one group that might not cheer — those desiring to be CEO and those who are the CEO. Actually, there’s another group which are the management consultants that thrive off them. Executives tend to be very busy people trying to deal with complex and complicated environments. They are always looking for quick answers and there’s an entire industry ready to the latest magic solution. Just look at the stuff that pervades our industry from Myers Briggs Personality Tests to (as reported in the HBR). Could we not do better? Would a machine care about earlobes? Unfortunately, such groups voting for robotic CEOs would be like turkeys voting for Christmas. If such a change was to happen then it would have to be through testing first in startups and then activist investors. Which, by the way, is why that . sell how earlobes can signify leadership potential WSJ article is so fascinating Given all the roles that could be replaced then on a basis of ability, cost and acceptability then the average CEO seems as good if not a far better bet than your average waiter. There are always exceptions to rules but it’s certainly worth thinking about. In my view it’s not the blue collar workers that need to be immediately worried about robots / machine intelligence but instead the white collar workers who think they’re not the target. Originally published at blog.gardeviance.org on December 22, 2016.

The robots are coming …

Software development goes through six main stages, otherwise known as the software development lifecycle. They are planning, design, development, testing, deployment, and maintenance. After development, the next phase involving quality assurance and testing is crucial to the success of the software. Proper QA testing identifies redundancies, inconsistencies, bugs, and defects in a newly developed product. Without this process, there’s no way to find out the ability of the product to survive in the real world and to work as intended for many years. Still, there have been instances when developers have either ignored this step or done it poorly for reasons such as cutting costs or beating a deadline. They have chosen to go straight to deployment, with grave consequences. For example, the Nest Learning Thermostat by Google that drained the device’s battery. As a result, people who had installed the device in their houses were unable to heat them. Nest declared that the issue was in the firmware update, incompatible boilers, and older air filters and released a new update that solved the issue for its 99.95% of affected users. experienced a software bug Another case is that of Cloudflare. In 2017, the web security company , dubbed Cloudbleed, that led to the leakage of sensitive personal information such as authentication tokens, passwords, private messages, and cookies from customers’ sites. Though the most severe issues were cleared up in hours, it took six days for the company to completely repair the bug. However, it is suspected that the breach could have started five months before. suffered a major software bug These two cases show that poor/ inadequate software testing is not only costly and time-consuming in the long run but also can damage a company’s reputation among customers. But before you decide to do QA and testing, let’s have a look at the difference between manual and automation QA. Key Differences Between Manual and Automation Testing Software testing is a broad aspect that can be categorized into two types. These are manual testing and automated testing. Manual testing, as the name suggests, is done by hand. The testers act as the end-users of the product and check that all the features and functionalities are working as intended before the software is released. As the testers check for defects manually, they don’t need to know how to use testing tools. On the other hand, the testers are expected to report on any discrepancies or errors for the developers to be able to fix them. Automation testing uses a set of testing tools or a pre-scripted framework to perform tests. It compares the collected results to the expected behavior of the product and creates a report. Automation eliminates the need for a manual tester, especially in a regression test. Projects That Can Do with Manual QA Only Manual QA works best for projects that require cognitive and behavioral capabilities during the testing of the software. The issues that can be tested include: The functionality of an app User interface Site and application activity Features User acceptance User experience. These are the projects that can do with the manual type of testing only. Consequently, manual QA should be exclusively used in the following cases: : Domain professionals predominantly implement it. They conduct their tests by examining the functionalities of a particular app without possessing the proper knowledge of the requirements. Exploratory testing : It is used for gauging if an app is user-friendly or comfortable for the end-user. Determining whether the end-user can comprehend and operate the application is the principal purpose of this testing. Usability testing : It is whereby testers try out an app at random without complying with any testing techniques or specifications in relevant documents. Ad-hoc testing is often conducted when the testers` knowledge regarding the subject app is excellent. Ad-hoc testing Projects That Only Need Test Automation to Succeed Sometimes, it is test automation that should be applied to achieve the best results. Automation testing is ideal for assessing things such as: Object information Function library Test data source Various reusable models. What are the cases in which this type of testing will work better than a manual one? Let`s observe these scenarios: Regression testing: It refers to the subsequent examination of a program that has been tested before and then modified accordingly. The purpose is to identify any problems that have come up due to the modifications or any other reason. Regular code changes make automated testing necessary in this case to ensure that regressions are run on time. Load testing: The goal of this process is to confirm whether the subject app can manage the projected number of transactions and gauge its behavior during peak and off-peak periods. Performance testing: It is used for understanding issues like the speed, scalability, and stability of a particular application. Final Thoughts To conclude, it is essential to appreciate that there is a QA talent shortage across the planet. Luckily, by leveraging QA R&D outsourcing solutions, companies can address the issue of scarce local talent by hiring offshore QA specialists. Things get even better since this model of cooperation is cost-effective. The reason is it often takes advantage of the disparity in salaries between the involved countries. Are you wondering about where to hire QA talent? Contact us for QA team outsourcing!

Automation Testing Versus Manual Testing Guidelines

A few weeks ago I decided to scratch an itch I’ve been having for a while — to participate in some bug bounty programs. Perhaps the most daunting task of the bug bounty game is to pick a program which yields the highest return on investment. Soon though, I stumbled upon a web application that executes user-submitted code in a sandbox. This looked interesting so I decided to pursue it. Python Let me out of this god-forsaken sandbox! After a bit of poking around, I discovered how to break out of the sandbox with some hacks at the Python layer. Report filed. Bugs fixed, and a nice reward to boot, all within a couple days. Sweet! A great start to my bug bounty adventures. But this post isn’t about that report. All in all, the issues I discovered are not that interesting from a technical perspective. And it turns out the issues were only present because of a regression. But I wasn’t convinced that securing a Python sandbox would be so easy. Without going into too much detail, the sandbox uses a combination of OS-level isolation and a locked-down Python interpreter. The Python environment uses a custom whitelisting/blacklisting scheme to prevent access to unblessed builtins, modules, functions, etc. The OS-based isolation offers some extra protection, but it is antiquated by today’s standards. Breaking out of the locked-down Python interpreter is not a 100% win, but it puts the attacker dangerously close to being able to compromise the entire system. So I returned to the application and prodded some more. No luck. This is indeed a tough cookie. But then I had a thought — Python modules are often just thin wrappers of mountainous C codebases. Surely there are gaggles of memory corruption vulns waiting to be found. Exploiting a memory corruption bug would let me break out of the restricted Python environment. Where to begin? I know the set of Python modules which are whitelisted for importation within the sandbox. Perhaps I should run a distributed network of AFL fuzzers? Or a symbolic execution engine? Or maybe I should scan them with a state of the art static analysis tool? Sure, I could have done any of those things. Or I could have just queried the some bug trackers. Turns out I did not have this hindsight when beginning the hunt, but it did not matter much. My intuition led me to discovering an exploitable memory corruption vulnerability in one of the sandboxes’ whitelisted modules via manual code review and testing. The bug is in Numpy, a foundational library for scientific computing — the core of many popular packages, including scipy and pandas. To get a rough idea of Numpy’s potential as a source of memory corruption bugs, let’ check out the lines-of-code counts. In the remainder of this post, I first describe the conditions which lead to the vulnerability. Next, I discuss some quirks of the CPython runtime which exploit developers should be aware of, and then I walk through the actual exploit. Finally, I wrap up with thoughts on quantifying the risk of memory corruption issues in Python applications. The Vulnerability The vulnerability which I am going to walk through is an integer overflow bug in Numpy v1.11.0 (and probably older versions). The issue has been fixed since v1.12.0, but there was no advisory issued. security The vulnerability resides in the API for resizing Numpy’s multidimensional array-like objects, and friends. is called with a tuple defining the array’s , where each element of the tuple is the size of a dimension. ndarray resize shape Sidenote: Yup, the array is leaking uninitialized memory, but we won’t be focusing on that in this post. Under the covers, actually 's a buffer, with the size calculated as the product of each element in the shape tuple and the element size. So in the prior snippet of code, boils down to C code . The next code snippet is the heavily paraphrased implementation of in C. resize realloc arr.resize((2, 3)) realloc(buffer, 2 * 3 * sizeof(int32)) resize Spot the vulnerability? You can see inside the for-loop (line 13) that each dimension is multiplied to produce the new size. Later on (line 25) the product of the new size and the element size is passed as the size to memory which holds the array. There is some validation on the new size prior to , but it does not check for integer overflow, meaning that very large dimensions can result in an array which is allocated with insufficient size. realloc realloc Ultimately, this gives the attacker a powerful exploit primitive: the ability to read or write arbitrary memory by indexing from an array with overflown size. Let’s develop a quick proof of concept that proves the bug exists. Quirks of the CPython runtime Before we walk through developing the exploit, I would like to discuss some ways in which the CPython runtime eases exploitation, but also ways in which it can frustrate the exploit developer. Feel free to skip this section if you want to dive straight into the exploit. Leaking memory addresses Typically one of the first hurdles exploits must deal with is to defeat address-space layout randomization (ASLR). Fortunately for attackers, Python makes this easy. The builtin function returns the memory address of an object, or more precisely the address of the structure which encapsulates the object. id PyObject In real-world applications, developers should make sure not to expose to users. In a sandboxed, environment there is not much you could do about this behavior, except perhaps blacklisting or re-implementing to return a hash. id(object) id id Understand memory allocation behavior Understanding your allocator is critical for writing exploits. Python has different allocation strategies based on object type and size. Let’s check out where our big string , little string , and numpy array landed. 0xa52cd0 0x7ffff7f65848 0x7ffff7e777b0 Python object structure Leaking and corrupting Python object metadata can be quite powerful, so it’s useful to understand how Python objects are represented. Under the covers, Python objects all derive from , a structure which contains a reference count and a descriptor of the object’s actual type. Of note, the type descriptor contains many fields, including function pointers which could be useful to read or overwrite. PyObject Let’s inspect the small string we created in the section just prior. Shellcode like it’s 1999 The library serves as a bridge between Python and C code. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. Many modules which have C bindings or require calling into shared libraries require importing ctypes. ctypes I noticed that importing ctypes results in the mapping of a 4K-sized memory region set with read/write/execute permissions. If it wasn’t already obvious, this means that attackers do not even need to write a ROP chain. Exploiting a bug is as simple as pointing the instruction pointer at your shellcode, granted you have already located the RWX region. Test it for yourself! Investigating further, I discovered that is for ing the RWX region. However, the region cannot be allocated RWX on certain platforms, such as systems with selinux enforced or PAX mprotect enabled, and there is code which works around this limitation. libffi’s closure API responsible mmap I did not spend much time trying to reliably locate the RWX mapping, but in theory it should be possible if you have an arbitrary-read exploit primitive. While ASLR is applied to libraries, the dynamic linker maps the regions of the library in a predictable order. A library’s regions include its globals which are private to the library and the code itself. Libffi stores a reference to the RWX region as a global. If for example you find a pointer to a libffi function on the heap, then you could precalculate the address of the RWX-region pointer as an offset from the address of the libffi function pointer. The offset would need to be adjusted for each library version. De facto exploit mitigations I tested out security-related compiler flags for the Python2.7 binary on Ubuntu 14.04.5 and 16.04.1. There are a couple of weaknesses which are quite useful for the attacker: Partial RELRO: The executable’s , which contains pointers to library functions dynamically linked into the binary, is writable. Exploits could replace the address of with for example. GOT section printf() system() No PIE: The binary is not a Position-Independent Executable, meaning that while the kernel applies ASLR to most memory mappings, the contents of the binary itself are mapped to static addresses. Since the GOT section is part of the binary, no PIE makes it easier for attackers to locate and write to the GOT. Road blocks While CPython is an environment full of tools for the exploit developer, there are forces which broke many of my exploit attempts and were difficult to debug. The garbage collector, type system, and possibly other unknown forces will break your exploit if you aren’t careful about clobbering object metadata. can be unreliable. For reasons I could not determine, Python appears sometimes to pass a copy of the object while the the original object is used . id() The region where objects are allocated is somewhat unpredictable. For reasons I could not determine, certain coding patterns led to buffers being allocated in the heap, while other patterns led to allocation in a python-specific ‘d heap. brk mmap The exploit Soon after discovering the numpy integer overflow, I submitted a report to the bug bounty with a proof of concept that hijacked the instruction pointer, but did not inject any code. When I initially submitted I did not realize that the PoC was actually pretty unreliable, and I wasn’t able to test it properly against their servers because validating hijack of the instruction pointer requires access to core dumps or a debugger. The vendor acknowledged the issue’s legitimacy, but they gave a less generous reward than for my first report. Fair enough! I’m not really an exploit developer, but I challenged myself to do better. After much trial and error, I eventually wrote an exploit which appears to be reliable. Unfortunately I was never able to test it in the vendor’s sandbox because they updated numpy before I could finish, but it does work when testing locally in a Python interpreter. At a high level, the exploit gains an arbitrary read/write exploit primitive by overflowing the size of a numpy array. The primitive is used to write the address of to 's GOT/PLT entry. Finally, Python’s builtin calls under the covers, so now you can call to get a shell, or replace /bin/sh with any command. system fwrite print fwrite print '/bin/sh' There is a bit more to it than the high-level explanation, so check out the exploit in full below. I recommend to begin reading from the bottom-up, including comments. If you are using a different version of Python, adjust the GOT locations for and before you run it. fwrite system Running the exploit gives you a “hacked” shell. Quantifying the risk It is well known that much of Python’s core and many third-party modules are thin wrappers of C code. Perhaps less recognized is the fact that memory corruption bugs are reported in popular Python modules all the time without so much as a CVE, a security advisory, or even a mention of security fixes in release notes. So yes, there are a lot of memory corruption bugs in Python modules. Surely not all of them are exploitable, but you have to start somewhere. To reason about the risk posed by memory corruption bugs, I find it helpful to frame the conversation in terms of two discrete use-cases: regular Python applications, and sandboxing untrusted code. Regular applications The types of applications we’re concerned with are those having a meaningful attack surface. Think web applications and other network-facing services, client applications which process untrusted content, privileged system services, etc. Many of these applications import Python modules built against mountains of C code from projects which do not treat their memory corruption bugs as security issues. The pure thought of this may keep some security professionals up at night, but in reality the risk is typically downplayed or ignored. I suspect there are a few reasons: The difficulty to remotely identify and exploit memory corruption issues is quite high, especially for closed-source and remote applications. The likelihood that an application exposes a path for untrusted input to reach a vulnerable function is probably quite low. Awareness is low because memory corruption bugs in Python modules are not typically tracked as security issues. So fair enough, the likelihood of getting compromised due to a buffer overflow in some random Python module is probably quite low. But then again, memory corruption flaws can be extremely damaging when they do happen. Sometimes it doesn’t even take anyone to explicitly exploit them to cause harm (re: cloudbleed). To make matters worse, it’s nigh impossible to keep libraries patched when library maintainers do not think about memory corruption issues in terms of security. If you develop a major Python application, I suggest you at least take an inventory of the Python modules being used. Try to find out how much C code your modules are reliant upon, and analyze the potential for exposure of native code to the edge of your application. Sandboxing There are a number of services out there that allow users to run untrusted Python code within a sandbox. OS-level sandboxing features, such as linux namespaces and seccomp, have only become popular relatively recently in the form of Docker, LXC, etc. Weaker sandboxing techniques can unfortunately still be found in use today — at the OS layer in the form of chroot jails, or worse, sandboxing can be done entirely in Python (see and ). pypy-sandbox pysandbox Memory corruption bugs completely break sandboxing which is not enforced by the OS. The ability to execute a subset of Python code makes exploitation far more feasible than in regular applications. Even pypy-sandbox, which claims to be secure because of its two-process model which virtualizes system calls, can be broken by a buffer overflow. If you want to run untrusted code of any kind, invest the effort in building a secure OS and network architecture to sandbox it.

Escaping a Python sandbox with a memory corruption bug

When mathematics and programing go apart Source: https://www.youtube.com/watch?v=884dnOYOb68 In the world that routinely shifts towards complete dependence on scientific computing, even a trivial mistake can have fatal consequences (check the yesterday’s headlines about ) Although I cannot give you precise statistics, quite likely >90% of errors are caused by us, humans. The issue, I want to write about, is far more frightening than a spontaneous or overlooked typo. Cloudbleed . Scientific computing and high impact factor journals are plagued with misuse and confusion between log(x) with ln(x)! Why should anybody bother about some obscure logarithmic operations? The established, mathematical definition: log(10) = 2.302585093 (…) The widespread definition used in physics, chemistry and derivative life-sciences: log(10) = 1 ln(10) = 2.302585093 (…) Take these numbers, multiply them by some ridiculously large integer (let’s say 10000000000000) and compare. The numerical difference between these two could easily translate into an incorrect trajectory for Space-X Falcon 9 rocket, abnormal termination of Instrument Landing System for any commercial airliner attempting CAT-3 autolanding, Tesla Model-X mistakenly speeding up and performing obstacle avoidance maneuver leading to accident, and countless other tasks, which rely on numerical optimization and intricate workings of embedded logarithmic operations. The computation of logarithm is at the very heart of potentially every problem that concerns computational physics, our dearly beloved , and all the possible permutations of numerical optimization. in silico machine learning The observed misuse of and is simply and by code developers and scientific writers. log(x) ln(x) INSTITUTIONALIZED AMPLIFIED Just to be clear here! The problem exists and it becomes by new generations of . On my side: I am not willing to play a “blame game”. perpetuated scientific programmers I have routinely observed in my students (still at University of Groningen), snippets of Python code with confused use of base-10 and natural logarithms. I observe the same problem in the code by my scientific collaborators. A quick review of research papers (from high impact peer reviewed journals, including ) on protein electrostatics from past 20 years, clearly demonstrates issues with proper use of ln(x). This coincides perfectly with the emergence of scientific computing, and a dominant place log(x) has as a representation of natural logarithm in all standard math libraries. PNAS I have made countless log(x) vs ln(x) mistakes . myself Looking for discrepancies in protein electrostatics calculation code How is the problem perpetuated? Simply by symbolical use of log(x), which performs ln(x) according to its mathematical definition. That’s how the “obvious” thing becomes “not so obvious” mistake, which in turn may have quite bizarre numerical consequences. I have attached below the snippets of documentation from different programing languages, which are widely used in scientific computing. : NumPy v1.12 documentation numpy.log( [, ]) x out = <ufunc ‘log’> Natural logarithm, element-wise. Matlab documentation : Syntax Y = log(X) Description example returns the natural logarithm ln(x) of each element in array . [Y](https://it.mathworks.com/help/matlab/ref/log.html?requestedDomain=www.mathworks.com#outputarg_Y) = log([X](https://it.mathworks.com/help/matlab/ref/log.html?requestedDomain=www.mathworks.com#inputarg_X)) X Mathematica documentation: [ ] gives the natural logarithm of (logarithm to base) Log z z C documentation (Tutorials point): Description The C library function returns the natural logarithm (base-e logarithm) of . double log(double x) x The ISO80000–2 way of dealing with it: ln(x) for natural logarithm lg(x) for base-10 logarithm What can we do about the log(x) vs ln(x) confusion? Enforce thorough reading of documentation on our students, collaborators and peers (wishful thinking). Simply introduce ln(x) and log(x) so mathematics,programing and life sciences go in concert together! development Follow the guidelines disclosed in ISO8000–2 standard http://www4.ncsu.edu/~jwilson/files/mathsigns.pdf Disclaimer: This post has been edited to address the comments and correct the issues pointed out in https://news.ycombinator.com/item?id=13731301#13732928 Useful post? If so, make my day by clicking the heart below! About the Author is the founder and the architect of . You can connect with him on , , , and . Kamil Tamiola Peptone — The Protein Intelligence Company AngelList LinkedIn Twitter Researchgate

log(x) vs ln(x) — The curse of scientific computing

Scaling Ethereum: Data Bloat, Data Availability, and the Cloudless Solution

স্কেলিং ইথেরিয়াম: ডেটা ব্লোট, ডেটা উপলব্ধতা এবং ক্লাউডহীন সমাধান

Skalierung von Ethereum: Datenaufblähung, Datenverfügbarkeit und die Cloudless-Lösung

Escalando Ethereum: exceso de datos, disponibilidad de datos y la solución sin nubes

Mise à l'échelle d'Ethereum : gonflement des données, disponibilité des données et solution sans cloud

एथेरियम का विस्तार: डेटा ब्लोट, डेटा उपलब्धता और क्लाउडलेस समाधान

Ethereum のスケーリング: データの肥大化、データの可用性、クラウドレス ソリューション

Ethereum 확장: 데이터 팽창, 데이터 가용성 및 클라우드리스 솔루션

Dimensionando o Ethereum: inchaço de dados, disponibilidade de dados e a solução sem nuvem

Масштабирование Ethereum: раздувание данных, доступность данных и безоблачное решение

Ethereum'u Ölçeklendirmek: Veri Şişmesi, Veri Kullanılabilirliği ve Bulutsuz Çözüm

Mở rộng quy mô Ethereum: Dữ liệu phình to, Tính sẵn có của Dữ liệu và Giải pháp Không có Đám mây

扩展以太坊：数据膨胀、数据可用性和无云解决方案

The software industry has witnessed a dynamic evolution, moving from monolithic applications to microservices, and now, a more comprehensive and sustainable approach: platform engineering. While microservices have served as a crucial step in improving scalability and flexibility, they bring new challenges that require an even more integrated solution. Platform engineering emerges as the answer, providing the stability, efficiency, and agility that modern software architectures demand. From Monolithic to Microservices: A Necessary Evolution The transition from monolithic architectures to microservices marked a significant leap in software development. Monolithic applications, characterized by their single codebase and tightly coupled components, often led to scalability and maintenance issues. As applications grew, the rigidity of these structures became apparent. In response, microservices offered a modular approach, allowing development teams to build, deploy, and scale individual services independently. According to a survey conducted by O’Reilly in 2021, 61% of organizations have implemented microservices, citing improved scalability and ease of deployment as the primary drivers . Leading tech companies like Netflix and Amazon were early adopters, demonstrating how microservices could handle complex, high-traffic applications. Netflix, for example, manages over 1,000 microservices to support its streaming platform, allowing for rapid development cycles and minimal downtime during deployments . Despite these benefits, microservices introduced new complexities. Issues with service orchestration, inter-service communication, and the increased overhead of managing numerous independent components have become apparent. This is where platform engineering enters the scene, offering a holistic solution that goes beyond the limitations of microservices. The Rise of Platform Engineering: Addressing Modern Software Challenges Platform engineering builds on the microservices concept by providing a unified framework for building, deploying, and managing services. It focuses on creating an internal developer platform (IDP) that streamlines infrastructure management and allows developers to focus on building features rather than wrestling with complexities. In a 2022 report by the Cloud Native Computing Foundation (CNCF), 65% of organizations stated that managing microservices became a significant operational burden, pushing the need for more comprehensive platform solutions . Platform engineering addresses these pain points by offering reusable components, tools, and processes within a centralized platform. This standardization reduces the operational overhead while providing the flexibility to handle various services, applications, and environments. Tech giants like Google and Spotify have pioneered platform engineering, utilizing tools like Kubernetes and Docker to create efficient and scalable internal platforms. Google’s Borg system, an early precursor to Kubernetes, was instrumental in transforming how services were managed across a vast infrastructure. Spotify’s "Backstage" platform allows their developers to discover and manage microservices, fostering greater efficiency and collaboration across teams. Kubernetes and Docker: Pillars of Platform Engineering Kubernetes and Docker have become synonymous with modern software development, forming the backbone of platform engineering. Kubernetes, an open-source container orchestration system, allows for automating the deployment, scaling, and operation of application containers across clusters of hosts. This ensures that services are highly available, resilient, and easily scalable. Docker, on the other hand, enables developers to package applications into containers, encapsulating everything needed to run the software reliably across different environments. Kubernetes adoption has surged in recent years, with a 2023 CNCF report revealing that 96% of organizations using containerized applications have adopted Kubernetes for container orchestration. This widespread adoption reflects the value that Kubernetes adds to platform engineering by providing an abstraction layer that simplifies resource management and deployment. By combining Kubernetes with Docker, platform engineering enables companies to create internal developer platforms that support various development, testing, and production environments. This approach allows developers to seamlessly move applications between environments, reducing friction in the software development lifecycle. As a result, companies can deliver software faster, more reliably, and with less operational strain. Case Studies: Platform Engineering in Action Several leading companies have embraced platform engineering to overcome the limitations of microservices. At Airbnb, the team developed an internal platform called “Nerve” to manage the complexities of their microservices architecture. This platform provided observability, fault-tolerance mechanisms, and seamless service discovery, streamlining the operational aspects and reducing the burden on individual teams. Another example is Shopify, which employs platform engineering to handle its vast infrastructure. Shopify’s internal platform, "Cloudbuddies," automates the deployment of services and manages Kubernetes clusters. This automation enables their developers to focus on building features and improving the customer experience, significantly enhancing productivity. According to Shopify’s engineering team, implementing platform engineering led to a 30% reduction in service downtime and improved developer efficiency by 40%. These statistics highlight the tangible benefits of moving beyond microservices toward a more integrated platform-centric approach. The Future: Why Platform Engineering Is Here to Stay As the complexity of software systems continues to grow, the need for a sustainable and future-proof architecture becomes more critical. Microservices laid the groundwork for modular software development, but the operational overhead they introduced necessitated the evolution to platform engineering. By centralizing infrastructure management, standardizing processes, and leveraging tools like Kubernetes and Docker, platform engineering addresses the scalability, flexibility, and efficiency challenges that modern software faces. For companies looking to stay ahead in the digital landscape, adopting platform engineering is not just a trend but a strategic imperative for long-term success. What’s Next? The journey from monolithic to microservices architectures revolutionized software development, but platform engineering is the next logical step to address the challenges of modern software ecosystems. By building internal developer platforms that leverage tools like Kubernetes and Docker, companies can streamline their operations, enhance developer productivity, and create a future-proof software architecture. As the tech industry continues to evolve, platform engineering stands as the key to unlocking the full potential of microservices and beyond. The software industry has witnessed a dynamic evolution, moving from monolithic applications to microservices, and now, a more comprehensive and sustainable approach: platform engineering. While microservices have served as a crucial step in improving scalability and flexibility, they bring new challenges that require an even more integrated solution. Platform engineering emerges as the answer, providing the stability, efficiency, and agility that modern software architectures demand. From Monolithic to Microservices: A Necessary Evolution From Monolithic to Microservices: A Necessary Evolution The transition from monolithic architectures to microservices marked a significant leap in software development. Monolithic applications, characterized by their single codebase and tightly coupled components, often led to scalability and maintenance issues. As applications grew, the rigidity of these structures became apparent. In response, microservices offered a modular approach, allowing development teams to build, deploy, and scale individual services independently. According to a survey conducted by O’Reilly in 2021, 61% of organizations have implemented microservices , citing improved scalability and ease of deployment as the primary drivers . Leading tech companies like Netflix and Amazon were early adopters, demonstrating how microservices could handle complex, high-traffic applications. Netflix, for example, manages over 1,000 microservices to support its streaming platform, allowing for rapid development cycles and minimal downtime during deployments . 61% of organizations have implemented microservices 61% of organizations have implemented microservices 1,000 microservices 1,000 microservices Despite these benefits, microservices introduced new complexities. Issues with service orchestration, inter-service communication, and the increased overhead of managing numerous independent components have become apparent. This is where platform engineering enters the scene, offering a holistic solution that goes beyond the limitations of microservices. The Rise of Platform Engineering: Addressing Modern Software Challenges The Rise of Platform Engineering: Addressing Modern Software Challenges Platform engineering builds on the microservices concept by providing a unified framework for building, deploying, and managing services. It focuses on creating an internal developer platform (IDP) that streamlines infrastructure management and allows developers to focus on building features rather than wrestling with complexities. In a 2022 report by the Cloud Native Computing Foundation (CNCF), 65% of organizations stated that managing microservices became a significant operational burden, pushing the need for more comprehensive platform solutions . Platform engineering addresses these pain points by offering reusable components, tools, and processes within a centralized platform. This standardization reduces the operational overhead while providing the flexibility to handle various services, applications, and environments. 65% of organizations 65% of organizations Tech giants like Google and Spotify have pioneered platform engineering, utilizing tools like Kubernetes and Docker to create efficient and scalable internal platforms. Google’s Borg system, an early precursor to Kubernetes, was instrumental in transforming how services were managed across a vast infrastructure. Spotify’s "Backstage" platform allows their developers to discover and manage microservices, fostering greater efficiency and collaboration across teams. Kubernetes and Docker: Pillars of Platform Engineering Kubernetes and Docker: Pillars of Platform Engineering Kubernetes and Docker have become synonymous with modern software development, forming the backbone of platform engineering. Kubernetes, an open-source container orchestration system, allows for automating the deployment, scaling, and operation of application containers across clusters of hosts. This ensures that services are highly available, resilient, and easily scalable. Docker, on the other hand, enables developers to package applications into containers, encapsulating everything needed to run the software reliably across different environments. Kubernetes adoption has surged in recent years, with a 2023 CNCF report revealing that 96% of organizations using containerized applications have adopted Kubernetes for container orchestration. This widespread adoption reflects the value that Kubernetes adds to platform engineering by providing an abstraction layer that simplifies resource management and deployment. 96% of organizations 96% of organizations By combining Kubernetes with Docker, platform engineering enables companies to create internal developer platforms that support various development, testing, and production environments. This approach allows developers to seamlessly move applications between environments, reducing friction in the software development lifecycle. As a result, companies can deliver software faster, more reliably, and with less operational strain. Case Studies: Platform Engineering in Action Case Studies: Platform Engineering in Action Several leading companies have embraced platform engineering to overcome the limitations of microservices. At Airbnb, the team developed an internal platform called “Nerve” to manage the complexities of their microservices architecture. This platform provided observability, fault-tolerance mechanisms, and seamless service discovery, streamlining the operational aspects and reducing the burden on individual teams. Another example is Shopify, which employs platform engineering to handle its vast infrastructure. Shopify’s internal platform, "Cloudbuddies," automates the deployment of services and manages Kubernetes clusters. This automation enables their developers to focus on building features and improving the customer experience, significantly enhancing productivity. According to Shopify’s engineering team, implementing platform engineering led to a 30% reduction in service downtime and improved developer efficiency by 40%. These statistics highlight the tangible benefits of moving beyond microservices toward a more integrated platform-centric approach. 30% reduction in service downtime 30% reduction in service downtime The Future: Why Platform Engineering Is Here to Stay The Future: Why Platform Engineering Is Here to Stay As the complexity of software systems continues to grow, the need for a sustainable and future-proof architecture becomes more critical. Microservices laid the groundwork for modular software development, but the operational overhead they introduced necessitated the evolution to platform engineering. By centralizing infrastructure management, standardizing processes, and leveraging tools like Kubernetes and Docker, platform engineering addresses the scalability, flexibility, and efficiency challenges that modern software faces. For companies looking to stay ahead in the digital landscape, adopting platform engineering is not just a trend but a strategic imperative for long-term success. What’s Next? What’s Next? The journey from monolithic to microservices architectures revolutionized software development, but platform engineering is the next logical step to address the challenges of modern software ecosystems. By building internal developer platforms that leverage tools like Kubernetes and Docker, companies can streamline their operations, enhance developer productivity, and create a future-proof software architecture. As the tech industry continues to evolve, platform engineering stands as the key to unlocking the full potential of microservices and beyond.

Beyond Microservices: Why Platform Engineering is the Key to Future-Proofing Software Architectures

Cryptographic Ransomware Encryption Detection: Survey: Detection of Encryption

Stories, News & Mentions about CLOUDBLUE

Cloudblue’s Stories on HackerNoon