Best match

Most recent

Classroom

Nostalgia

Classrooms have evolved over time as technology has advanced, leaving behind outdated tech that used to be essential tools for students and teachers.

https://www.bing.com/th?id=OVFT.z-Vi6nPtZhFiFUeMAz5o7y&pid=News

Technology

These Technological Relics Will Have You Nostalgic About Your Time In The Classroom

https://www.bing.com/th?id=ODF.cEtC7h8RK7I-UHn7jMddSA&pid=news

SlashGear

ThoughtWorks

Thoughtworks Holding Inc. (NASDAQ:TWKS) concluded the trading at $8.21 on Friday, June 02 with a rise of 4.32% from its closing price on previous day. Taking a look at stock we notice that its last check on previous day was $7.

Thoughtworks Holding Inc. (TWKS)’S 4.32% Increase Makes It Worth Considering Again

stocksregister

Information Age

Featuring 70+ speakers to share their knowledge and expertise on the changing face of the banking sector though innovation and technology developments, the 14th Edition of the Middle East Banking Innovation Summit (MEBIS 2023) once again reinforced itself as being the leading banking technology event in the Middle East region bringing together the shining minds of the industry!

https://www.bing.com/th?id=OVFT.xJcDuL8ln2qgcWCZALMygC&pid=News

MEBIS 2023 - Where the banking elite convened to deliberate on the challenges and opportunities of the digital age

https://www.bing.com/th?id=ODF.bZ09D360_Tp9_qP3MwPrrw&pid=news

Zawya

Revolution

Finance

Embedded system

Nima Montazeri, CPO of embedded finance solutions platform, Liberis, discusses how embedded finance and AI will revolutionise financial services The hype around artificial intelligence (AI) isn’t going to die down any time soon,

https://www.bing.com/th?id=OVFT.4EBlh4n3owXFd_QW2ujmri&pid=News

The marriage of embedded finance and AI

fintechmagazine

Headquartered in Jacksonville, Florida, Fidelity National Information Services, Inc. (NYSE:FIS) is a financial technology solutions provider. On May 16, 2023, Fidelity National Information ...

Do You Think Fidelity National Information Services (FIS)’s Current Valuation is Attractive?

Yahoo Finance

Information and ... as well as the various services and appliances with them such as video conferencing and distance learning. ICT also includes analog technology, such as paper communication ...

INFORMATION AND COMMUNICATION TECHNOLOGY(ICT

LinkedIn

Information technology

Software designed to detect child abuse images on smartphones, which could be mandated by governments in some counties, can be covertly repurposed to intrude on personal privacy, according to research at Imperial college. ‘Client-side scanning’ (CSS ...

Information Technology

Electronics Weekly

NASDAQ

Thursday

The best performing sector as of midday Thursday is the Technology & Communications ... BNK Invest Inc. provides investment services and information. BNK Invest owns and operates a market news ...

Thursday Sector Leaders: Technology & Communications, Services

Nasdaq

At the most recent Berkshire Hathaway Inc. (NYSE: BRK-B) investors conference in early May, Warren Buffett offered some insights into his most recent buys and sells, but more detail was released ...

Benzinga.com

Nigeria

“It will function primarily as a payment solutions provider (PSP), focusing on developing innovative technology solutions to enhance Stanbic IBTC’s existing financial services offerings.

Stanbic IBTC’s financial technology subsidiary starts operation

The Guardian Nigeria

Get the most recent info and news about ThoughtWorks on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

Thoughtworks Holding, Inc. provides technology consultancy services in North America, the Asia Pacific, Europe, and Latin America. The company's consultancy services integrate strategy, design, and software engineering to enable enterprises and technology disruptors to thrive as modern digital businesses. It offers enterprise modernization, platforms, and cloud services, including modernization strategy and execution, digital platform strategy and engineering organization transformation, cloud modernization, cloud-native application support and evolution, and platform assessment and remediation; and customer experience, product, and design services comprising idea to market, customer experience strategy, product management transformation, and product design and delivery. The company also provides data and artificial intelligence services, such as data strategy, intelligent products, data platforms and data mesh, continuous delivery for machine learning, and data governance; and digital transformation and operations services, which include delivery transformation, digital fluency, organization transformation, value-driven portfolio management, technology strategy, executive advisory, and digital foundations training. It serves its clients in various industry verticals, including technology and business services; energy, public, and health services; retail and consumer; financial services and insurance; and automotive, travel, and transportation. The company was founded in 1993 and is headquartered in Chicago, Illinois.

is a regular time to take a look at what experienced industry experts think might be the next wave of tools, practices, and technologies to consider or stop using. Thoughtworks’ Technology Radar I always look forward to adding many new “blips” to my list, or seeing which are growing in adoption since you last took a look. Version 27 is out now, and it continues some trends that the radar has been following for a while, such as “platform as a service” or forms of group programming. This round-up is far from comprehensive, but just the blips that appealed to me or that, I thought, might appeal to you. If any of them take your interest, or you have an opinion on anything I’ve featured, ! head over to your profile and let our readers know Mainstreaming of Machine Learning It really does feel like that in the past 6 months or so, machine learning went from something that mostly only developers and data scientists interacted with to something that the public had awareness of. There was announcement after announcement of another ML or AI-based tool that spat out some sort of media after you fed something into it, and we were swamped with thousands of strange-looking images and videos of celebrities. This also means that developers interested in experimenting with ML also have a lot more resources available to them in the shape of data sets and frameworks to use those data sets. Of relevance to experimenting developers is , a field of machine learning enabling a variety of always-on use-cases and targeting battery-operated devices. TinyML Another is , a method for creating ML pipelines that might be more familiar to developers from other backgrounds. feature stores This radar also covered ML-related tools for improving the developer experience of building and training models, for example, the practice of “federated machine learning” for aggregating model data across machines for processing or synthetic data for initial model testing. Continuous Observability Despite being a relatively new term, “observability” has rapidly grown from a loose collection of practices and standards for building a better picture of a running system to become a broader term to mean, well, to mean all sorts of things. The latest area of the developer workflow to receive insights is one that I am surprised took so long, observability for continuous integration and delivery pipelines. While CI and CD certainly bring benefits, processes frequently go wrong with very little insight into what until you try, try again. Now, a handful of existing observability vendors and open-source projects are bringing tracing-style visualizations of continuous processes to hopefully help you debug quite what was going on. Software Bill of Materials As developers lean increasingly on semi-anonymous containers and packages for their work, those focused on security and legalities looked on and wondered, “do we trust this?”. This concern has led to a rush to standardize and normalize (in enterprise, especially) a “software bill of materials”, often referred to by the comical acronym, SBOMs. While, in theory, “anyone” can dig into an open-source project and see what its components are, this is generally easier said than done and isn’t possible with closed-source projects at all. Recent high-profile vulnerabilities in software revealed how small dependencies can bring such large problems. While SBOMs could go a long way to help, I wonder how many companies will want to reveal their inner workings and be honest about them. Sustainable Coding This is a topic that I was inspired to cover myself after speaking with someone from Thoughtworks about their Carbon cloud footprint tool. The tendency to endlessly throw new frameworks, tools, and cloud frameworks at a problem has led to many of us forgetting that all of these lines of code actually run somewhere on something. And that leads to carbon emissions. Water motivated by altruistic reasons or financial, and an ecumenic downturn leads to cost savings, this is a topic more are talking and thinking about. Closely related to the environmental cost of projects and their infrastructure, and an option I’ve seen used by production teams is which estimates the cost impact of changes you make to Terraform definitions. infracost Responsibilities as Code Any SaaS company defines service-level indicators (SLIs) and service-level objectives (SLOs) that their clients can expect. However, how companies defined these have often been a bit haphazard and non-standard, and with so much else defined in codified pipelines, why not this fundamental component too? Several Observability providers now offer this option and of course, a group of people started a standard, . OpenSLO A No to Node I dabble in JavaScript and TypeScript and have noticed a handful of emerging runtime alternatives to Node recently. I am not sure what is wrong with Node, aside from being fairly old now, to warrant alternatives, but I guess competition is good 🤷‍♂️? is one newer alternative that I tried a few months ago, and I’ll admit I wasn’t really sure what benefits it brought me, but it uses WebKit’s JavaScriptCore instead of Chrome’s V8 engine. Bun Again, variety isn’t a bad thing. Bun is written in , which I had never heard of before, and it claims to be a “drop-in” C/C++ replacement. Zig Which leads nicely to… C+++ If Zig, Rust, and other languages are anything to go by, numerous people feel that C++ needs a more modern replacement. Google has thrown another option into the list of contenders with . Carbon The language is still in experimental stages but claims to be easier to understand and onboard with. Google has a tendency to spin up many experimental projects and then dump them again, so Thoughtworks’ suggestion is currently “wait and see”.

Developer Trends from Thoughtworks’ Technology Radar 27

Tendencias de desarrolladores del Radar de tecnología de Thoughtworks 27

Tendances des développeurs du radar technologique de Thoughtworks 27

थॉटवर्क्स के टेक्नोलॉजी रडार 27 से डेवलपर रुझान

ソートワークスのテクノロジー レーダー 27 からの開発者トレンド

Tendências de desenvolvedores do Technology Radar 27 da Thoughtworks

Xu hướng dành cho nhà phát triển từ Radar công nghệ của Thoughtworks 27

Thoughtworks 技术雷达的开发者趋势 27

A while ago released an for anyone to host their own personal radar. ThoughtWorks open source repository technology I was curious to see how the reputable company implemented it and strangely I noticed a lot of concepts that reminded me of Java, such as and error handling with . stateful objects try/catch/exception All the application really does is take a single input and produce a tech radar. A Java-esque approach seemed inappropriate for that use case so I decided to rewrite some of it in ! Elm Step 1: Model the radar Let’s start things off by thinking about how to model the radar itself. If we model our problem carefully it can simplify the work ahead of us. First let’s consider what we know about the data. The radar is split into and . These quadrants and rings represent different categories that several can be placed into. four quadrants four rings blips Since we know there are only four quadrants and rings let’s represent those as . union types So far so good. Next up, the blips. Modeling the blips requires some extra thinking because there are two representations of a blip. The first representation is the blip data from the Google Sheet ( ). The information included is the , , , an flag, and a . example name ring quadrant isNew description The blips that are visualized on the radar need those same 5 properties but in order to be displayed on the radar they also need an . x-y position Several ways to model these blips include: This option uses fewer types but will inevitably lead to more . Throw all properties into a single record and make the type of the position field a . Maybe maybe handling This option removes the but at the cost of in two separate record fields. Treat the blips from the Google Sheet as a distinct type and record from the blips visualized on the radar. maybe verbosity duplicating data Since the properties of the radar blip are a superset of the blips from the Google Sheet we can . A blip positioned on the radar will have the Google Sheet blip data in addition to a defined position. represent the data with composition There are other options worth considering, especially options that leverage union types, but the third option will suffice for now and we can refactor later if needed. Next up is visualizing the radar. If you’re curious on how we’re doing in comparison to ThoughtWorks’ open source example then swing on over to their OO equivalent modeling . Step 2: Visualize the radar Now that we have the basic building blocks to construct a view let’s start building the radar visualization. The first thing I want to do is get the quadrants and rings visualized. We’ll heavily use SVG and borrow a lot of the styles from the ThoughtWorks open source project. I’m sure they won’t mind. Creating an arc The radar isn’t a circle. It’s technically four quarter-circles put together, a.k.a arcs. Circles in SVG are not that hard. Arcs, on the other hand, suck big time. Here’s a demonstration of what I mean. With the above code working it will be exiled to a utility module so that we don’t have to think about the implementation details for creating an arc in SVG. Visualizing quadrants Visualizing a quadrant only requires that we visualize 4 arcs (rings), each with a different radius. Not bad. If we do this three more times and provide different numbers we get the entire radar background. Visualizing blips There are two types of blips that can be visualized which is determined by the flag. One is a rounded triangle and the other resembles a smudged circle. isNew I’m going to simply transcribe ThoughtWorks’ for these blip visualizations to Elm. code ..more SVG implementation details that we’re not going to look at ever again. Next up, visualizing mocked blips on the radar. Step 3: Wire up the view with mock data Before we consume actual data from a Google Sheet we’re going to mock up some data to make sure our blips are correctly displayed on the radar. The above function is but it will produce 16 blips on the radar, one in each ring in each quadrant. However, this brings up the question of where exactly to position the blips. temporary I went ahead and peeked at how . In short, they produce a sinusoidal curve that a blip is to be positioned on and if it collides with another blip it will try another random position on this curve. ThoughtWorks did it We can do the same thing in Elm but it brings up an interesting topic.. how to do randomness in a pure functional language. <quick detour> Pure randomness with generators In Elm, every function is a pure function. In order to produce randomness we can ask Elm to generate us a random value for a given seed. A random value and another seed is returned so that we can use that seed to ask for yet more random values. However, if we’re producing many random values then managing these seeds becomes extremely tedious and prone to mistakes. Fortunately, Elm gives us the concept of a . According to its documentation, Generator a generator is like a recipe for generating certain random values. Here’s an example generator that can be used to produce random integers from 3 to 7. Random.int 3 7 Here’s another generator for producing a random list of even values under 100. Simply creating a generator doesn’t actually produce a random value. It still has to be passed to . Random.step (randomEvens, nextSeed) =Random.step (generateEvenValues 10) (Random.initialSeed 123) Given this knowledge we can compose a few generators to randomly position blips along a sinusoidal curve and keep producing values if there is a collision. This will behave identically to the ThoughtWorks algorithm and it also happens to be where we convert the to . List GoogleSheetBlip List PositionedBlip Randomly positioned blips with Generators I realize we went from 0 to 60 with generators in about 90 seconds. If the above code is not immediately understandable, don’t worry. Like much with Elm, it is initially unfamiliar but it very quickly becomes second nature to read that code. </quick-detour> Blips visualized with randomness When we run the mock data at the beginning of this section into our view function we get a radar with blips on it. At this point we can keep throwing data at the radar and verify that the algorithm for determining blip positions is working. The positioning here seems reasonable. Moving on. If you’re tracking progress here’s how ThoughtWorks is doing the same . It’s also worth noting that the radar is mirrored along the y-axis because we hard coded the random seed. That’s fine. Step 4: Model the app state Just as before, modeling the app state is incredibly important. When writing your own Elm applications it’s worth pausing, discussing, and considering the trade-offs for how to model your data. For this application we already have a radar but we’re missing a page to enter a Google Sheet url. I chose to model that page state like this: It will have a form with an initially empty url. The form might have an error and we’ll want to know if we’re actively retrieving the Google Sheet. There are to this approach, but we’ll stick with these three properties for now. alternatives That covers what I’m calling the . Lastly, there’s the that needs to be provided to The Elm Architecture. Landing Page Model This is loosely based on except with a few things cut out for the sake of simplicity, such as not dealing with routes for such a small application. Richard Feldman’s SPA example It’s important to point out that at a high level this application is in one of two states: We’re on the landing page and interacting with the url input form We’re on the radar page interacting with a tech radar Step 5: User input, regex, data retrieval, and error handling The landing page involves , , , , and in all of those cases. I’m going to skip the user input part because if you made it this far you know how to do that already. receiving input from the user parsing out a Google Sheet ID performing an HTTP request parsing the response handling errors If I’m wrong and handling user input in Elm is new to you I suggest pausing and checking out this example . Regular expressions in Elm We’re about to go 0 to 60 again. Hang on. The user will supply us with a link to their published Google Sheet, such as “ ”. https://docs.google.com/spreadsheets/d/11Fd0lwNIEUs2ymxNEiTfpM5CoQHQbMuOId8TjrQHDn0/edit?usp=sharing Hidden within this URL is an ID that we need to perform an HTTP request. It’s that unreadable string between “/d/” and “/edit?”. First we’ll declare a regular expression that will find that ID. It goes without saying that credit for this regex goes to ThoughtWorks again. Within that regex are groups and we care about the first group specifically. In order to apply a regex to a string we use the function. find matches =find All sheetIdRegex url That returns a list of matches that take the following shape. Our Google Sheet ID is hidden within the list. So what we want to do is get the first match in the list of matches returned from , and on that first match find the first item in its . submatches find submatches If you’ve worked with Elm lists and before you’re probably already aware that we’re about to go head first into . Thanks to the code to pluck off that ID and produce a Result isn’t too bad. List.head MaybeLand elm-community/maybe-extra The first collapses the double Maybe that occurs when you do a on a List of Maybes. However, it’s still a Maybe returned from so the second collapses it one more time before converting it to a Result. MaybeExtra.join List.head Maybe.map MaybeExtra.join Retrieving the Google Sheet data Once we have the ID we can make a request to retrieve the Google Sheet data in CSV format. When this Cmd is performed it’ll call with the Result. httpResultToMsg The above function does the following On success, split the string by new lines Iterate over each row and attempt to parse the CSV as a blip Successfully parsing data puts it in the list while parse errors are added to the list. blips errors Send off a Msg with the blips and any errors Parsing each CSV row is as simple as attempting to map the data onto our data model defined earlier. Conclusion This post is getting long so I’ll wrap things up by making a few observations in rewriting some of the ThoughtWorks Tech Radar. The BYO radar project used for , for graphics and HTML templating, and for Google Sheets. It uses to model its data and incorporated rather than gracefully handling errors. chance randomness d3 tabletop parsing stateful object instances exception handling The code is fairly complicated and even with understanding the desired behavior it’s difficult to track the implementation details. Most of all, after spending so much time in that code I still don’t understand what much of the code does. On the other hand, a functional approach with Elm yielded more comprehensible code with similar functionality. In fact, once you grasp the way Elm does randomness and regular expressions you’ll realize the code is much more simple too. To see the partial rewrite in action check out these two links: http://ckoster22.github.io/elmradar/ https://github.com/ckoster22/techRadar/tree/blogpost

Rewriting the ThoughtWorks Tech Radar in Elm

How are you, hacker?
🪐 has got you covered with fresh content from our top 5 stories of the day, every day at noon your local time! Set email preference . What's happening in tech this week: The Noonification by HackerNoon here 4 Steps to Build a Web Analytics Measurement Plan By [ 6 Min read ]
If you have been using web analytics for a long time, then you know that it can hugely impact how you do digital marketing. @mikhailkirilin Read More. 4 Technology Developments for the End of 2022 and Christmas By [ 4 Min read ]
A look at four technology developments for the end of 2022 and Christmas. @gabrielmanga Read More. Its Time for Individual Carbon Credit Allowance By [ 5 Min read ]
Carbon Credits are defined as 1 ton of CO2 Emissions traded in the market to allow buyers to emit CO2 into the atmosphere. @m_muslimi Read More. Developer Trends from Thoughtworks’ Technology Radar 27 By [ 4 Min read ]
Thoughtworks’ Technology Radar is a regular time to take a look at what trends are changing our industry. @ChrisChinchilla Read More. Effective Techniques Against Disinformation By [ 6 Min read ]
We need to be aware of our news diet and critical thinking (being able to objectively follow news and change our minds if we come across relevant information). 🧑‍💻 What happened in your world this week?It's been said that , , . Feeling stuck? We got you covered ⬇️⬇️⬇️ We hope you enjoy this worth of free reading material. Feel free to forward this email to a nerdy friend who'll love you for it.See you on Planet Internet! With love,
The Hacker Noon Team ✌️ @roxanamurariu Read More. writing can help consolidate technical knowledge establish credibility and contribute to emerging community standards ANSWER THESE GREATEST INTERVIEW QUESTIONS OF ALL TIME

The Noonification: Effective Techniques Against Disinformation (11/15/2022)

Let's learn about via these 74 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Observability /Learn Repo 1. So, How are Observability and Monitoring Different, Actually? The similarities and differences between monitoring and observability and how to pair the two strategies 2. Observability Tips and Tricks For Using Grafana and Prometheus Rapid prototyping tips when using Grafana and Prometheus observability and monitoring stack. 3. Observability and Monitoring in a nutshell A guide to understanding the concepts and differences between monitoring and observability. 4. Debugging AWS Lambda Runtime Errors Debugging AWS Lambda can get tricky in Amazon CloudWatch. KloudMate lets you get to Lambda errors quickly and effectively using Lambda error logs. 5. An Intro to Dynamic CDK Dashboards How to create a construct based on other constructs in the same stack without forcing the developer to call a method for each? That's exactly what aspects do! 6. N/A please delete this 7. Developer Trends from Thoughtworks’ Technology Radar 27 Thoughtworks’ Technology Radar is a regular time to take a look at what trends are changing our industry. 8. How Observability and Monitoring Produces Better Software An article focused on deep diving into observability and its significance in software. Its history, goals, the importance of observability, and the issues that 9. How to Build Your Own Observability Platform on Kubernettes End-to-end full observability using Odigos and other back-end observability tools. 10. Maintaining Quality When Transitioning from Monolith to Microservices Piece by piece, legacy monolith applications are being broken down and replaced by microservices. 11. How to Monitor Apache Flink with OpenTelemetry Apache Flink monitoring support is now available in the open source OpenTelemetry collector. 12. Observability vs Monitoring: What's the Difference? Monitoring has been a basic system to track the health of servers for years. Now it is not enough. 13. Observability and Monitoring Have a Symbiotic Relationship, but They Are Different Observability vs. monitoring, what is the difference? Monitoring is the what to observability’s why. Here we dig into the differences. 14. Microservice.add(observability) != Microservice.add(monitoring) You are reading this content, which means that you are not novice to the microservices field. So let me just  scratch the surface of it before moving to Observable Microservices.  Once upon a time Monolith Application was now transformed into Microservices based application. 15. How Do You Make Cloud Migration Easier? Engage in intelligent change. 16. Application Monitoring: Closing Observability Gaps with Custom Metrics Which application metrics should you collect for your microservices? 17. The Challenges of Modern Observability It’s tough, impossible, to get a holistic view of your entire application when it’s running on 200 pods deployed to 12 nodes distributed across the world. 18. Why OpenTelemetry Should Matter to Network and Systems Admins OpenTelemetry is young, even by internet standards. Born out of the merger of OpenTracing and OpenCensus projects at the Cloud Native Computing Foundation CNCF 19. What is the ideal memory size to lower costs of running a task on Lambda? Should you increase Lambda memory? It might sound crazy, but increasing your AWS Lambda memory could actually lower your bills. Find out how. 20. What "Shifting Left" in Software Really Means for Blameless DevOps A truly "blameless culture" in software must to evolve from incident reporting, to telemetry aimed at Proactive Observability in DevOps 21. CI/CD/CF — The "Missing-Link" in the DevOps Toolchain? Everyone is familiar with CI and CD processes, but whatever happened to Continuous Feedback? Leveraging observability in dev creates a new type of dev process 22. Why Your Monitoring Dashboard May Be Feeding You Phantom Metrics We trust our metrics to show us the status of our system and where it misbehaves. But do our metrics show us what really happened? 23. Cloud Native Applications: 4 Building Blocks You Need in Your Platform To create amazing cloud native applications your platform needs these components: Kubernetes, Continuous Delivery Pipelines, Edge Stack, and Observability. This article will explain the importance of each component. 24. Comparing Different Serverless Monitoring Platforms Technology touches almost every corner of the world economy. Even when it’s an indirect relation, in many cases tech is an essential, vital part of our societies. It just can’t fail without causing too much distress and losses. Not only financially, but especially to the human aspect. 25. Get Started With Sidekick Open Source Live Debugger in 5 Mins Sidekick is a live application debugger that lets you troubleshoot your applications while they keep on running. Here is how you can start using it in 5 minutes 26. Logging in Observability - Part 1 rsyslog vs journald vs filebeat 27. Managing Collectors on K8 – Why We Chose the OpenTelemetry Collector for Odigos Without proper implementation and continuous configuration of your collectors, observability tools will be limited at best at best and many times ineffectual. 28. On Dynamic Observability and Team Culture with Liran Haimovitch, Rookout CTO Rookout Co-Founder and CTO, Liran Haimovitch, shares the origin story of their debugging tool, what excites him about the startup life, PLG, and more. 29. 9 Data Trends You’ll See in 2023 2022 saw the data space grow by leaps and bounds. Here are the top 9 things our team of data experts expects to see in 2023. 30. Observability and AI Partner Well to Tackle All of That Data Gone are the days when operations and development were a back-office affair. 31. Logging in Observability - Part 2 In this article, I want to describe how to work with logs, analyze them in the command line, and we will consider new modern tools to visualize logs. 32. Tracing: An Essential Component of Observability In this article, I am going to shed light on one important aspect of observability - tracing. 33. Microservice Observability Patterns [Part 1] Logging is one of the most important parts of software systems. Whether you have just started working on a new piece of software, or your system is running in a large scale production environment, you’ll always find yourself seeking help from log files. Logs are the first thing people look for when something goes wrong, or something doesn’t work as expected. 34. What is Observability? Is it Cultural? In order to leverage observability we need a significant shift in our corporate culture that encapsulates the entire company & goes beyond the tools. 35. Murre - the lightweight K8s metrics monitoring tool 36. Introductory Guide to Debugging Lambda Configuration Errors The “Lambda configuration error” is as generic as it gets but at the end of the day, it's a pathing issue. Let's go over how to resolve this. 37. Contextual Intelligence and Observability: Without the Former, You Really Don’t Have the Latter Observability is a hot term in the industry, but don’t let it fool you: having visibility into your organization's apps and services only gives you a piece. 38. The Everything Guide to Data Collection in DevSecOps Collecting wide swaths of observability and security data is key to a high-quality digital experience. Find out what you need to know to get started. 39. Better Performance and Security by Monitoring Logs, Metrics, and More Monitoring is a crucial part of observability. Learn how monitoring can specifically improve security, performance, and reliability. 40. Goodbye, Cost-depth Trade Off: Cloud-Native Observability, Redefined It turns out that, by rethinking the fundamentals of your approach to monitoring, you can have it all 41. Prometheus — Easy In, Difficult Out Setting up a Prometheus server can be easy, but getting actionable information out is a very long and never-ending journey. 42. Choosing a Computing Method: a Serverless SWOT Analysis If you ever find yourself deciding for or against serverless the following tries to make the decision easier for you. 43. Setting Up AWS CloudWatch Alerts (vs Dashbird Alerts) To Monitor Your Applications Learn about the best practices for AWS Cloudwatch Alerts and Dashbird Alarms, to not miss out on critical info about your serverless app. 44. Exploring Differences Between Monitoring And Observability Monitoring vs Observability: in this article, we're explaining what is observability exactly and how does it differ from monitoring. 45. Kuma and Prometheus for Observability in Kubernetes Microservices Clusters A year ago, Harry Bagdi wrote an amazingly helpful blog post (link at bottom of article) on observability for microservices. And by comparing titles, it becomes obvious that my blog post draws inspiration from his work. 46. To Log Or Not To Log, Is The (Eternal) Question An alternative logging strategy to make loggers your friends, not enemies 47. 2023 Observability Predictions: Open Source is the Key to Explosive Growth The observability industry is projected to grow about 8% in 2023, a continuation of a strong growth trend since the 90’s. 48. How Goji Investments Enhances Developer Experience via Observability This post was written by Dean Record, Engineer at Goji Investments. 49. What is API Observability? APIs are the blood vessels of digital business. 50. A Guide to Deploying Jaeger on Kubernetes in Production Logs, metrics, and traces are the three pillars of the Observability world. The distributed tracing world, in particular, has seen a lot of innovation in recent months, with OpenTelemetry standardization and with Jaeger open source project graduating from the CNCF incubation. 51. How To Use a FastAPI Abstraction To Query AWS Data Lake Our guide in creating FastAPI abstraction that allows us to query the AWS Glue metadata catalog - a Data Discovery API for AWS Data Lake. 52. Sizing Your Cloud Resources: 10 Common Mistakes to Avoid Learn the common pitfalls of moving to the cloud and how you can avoid them to truly benefit from the cloud’s elasticity. 53. CNCF Tools Overview: Are You Cloud-Native? Cloud computing is becoming more and more of a household name, with even the most conservative fields of business figuring out how to make the best use of it. Cloud computing usually starts with running a private cloud solution on premises before venturing onto the public cloud. Of course, the cloud is not a single uniform being. It may come from different providers, Amazon Web Services, Google Cloud Platform, and Microsoft Azure being the biggest players here. Or it may come with different visibility and hosting, that is, public (resides with the provider), private (self-hosted), or hybrid (which uses a bit of both). And the cloud can use different tools and APIs for management as well. 54. Microservice Observability Patterns [Part 2] In my previous article, I talked about the importance of logs and the differences between structured and unstructured logging. Logs are easy to integrate into your application and provide the ability to represent any type of data in the form of strings. 55. Improving Observability Through Service Level Metrics 56. Observability on Heroku: How to Monitor Apps on a Managed Infrastructure Cloud platforms like Heroku make it easier than ever to host applications: just upload your code, and they’ll deploy it for you. But a common misconception is that, because you don’t own the infrastructure, you can’t really monitor your applications or see under the hood. 57. Setting Up Notification Systems to Observe and Analyze Your Application By building observability and analytics into your notification system, you can identify and quickly resolve issues by monitoring how your product is performing. 58. Better Failure Detection in Serverless Apps Having reliable failure detection in place is a must for any application in production. Here's how to best achieve it for AWS Lambda. 59. 4 Key Observability Metrics for Distributed Applications A common architectural design pattern these days is to break up an application monolith into smaller microservices. Each microservice is then responsible for a specific aspect or feature of your app. For example, one microservice might be responsible for serving external API requests, while another might handle data fetching for your frontend. 60. Lua-Land CPU Flame Graphs in OpenResty XRay This post will introduce the idea of Lua-land CPU flame graphs and use OpenResty XRay to produce real flame graphs for several small and standalone Lua examples 61. 5 Key Takeaways from New Relic’s 2022 State of Logs Report The 2022 State of Logs Report provides a detailed view of how this practice is shaping engineering and the technologies of the future. 62. Introduction to Observability in ITOM and AIOps Observability is a best practice implemented by AIOps, enabling automation and expanding visibility into the entire organizational ecosystem. 63. Internal AWS Monitoring is Hard to Grasp Learm how a serverless monitoring solution can catch problems for you without the painful learning curve connected to serverless failure detection. 64. Jaeger Persistent Storage with Elasticsearch, Cassandra and Kafka Running systems in production involve requirements for high availability, resilience and recovery from failure. When running cloud-native applications this becomes even more critical, as the base assumption in such environments is that compute nodes will suffer outages, Kubernetes nodes will go down and microservices instances are likely to fail, yet the service is expected to remain up and running. 65. How to Monitor Serverless Applications With AWS CloudWatch Alarms Running any application in production assumes reliable monitoring to be in place and serverless applications are no exception. 66. How to Optimize Large Scale Serverless Applications for Operational Excellence Managing applications at scale often comes up as one of the biggest concerns for businesses; How can it work smoothly? How do we monitor so many resources? How do we maintain best practices with constantly evolving infrastructure? In this article, we run through the best approach for operational excellence looking at serverless monitoring strategy, serverless alerting strategy, and security and compliance best practices. 67. Fixing The ClickHouse Node Failure On Distributed Systems - A How-To Guide Part One: ClickHouse Failures, by Marcel Birkner 68. Introduction To Distributed Tracing Pattern A distributed architecture brings in several challenges when it comes to operability and monitoring. Here, one may be dealing with tens if not hundreds of microservices, each of which may or may not have been built by the same team. 69. 10 Threats to an Open API Ecosystem Despite tight economic situations worldwide, the API economy continues to grow. 70. Data Access for Microservices If you want to access data in a distributed environment such as in a microservice architecture, then data services are the way to go. The idea is to create a data abstraction layer (DAL) that the rest of the system’s applications and services can share. Thus, a data service gives you a generalized interface to the data you’re exposing and provides access to it in a standard manner. This would be in a well-understood protocol and a known data format. For example, a popular approach is to use JSON via HTTP/S. 71. Rethinking Programming: Automated Observability Introduction 72. Watching the Requests Go By: Reconstructing an API Spec with APIClarity In this post, we’re going to look at what API reconstruction is and how APIClarity solves the API observability problem. 73. Navigate Your Way to Production Bliss with Caretta The easiest way to map our cluster: Caretta - a standalone OSS tool, leveraging eBPF technology to be lightweight and frictionless 74. The Importance of Monitoring Big Data Analytics Pipelines In this article, we first explain the requirements for monitoring your big data analytics pipeline and then we go into the key aspects that you need to consider to build a system that provides holistic observability. Thank you for checking out the 74 most read stories about Observability on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

74 Stories To Learn About Observability

86 Stories To Learn About C++

140 Stories To Learn About Agile Software Development

Automated tests can be run on multiple environments, multiple browsers, multiple devices and multiple data sets in parallel. These tests are less prone to human error than manual tests, which can help improve the accuracy and reliability of the testing process. Furthermore, they are repeatable, consistent and reliable which helps in reducing false positives and false negatives. In this article, I’ll talk about why you should be using automated tests in Selenium, JUnit5 and Gauge Framework. Increases Efficiency Automation tests can be run faster than manual tests, allowing more tests to be run in a shorter time period. This can help identify issues and bugs more quickly, saving time and resources in the long run. Automation tests can run on multiple environments, multiple browsers, multiple devices and multiple data sets in parallel which increases the overall efficiency of the testing process. import com.thoughtworks.gauge.Step
import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver

class AutomationTest {
private lateinit var driver: WebDriver
@Step("Open the browser and navigate to the website")
fun openBrowser() {
    System.setProperty("webdriver.chrome.driver", "/path/to/chrome/driver")
    driver = ChromeDriver()
    driver.get("https://www.example.com")
}

@Step("Search for <searchTerm>")
fun search(searchTerm: String) {
    driver.findElement(By.name("q")).sendKeys(searchTerm)
    driver.findElement(By.name("btnK")).click()
}

@Step("Close the browser")
fun closeBrowser() {
    driver.quit()
  }
} This is an example of a simple test automation script written in which is using the library to interact with a web page, and the to organize and run the tests. This script opens a browser, navigates to a website, performs a search, and then closes the browser. Kotlin Selenium Gauge framework Because this test is automated, it can run faster than if it were performed manually. This is just a simple example, in a real-world use-case, you can have multiple scenarios, multiple test cases, and multiple test suites which can be automated and run in parallel, increasing the overall efficiency of the testing process. It's worth noting that, if you are working with a web application, there are other frameworks and tools that can be used in conjunction with or instead of Selenium, such as Cypress, WebDriverIO, and TestCafe, each one having its own specific features and advantages. Improves Accuracy Automated tests are less prone to human error than manual tests, which can help improve the accuracy and reliability of the testing process. Automated tests are repeatable, consistent and reliable which helps in reducing false positives and false negatives. import com.thoughtworks.gauge.Step
import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver

class AutomationTest {
private lateinit var driver: WebDriver

@Step("Open the browser and navigate to the website")
fun openBrowser() {
    System.setProperty("webdriver.chrome.driver", "/path/to/chrome/driver")
    driver = ChromeDriver()
    driver.get("https://www.example.com")
}

@Step("Search for <searchTerm> and verify the result")
fun search(searchTerm: String) {
    driver.findElement(By.name("q")).sendKeys(searchTerm)
    driver.findElement(By.name("btnK")).click()
    val searchResult = driver.findElement(By.xpath("//div[@class='g']"))
    assert(searchResult.text.contains(searchTerm))
}

@Step("Close the browser")
fun closeBrowser() {
    driver.quit()
  }
} In this example, the script not only performs a search, but also verifies the results by asserting that the returned search results contain the search term. This helps ensure the test is accurate and will catch any issues that may occur with the search functionality. Because the script is automated, it will run the same way every time it is executed with the same inputs, ensuring the test is repeatable, consistent and reliable. This helps in reducing false positives and false negatives, which can occur with manual testing. Additionally, you can use test reporting tools like Gauge report, Allure, TestNG, JUnit, etc. to track test results, allowing you to see if a test has failed or passed. If it has failed, it will reveal the reason behind the failure, which helps in identifying issues early in the development process and prevents them from becoming more serious problems later on. 3. Enables Frequent Testing Automated tests can be run on a regular basis, such as every time code changes are made, which can help identify issues early in the development process and prevent them from becoming more serious problems later on. This aids in catching bugs early in the development cycle and helps in reducing the overall cost of fixing bugs. import com.thoughtworks.gauge.Step
import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.BeforeEach

class AutomationTest {
private lateinit var driver: WebDriver
@BeforeEach
fun openBrowser() {
    System.setProperty("webdriver.chrome.driver", "/path/to/chrome/driver")
    driver = ChromeDriver()
    driver.get("https://www.example.com")
}

@Test
fun testSearch() {
    driver.findElement(By.name("q")).sendKeys("searchTerm")
    driver.findElement(By.name("btnK")).click()
    val searchResult = driver.findElement(By.xpath("//div[@class='g']"))
    assert(searchResult.text.contains("searchTerm"))
}

@AfterEach
fun closeBrowser() {
    driver.quit()
  }
} This script uses JUnit5 framework and the annotation, which marks a method as a test method. This allows the test to be run automatically as part of a test suite. Additionally, the script uses the and annotations that marks a method to be executed before or after each test method. @Test @BeforeEach @AfterEach You can also use a CI/CD pipeline and integrate the automation test with it, allowing the tests to be run automatically as part of a build process and with each code change. This helps in ensuring that the code changes are tested and are of good quality before being deployed to the production. 4. Cost-effective Automated tests can run without human intervention, which reduces the cost of testing over time. Automated tests can be run on a regular basis and can be scheduled to run at a specific time which reduces the need for human resources. import com.thoughtworks.gauge.Step
import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.BeforeEach

class AutomationTest {
private lateinit var driver: WebDriver

@BeforeEach
fun openBrowser() {
    System.setProperty("webdriver.chrome.driver", "/path/to/chrome/driver")
    driver = ChromeDriver()
    driver.get("https://www.example.com")
}

@Test
fun testSearch() {
    driver.findElement(By.name("q")).sendKeys("searchTerm")
    driver.findElement(By.name("btnK")).click()
    val searchResult = driver.findElement(By.xpath("//div[@class='g']"))
    assert(searchResult.text.contains("searchTerm"))
}

@Test
fun testSignUp() {
    driver.findElement(By.linkText("Sign Up")).click()
    driver.findElement(By.name("username")).sendKeys("myusername")
    driver.findElement(By.name("password")).sendKeys("mypassword")
    driver.findElement(By.name("submit")).click()
    val message = driver.findElement(By.xpath("//div[@class='message']"))
    assert(message.text.contains("Welcome myusername"))
}

@AfterEach
fun closeBrowser() {
    driver.quit()
  }
} In this example, we have two test cases. The first one is for testing search functionality and the second one is for testing the sign-up functionality. By having multiple test cases in a single script, it increases the overall efficiency of the testing process, which can help reduce the overall cost of testing. Additionally, you can use test parameterization where you can run the same test case with multiple inputs and data sets. This increases overall test coverage and helps find more bugs,  reducing the overall cost of testing. Further, you can use cloud-based test environments such as SauceLabs, BrowserStack, and TestingBot, allowing you to run your tests on a variety of browsers and operating systems without having to maintain your own test infrastructure. This can reduce the costs associated with maintaining and scaling test infrastructure. Overall, automated tests can be cost-effective by reducing the need for human resources, increasing the efficiency of the testing process, and by using cloud-based test environments. 5. Increases Coverage Automated tests can cover a wide range of scenarios, inputs and use cases which are difficult to cover manually, thus increasing the overall coverage of the testing. Automated tests can cover different browsers, different devices, different versions of the operating system, different data sets and different scenarios which increases the overall coverage of the testing. import com.thoughtworks.gauge.Step
import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.CsvSource

class AutomationTest {
private lateinit var driver: WebDriver

@BeforeEach
fun openBrowser() {
    System.setProperty("webdriver.chrome.driver", "/path/to/chrome/driver")
    driver = ChromeDriver()
    driver.get("https://www.example.com")
}

@ParameterizedTest
@CsvSource(value = ["searchTerm1, expectedResult1", "searchTerm2, expectedResult2", "searchTerm3, expectedResult3"])
fun testSearch(searchTerm: String, expectedResult: String) {
    driver.findElement(By.name("q")).sendKeys(searchTerm)
    driver.findElement(By.name("btnK")).click()
    val searchResult = driver.findElement(By.xpath("//div[@class='g']"))
    assert(searchResult.text.contains(expectedResult))
}

@AfterEach
fun closeBrowser() {
    driver.quit()
  }
} In this example, we are using the JUnit5 annotation and to run the same test case with multiple inputs, increasing the overall test coverage. The test will run three times with different input values, and each time it will check if the output matches the expected result. @ParameterizedTest @CsvSource This way you can test the same functionality with multiple data sets, which helps in finding more bugs and increases the overall test coverage. Additionally, you can also use data-driven testing frameworks like TestNG, JUnit5, etc. to run the test cases with multiple inputs,  increasing the overall test coverage. Automated tests run in the same way every time, ensuring the same issues are not missed repeatedly. Automated tests are repeatable, consistent and reliable which reduces false positives and false negatives. 6. Improves Consistency import com.thoughtworks.gauge.Step
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.Response

class API_AutomationTest {

@Step("GET request to <endpoint> and verify the response")
fun testAPI(endpoint: String) {
    val client = OkHttpClient()
    val request = Request.Builder()
        .url(endpoint)
        .get()
        .build()
    val response = client.newCall(request).execute()
    val json = response.body()?.string()
    assert(json!!.contains("\"userId\": 1"))
  }
} This script uses the Gauge framework and the annotation, which marks a method as a step in the test scenario. This allows the test scenario to be written in a readable and understandable format, making the flow of the test easy to understand. @Step You can also use the gauge's concept of data-driven testing, where you can run the same test scenario with multiple inputs, increasing the overall test coverage and potentially finding more bugs. Additionally, you can use the gauge's reporting capability, which provides a clear and concise report of the test scenarios, test results and the test execution time. 7. Enables Continuous Integration and Continuous Delivery Automated testing is a key enabler for continuous integration and delivery, which is a critical aspect of modern software development. Automated tests can be integrated with the CI/CD pipeline and can be run automatically on every code change, ensuring the code changes are tested and are of good quality before being deployed to the production. You can use a CI/CD pipeline and integrate the script with it, allowing the tests to be run automatically as part of a build process, and with each code change. For example, you can use Jenkins, Travis, CircleCI, etc. as a CI/CD tool. Then you can set up the job to build the project, run the test cases and deploy the application to the production if the tests pass. This is a critical aspect of modern software development, and it helps in achieving a faster and more reliable software delivery. pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh './gradlew build'
            }
        }
        stage('Test') {
            steps {
                sh 'gauge run specs/'
            }
        }
        stage('Deploy') {
            steps {
                sh './deploy.sh'
            }
        }
    }
} This Jenkinsfile defines a pipeline with three stages: Build, Test and Deploy. In the , the pipeline runs the command './gradlew build'. This command will build the project and generate the necessary artifacts. Build stage In the , the pipeline runs the command 'gauge run specs/'. This command will execute all the gauge test cases in the specs folder. Test stage In the , the pipeline runs the command './deploy.sh'. This command will deploy the application to the production environment. Deploy stage You can set up the Jenkins job to run this Jenkinsfile and it will build, test and deploy the application if the tests pass. With this example, you can see how you can use the Gauge tests in a Jenkinsfile to enable continuous integration and continuous delivery. This allows you to automate the testing process. Final Thoughts Increases efficiency by allowing multiple test cases to be run in a single script. Improves accuracy by reducing human error and providing consistent results. Enables frequent testing by making it easy to run tests on a regular basis. Is cost effective by reducing the overall cost of testing and increasing test coverage. Increases coverage by allowing the same test case to be run with multiple inputs and data sets. Improves consistency by ensuring that the software is working as expected. Enables Continuous integration and Continuous delivery by allowing the tests to be run automatically as part of a build process, and with each code change, ensuring the code changes are tested and are of good quality before being deployed to the production. Overall, automation testing helps in improving the overall quality of the software, reducing the overall cost of testing, increasing the test coverage, improving consistency and enabling continuous integration and continuous delivery which results in faster and more reliable software delivery.

7 Reasons to Use Automated Tests in Selenium, JUnit5, and Gauge Framework 

7 razones para usar pruebas automatizadas en Selenium, JUnit5 y Gauge Framework

7 raisons d'utiliser des tests automatisés dans Selenium, JUnit5 et Gauge Framework

सेलेनियम, JUnit5 और गेज फ्रेमवर्क में स्वचालित परीक्षण का उपयोग करने के 7 कारण

Selenium、JUnit5、Gauge Framework で自動テストを使用する 7 つの理由

7 razões para usar testes automatizados em Selenium, JUnit5 e Gauge Framework

7 lý do nên sử dụng Kiểm thử tự động trong Selenium, JUnit5 và Gauge Framework

在 Selenium、JUnit5 和 Gauge Framework 中使用自动化测试的 7 个理由

Tech is both the future and the road that will get us there. Whether it’s an adjacent digital reality- that integrates seamlessly with a tangible one- or an app that tracks your heart rate, we depend more and more on innovative solutions to problems we didn’t even know we had. The 2022 Noonies’ Award Category, presented by ,  puts the spotlight on the builders that improve our lives on a daily basis, the creative minds that help build these solutions, and by doing so light our path to an exciting future. Programming Slogging Here are the stats: 4,743 nominees. 371 Hacker Noon awards for 2022 Across 5 award categories: Web3, Emerging Tech, Gaming, Internet Heroes, and Programming. And a monumental 143,156 votes cast over at noonies.tech To see the winners simply visit , pick an award, and discover which name appears at the top. That’s the winner! noonies.tech The category alone is announcing 110 winners. Let’s congratulate the winners! Programming HackerNoon Contributor of the Year- JAVA is a Software Engineer and published author currently working as a Developer Advocate at MariaDB Corporation. Alejandro Duarte HackerNoon Contributor of the Year- KUBERNETES Aside from being a Content Strategist at Valasys Media, is  a Psychic, Tarot Reader and Violet Flame Alchemist. Priya Kumari HackerNoon Contributor of the Year- CODING is a Technical Leader with a hobby of competitive programming. Ruslan Rakhmedov HackerNoon Contributor of the Year- SQL is a Software Engineer and published author currently working as a Developer Advocate at MariaDB Corporation. Alejandro Duarte HackerNoon Contributor of the Year- DEVOPS Aside from being a Content Strategist at Valasys Media, is  a Psychic, Tarot Reader and Violet Flame Alchemist. Priya Kumari HackerNoon Contributor of the Year- CODING-SKILLS is a Senior Ruby on Rails Developer with 2+ YoE in developing e-commerce, travel, and blockchain projects. Alexey Shepelev HackerNoon Contributor of the Year- DATABASE is a Software Engineer and published author currently working as a Developer Advocate at MariaDB Corporation. Alejandro Duarte HackerNoon Contributor of the Year- CICD has 9+ years of professional experience as a Solutions architect, Tech lead, Cloud Architect and in DevOps. Viachaslau Matsukevich HackerNoon Contributor of the Year- CLOUD-COMPUTING Congratulations to , who loves coding in C# and Cloud Computing. Uladzislau Baryshchyk HackerNoon Contributor of the Year- AWS Congratulations to once again! Uladzislau Baryshchyk HackerNoon Contributor of the Year- MICROSERVICES brings over 19 years of experience in Software Development and a University degree in Computer Science. Oleg Melnic HackerNoon Contributor of the Year- GOLANG is a seasoned Software engineer Skilled in Go, Java, and JavaScript. Yuda Prasetiya Haqqy HackerNoon Contributor of the Year- CSS is a backend software engineer at Snap Inc,  interested in distributed systems, web security, and building cool things. Stanley Lim HackerNoon Contributor of the Year- PROGRAMMING is an Experienced Software Quality Assurance Tester at Libertex Group. Sergei Shaikin Coding Guru of 2022 is a Senior Software Engineer at Layermark. Alexander Makeev HackerNoon Contributor of the Year- MONITORING is a backend software engineer at Snap Inc,  interested in distributed systems, web security, and building cool things. Stanley Lim HackerNoon Contributor of the Year- GITHUB is a Software engineer with 11 years of experience focused on fault tolerant, distributed systems, PaaS, golang, HA. Anton Gubarev HackerNoon Contributor of the Year- DOCKER is a Software Engineer and published author, winning this year Noonies! Alejandro Duarte Tech Tutorial Teacher of the Year will solve all of your coding issues! Code basics HackerNoon Contributor of the Year- REACT is a Skilled Front-End engineer with 7 years of experience in developing Web and SmartTV applications. Yan Tsishko HackerNoon Contributor of the Year- FLUTTER is the Chief Technology Officer at Estatic Infotech Pvt. Ltd. Kiran Beladiya HackerNoon Contributor of the Year- API is an amazing data engineer and rockstar python teacher, winning this year's Noonies. Veronika Vasileva HackerNoon Contributor of the Year- CLEAN-CODE is a Senior software engineer & agile lover who specializes in declarative designs and S.O.L.I.D. Maximiliano Contieri HackerNoon Contributor of the Year- HOW-TO is a Senior Software Engineer at Akma Trading. Vladislav Gukasov HackerNoon Contributor of the Year- FUNCTIONAL-PROGRAMMING is a Staff Software Engineer at Lab49 with large hands-on experience on efficiently coding web apps. Georgii Kliukovkin Software Developer of The Year Asides managing Developer Experience & Relations at StepZen, is a Tech Author, Speaker and Entrepreneur. Roy Derks HackerNoon Contributor of the Year- SOFTWARE-ARCHITECTURE is a Senior Software Engineer with 7+ YoE building massively scalable systems both from scratch and diving into a codebase. Sergey Golitsyn HackerNoon Contributor of the Year- KOTLIN is a Senior Software Engineer at Ultimeta with over 10+ years of experience in software development with Java, Kotlin, and Spring. Fedor Yaremenko Top Programming Guru is a Principal Product Engineer at Esri. Hussein Nasser HackerNoon Contributor of the Year- JAVASCRIPT is a Frontend Developer and Technical Writer. Amarachi Iheanacho HackerNoon Contributor of the Year- GIT is a Software Engineer and published author currently working as a Developer Advocate at MariaDB Corporation. Alejandro Duarte iOS Writer of the Year Congratulations to , the Chief Technology Officer at Estatic Infotech Pvt. Ltd. Kiran Beladiya HackerNoon Contributor of the Year- DATA-SCIENCE Congratulations to , an avid data science writer currently studying Computer Science at Penn State University. Max Vidavsky HackerNoon Contributor of the Year- PRODUCT-DEVELOPMENT Congratulations to , the Chief Technology Officer at Estatic Infotech Pvt. Ltd. Kiran Beladiya Android Advocate of The Year Congratulations to once again!. Kiran Beladiya HackerNoon Contributor of the Year- SOFTWARE-TESTING Congrats to , a software engineer with a love for backend testing! Sergei Shaikin HackerNoon Contributor of the Year- MONGODB , Senior Ruby on Rails Developer, wins this award! Alexey Shepelev HackerNoon Contributor of the Year- LINUX The winner is , on his way to enlightenment! Aleksandr Tabakov HackerNoon Contributor of the Year- TESTING Congratulations to , Head of Product Marketing at Livecycle! Zevi Reinitz HackerNoon Contributor of the Year- DATA The award goes to… l, a Data Scientist, applied scientist, research consultant, freelancer and startup builder. Shaurya Uppa HackerNoon Contributor of the Year- DOTNET is the winner this year! Rahul Rai HackerNoon Contributor of the Year- DEBUGGING Congratulations to ! He is a professional fullstack software engineer, currently working for REVOLUT as a software backend engineer. Artem Sutulov HackerNoon Contributor of the Year- AUTHENTICATION is the winner! Rodrigo Kamada HackerNoon Contributor of the Year- FRONTEND Congratulations to , Frontend Developer And Technical Writer. Amarachi Iheanacho HackerNoon Contributor of the Year- AZURE Congratulations to , a solutions architect with expertise in public cloud, kubernetes, PAAS and CI\CD. Viachaslau Matsukevich HackerNoon Contributor of the Year- BACKEND wins his second award! Artem Sutulov HackerNoon Contributor of the Year- SERVERLESS is a solutions architect with expertise in public cloud, kubernetes, PAAS and CI\CD. Viachaslau Matsukevich HackerNoon Contributor of the Year- QUALITY-ASSURANCE is on fire! Congratulations! Ruslan Muradov HackerNoon Contributor of the Year- COMPUTER-SCIENCE Congratulations to ! Rishabh Agarwal HackerNoon Contributor of the Year- HEROKU , senior python developer, wins this award. Maksim Kuznetsov HackerNoon Contributor of the Year- REST-API Congratulations to ! Alexey Shepelev HackerNoon Contributor of the Year- NPM Congratulations to , Web/Blockchain Engineer, Technical Writer. Idris Abdul-Lateef HackerNoon Contributor of the Year- PERFORMANCE Congratulations to , a Senior Software Engineer. Yura Abharian HackerNoon Contributor of the Year- LOW-CODE , a passionate software engineer, wins this award. Aleksandr Zakharov HackerNoon Contributor of the Year- ANGULAR is the winner! Rodrigo Kamada HackerNoon Contributor of the Year- AGILE Congratulations to , a Software Engineering Manager at UTMOST, a Devops Advocate, passionate about Startups and continuous improvement pursuer Alessandro Diaferia HackerNoon Contributor of the Year- AGILE-SOFTWARE-DEVELOPMENT Congratulations to once again! Alessandro Diaferia Open Source Writer of the Year ’s motto is "EVERY DAY IS A LEARNING DAY". She is a Growing Developer, Python and Open-Source and a Tech Enthusiasts & Writer. Hillary Nyakundi Data Science Demon Congratulations to , a Data Scientist, a Problem Solving Expert and a Research Consultant. Shaurya Uppal HackerNoon Contributor of the Year- WEBSITE-DEVELOPMENT brings home another award! Rodrigo Kamada HackerNoon Contributor of the Year- OPTIMIZATION , a Data Engineer and a Python Tutor wins! Nikita Vasilev Master on the Edge of Computing Congratulations to for another win! Viachaslau Matsukevich HackerNoon Contributor of the Year- SOFTWARE-ENGINEERING Congratulations to ! Artem Sutulov HackerNoon Contributor of the Year- SCALING wins! Congratulations! Ralph Benko HackerNoon Contributor of the Year- CONTAINERS has been a pioneer in video software development since 2005. Congrats! Fora Soft HackerNoon Contributor of the Year- TYPESCRIPT , currently developing and enjoying life, and meanwhile winning Noonies. Anatolii Kabanov Rawr for ROR , a 6+ years Java Developer, crashed this award! Alexander Makeev HackerNoon Contributor of the Year- SWIFT ,  a senior IOS Software Engineer at BestDoctor has won the HackerNoon Contributor of the Year- Swift. Swift is designed to function as the intuitive language for ios, ipados, macos, watchos, and tvos. Pavel Plotnikov Most Authentic Developer Advocate of The Year has your back! Dev Advocate at DraftBit and ReactNative & Expo Enthusiast and now, the most authentic dev advocate of the year. Aman Mittal HackerNoon Contributor of the Year- NEURAL-NETWORKS The crux of deep learning algorithms: Neural Networks is taken over by , a researcher / engineer with 10+ YoE in deep learning, computer vision, and autonomous driving. Dmitrii Khizbullin HackerNoon Contributor of the Year- HTML Build better websites with , a Senior Ruby on Rails Developer who has won the HTML Contributor of the Year at HackerNoon. Alexey Shepelev HackerNoon Contributor of the Year- HACKING wins the HackerNoon Contributor of the Year: Hacking. They write tutorials on Cybersecurity and Bug Bounty on our website and HackerNoon. Kali Linux Tutorials HackerNoon Contributor of the Year- INFRASTRUCTURE Build better websites with , a Senior Ruby on Rails Developer who has won the HTML Contributor of the Year at HackerNoon. Alexey Shepelev HackerNoon Contributor of the Year- REDUX takes the award for Contributor of the Year- Redux. Redux is an open source javascript library, commonly used with libraries like react or angular for building user interfaces. Anatolii Kabanov No No No NodeJs wins their second title for NodeJs. Congratulations X2! Anatolii Kabanov HackerNoon Contributor of the Year- ENCRYPTION is an eater of bitcoin stories, spinner of blockchain tales, and lastly the winner of the HackerNoon Contributor of the Year- Encryption. StevieSats HackerNoon Contributor of the Year- REFACTORING This award is for the highest quality refactoring stories on hackernoon and it goes to . Abbey is a full-stack web developer, crafter, blogger, cosplayer, fiber artist, yoga teacher, and gamer. Abbey Perini HackerNoon Contributor of the Year- ENGINEERING is not only a Tech Enthusiast, he is also the HackerNoon Contributor of the Year for Engineering. Rishabh Agarwal HackerNoon Contributor of the Year- RASPBERRY-PI loves Robotics and IoT. Nazar has provided us with the highest quality raspberry-pi stories for HackerNoon and has won the HackerNoon Contributor of the year award. Chingiz Nazar HackerNoon Contributor of the Year- DEVELOPER-TOOLS is a founder at Adadot.com, the world’s first fitness tracker for work, helping developers improve the way they work and feel. His contributions for HackerNoon will be remembered forever. Congratulations Alex! Alex Harris Developer Defying Patriarchy (While Being an Awesome Developer) is a software Engineer at Mezmo and mentor for female tech-newbies and advocate of slaying the imposter syndrome dragon. Sullivan, the defier of patriarchy and the HackerNoon Contributor of the Year, Congratulations! Shanna Sullivan HackerNoon Contributor of the Year- NATURAL-LANGUAGE-PROCESSING is a coder and a writer, with a goal of becoming a streamer someday. Tan is surely a step closer after winning HackerNoon Contributor of the Year for Natural Language Processing. Liling Tan Bug Finder of the Year is the HackerNoon Bug Finder of the Year, reporting bugs and significant vulnerabilities in a tech project. Andrej is also a dedicated writer and digital evangelist. Andrej Kovacevic HackerNoon Contributor of the Year- OPEN-SOURCE is focused on the open source software supply chain to build a better digital future for all of us. We can all appreciate Sal for his contributions and congratulate them for winning the HackerNoon Contributor of the year for Open Source. Sal Kimmich AI Artist Award is a trained neural network called DALL·E that creates images from text captions for a wide range of concepts expressible in natural language. 2022 has seen AI drawing software proliferate like no other and Dall E is one of them. Congratulations! Dall E HackerNoon Contributor of the Year- CODE-QUALITY Whatever your metrics for judgment are, code quality stories on hackernoon provide pointers that ease your way. helps make your Python code easier to read, understand, and work with through automatic refactoring suggestions. Let's congratulate them for being the HackerNoon Contributor of the Year for Code Quality! Sourcery HackerNoon Contributor of the Year- MOBILE is the CTO and Co-Founder of @incorainc, where we can turn your ideas into products. Incorainc is THE website that caters to your short time span. Congratulations on being the HackerNoon Contributor of the Year for Mobile. Tetiana Stoyko Pythonistas' Paradise is THE paradise for pythonistas.  They are a professional provider of data protection solutions for enterprises. Congrats on being the HackerNoon Contributor of the Year! Vinchin Backup & Recovery HackerNoon Contributor of the Year- ARCHITECTURE is a full-stack engineer and architect. Michelangelo would be proud of Andronchik’s contribution and being the HackerNoon Contributor of the Year. George Andronchik The Skynet Award Unconcerned with a global takeover at the hands of AI, this award honors the writers, inventors, or companies that brought us closer to the singularity this year. , an AI research and deployment company with the mission to ensure that artificial general intelligence benefits all of humanity has won the Hackernoon’s Skynet Award. OpenAI HackerNoon Contributor of the Year- APPLICATION Application is a set of instructions encoded in the language a computer understands. This was best reflected by , a Smart Web Vulnerability Scanner and their contributions to HackerNoon. Congratulations on winning the HackerNoon Contributor of the Year for Applications. SmartScanner HackerNoon Contributor of the Year- SCRUM Scrum represents complex products, sustainability and adaptive solutions, in the constantly evolving work and technological landscape. , the principal data consultant at ThoughtWorks has proven themselves to be the HackerNoon Contributor of the Year- Scrum. Ryan Dawson HackerNoon Contributor of the Year- ENGINEERING-MANAGEMENT is a human specialist and former VPE at New Relic, Gremlin. Jade wins the award  for the highest quality engineering-management stories on HackerNoon Jade Rubick HackerNoon Contributor of the Year- FIREBASE Congratulations to , a software engineer at Udgama for winning the HackerNoon Contributor of the Year for Firebase. Snehal Narkar HackerNoon Contributor of the Year- PROJECT-MANAGEMENT Project managers are so meticulous that they remember all the details. is one of them and wins this award with grace! Jade Rubick The Calmer of Storms Debugging is often the most grueling sweat of one’s brow but it easier by providing open-source error tracking to monitor and fix bugs and crashes anywhere in your application in real time. Sentry makes HackerNoon Contributor of the Year- APP-DEVELOPMENT Congratulations to , a Layer-3 peer-to-peer network to securely exchange liquidity & facilitate trading, clearing, settlement, and compliance. Yellow Network SaaSiest CEO of the year , SaaSiest Ceo of Notion, a project management software designed to help members of a company and individual users coordinate objectives, working processes and so forth is the winner! Ivan Zhao Tip your Hat Tutorials - Best Tutorials has some of the best tutorials on backend development, deployment, frameworks and much more. DigitalOcean Blog HackerNoon Contributor of the Year- CODE crashed this award! Oscar Daum HackerNoon Contributor of the Year- DATA-ANALYSIS Congratulations to , a global data labeling company. Toloka HackerNoon Contributor of the Year- GOOGLE-CLOUD-PLATFORM Congratulations to , a Java programmer and blogger! Javin Paul HackerNoon Contributor of the Year- PHP is an experienced Full-Stack Developer and CEO of BootesNull. Surinder Rawat The Best Alternatives to Notepad++ is a highly configurable text editor built to make creating and changing any kind of text very efficient. It is included as "vi" with most UNIX systems and with Apple OS X. Vim HackerNoon Contributor of the Year- CSS3 We all need a Sensei to guide us through our learning journey and has been our most favorite in CSS3 voyage. Congratulations! Anderson Osayerie HackerNoon Contributor of the Year- SOFTWARE is a Chief Product Officer for Chargify and SaaSOptics. He is passionate about creating products that can maximize his customer’s ease. Barrow Hamilton HackerNoon Contributor of the Year- CMS is a passionate writer interested in new technologies, a dog lover and a digital nomad. Kat HackerNoon Contributor of the Year- DJANGO Congratulations to who is a Software engineer, blogger, lifelong learner with an added badge to his armor as this year’s DJANGO master . Ismail Tlemcani HackerNoon Contributor of the Year- INFORMATION-SECURITY Congratulations to for keeping us well informed on how to stay safe in the virtual world jasonhiggins HackerNoon Contributor of the Year- LARAVEL To the artisan that made learning Laravel easier for all of us. Congratulations ! Adam Schmideg

#Noonies2022 Awards: The List of Winners in the Programming Category

Let’s become the architects of a cleaner, greener world The cloud has a than the airline industry. With the increasing demand for software to drive commerce and industry, there is a greater urgency for data collection and storage, cloud computing, and AI to source electricity from greener energy sources and generate significantly fewer carbon emissions. bigger carbon footprint By shifting electricity consumption to times and places where electricity is cleanest, programming can mitigate climate change and ensure a cleaner and safer future for us and future generations. This is Carbon Aware Computing, and it has the potential to make software development a critical force in mitigating climate change by helping us choose and optimize carbon-free energy with climate-conscious code. Carbon Aware Computing 101 There aren’t many examples of carbon-aware computing out there - which is why events like __ __are significant in crowd-sourcing new ideas and accelerating innovation in software programming to build applications that are both climate-conscious and energy-responsive. Carbon Hack 22 Green Software Foundation __ __carbon awareness as the idea of doing more when more energy comes from low-carbon “clean” sources and doing less when more energy comes from high-carbon “dirty” sources. As developers, we can do so by measuring and responding to the carbon intensity of electricity (gCO2/kWh), which can help create code to reduce the amount of carbon emissions generated through software. discusses Carbon Computing Approaches & Application There are broadly 3 approaches: Running our software in a cleaner location; sourcing electricity from a region where there is a greater supply of cleaner energy (e.g., wind, solar, water) than others. Location Shifting: 2. Running our software at a cleaner time; when clean energy is being generated by wind, water, or solar power. Time Shifting: Running our software so it does more when electricity is clean and less when it’s dirty. Demand Shaping: Building carbon-aware applications is challenging, despite various data sources, partly because some of these sources are hard to navigate. The , a project of the Green Software Foundation’s Open Source Working Group, makes it easier. This API helps us identify, analyse and predict the carbon intensity (how green or dirty the electricity is) of our application at any given time and in any given place and create code that allows our software to pull clean energy based on location, run when the wind is blowing or enable systems to follow the sun’s rays. open source Carbon Aware API The API is extremely simple. If you visit you will see the web interface like so: https://grnsft.org/hack22/api We can hook it to APIs like and , or to our own datasets. For the duration of the Carbon Hack 22 (now until Nov 10th, 2022), the Carbon Aware API will be for developers to use. Watttime Electricty Maps hooked up to a WattTime data source What time is best to run my software? Time Shifting: When time shifting, you can use the API endpoint, providing it with the location you are interested in in the field. Currently, the location names are mapped to Azure region names. You also need to provide the time window during which you are able to move the job. The start time needs to be entered in the field, and the end of it in the field. The dates provided should be in the ISO 8601 format. /emissions/bylocations/best location time toTime You can directly request data from the API via a Curl request like so: curl -X 'GET' \

'https://carbon-aware-api.azurewebsites.net/emissions/bylocations/best?location=uksouth&time=2022-10-08T00%3A00%2B01%3A00&toTime=2022-10-08T23%3A59%2B01%3A00' \

-H 'accept: application/json' Or even just a normal HTTP GET request in the . browser As output, you will receive an array of JSON objects, with the WattTime region name for the location, time of the data, carbon intensity rating, and duration for which this data is valid (WattTime provides data in 5-minute intervals). This data will be the lowest carbon ratings for the requested location. This query looks at data from a single 24-hour window, so to get an accurate result you might want to request data for multiple days (e.g., by looking at the endpoint) and check if this is not a one-time drop in carbon intensity! /emissions/bylocations [

{

"location": "UK",

"time": "2022-10-08T00:45:00+00:00",

"rating": 351.98767912,

"duration": "00:05:00"

},

{

"location": "UK",

"time": "2022-10-08T00:40:00+00:00",

"rating": 351.98767912,

"duration": "00:05:00"

},

{

"location": "UK",

"time": "2022-10-08T00:35:00+00:00",

"rating": 351.98767912,

"duration": "00:05:00"

}

] Where is the best location to run my software? Location Shifting: When location shifting, you can use the same endpoint as previously ( ), but for this time, provide multiple locations you are interested in and a single time during which you are looking. The example query below has multiple locations provided, taking under consideration. You can provide it with a time window (similarly to time shifting) or a single time point instead, filling in the and fields. /emissions/bylocations/best uksouth, eastus, and francecentral time toTime You can directly request data from the API via a Curl request like so: curl -X 'GET' \

'https://carbon-aware-api.azurewebsites.net/emissions/bylocations/best?location=uksouth&location=eastus&location=francecentral&time=2022-10-08T09%3A00%2B01%3A00' \

-H 'accept: application/json' Or even just a normal HTTP GET request in the . browser As output, you will receive the same format of a response, with locations having the lowest carbon intensity, along with the time for that reading, the actual value of the carbon intensity, and the duration for which this data is valid. [

  {

    "location": "FR",

    "time": "2022-10-08T08:00:00+00:00",

    "rating": 242.21832558,

    "duration": "00:05:00"

  }

] How do I adjust user behaviour? Demand shaping: To implement demand shaping, use a combination of the above queries and others to determine how best to adjust the behaviour of your application. Imagine this! Imagine if you still carried out those energy-hogger ML training jobs... but only when the electricity grid powering them was running on renewables. Imagine if your washing machine only ran at the time your local grid was fully wind-powered. Imagine you wanted to run your CI pipeline simultaneously each day, but ensured it runs only in an availability zone (country) currently powered by green energy. In all these cases, you would still consume as much energy as before, but you would generate much fewer greenhouse gas emissions. Whether you are polling the Carbon Aware API, running it in a local container, or as a Command Line Tool (CLI), you can use the patterns of event-driven development to connect events in your app to triggers in the electricity grid. Your own local grid or a greener, distant one. Depending on your use cases, you can allow an entire device, computing job, or application to run exclusively on renewable energy, either by only running when the local grid is green or running only from zones currently powered by green energy. Instead of being mechanics who keep the engine of digital emissions running, let’s be the engineers who help slow it down while maybe even contributing to the rebuild. Carbon-aware computing is the next frontier of sustainable software engineering. Some of the most influential decision-makers in the industry are interested in reducing carbon emissions from software and showing that by lending their resources and contributing significant prize money to . Key players like Accenture, Avanade, Boston Consulting Group, Globant, Goldman Sachs, Intel Corporation, Thoughtworks, UBS and VMWare have their best and brightest in sustainable technology participating and are collectively contributing 100,000 USD in prize capital. Carbon Hack 22 It is now also the job of software engineers to build solutions that do less harm to both people and the planet. Carbon-aware computing is how current and future generations of software engineers can help achieve this goal. and to join nearly 350 hackers and 70 innovations in identifying and resolving barriers to software that reduces the carbon intensity of electricity. Explore the Carbon Aware API participate in Carbon Hack Written by Asim Hussain, Executive Director and Chairperson, Green Software Foundation

Our Code Is Harming The Planet, We Need Carbon Aware Design Patterns

Nuestro código está dañando el planeta, necesitamos patrones de diseño conscientes del carbono

Notre code nuit à la planète, nous avons besoin de modèles de conception sensibles au carbone

हमारा कोड ग्रह को नुकसान पहुंचा रहा है, हमें कार्बन जागरूक डिजाइन पैटर्न की आवश्यकता है

私たちの行動規範は地球に害を及ぼしています。炭素を意識した設計パターンが必要です

Nosso código está prejudicando o planeta, precisamos de padrões de design com consciência de carbono

Quy tắc của chúng tôi đang gây hại cho hành tinh, chúng tôi cần các mẫu thiết kế nhận biết carbon

我们的代码正在危害地球，我们需要碳感知设计模式

Using APIs and vCards To Make Startup Swag Cool Again 

200 Black Women in Tech to Follow on Twitter in 2022

200 mujeres negras en tecnología para seguir en Twitter en 2022

200 femmes noires en technologie à suivre sur Twitter en 2022

टेक में 200 अश्वेत महिलाएं 2022 में ट्विटर पर फॉलो करेंगी

2022 年に Twitter でフォローするテクノロジー業界の黒人女性 200 人

200 mulheres negras em tecnologia para seguir no Twitter em 2022

200 phụ nữ da đen trong công nghệ để theo dõi trên Twitter vào năm 2022

2022 年将在 Twitter 上关注 200 名科技黑人女性

In the age of self-service , nearly every company considers themselves a data-first company, but not every company is treating their data architecture with the level of democratization and scalability it deserves. business intelligence Your company, for one, views data as a driver of innovation. Your boss was one of the first in the industry to see the potential in and Looker. Or maybe your CDO spearheaded a cross-functional initiative to educate teams on data management best practices and your CTO invested in a data engineering group. Snowflake Most of all, however, your entire data team wishes there were an easier way to manage the growing needs of your organization, from fielding the never-ending stream of ad hoc queries to wrangling disparate data sources through a central ETL pipeline. Underpinning this desire for democratization and scalability is the realization that your current data architecture (in many cases, a siloed data warehouse or a data lake with some limited real-time streaming capabilities) may not be meeting your needs. Fortunately, teams seeking a new lease on data need look no further than a data mesh, an architecture paradigm that’s taking the industry by storm. What is a data mesh? Much in the same way that software engineering teams transitioned from , the data mesh is, in many ways, the data platform version of microservices. monolithic applications to microservice architectures As first defined by Zhamak Dehghani, a ThoughtWorks consultant and the original architect of the term, is a type of data platform architecture that embraces the ubiquity of data in the enterprise by leveraging a domain-oriented, self-serve design. a data mesh Borrowing Eric Evans’ theory of , a flexible, scalable software development paradigm that matches the structure and language of your code with its corresponding business domain. domain-driven design Unlike traditional monolithic data infrastructures that handle the consumption, storage, transformation, and output of data in one central data lake, a data mesh supports distributed, domain-specific data consumers and views “ ,” with each domain handling their own data pipelines. The tissue connecting these domains and their associated data assets is a universal interoperability layer that applies the same syntax and data standards. data-as-a-product Instead of reinventing Zhamak’s very thoughtfully built wheel, we’ll boil down the definition of a data mesh to a few key concepts and highlight how it differs from traditional data architectures. (If you haven’t already, however, I highly recommend reading her groundbreaking article, , or watching Max Schulte’s tech talk on . You will not regret it). How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh why Zalando transitioned to a data mesh Domain-oriented data owners and pipelines Data meshes federate data ownership among domain data owners who are held accountable for providing their data as products, while also facilitating communication between distributed data across different locations. While the data infrastructure is responsible for providing each domain with the solutions with which to process it, domains are tasked with managing ingestion, cleaning, and aggregation to the data to generate assets that can be used by business intelligence applications. Each domain is responsible for owning their ETL pipelines, but a set of capabilities applied to all domains that stores, catalogs, and maintains access controls for the raw data. Once data has been served to and transformed by a given domain, the domain owners can then leverage the data for their analytics or operational needs. Self-serve functionality Data meshes leverage principles of domain-oriented design to deliver a self-serve data platform that allows users to abstract the technical complexity and focus on their individual data use cases. As outlined by Zhamak, one of the main concerns of domain-oriented design is the duplication of efforts and skills needed to maintain data pipelines and infrastructure in each domain. To address this, the data mesh gleans and extracts domain-agnostic data infrastructure capabilities into a central platform that handles the data pipeline engines, storage, and streaming infrastructure. Meanwhile, each domain is responsible for leveraging these components to run custom ETL pipelines, giving them the support necessary to easily serve their data as well as the autonomy required to truly own the process. Interoperability and standardization of communications Underlying each domain is a universal set of data standards that helps facilitate collaboration between domains when necessary — and it often is. It’s inevitable that some data (both raw sources and cleaned, transformed, and served data sets) will be valuable to more than one domain. To enable cross-domain collaboration, the data mesh must standardize on formatting, governance, discoverability, and metadata fields, among other data features. Moreover, much like an individual microservice, each data domain must define and agree on SLAs and quality measures that they will “guarantee” to its consumers. Why use a data mesh? Until recently, many companies leveraged a single data warehouse connected to myriad business intelligence platforms. Such solutions were maintained by a small group of specialists and frequently burdened by significant technical debt. In 2020, the architecture du jour is a data lake with real-time data availability and stream processing, with the goal of ingesting, enriching, transforming, and serving data from a centralized data platform. For many organizations, this type of architecture falls short in a few ways: A central ETL pipeline gives teams less control over increasing volumes of data As every company becomes a data company, different data use cases require different types of transformations, putting a heavy load on the central platform Such data lakes lead to disconnected data producers, impatient data consumers, and worse of all, a backlogged data team struggling to keep pace with the demands of the business. Instead, domain-oriented data architectures, like data meshes, give teams the best of both worlds: a centralized database (or a distributed data lake) with domains (or business areas) responsible for handling their own pipelines. , data architectures can be most easily scaled by being broken down into smaller, domain-oriented components. As Zhamak argues Data meshes provide a solution to the shortcomings of data lakes by allowing greater autonomy and flexibility for data owners, facilitating greater data experimentation and innovation while lessening the burden on data teams to field the needs of every data consumer through a single pipeline. Meanwhile, the data meshes’ self-serve infrastructure-as-a-platform provides data teams with a universal, domain-agnostic, and often automated approach to data standardization, data product lineage, data product monitoring, alerting, logging, and data product quality metrics (in other words, data collection and sharing). Taken together, these benefits provide a competitive edge compared to traditional data architectures, which are often hamstrung by the lack of data standardization between both ingestors and consumers. To mesh or not to mesh: that is the question Teams handling a large amount of data sources and a need to experiment with data (in other words, transform data at a rapid rate) would be wise to consider leveraging a data mesh. Please answer each questions, below, with a number and add them all together for a total, in other words, your data mesh score. We put together a simple calculation to determine if it makes sense for your organization to invest in a data mesh. . How many data sources does your company have? Quantity of data sources . How many data analysts, data engineers, and product managers (if any) do you have on your data team? Size of your data team . How many functional teams (marketing, sales, operations, etc.) rely on your data sources to drive decision making, how many products does your company have, and how many data-driven features are being built? Add the total. Number of data domains How frequently is the data engineering team a bottleneck to the implementation of new data products on a scale of 1 to 10, with 1 being “never” and 10 being “always” ? Data engineering bottlenecks. How much of a priority is data governance for your organization on a scale of 1 to 10, with 1 being “I could care less” and 10 being “it keeps me up all night”? Data governance. Data mesh score In general, the higher your score, the more complex and demanding your company’s data infrastructure requirements are, and in turn, the more likely your organization is to benefit from a data mesh. If you scored above a 10, then implementing some data mesh best practices probably makes sense for your company. If you scored above a 30, then your organization is in the data mesh sweet spot, and you would be wise to join the data revolution. Here’s how to break down your score: : Given the size and unidimensionality of your data ecosystem, you may not need a data mesh. 1–15 : Your organization is maturing rapidly, and may even be at a crossroads in terms of really being able to lean into data. We strongly suggest incorporating some data mesh best practices and concepts so that a later migration might be easier. 15–30 : Your data organization is an innovation driver for your company, and a data mesh will support any ongoing or future initiatives to democratize data and provide self-service analytics across the enterprise. 30 or above As data becomes more ubiquitous and the demands of data consumers continue to diversify, we anticipate that data meshes will become increasingly common for cloud-based companies with over 300 employees. A data mesh can reduce risks The vast potential of using a data mesh architecture is simultaneously exciting and intimidating for many in the data industry. In fact, some worry that the unforeseen autonomy and democratization of a data mesh introduces new risks related to data discovery and health, as well as data management. Given the relative novelty around data meshes, this is a fair concern, but I would encourage inquiring minds to read the fine print. Instead of introducing these risks, . a data mesh actually mandates scalable, self-serve observability into your data According to Zhamak, such self-serve capabilities inherent to any good data mesh include: Encryption for data at rest and in motion Data product versioning Data product schema Data product discovery, catalog registration, and publishing Data governance and standardization Data production lineage Data product monitoring, alerting, and logging Data product quality metrics When packaged together, these functionalities and standardizations provide a robust layer of . data observability Interested in learning more about the data mesh? In addition to Zhamak and Max’s resources, check out some of our favorite articles about this rising star of data engineering: Data Mesh Applied — Sven Balnojan The Data Mesh: Re-Thinking Data Integratio n — Kevin Petrie Should Your Application Consider Data Mesh Connectivity? — Joe Gleinser Also Published Here

What Is A Data Mesh — And Is It Right For Me?

From the evolution of successful software-powered companies, the following five building characteristics have emerged: Speed, Agility, Automation, Security, and Quality. All five of these can be developed with DevOps Principles and new technologies. But with so many DevOps technologies on the market, software development has become more complicated than ever. CI/CD approach help reduce complexity while helping you deploy your software smoothly. CI/CD is about continually packaging, testing, and efficiently deploying code into a production environment. This approach has the advantage of eliminating integration issues and mitigating risk in deployments by making them frequent. Breakdown of CI/CD is a technique where developers connect and share their work as they progress on tasks. In the process, they continuously make changes to the entire system instead of making them one by one. The result is a reduction in bugs and a higher quality product. One of the main benefits of using Continuous Integration is sharing code across teams. When teams can see what other teams are doing, they can collaborate more efficiently. Again, this leads to higher quality products and fewer or no bugs. Lastly, Continuous Integration increases the speed at which developers push code changes by reducing the chance for conflict or error. It's like an assembly line, where all members of the team pitch in and assemble a car in parallel, rather than one person working on one part of the car at a time. Continuous Integration is a similar technique and includes delivering software after every successful build. Teams that practice Continuous Delivery will usually automate their release process. This eliminates errors that can occur when human intervention is required. In addition, by practicing Continuous Delivery, teams can update their software more often without worrying about any errors getting through. You can also use this technique for events or seasons when customers are less active to lower your risk for downtime in production environments. Continuous Delivery is the final step where the teams if they want to, should be able to deploy to production at any given moment. This process can be completely automated, and DevOps aims at zero-touch automation of deployment. Still, sometimes that may not be practically easy, and hence we rarely see companies employing Continuous Deployment. Continuous Deployment While Continuous Deployment may not be practical across all departments or products, it has helped some companies improve their development process significantly by decreasing the time spent on bug fixes and speeding up the release cycle by deploying software frequently. What is CI/CD pipeline? can be thought of as a set of stages in the software release process. The pipeline represents a workflow that includes various steps such as code compilation, unit testing, packaging, and deployment from development to production. An automated CI/CD system must be in place for this process to happen successfully. It will notify developers when new commits are made into the repository and trigger jobs accordingly. The CI/CD pipeline should be designed such that it's easy to add new steps or improve existing ones without impacting other parts of the process. CI/CD pipeline Why CI/CD Every company is viewed as a software company today, and enterprises are going through a massive digital transformation, with automation being an integral part. Well, automation is supercritical these days to tackle ever-growing technology trends. CI/CD enhances the . It aims to empower developers to deploy software at any point to any environment on-demand with the push of a button. developer experience CI/CD is Much More Than Just Automation It lets the stakeholders and end-users get new solutions and features quickly and efficiently with quality. CI/CD is a competitive advantage for enterprises embracing it thoroughly. If you think of software as a set of Lego pieces, CI/CD lets these pieces integrate and sync well with each other and deploy with ease. CI/CD reduces the MTTR [Mean Time to Resolution] because code changes are smaller, iterative, and fault isolations are easier to detect. CI/CD improves the release rate: continuous code merges and Continuous Deployment to production after precise testing keeps the code in a release-ready condition. Easy feedback, transparency, and collaboration are made simple with CI/CD practices. The Business Impact of CI/CD The cost, effort, and resources required to fix a bug increase exponentially if we find them in the later stages of the , and this also increases risk. With Continuous Integration, we are able to find bugs as soon as they are introduced. This leads to shipping releases faster and with better quality. In addition, Continuous Integration eliminates manual handoffs and ensures that releases are frequent. This way, developers can focus on writing code for new features rather than fixing bugs manually all day long. It saves time, effort and increases developer morale towards the work. SDLC In most organizations, the friction in software delivery is because most of the tasks are manual and error-prone. For example, someone provisions/updates the environment as needed, a team deploys a specific version of the software, and another team keeps track of what’s running where. This creates a dependency hell and lack of visibility. Ultimately, these inefficiencies lead to slower software releases and lost revenue and opportunities. The best approach to achieve velocity is to automate all steps in the deployment pipeline. This leads to more frequent and rapid release cycles that are predictable and error-free and will lead to happy and productive engineering teams. In addition, Continuous Deployment helps you create and promote manifests across environments and flexible deployments to any cloud. CI/CD forms the building blocks of your software development and delivery process. Therefore, it is essential to understand and apply it thoughtfully to streamline your software delivery. Notable CI/CD Solutions Available in the Market Jenkins is one of the first tools in the CI/CD game. It is an open-source automation server where the central build and Continuous Integration processes take place. It is a Java-based program with packages available for operating systems like Windows, macOS, and other Unix-like operating systems. With numerous plugins available, Jenkins supports building and helps in deploying and automating software development projects. Jenkins Harness With its recent acquisition of Drone CI, is a modern and complete CI/CD solution - a unique tool that incorporates the power of AI and inbuilt vulnerability scanning. Harness enables developers to build complex deployments like canary or blue/green. With Harness, you can have a single integrated Pipeline for both CI/CD. Feature Flags and Cloud Cost Management are added advantages. Developers can observe what’s happening to their pipelines with strong debugging capabilities. natively and eliminates scripting toil to achieve highly automated and true CI/CD. Harness Harness provides powerful GitOps capabilities CircleCI is yet another CI/CD solution. It is a highly sophisticated platform with powerful caching capabilities and customizable workflows. It supports many languages including Java, Ruby/Rails, Python, JS, PHP, Haskell, Scala, etc. This tool is easy to set up and run. CircleCI helps companies run extremely complicated pipelines efficiently. CircleCI builds, tests, does security scans, approvals, and deployments. CircleCI Travis CI is considered one of the oldest CI solutions that exist. Travis has a large community, and there are many tutorials available. Travis can not only be used as SaaS, but it also has an on-prem version. Travis automatically detects new commits made by developers and pushes them to a GitHub repository. After every new code commit, Travis CI will build the project and then run tests accordingly with high efficiency. Travis GitLab is a widely known DevOps tool, and GitLab CI/CD is an inbuilt tool/feature that every GitLab user will get. It helps companies test, build, and publish their software applications seamlessly. In addition, it allows users to host many GitLab runners on servers and also allocate labels to them. This gives users a farm of build servers where builds can be assigned to any server as required. GitLab Codeship is a hosted CI/CD platform that supports early and automatic software releases. It helps companies develop more reliable products faster by optimizing testing and release processes. It runs your automated tests and configured deployment when you push to your repository. It also takes care of managing and scaling infrastructure as required so that you can test and release more frequently. Codeship GoCD is an open-source CI/CD tool that helps developers to build and release software that supports modern and advanced infrastructure. GoCD comes from the renowned company Thoughtworks. GoCD's major product differentiator is the Value Stream Map feature that helps companies visualize their CD pipelines, but every CD tool provides this these days. GoCD is available for Windows, Linux, and Mac and runs on a system of pipelines that help keep the workflow moving. GoCD TeamCity is another modern CI/CD server from . It is known for its straightforward setup and magnificent user interface. It includes a robust set of features and a growing plugin ecosystem. TeamCity permits users to integrate code and configure it with simple steps. In addition, TeamCity makes it possible for parallel builds to run on different platforms and environments with more efficiency. TeamCity JetBrains Bamboo is one more traditional CI/CD server from . Like any other CI/CD server, Bamboo lets developers automatically build, integrate, and test source code and prepare the application for the deployment-ready state. In addition, Bamboo operates and integrates seamlessly with Atlassian’s other tools like Jira and Hipchat. Bamboo Atlassian Semaphore is a cloud-based CI/CD solution for testing and deploying software applications. In other words, this tool helps developers with setting up the build, running tests, and deploying applications. Semaphore sets CI/CD norms with a pull request-based development process. Semaphore has some great features, such as native Docker support, secrets and dependency management, parallel execution, customizable stages, and control flow switches. Semaphore Codefresh is a powerful CI/CD platform, especially for Docker and Kubernetes. It boasts many critically valuable features, such as allowing developers to get faster builds, native testing for microservices, and smooth deployments with Kubernetes and GitOps. In Codefresh, each pipeline step runs efficiently in its own container, having a shared volume and variables between steps. Codefresh Conclusion There are many tools in the cloud-native space for CI/CD. Some firms claim they help people do CI/CD, but they only do the CI part well and use scripting for CD. As CI/CD approaches become an integral part of the DevOps process, it is highly recommended to invest your time and effort in selecting the tool that fits your needs and has a future vision. In addition, something that is well equipped with all the modern technologies like AI, ML, security, etc. Ultimately, developers should take charge and pick the right tool. But in this highly scattered market, choosing the right tool can be daunting. We have plenty of resources at Harness to help. Familiarizing yourself with should make the process easier. CI/CD Best Practices Also Published Here

Best CI/CD Tools to Consider in 2022

Companies that have pioneered the application of AI at scale did so using their own in-house ML platforms (uber, LinkedIn, Facebook, Airbnb). Many vendors are now making these capabilities available to purchase off-the-shelf products. There’s also a range of open-source tools addressing MLOps. The rush to the space has created a new problem - . There are now hundreds of tools and at least 40 platforms available: too much choice This is a very difficult landscape to navigate. But organizations have to figure it out because there is an imperative to get value from ML. Let’s understand the big challenges and then we’ll introduce some new free material that aims to address these problems. Challenge #1: Overwhelming Choice Chip Huyen gathered data on the . Chip found . Evaluations don’t have to consider all of these tools provided we can narrow them down to the ones that are most relevant to us... but that’s not easy to do. ML tool scene in 2020 284 tools and the number keeps on growing Challenge #2: Blurred Categories Typically we get a picture of which software does what by putting products in categories. There are attempts to do this with the , the , and . But often ML software does more than one thing and could fit into any of several categories. Platforms are especially hard to categorise as they explicitly aim to do more than one thing. LFAI Landscape diagram MAD landscape GitHub lists Because software that does several things is hard to categorize, ML platforms all tend to end up in a single category of ‘platform’. This obscures what emphasis each platform has and also loses all of the . nuances of how different platforms do similar things in different ways Challenge #3: Shifting Landscape ML categories are hard to keep track of in part because . Feature stores, for example, haven’t been around very long but are now a significant category. This affects platforms too as platforms are introducing big new features and changing their emphasis (in part in response to what new tools appear). new categories keep appearing Challenge #4: Complex Problems ML is complex. It’s also a big field. so that means understanding regression, classification, nlp, image processing, reinforcement learning, explainability, AutoML and a lot more. Platforms do a wide range of things Challenge #5: Range of Roles and Stakeholders Not only are the problems various and complex, there’s also a . Data Scientists, ML Engineers, Data Engineers, SRE, Product Managers, Architects, Application Developers, Platform Developers, End Users etc. Different roles have different points of interaction with the software and different needs from it. range of roles involved in the ML lifecycle Challenge #6: Build vs Buy and other controversies There’s a lot of discussion of build vs buy trade-offs in the ML space. How much control do organisations need over their stack? How much does that control cost? Build vs buy is often presented as an either-or but it is more of a spectrum. This is just one of the (consider how controversial AutoML is). confusing controversies in the ML space How do we get on top of all this? We’ve launched two new resources to help. For understanding the landscape and dealing with the trade-offs we’ve launched a Guide to Evaluating MLOps Platforms: https://www.thoughtworks.com/what-we-do/data-and-ai/cd4ml/guide-to-evaluating-mlops-platforms This is available for free without any sign-up. It addresses and . how to distinguish MLOps platforms how to structure an evaluation to suit the needs of your organisation We also need to apply this knowledge and see how to compare platforms against each other. For this we’ve released an open-source comparison matrix: https://github.com/thoughtworks/mlops-platforms The matrix is structured to highlight how vendors do things in their own ways and also point to more detail in the product documentation. We’ve also included in the repository a series of profiles that describe the product directions of popular platforms concisely and in a marketing-free way. We hope you find this material helpful and welcome contributions in GitHub. Feel free to ask any questions either on or to me directly on , . github twitter linkedin Title image by chenspec on pixabay.

How to Evaluate MLOps Platforms

Hey Hackers! I’m Ryan Dawson, and I’m Principal Data Consultant at Thoughtworks. First of all, a huge thank you to the HackerNoon community and staff for nominating in the 2021 Noonies awards! I’ve been nominated in the following categories - please do check out these award pages and vote: Contributor of the Year - Data Contributor of the Year - Scrum As someone that works a lot with data, I’m currently excited about Data Mesh. It’s an approach to structuring and using data within an enterprise that contrasts with the Data Warehouse or Data Lake model. The Lake and Warehouse centralized models with a single team of gatekeepers is creating bottlenecks when many organizations want to be moving quickly to get value from the data. Data Mesh is a bit like microservices for data. Learn more about my thoughts and opinions on getting value from data and my journey in the tech industry via the interview below. 1. What do you do, and why do you do it? My role at ThoughtWorks is Principal Data Consultant, which is pretty broad. I work with clients to help them get the most value out of data. This can mean organizing the way data is captured and shared, selecting the right data technologies for working with data, and advising on optimal team structures. It also includes some AI work as AI is key to the value that many organizations are looking to get from data. Taking machine learning to production is one of my favorite topics. To me, the big challenge in tech right now is for organizations to become more data-driven. Certain very famous companies have shown how impressive software products can be when they leverage data in intelligent ways. The likes of Amazon, Google, Netflix, and Spotify and their search and recommendation features especially have had a big impact on people’s lives. Lots of other companies see that data has a lot of potential but don’t yet see how to make full use of it. 2. Tell us more about the things you create / write / manage / build! My day-to-day is working with customers on whatever their problems might be. That might be about putting together a strategy document or refining a plan, or advising on a choice of technology or some team structures. Or sometimes, it can mean getting into the details of system code or system architecture. I also write about the challenges that I encounter, and I publish some of that on hackernoon. 3. How did you end up on your current career path? Do you like it? I started as a software consultant, building software projects for big banks. Those were mostly Waterfall projects, and I learned about the shortcomings of the Waterfall delivery model first-hand. It got me very interested in Agile and in the big picture of how we approach software delivery. Then I moved into open-source software at Alfresco. It was interesting to work at a company selling software products. You tend to get a different culture when the software is what makes the company money when compared with software developed for in-house use. Software product companies are more conscious that they have to nurture the software over time. My next job was at another open-source software product company, Seldon. That was working on tools to get value out of machine learning - MLOps tools. MLOps is a new field - it's all about making the machine learning build-deploy-monitor lifecycle as smooth as possible. This is surprisingly challenging, as machine learning uses data in very different ways from most software. You extract patterns from data and reapply those patterns to make predictions. So, if the new data you're applying the patterns to is different, or it changes, then you can get really bad predictions. I’m still involved in MLOps in my current role at Thoughtworks. Now I work on the more general problem of how to get value from data. One aspect of this is enabling AI, but there are also lots of challenges about how to structure and store data, how to ensure quality and keep the data fresh, and how to enable the various use cases that arise for the data. 4. What tech are you most excited or passionate about right now and why? I’m very excited about Data Mesh. It’s an approach to structuring and using data within an enterprise that contrasts with the Data Warehouse or Data Lake model. The Lake and Warehouse centralized models with a single team of gatekeepers are creating bottlenecks when many organizations want to be moving quickly to get value from the data. Data Mesh is a bit like microservices for data. There’s a lot of enthusiasm about Data Mesh right now, and Thoughtworks are leaders in developing the concept and bringing it to life. It’s an exciting thing to be part of. 5. What tech are you most worried about right now and why? I’m most worried about the Attention Economy and how much business relies on getting people to scroll or click on a screen. I love software, but some software design practices strike me as manipulative, and I am worried about a tendency to play to the subconscious. These days everyone seems to feel that they spend more time with their screens than they want to, and I suspect certain design practices are a big part of why that’s happening. 6. If we gave you 10 million dollars to invest in something today, what would you invest in and why? Emitwise, if they wanted the money. I'm an organizer of Tech Ethics London, and Mauro Cozzi told us all about Emitwise. They help companies better track and improve their overall carbon footprint. I’ve previously talked about how emphasizing ethics is a sound business investment, but I'm also pleased to see that there's more legislation coming to force companies to do this kind of reporting (at least in the UK). 7. What are you currently learning? I'm looking into the various data platforms out there and trying to work out how best to use them for Data Mesh. Most recently, I did the Azure Data Engineer Certification, and I'm looking to get deeper into more of the big data platforms. 8. What's the best advice you've ever given someone? Years ago I was doing some teaching and giving students feedback on their essays. There was one essay for which I told the student that it was a very good piece that they should be proud of but that to do best on the exam, they should be less interesting. Exams have a mark scheme, and there's a game you have to play to tick all the right boxes. I was pretty pleased to be able to give that advice as it was something I had to learn for myself when I was a student. 9. What's the best advice you've ever received? It was basically to focus on something and do it really well. At the time, I was writing a dissertation, and I was trying to do too many things. Narrowing the scope really helped me a lot. It became something much better after it became more focused, and I realized that I could always return to the other questions that I'd set aside later. It was specific advice at the time, but I think it generalizes. It's tempting to think that you have to do something big in order for it to be useful to others but often, what matters to others is actually the quality. Also, if you do something really well, then you gain the skills to do that thing well, and in the long run, those skills may be more valuable than whatever you're working on. AboutHackerNoon's 2021 Noonie Awards The annual Noonie Awards celebrate the best and brightest of the tech industry, bringing together all who are making the Internet and the world of tech what it is today. Please be sure to , nominate, and vote for the people and companies who you think are making the biggest impact on the tech industry today. check out our award categories The 2021 Noonies are sponsored by: , , and . Thank you so much to these sponsors who are helping us celebrate the accomplishments of all our nominees. bybit Dottech Domains Avast

Notes On Data-Driven Value Creation with Noonies Nominee Ryan Dawson

Among all the types of tests which make up , the end-to-end UI tests are considered by far the most difficult to manage, slow to execute, unstable, flaky, and sometimes even downright impractical. Just consider this recent post from the –  the author recalls a discussion with an engineering team about the state of their UI testing: the pyramid of software testing ThoughtWorks blog "Well, we have a couple of Selenium tests, but they're pretty brittle. They always seem to be broken, so we rarely run them." Elsewhere across the industry, the seems to think that software testing doesn't enjoy a favorable reputation among software developers: team behind the Cypress project "Until now, end-to-end testing wasn't easy. It was the part developers hated." Coding vs Testing Convincing developers that software testing is exciting and cool, the same way as frontend development is perceived to be, will take some stretching of the imagination. Testing is “dreaded” by developers simply because the entire software industry has adopted a dualist view of the software development process. We have developers on this side, who are always promised to build cool stuff, to experiment with the latest technologies, and so on, and we have testing and quality assurance on this other side, which is often regarded as too tedious and unexciting by developers. As long as this duality between development and testing exists, with developers dreading the testing part (and perhaps also quality engineering dreading the developers’ obsession with constant change and slickness), the tests will continue to be flaky and brittle, which will lead to overall poor quality. What I will try to suggest is that coding and testing are not two separate activities that happen independently of each other. Testing is an integral part of software development, in the same way as coding is. Every developer who thinks that testing is not their thing is still doing some form of testing. They are just not doing it in an automated fashion, but rather in a manual, tedious, and somewhat random one. The Evolution of Testing About ten years ago, when Node.js had been released, there were barely any tools available for end-to-end testing using JavaScript, and the ones that existed required significant effort to set up and use in an effective manner. However, since the release of in 2014, several other testing frameworks powered by Node.js were published (like , , or , among others), all of them promising to deliver fast installation and a way to write stable tests in JavaScript. Nigthwatch.js WebdriverIO Cypress Playwright Surely the days when you had to spend hours, if not days, to set up a UI automation project are behind us. With Nightwatch, it only takes a minute or two now to be up and running with a testing automation project using Chrome, Firefox, Safari, including built-in support for and cloud testing with . Additional services are easy to add. Selenium Grid Browserstack <iframe title="vimeo-player" src="https://player.vimeo.com/video/376838936" width="640" height="360" frameborder="0" allowfullscreen></iframe> But are the tests still brittle and/or flaky? Right, the two most popular adjectives used with UI tests, especially with Selenium tests. In Nightwatch, there is now pretty much all you can expect from an open-source framework to mitigate brittleness and flakiness in the test runs, like implicit waits for elements, automatic retries on failed assertions, retries on failed test cases, retries on network errors, and so on. Our plan is not just to try to detect and mitigate flakiness, but to make it impossible. The Internet Is Faster, but Websites Aren’t On the evolution of web development over the past decade, the story is not that convincing though. Or it all depends on what you mean by “evolved”. Sure, there’s webpack and React and ES6, but according to data from the median page load time has remained about the same over the past 10 years, even though internet speeds have been steadily increasing, along with rapid advancements in hardware. httparchive.org It’s safe to say that . In addition, as the team behind the utility has pointed out, “building for the web has never been more complicated.” the internet is faster, but websites aren’t Skypack With this in mind, I’d be quite confident in affirming that actually web development hasn’t really evolved all that much, even though we now have better and more sophisticated tools. On the other hand, frontend testing did evolve and is still evolving on multiple fronts, maybe even more rapidly than web development itself. What are Feedback Loops? The feedback loop is an iterative activity made of the following steps: plan, code, verify, exit – the process of planning a code change, implementing it, and then verify if the outcome is as intended. Feedback loops are everywhere in software development and beyond. I’ve first heard about it from a colleague who delivered at a jQuery conference, back in 2010. Feedback loops are a common concept in Agile software methodologies, but the same concept is found also in control theory, biology, math, engineering, etc. Even the Earth’s has feedback loops. a presentation climate system One of the founding texts of cognitive science, , introduced an early form of the feedback loop as a fundamental unit of behavior in humans. The authors identified the following steps: Test-Operate-Test-Exit which makes a TOTE unit. As they described, “in its weakest form, the TOTE asserts simply  that the operations an organism performs are constantly guided by the outcomes of various tests.” [1] Plans and the Structure of Behavior (1960) According to the “cybernetic hypothesis”, the feedback loop is the fundamental building block of the nervous system [2]. This brings me to my earlier point that every developer saying they don’t do testing in their process is still doing some form of testing. Any task that we have to complete, either being implementing a new feature or fixing a bug, the process involves at least one feedback loop – a TOTE unit, usually with multiple iterations. Completing a Task Here’s an example task. is a popular choice for users of Nightwatch who are looking to run their tests on a distributed cloud infrastructure that contains multiple desktop and mobile browsers. Browserstack Say you’re a developer building the Browserstack Automate UI dashboard and your task is to make a new Nightwatch test appear in the list, in real-time. Considering the above TOTE unit, the phase will be quite complex and will involve a few different operations that are needed before asserting if the feature is implemented successfully (the condition which will stop the loop). There are even some sub-feedback loops in there. Test Once you implement the changes, you need to perform the following manual steps of the verification process: ensure the local dev server is running execute a sample test script against the local dev backend open the browser and navigate to the local dev URL (or reload the page) check the test script that appears in the dashboard list How to Shorten the Feedback Loop The implementation phase is variable at each iteration, but the testing phase is quite fixed, involving the same steps and roughly the same amount of effort every time. Therefore if we manage to reduce it, not only it will be less complex to execute, but it will be shorter. Thankfully now we can automate all the manual steps involved in the preparation phase and we can also add a test assertion to verify if the condition was met (the test script that was executed appears in the dashboard list). Then the actual phase of the feedback loop will consist of only running this newly created automated script. Test Demo Project If you’re still not convinced, maybe will help, which is available on Github. The project contains all the code for the experiment I’ve described above. this demo project I've included the public URL here in order to have a working example, but of course, during the implementation phase, a dev server would be used instead. Here's what the main end-to-end test does: perform a login against the service browserstack.com save the cookies needed to maintain the session state for subsequent test runs start the second Nightwatch test inside a child process assert if the second test was found in the list, in the Browserstack dashboard. By using this technique, not only you will have shorter feedback loops, but you’ll also have an end-to-end test as well when the task is done, which can be used for the purpose of regression testing and continuous integration. [1] George A. Miller, Eugene Galanter, Karl H. Pribram – p. 39 Plans and the Structure of Behavior (Henry Holt and Co, 1960) [2] The “cybernetic hypothesis” formulated by Norbert Wiener in ) Cybernetics or Control and Communication in the Animal and the Machine (New York: Wiley, 1948

How To Make End-to-End Testing Your Friend

The last decade has seen an acceleration of software technology such as progressive web apps, cloud computing, and APIs. This has led to fundamental changes in software engineering roles, teams, tools, and processes. As teams have shifted from monoliths to microservices and seek to streamline and automate processes for speed, continuous delivery, competitive advantage, and satisfied end-users, developers are reconfiguring not only their workflow but their workplace. An example of this is the use of Internal Developer Platforms (IDPs). What is an Internal Developer Platform? Evan Bottcher, Head of Engineering at defines an IDP as: ThoughtWorks Australia "A foundation of self-service APIs, tools, services, knowledge, and support which are arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace, with reduced coordination." IDPs aim to improve the developer workflow and internal development processes. They use tools that integrate with each other and typically include the use of a distinct from the operations team. This creates a self-service environment where developers can work autonomously on feature development instead of waiting for the operations team to answer tickets. This leads to increased deployment frequency, greater visibility and transparency across teams, and reduced lead times. dedicated platform team However, as Scott Carey, UK Group Editor at InfoWorld , "Internal developer platforms are like snowflakes, in that no two are the same. Each platform varies from organization to organization depending on their stack, culture, codebase, and toolset." notes What are the benefits of an IDP? An IDP streamlines the internal development process to relieve pressure on Ops, improving productivity and speeding up business processes. It offers a way to compartmentalize the complexity involved in DevOps by creating a standardized developer workflow on top of all infrastructure and tools. This allows for developer self-serving and tasks are automated where practical, leaving developers to focus on the most mission-critical tasks. The importance of self-service The IDP should provide a team with self-service provisioning, configuration and management of the platform capabilities. According to a whitepaper by , self-service is fundamental to internal developer platforms. Container Solutions "This can be eased by providing the developers with practical user interfaces, either CLI or GUI. The most obvious way is to use, build, or buy some sort of interface that can help users provision services or infrastructure, check their real-time stats and logs, as well as simplifying debugging." The platform model is often associated with cloud-native environments but is also appropriate for many other types of architecture, ranging from modern to legacy. Who uses Internal Developer Platforms? Top engineers in companies such as Google, Netflix, , and Amazon built their own IDPs. Puppet and CircleCI's ninth annual of over 2,400 participants found that 63 percent of respondents say  their company has at least one self-service internal platform, with almost a third of respondents having 25 to 50 percent of developers using an internal platform. It also found that 48% of teams with a high DevOps evolution have IDPs. Github State of DevOps annual report IDP use correlates strongly with high-functioning teams who have evolved to self-serving where: Incident responses are automated Resources are available via self-service Applications are rearchitected based on business needs Security teams are involved in technology design and deployment At last December's KubeCon, 's Senior DevOps Engineer, David Sufia, his team's experience building a Kubernetes-based IDP. He used (CNCF) tools such as Buildpacks, Helm, OpenTelemetry, Prometheus, Envoy, LinkerD, and gRPC to make a much smoother experience for developers. GoSpotCheck shared Cloud Native Computing Foundation David shared that his company had outgrown Heroku PaaS which was causing all kinds of problems. He notes: "We could deploy fast, but we maxed out the number of servers and needed more control and customization. We're too big for platform-as-a-service, but we need to buy as we can't be running it all ourselves." For many companies, Heroku PaaS is/was a traditional place to start building services due to being quick and easy, but the costs build very quickly. There comes a point that many companies outgrow their one-size-fits-all PaaS. They may be planning to adopt a microservices architecture. Their number of developers may have increased with the introduction of a dedicated DevOps engineer. Features such as service discovery, traffic routing, and secret management may have become a priority. People want personalised control, but that takes time to build and resources to maintain. Despite the initial look of high cost, does something-as-a-service that gives you control but ties with common patterns fit into that gap perfectly? You also need to know what outcome you are trying to achieve. According to David, "Just because something is a good idea doesn't mean the process is easy or painless." He shared, "We could do whatever we want, but what do we actually want, and do we have the resources to do it all?" Companies have the option to either their own internal developer platform with a plethora of pros and cons for each choice. buy or build As Kelsey Hightower asserts “I'm convinced the majority of people managing infrastructure just want a PaaS. The only requirement: it has to be built by them.” Internal Developer Platform as a product Global software consultants Thoughtworks that companies that are most successful in introducing an Internal Development Platform are those that apply product management to internal platforms: “This means establishing empathy with internal consumers (the development teams) and collaborating with them on the design.” assert David Sufia also emphasizes the importance of the platform as a product. He notes that where any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure, is a problem in platform teams. Conway's Law, "One of the biggest pitfalls of platform teams that I've seen is that they build things for themselves and platform teams tend to be composed of some of the wonkiest teams who build things that are not what everyone consuming the platform wants to use." In response, GoSpotCheck viewed their IDP as something that needed to be designed, managed, and operated just like any other product within an organization. This correlates with Puppet's research that found highly evolved firms are nearly twice as likely to be highly product-oriented as firms in the middle of their DevOps evolution. They assert that organizational buy-in is crucial to the success of a platform, such as ongoing development and funding. "Over the longer term, you'll build a competency that will become a serious competitive advantage for rolling out future revenue-producing products that can drive your business forward." David's team utilized cross-functional reference groups and made an effort to represent user personas, including senior and junior developers, and created a library of tools. He asserts that CNCF tooling "is at a point of maturity and interoperability where you can dive in as a midsize org and compose something that is going to serve your needs probably pretty well." Getting started with your own Internal Developer Platform Moving to an IDP is best achieved through gradual, incremental processes where the platform capabilities you need will emerge as you test various approaches. Evan Bottcher suggests starting small based on genuine proven needs: "Harvest already proven solutions from application teams, and try joint-ventures to create and test capabilities with the teams that will use them." Further, any platform is only as good as its makers and users, and the decision to build an IDP may result in a company realizing a lack of capabilities or specialist skills. Implementing an IDP is all about interpersonal team relationships. Resources such as Manuel Pais and Matthew Skelton's bring together different frameworks, models, and case studies to provide a functional and team-centered approach to building complex software systems. Team Topologies Container Solutions suggest starting with a system's less critical, smaller applications. "By looking at these components, we can examine the dependencies and create a structure that eliminates many of them or consider tools to achieve the self-service aspects (such as a mixture of Kubernetes IDEs and Operators). Once all the core components are in place for this kind of setup, we can look at migrating this component, working with the engineers in charge of it to understand the engineers' needs and workflows. Once those steps are undertaken, you now have a starting point for your platform." They suggest using a dedicated team that will gradually move more significant, more critical components to the internal developer platform as the Platform Team increases their understanding of the company's specific needs. Looking for more information? The crowdsourced, curated site Internal Developer Platform provides a central repository for IDP resources including articles, experts, and companies, and tools that can help.

How Internal Developer Platforms Impact Your DevOps Team Success

Being an automation tester, we do realize that in a release cycle, time is always of the essence! Selenium test automation helps to save us a considerable amount of time in our test cycles. However, it is pivotal to note the way through which you are executing your Selenium testing scripts. Which frameworks are you using? Are you doing it with an in-house infrastructure or with an online Selenium Grid? Are you making use of build automation tools or not?! Build automation tools like Maven, Gradle and ANT provide you to accelerate the Selenium test automation even further. Not only do they help you manage build lifecycles, dependencies but they also allow you to perform parallel test execution. In this post, we are going to understand why every automation tester needs a build management tool for Selenium testing. What Are Build Automation Tools? Build automation tools allow us to orchestrate our project builds by automating the processes for handling Selenium dependencies, compiling source code to binary & then later packages the binary. All in order to run automation testing. Build automation tools have become pivotal for the software development & testing process. These tools help developers in completing day to day activities like. Downloading dependencies Generation of the source code and extracting documentation from it Compiling the source code Packaging the compiled source code Installing packaged code into a server, local or a central repository Running Tests What Makes Build Tools Special For Selenium Test Automation? Build automation tools play a vital role in Selenium test automation. In most of the small projects, developers usually do the build process manually. But it becomes quite difficult as the projects become bigger. That is why we need to automate these processes and save ourselves some unwanted manual labor. Let us take a look at some striking reasons to build tools that are compelling for every automation tester who is working with Selenium test automation. 1. Supports Continuous Integration and Continuous Delivery Most companies are now moving forward to ensuring the participation of as many teams as possible in a project. Thus, the increase in the adoption of DevOps culture has led to the rise in Continuous Integration. Every firm has a shared repository where the developers integrate the code. But when they do so several times a day, we call this Continuous Integration. You can also verify each integration with the help of an automated build. The practice of Continuous Delivery usually goes hand in hand with CI. Build automation tools easily integrate with systems including Hudson, Jenkins, Bamboo, TravisCI, TeamCity, etc. This way, they help you to not only build a robust project but also help you to support CI/ CD processes which triggers these projects. Let’s take an example of Maven and Jenkins, a build automation tool & a CI/CD tool going hand-in-hand. Maven is a build automation tool that helps you to compile the Selenium dependencies so you don’t end up getting stuck with incorporating JAR files into your Selenium test automation project. It also helps you to run . Selenium testing in parallel Jenkins, on the other hand, offers you a way to create a pipeline to help you trigger these build projects from one testing environment to another. Using Jenkins you can trigger your build automatically along with tests and deployment. 2. Enables Management of Build Lifecycle A build lifecycle defines a sequence of phases to achieve company goals. It defines the process of building and deploying a project. In this case, phases represent different stages in the life cycle. Every firm needs to follow a sequence of steps before they can deliver a product. Since build management comprises hooks or cycle phases, it helps in handling the sequential steps. Default build lifecycle include the following phases mentioned below: validate – to validate the project is correct and all required information is available compile – compile the source code test – test the compiled source code package – to package the compiled code in distributable format, such as a WAR. verify – run any checks on results of integration tests to ensure their quality install – install the package into the local repository deploy – done in the build environment, copies the final package to the remote repository. For instance, you have to compile after completing the coding part. Then comes the packaging phase into a bundle such as WAR, JAR, or EAR. However, you might require some pre-packaging tasks beforehand. Build management enables the use of cycle phases for achieving all the above-mentioned tasks. 3. Comprises Impeccable Plugin Support Most build automation tools contain add-ons and other tools which provide benefits in addition to the existing capabilities. Some of these plugins are crucial for core tasks. Plugins can help in adding goals to our phases in our projects. For example, the has two goals, compile and testcompile. ‘ ’ , helps to compile the source code of the main code, while ‘ ’ helps to compile the source code of the test code. Compiler Plugin Compile testcompile On the other hand, we can customize some for performing tasks according to individual needs. They also automate common functions such as open, copy, zip or rename files, and reconstruct folders. In addition to automating these common functions, these plugins reduce manual labor in the area of text modification and validation of the project structure. Plugin support also includes static code analysis, compiling tests, running tests, generating test reports, etc. Not only that, but these plugins also help other teams use the Selenium test automation frameworks over different programming languages. They automate the creation, assembly, and packaging of executables with the help of metadata. After that, we can install these into our local repository and deploy them to a remote repository for other teams to use. 4. Execution Of Parallel Testing One of the most lucrative reasons to opt to build tools for Selenium test automation is to leverage parallel testing. Automation testing is a big time-saver for developers and testers! However, if executed in a sequential manner, it can slow the team rather than accelerating it. Your websites or web-applications are bound to grow over time, and with it grows your Selenium test automation suites or requirements. Eventually, you will reach a point where you would have to leverage the parallel testing in Selenium, to deliver your projects before the deadline. Build automation tools can help you do that with ease. In fact, there are few test automation frameworks such as Gauge which allows parallel testing with Selenium through Maven plugin. Here is an example from one of our . GitHub repositories to run Gauge with Selenium com.thoughtworks.gauge.maven gauge-maven-plugin 1.3.3 test-chrome test chrome true 2 specs execute < > groupId < / > groupId < > artifactId </ > artifactId < > version </ > version < > executions < > execution < > id < / > id < > phase < / > phase < > configuration < > env < / > env < > inParallel < / > inParallel < > nodes </ > nodes < > specsDir < / > specsDir < / > configuration < > goals < > goal < / > goal </ > goals < / > execution 5. Enables Management of Dependencies When one piece of software depends on another, we call that a dependency. They include files, a group of files in a package, jars, internal and external libraries present on the network. It’s important to resolve and manage these dependencies. This is where build tools play a crucial rule. Build tools enable an automated resolution of dependencies. For instance, you wish to use the current and updated version of a library. Now, if you do this manually, you will have to remove the current JAR file first. After that, you have to download the latest version from a repository and add it to the project. In some cases, you might even need to download transitive dependencies and add them to your project as well. Build tools help us modify dependency configurations which are flexible. These configurations are in the form of files such as build.Gradle, pom.xml, etc. Build tools can also create local structures or local repositories. After that, they download and cache the dependencies and do this again if needed. 6. Correctly Executes Order of Commands Commands with dependencies are the most time-consuming to execute. But knowing how you can order command execution is even more challenging. This is because in most cases, we need to feed the output of one command as an input to another. Thus, if the project is massive, it becomes quite overwhelming to manage it all manually. Build tools to automate these tasks and help employees concentrate on other important work. These tools also automate the building of executables. This way, every developer can build as well as run the software on their machine. List Of Renowned Build Automation Tools I hope you now know the perks of using build tools for Selenium test automation. Now, the next question that arises is. Which one to use? Well, that depends upon which for testing are you comfortable with? programming language Java : An open-source build automation tool, Maven helps to report and document from a central piece of information. Maven can also be used to manage projects written in C# and ruby. Maven : Also an open-source builds automation tool. Instead of using the XML form used by Maven, Gradle uses Groovy-based domain-specific language. Gradle Python : PyBuilder is a build tool written purely in Python.PyBuilder is based on the concept of dependency based programming, but it also comes with a powerful plugin mechanism, allowing the construction of build life cycles similar to those known from other famous (Java) build tools. Pybuilder Ruby : With 1,700+ GitHub Stars, 530+ Forks, Rake is a build automation tool. Rake helps the user to define tasks and describe dependencies. Rake PHP : PHP tool based on Apache ANT, It uses XML build files and extensible PHP task classes which make it easy to use. Phing C# : CMake is an open-source, build automation tool designed to build, test and package software. CMake : Microsoft Build Engine, better known as MSBuild, is a free and open-source build automation tool. MS Build JavaScript : With 31,600+ GitHub Stars, 4,400+ Forks and more than 3300+ plugins, Gulp is a streaming build system for node.js to help developers automate & enhance their development workflows. Gulp : With 12,000+ GitHub Stars, 1500+ Forks, and more than 6300+ plugins, Grunt is a renowned task runner for JavaScript, which was originally designed with the purpose of delivering a build tool interface to JavaScript developers through the command line. Grunt That Is All! With an increased focus on Selenium test automation, the lack of awareness about build tools can hold you back. Automating repetitive tasks is a major factor for delivering your projects on time. Doing so in parallel with the right build automation tool can make your release past like a breeze! If you are a novice automation tester, then it is important for you to keep in mind the benefits of using build tools for your upcoming Selenium test automation projects. If you are finding it a little overwhelming then it is best recommended to start small. Start off by tweaking little things in bits to automate smaller test cases. Gradually, you & your team can have a more streamlined process for fully utilizing the potential of build automation tools. Previously published at https://www.lambdatest.com/blog/why-you-need-build-automation-tools-for-selenium-automation-testing/

ThoughtWorks News on
HackerNoon and Around the Web

thoughtworks.com

12500 employees

Since 1993

55503 at Alexa

📉 Stock Prices

ThoughtWorks Story Mentions

Articles Around the Web

Read More Tech Stories Related to #ThoughtWorks

ThoughtWorks News on HackerNoon and Around the Web