2TEST is the datawarehouse test automation tool

2TEST

2test

Dushyant Chautala, the friend-turned-foe of the BJP who is out to fell the government, should police his party, sources have indicated.

Amid Haryana Crisis, 4 MLAs Of Dushyant Chautala Meet BJP's ML Khattar

Donald Trump is heading into the most perilous phase of his hush money trial aside from the moment when the jury retires to consider its verdict.

Analysis: Critical days of Trump trial will test whether he can exercise discipline and restraint | CNN Politics

OTTAWA — The federal government will test its capacity to issue emergency alerts today. That means Canadians in most provinces and territories will receive a test alert on their cellphones, as well as through TV and radio broadcasts. The sole exception is Ontario, where the test will take ...

Most of Canada to receive emergency alert test today

Canada&#x27;s national public alert system is running tests on television, radio and compatible wireless devices across most of the country on May 8.

There's a test of the national emergency alert system in most of Canada Wednesday | CBC News

The federal government will test its capacity to issue emergency alerts today, with the exception of Ontario, where the test will take place on May 15.

Cellphone emergency alert system testing in Canada | CTV News

On May 8, the National Public Alerting System, Alert Ready, will send out an emergency alert test message at 1:55 p.m.

Not the real deal: Manitobans to receive emergency alert test - Winnipeg | Globalnews.ca

"How Many Colors Can the Human Eye See?": The Application

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Andrey Zhmoginov, Google Research & {azhmogin,sandler,mxv}@google.com;
(2) Mark Sandler, Google Research & {azhmogin,sandler,mxv}@google.com;
(3) Max Vladymyrov, Google Research & {azhmogin,sandler,mxv}@google.com. Table of Links Abstract and Introduction
Problem Setup and Related Work
HyperTransformer
Experiments
Conclusion and References
A Example of a Self-Attention Mechanism For Supervised Learning
B Model Parameters
C Additional Supervised Experiments
D Dependence On Parameters and Ablation Studies
E Attention Maps of Learned Transformer Models
F Visualization of The Generated CNN Weights
G Additional Tables and Figures C ADDITIONAL SUPERVISED EXPERIMENTS While the advantage of decoupling parameters of the weight generator and the generated CNN model is expected to vanish with the growing CNN model size, we compared our approach to two other methods, LGM-Net (Li et al., 2019b) and LEO (Rusu et al., 2019), to verify that our approach can match their performance on sufficiently large models. For our comparison with the LGM-Net method, we used the same image augmentation technique that was used in Li et al. (2019b) where it was applied both at the training and the evaluation stages (Ma, 2019). We also used the same CNN architecture with 4 learned 64-channel convolutional layers followed by two generated convolutional layers and the final logits layer. In our weight generator, we used 2-layer transformers with local feature extractors that relied on 48-channel convolutional layers and did not use any global features. We trained our model in the end-to-end fashion on the MINIIMAGENET 1-shot-5-way task and obtained a test accuracy of 69.3% ± 0.3% almost identical to the 69.1% accuracy reported in Li et al. (2019b). We also carried out a comparison with LEO by using our method to generate a fully-connected layer on top of the TIEREDIMAGENET embeddings pre-computed with a WideResNet-28 model employed by Rusu et al. (2019). For our experiments, we used a simpler 1-layer transformer model with 2 heads that did not have the final fully-connected layer and nonlinearity. We also used L2 regularization of the generated fully-connected weights setting the regularization weight to 10−3. As a result of training this model, we obtained 66.2% ± 0.2% and 81.6% ± 0.2% test accuracies on the 1-shot-5-way and 5-shot-5-way TIEREDIMAGENET tasks correspondingly. These results are almost identical to 66.3% and 81.4% accuracies reported in Rusu et al. (2019). This paper is available on arxiv under CC 4.0 license. Authors: (1) Andrey Zhmoginov, Google Research & {azhmogin,sandler,mxv}@google.com; (2) Mark Sandler, Google Research & {azhmogin,sandler,mxv}@google.com; (3) Max Vladymyrov, Google Research & {azhmogin,sandler,mxv}@google.com. This paper is available on arxiv under CC 4.0 license. Authors: (1) Andrey Zhmoginov, Google Research & {azhmogin,sandler,mxv}@google.com; (2) Mark Sandler, Google Research & {azhmogin,sandler,mxv}@google.com; (3) Max Vladymyrov, Google Research & {azhmogin,sandler,mxv}@google.com. Table of Links Abstract and Introduction Problem Setup and Related Work HyperTransformer Experiments Conclusion and References A Example of a Self-Attention Mechanism For Supervised Learning B Model Parameters C Additional Supervised Experiments D Dependence On Parameters and Ablation Studies E Attention Maps of Learned Transformer Models F Visualization of The Generated CNN Weights G Additional Tables and Figures Abstract and Introduction Abstract and Introduction Problem Setup and Related Work Problem Setup and Related Work HyperTransformer HyperTransformer Experiments Experiments Conclusion and References Conclusion and References A Example of a Self-Attention Mechanism For Supervised Learning A Example of a Self-Attention Mechanism For Supervised Learning B Model Parameters B Model Parameters C Additional Supervised Experiments C Additional Supervised Experiments D Dependence On Parameters and Ablation Studies D Dependence On Parameters and Ablation Studies E Attention Maps of Learned Transformer Models E Attention Maps of Learned Transformer Models F Visualization of The Generated CNN Weights F Visualization of The Generated CNN Weights G Additional Tables and Figures G Additional Tables and Figures C ADDITIONAL SUPERVISED EXPERIMENTS While the advantage of decoupling parameters of the weight generator and the generated CNN model is expected to vanish with the growing CNN model size, we compared our approach to two other methods, LGM-Net (Li et al., 2019b) and LEO (Rusu et al., 2019), to verify that our approach can match their performance on sufficiently large models. For our comparison with the LGM-Net method, we used the same image augmentation technique that was used in Li et al. (2019b) where it was applied both at the training and the evaluation stages (Ma, 2019). We also used the same CNN architecture with 4 learned 64-channel convolutional layers followed by two generated convolutional layers and the final logits layer. In our weight generator, we used 2-layer transformers with local feature extractors that relied on 48-channel convolutional layers and did not use any global features. We trained our model in the end-to-end fashion on the MINIIMAGENET 1-shot-5-way task and obtained a test accuracy of 69.3% ± 0.3% almost identical to the 69.1% accuracy reported in Li et al. (2019b). We also carried out a comparison with LEO by using our method to generate a fully-connected layer on top of the TIEREDIMAGENET embeddings pre-computed with a WideResNet-28 model employed by Rusu et al. (2019). For our experiments, we used a simpler 1-layer transformer model with 2 heads that did not have the final fully-connected layer and nonlinearity. We also used L2 regularization of the generated fully-connected weights setting the regularization weight to 10−3. As a result of training this model, we obtained 66.2% ± 0.2% and 81.6% ± 0.2% test accuracies on the 1-shot-5-way and 5-shot-5-way TIEREDIMAGENET tasks correspondingly. These results are almost identical to 66.3% and 81.4% accuracies reported in Rusu et al. (2019).

HyperTransformer: C Additional Supervised Experiments

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Sebastien Ferr ´e, Univ Rennes, CNRS, Inria, IRISA Campus de Beaulieu, 35042 Rennes, France and ferre@irisa.fr. Table of Links Abstract & Introduction Abstraction and Reasoning Corpus (ARC) Related Work Object-centric Models for ARC Grids MDL-based Model Learning Evaluation Conclusion and Perspectives, References & Supplementary Materials 2 Abstraction and Reasoning Corpus (ARC) ARC is a collection of tasks3 , where each task is made of training examples (3.3 on average) and test examples (1 in general). Each example is made of an input grid and an output grid. Each grid is a 2D array (with size up to 30x30) filled with integers coding for colors (10 distinct colors). For a given task, the size of grids can vary from one example to another, and between the input and the output. Each task is a machine learning problem, whose goal is to learn a model that can generate the output grid from the input grid, and so from a few training examples only. Prediction is successful only if the predicted output grid is strictly equal to the expected grid for all test examples, there is no partial success. However, three trials are allowed for each test example to compensate for potential ambiguities in the training examples. Figure 1 shows two ARC tasks (with the expected test output grid missing). The first is used as a running example in this paper. We now more formally define grids, examples, and tasks. ARC tasks have 3.3 training examples on average, and 1 or 2 test examples (most often 1). As illustrated by Figure 1, the different input grids of a task need not have the same size, nor use the same colors. The same applies to test grids. ARC is composed of 1000 tasks in total: 400 “training tasks”4 , 400 evaluation tasks, and 200 secret tasks for independent evaluation. Figure 1 shows two of the 400 training tasks. Developers should only look at the training tasks, not at the evaluation tasks. The latter should only be used to evaluate the broad generalization capability of the developed systems. This paper is available on Arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. Authors: (1) Sebastien Ferr ´e, Univ Rennes, CNRS, Inria, IRISA Campus de Beaulieu, 35042 Rennes, France and ferre@irisa.fr. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Sebastien Ferr ´e, Univ Rennes, CNRS, Inria, IRISA Campus de Beaulieu, 35042 Rennes, France and ferre@irisa.fr. Table of Links Abstract & Introduction Abstract & Introduction Abstraction and Reasoning Corpus (ARC) Abstraction and Reasoning Corpus (ARC) Related Work Related Work Object-centric Models for ARC Grids Object-centric Models for ARC Grids MDL-based Model Learning MDL-based Model Learning Evaluation Evaluation Conclusion and Perspectives, References & Supplementary Materials Conclusion and Perspectives, References & Supplementary Materials 2 Abstraction and Reasoning Corpus (ARC) ARC is a collection of tasks3 , where each task is made of training examples (3.3 on average) and test examples (1 in general). Each example is made of an input grid and an output grid. Each grid is a 2D array (with size up to 30x30) filled with integers coding for colors (10 distinct colors). For a given task, the size of grids can vary from one example to another, and between the input and the output. Each task is a machine learning problem, whose goal is to learn a model that can generate the output grid from the input grid, and so from a few training examples only. Prediction is successful only if the predicted output grid is strictly equal to the expected grid for all test examples, there is no partial success. However, three trials are allowed for each test example to compensate for potential ambiguities in the training examples. Figure 1 shows two ARC tasks (with the expected test output grid missing). The first is used as a running example in this paper. We now more formally define grids, examples, and tasks. ARC tasks have 3.3 training examples on average, and 1 or 2 test examples (most often 1). As illustrated by Figure 1, the different input grids of a task need not have the same size, nor use the same colors. The same applies to test grids. ARC is composed of 1000 tasks in total: 400 “training tasks”4 , 400 evaluation tasks, and 200 secret tasks for independent evaluation. Figure 1 shows two of the 400 training tasks. Developers should only look at the training tasks, not at the evaluation tasks. The latter should only be used to evaluate the broad generalization capability of the developed systems. This paper is available on Arxiv under CC 4.0 license. Arxiv Arxiv

The Definition of Abstraction and Reasoning Corpus (ARC): A Beginner's Lesson

Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the gpt turbo model for code generation. Introduction The Microsoft team has recently launched an challenge in which a developer can learn how to build Azure AI solutions and apps. Open AI Prerequisite Experience working with Azure and Azure portals. An understanding of generative AI. Experience in one high-level programming language like C# or Python Steps to Create Open AI Service on Azure with “Dall-E” model deployed. Day 1 — Azure Open AI Challenge If you would like to explore image generation, follow this link Day 2 — Azure Open AI Challenge: Image Generation Getting Started Considering Azure Open AI Service is running on the Azure portal and the gpt-turbo model is deployed successfully. Step 1: Create a console application To test image generation, create a console application in Visual Studio or Visual Studio Code. dotnet new console Step 2: Read the configuration Read the configuration from appsettings.json file // Build a config object and retrieve user settings.
    IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .Build();
    string? oaiEndpoint = config["AzureOAIEndpoint"];
    string? oaiKey = config["AzureOAIKey"];
    string? oaiDeploymentName = config["AzureOAIDeploymentName"]; Step 3: Add sample code Create a file under the console application project as follows: sample-code> function > function.cs namespace Function
    {
        class Program
        {
            int AbsoluteSquare(int num1, int num2)
            {
                int result = Math.Abs(num1 - num2);
                result *= result;
                return result;
            }
        }
    } Another file as sample-code > go-fish > go-fish.cs => Reference file Step 3: Prepare Console Menu The next step is to prepare a menu on the console application using digits. If the console message is “quit” the console application is exited. If the console input is 1 or 2, then the sample file function.cs is used If the console input is 3, then the sample file go-fish.cs is used Else it will return a console message saying invalid input do
    {
        Console.WriteLine("\n1: Add comments to my function\n" +
        "2: Write unit tests for my function\n" +
        "3: Fix my Go Fish game\n" +
        "\"quit\" to exit the program\n\n" +
        "Enter a number to select a task:");
    
        command = Console.ReadLine() ?? "";
    
        if (command == "quit")
        {
            Console.WriteLine("Exiting program...");
            break;
        }
    
        Console.WriteLine("\nEnter a prompt: ");
        string userPrompt = Console.ReadLine() ?? "";
        string codeFile = "";
    
        if (command == "1" || command == "2")
            codeFile = System.IO.File.ReadAllText($@"{AppDomain.CurrentDomain.BaseDirectory}sample-code\function\function.cs");
        else if (command == "3")
            codeFile = System.IO.File.ReadAllText($@"{AppDomain.CurrentDomain.BaseDirectory}sample-code\go-fish\go-fish.cs");
        else
        {
            Console.WriteLine("Invalid input. Please try again.");
            continue;
        }
    
        userPrompt += codeFile;
        await GetResponseFromOpenAI(userPrompt);
    } while (true); Step 4: Code generation from AI Please find below the method GetResponseFromOpenAI, which generates the code using gpt-turbo. Read the configuration values required, if not present don’t execute. Prepare the request for chat completion response. Finally, write the response back to result > app.txt file async Task GetResponseFromOpenAI(string prompt)
    {
        Console.WriteLine("\nCalling Azure OpenAI to generate code...\n\n");
    
        if (string.IsNullOrEmpty(oaiEndpoint) || string.IsNullOrEmpty(oaiKey) || string.IsNullOrEmpty(oaiDeploymentName))
        {
            Console.WriteLine("Please check your appsettings.json file for missing or incorrect values.");
            return;
        }
    
        // Configure the Azure OpenAI client
        OpenAIClient client = new OpenAIClient(new Uri(oaiEndpoint), new AzureKeyCredential(oaiKey));
    
        // Define chat prompts
        string systemPrompt = "You are a helpful AI assistant that helps programmers write code.";
        string userPrompt = prompt;
    
        // Format and send the request to the model
        var chatCompletionsOptions = new ChatCompletionsOptions()
        {
            Messages =
            {
                new ChatRequestSystemMessage(systemPrompt),
                new ChatRequestUserMessage(userPrompt)
            },
            Temperature = 0.7f,
            MaxTokens = 1000,
            DeploymentName = oaiDeploymentName
        };
    
        // Get response from Azure OpenAI
        Response<ChatCompletions> response = await client.GetChatCompletionsAsync(chatCompletionsOptions);
    
        ChatCompletions completions = response.Value;
        string completion = completions.Choices[0].Message.Content;
    
        // Write full response to console, if requested
        if (printFullResponse)
        {
            Console.WriteLine($"\nFull response: {JsonSerializer.Serialize(completions, new JsonSerializerOptions { WriteIndented = true })}\n\n");
        }
    
        // Write the file.
        System.IO.File.WriteAllText(@"result\app.txt", completion);
    
        // Write response to console
        Console.WriteLine($"\nResponse written to result/app.txt\n\n");
    } Test Case 1: Add comments to the code Choose option 1 Then add the following prompt to add comments to the code Add comments to the following function. Return only the commented code. Output Test Case 1 Refer to the sample on . function.cs file GitHub Please find below the convert function with the single line comments added as shown below. // This function calculates the absolute square of the difference between two numbers
    
    namespace Function
    {
        class Program
        {
            // Function to calculate the absolute square of the difference between two numbers
            int AbsoluteSquare(int num1, int num2)
            {
                // Calculate the absolute difference between the two numbers
                int result = Math.Abs(num1 - num2);
    
                // Square the result
                result *= result;
    
                // Return the absolute square of the difference between the two numbers
                return result;
            }
        }
    } Additionally, the user can suggest gpt model to add multi-line comments instead of single lines using the following prompt Add comments to the following function. Return only the commented code. — namespace Function
    {
        class Program
        {
            // This function calculates the absolute square of the difference between two integers.
            // Parameters:
            //   num1: the first integer
            //   num2: the second integer
            // Returns:
            //   The absolute square of the difference between num1 and num2
            int AbsoluteSquare(int num1, int num2)
            {
                int result = Math.Abs(num1 - num2);
                result *= result;
                return result;
            }
        }
    } Test Case 2: Add comments to the code Choose option 2 Then add the following prompt to add comments to the code Write four unit tests for the following function. — Output Test Case 2 Refer to the sample on . Please find below the test cases added function.cs file GitHub Sure! Here are four unit tests for the `AbsoluteSquare` function:
    
    1. Test case where `num1` is greater than `num2`:
    
    [Test]
    public void TestAbsoluteSquare_GreaterNum1_ReturnsSquaredDifference()
    {
        int num1 = 5;
        int num2 = 2;
        int expected = 9;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    }
    
    2. Test case where `num2` is greater than `num1`:
    
    [Test]
    public void TestAbsoluteSquare_GreaterNum2_ReturnsSquaredDifference()
    {
        int num1 = 2;
        int num2 = 5;
        int expected = 9;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    }
    
    3. Test case where `num1` and `num2` are the same:
    
    [Test]
    public void TestAbsoluteSquare_SameNumbers_ReturnsZero()
    {
        int num1 = 4;
        int num2 = 4;
        int expected = 0;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    }
    
    
    4. Test case where `num1` and `num2` are negative:
    
    [Test]
    public void TestAbsoluteSquare_NegativeNumbers_ReturnsSquaredDifference()
    {
        int num1 = -3;
        int num2 = -6;
        int expected = 9;
        int result = Function.Program.AbsoluteSquare(num1, num2);
        Assert.AreEqual(expected, result);
    } Test Case 3: Fix my Go Fish game Choose option 3 Then add the following prompt to add comments to the code Fix the code below for an app to play Go Fish with the user. Return only the corrected code. — Output Test Case 3 Refer to the sample on . go-fish.cs file GitHub Please find below the code that is fixed. using System;  
    using System.Collections.Generic;  
    using System.Linq;  
      
    namespace GoFish  
    {  
        class Program  
        {  
            static void Main(string[] args)  
            {  
                // Define the deck of cards  
                List<string> deck = new List<string> { "A", "2", "3", "4", "5", "6", "7", "8", "9", "10", "J", "Q", "K" };  
                deck.AddRange(deck);  
                deck.AddRange(deck);  
                deck.AddRange(deck);  
      
                // Shuffle the deck  
                var rand = new Random();  
                deck = deck.OrderBy(x => rand.Next()).ToList();  
      
                // Deal the cards  
                List<string> playerHand = deck.Take(5).ToList();  
                List<string> computerHand = deck.Skip(5).Take(5).ToList();  
      
                // Define the initial score  
                int playerScore = 0;  
                int computerScore = 0;  
      
                // Define the main game loop  
                while (deck.Count > 0)  
                {  
                    // Print the player's hand  
                    Console.WriteLine("Your hand: " + string.Join(", ", playerHand));  
      
                    // Ask the player for a card  
                    Console.Write("Do you have any... ");  
                    string card = Console.ReadLine();  
      
                    // Check if the player has the card  
                    if (playerHand.Contains(card))  
                    {  
                        // Remove the card from the player's hand  
                        playerHand.Remove(card);  
      
                        // Add a point to the player's score  
                        playerScore += 1;  
      
                        // Print the player's score  
                        Console.WriteLine("You got a point!");  
                        Console.WriteLine("Your score: " + playerScore);  
                    }  
                    else  
                    {  
                        // Go fish!  
                        Console.WriteLine("Go fish!");  
      
                        // Draw a card from the deck  
                        string drawnCard = deck.First();  
                        deck.RemoveAt(0);  
      
                        // Add the card to the player's hand  
                        playerHand.Add(drawnCard);  
      
                        // Print the card that was drawn  
                        Console.WriteLine("You drew a " + drawnCard);  
                    }  
      
                    // Check if the player has won  
                    if (playerScore == 5)  
                    {  
                        Console.WriteLine("You win!");  
                        break;  
                    }  
      
                    // Computer's turn  
                    string computerCard = computerHand[rand.Next(computerHand.Count)];  
                    Console.WriteLine("Do you have any " + computerCard + "?");  
                    if (playerHand.Contains(computerCard))  
                    {  
                        // Remove the card from the player's hand  
                        playerHand.Remove(computerCard);  
      
                        // Add a point to the computer's score  
                        computerScore += 1;  
      
                        // Print the computer's score  
                        Console.WriteLine("The computer got a point!");  
                        Console.WriteLine("Computer score: " + computerScore);  
                    }  
                    else  
                    {  
                        // Go fish!  
                        Console.WriteLine("The computer goes fishing!");  
      
                        // Draw a card from the deck  
                        string drawnCard = deck.First();  
                        deck.RemoveAt(0);  
      
                        // Add the card to the computer's hand  
                        computerHand.Add(drawnCard);  
      
                        // Print the card that was drawn  
                        Console.WriteLine("The computer drew a " + drawnCard);  
                    }  
      
                    // Check if the computer has won  
                    if (computerScore == 5)  
                    {  
                        Console.WriteLine("The computer wins!");  
                        break;  
                    }  
                }  
            }  
        }  
    } Finally, if you type quit, the console application will be terminated. Complete code on GitHub Make sure to give it a star on GitHub and provide feedback on how to improve the tool further..!! ssukhpinder/AzureOpenAI C# Programming🚀 Thank you for being a part of the C# community! Before you leave: If you’ve made it this far, please show your appreciation with a clap and follow the author! 👏️️ Follow us: | | | | | X LinkedIn Dev.to Hashnode Newsletter Tumblr Visit our other platforms: | | | | GitHub Instagram Tiktok Quora Daily.dev More content at C# Programming Also published . here

Taking the Azure Open AI Challenge, Day 3: Mastering Code Generation

This paper is available on arxiv under CC 4.0 license. Authors: (1) Ali Ghanbari, Dept. of Computer Science, Iowa State University; (2) Deepak-George Thomas, Dept. of Computer Science, Iowa State University; (3) Muhammad Arbab Arshad, Dept. of Computer Science, Iowa State University; (4) Hridesh Rajan, Dept. of Computer Science, Iowa State University. Table of Links Abstract & Introduction Background Motivating Example Proposed Approach Supported DNN Bugs Evaluation Threats to Validity Related Work Conclusion, Acknowledgments & References IV. PROPOSED APPROACH Our technique deepmufl comprises four components: (1) mutation generator, (2) test case splitter, (3) mutation executor/tester, and (4) suspiciousness calculator. Figure 2 depicts these components as processes, numbered accordingly, taking inputs and producing outputs. Mutation generator (marked ⃝1 in Figure 2), applies 79 mutators on all the layers of the input buggy DNN, so as to generate a pool of mutants, i.e., variants of the original, buggy DNN model with small perturbations, e.g., replacing activation function of a layer. Test case splitter (marked ⃝2 in the figure), applies the original buggy DNN on a given set of test data points, i.e., test cases (or input values) paired with their expected output values, so as to partition the set into two subset, namely passing test cases and failing test cases. Passing test cases are referred to as those input values for which the expected output matches that of produced by the original model, whereas failing test cases are referred to as those input values for which the expected output does not match that of produced by the model. This component also stores the output of the original model on each of the test cases. Next, mutation executor (which is also called mutation tester, marked ⃝3 in the figure) applies the generated mutants on each of the passing and failing test cases and the results are compared to that of the original model recorded in the previous step. This amounts to a mutation execution matrix that is used to calculate suspiciousness values for each layer in the model (marked ⃝4 in the figure). The user may instruct deepmufl to use a specific fault localization formula, e.g., MUSE or Metallaxis with SBI or Ochiai, for calculating suspiciousness values. The layers are then ranked based on the calculated suspiciousness values for user inspection. The ranked list is accompanied with the information about the mutations conducted on each layer to facilitate debugging. A. Mutation Generator Mutation generator component receives the original, buggy DNN model and generates as many variants of the model, i.e., mutants, as possible, by systematically mutating every elements of the input model. This component implements 79 mutators. Mutators can be viewed as transformation operators that when applied to a given element, e.g., a neuron or a layer, in the model, returns a new variants of the model with that particular element mutated. Table 2 lists all the mutators implemented in deepmufl, the types of model elements on which they can operate, and the way each mutator affects the target element. These mutators are inspired by the ones implemented in the existing mutation analysis systems, e.g., [27], [28], [29], [49], [50], [51], [52], [53], to name a few. Ma et al. [29], and Hu et al. [28], define so-called model-level mutators that also operate on pre-trained models. Direct reuse of all of their mutators was not possible, as those mutators depend on random values which would introduce a source of nondeterminism in deepmufl: mutating the same model element with random values, e.g., Gaussian fuzzing, as in [29], would yield a different mutant each time, making deepmufl to produce different outputs on each run for the same model. In general, as far as MBFL is concerned, using variable values (whether it is deteministic or not), instead of the current hard-coded ones, for mutation of weights would not bring about any benefit, as the goal here is to break the model in some way and observe its impact on originally failing and passing test cases. We argue that not all model bugs could be emulated using mutators at the level of pre-trained models, e.g., missing batch normalization, but the mutators listed in Table 2 are sufficient for emulating a subset of such bugs, e.g., wrong activation function or missing/redundant layer. Please see §V for a more detailed discussion on supported bugs. Mutation generation in deepmufl is done directly on the trained model and there is no need for retraining the model. This makes deepmufl quite fast and perhaps more attractive from a practical point of view. However, this comes with a risk of not being traceable, i.e., a mutation on pre-trained .h5 model does not directly correspond to a line of source code for the user to inspect. In the Keras programs that we studied, this limitation was mitigated by the fact that the models with Sequential architecture were implemented using a wellunderstood structure and mapping layer numbers/identifiers in deepmufl’s reports to source code was trivial. In a future work, with the help of simple auto-generated annotations, e.g., for lexical scoping of the code snippet for model declaration, we will extend deepmufl to automatically map layer numbers/identifiers in its reports to actual lines of source code. Humbatova et al. [27] argue about the importance of mutators emulating real DNN faults. We acknowledge that mutators emulating real faults would help generating more informative reports that would also give hints on how to fix the program. However, unlike mutation analysis, the main objective of an MBFL technique is to assign suspiciousness values to the program elements which can, in theory, be done using any kind of mutator, whether or not it makes sense from the standpoint of a developer. It is worth noting that the alternative design decision of using DeepCrime [27] as a mutation generation engine for deepmufl would result in finding more bugs than the current version of deepmufl, e.g., bugs related to training hyper-parameters or training dataset, but such a design is expected to be impacted by the nondeterminacy inherent in training process and, given the fact that we do not employ any training data selection, would be significantly slower due to numerous re-training. Nevertheless, finding more bugs would be an incentive for exploring this alternative as a future work. B. Test Case Splitter Before we introduce this component, we need to clarify certain terminology. .** Definition 1 A data point in a testing dataset for a DNN model is defined to be a pair of the form (I, O), where I ∈ R m and O ∈ R n, with m and n being positive integers. In this paper I is called test case, test input, or input, while O is called expected output or ground-truth value. Given a test dataset, test case splitter component applies the original model on each of the test cases for the data points and checks if the model output matches to the expected output. If the two outputs match, then the corresponding test case will be marked as passing, otherwise it will be marked as failing. This component also records the output produced by the original model to be used during impact analysis, described below. C. Mutation Executor (Mutation Tester) We start describing this component with a definition. Definition 2. A mutation execution matrix E is a k×l matrix, where k is the number of generated mutants, while l is the number of test cases. Each element Ei,j in the matrix is a member of the set {✓, ✗, ❍}, wherein ✓ indicates that the i th mutant impacts j th test case, whereas ✗ indicates that the mutant does not affect the test case. ❍ denotes a nonviable mutant, i.e., a mutant that fails when loading or applying it on a test case. Such mutants might be generated, e.g., due to creating a shape error [32] after the mutation. Mutation executor component construct mutation execution matrix by applying each of the generated mutants (see §IV-A) on the failing and passing test cases to determine which of the test cases are impacted by which mutants. The impact of mutation on test cases is measured using two types of impacts, i.e., type 1 impact and type 2 impact, defined below. Definition 3. Given a DNN model M, its mutated version M′ , and a data point (I, O), we define the two types of impacts: • Type 1: Mutant M′ impacts the test case I if M(I) = O but M′ (I) ̸= O, or M(I) ̸= O but M′ (I) = O. In other words, type 1 impact tracks pass to fail and fail to pass test cases, a la MUSE [17]. ` • Type 2: Mutant M′ impacts the test case I if M(I) ̸= M′ (I). In this definition, M(I) or M′ (I) denotes the operation of applying model M or M′ on the test case I. It is worth noting that checking for equality of two values can be tricky for regression models, as those models approximate the expected values. To work around this problem, deepmufl compares values obtained from regression models using a user-defined delta threshold, i.e., the values are deemed equal if their absolute difference is no more than a threshold. By default, deepmufl uses a threshold of 0.001. This is the approach adopted by well-known testing frameworks for comparing floating-point values [54], [55]. Also, whether deepmufl uses type 1 or type 2 impact is a user preference and is determined alongside the threshold. D. Suspiciousness Value Calculator Armed with the above definitions, we can now give concrete definitions for the terms used in Eq. 1, 2, and 3, specialized to DNNs. • Given a model element, i.e., neuron or a layer, e, M(e) is defined to be the set of all mutants generated by mutating e. These sets are produced by mutation generator process. • Assuming that m is a mutant on the element e, Tf (m, e) (or Tp(m, e)) is defined as the set of failing (or passing) test cases that are impacted in type 1 or type 2 fashion by m. More concretely, Tp(m, e) = {t | Em,t = ✓ ∧ t is passing}, similarly Tf (m, e) = {t | Em,t = ✓ ∧ t is failing}. These two sets are defined using a quite similar notation; the readers are encouraged to review Definitions 2 and 3 to avoid confusion. • Tf (or Tp) is the set of originally failing (or passing) test cases. These sets are constructed by test case splitter. • F ⇝ P (or P ⇝ F), for a model element e, is defined as the set of originally failing (or passing) test cases that turned into passing (or failing) as a result of some mutation on e. More concretely, assuming the execution matrix E is constructed using type 1 impact, F ⇝ P is defined as {t | t is failing ∧ ∃m ∈ M(e) · Em,t = ✓}. Similarly, P ⇝ F is defined as {t | t is passing ∧ ∃m ∈ M(e)·Em,t = ✓}. In other words, these sets track all the failing/passing test cases that are type 1 impacted by some mutant of a given element. These two sets are defined using a quite similar notation; the readers are encouraged to review Definitions 2 and 3 to avoid confusion. Having specialized definitions for the terms used in the fault localization formulas described earlier, we are now able to calculate suspiciousness values for the elements in a DNN model. Guided by the user preferences, deepmufl calculates all the values for |Tf (m, e)|, etc., and plugs into the specified formula to calculate suspiciousness values for the elements. It is worth noting that if all the mutants generated for a given element are nonviable, MUSE formula (Eq. 3) and all the variants of Metallaxis (e.g., Eq. 1), by definition, will return 0 as the suspiciousness value for the element. Nonviable mutants do not contribute toward localizing the bug, therefore they are considered overhead to the fault localization process. Fortunately, based on our observations in our dataset of buggy DNNs, nonviable mutants are rare. Equivalent mutants are another source of overhead for deepmufl. Currently, we do not have any means of detecting equivalent mutants, but we argue that these mutants do not impact MBFL results, as they are equivalent to the original model and do not impact any passing or failing test cases.

Mutation-based Fault Localization of Deep Neural Networks: Proposed Approach

The market offers a wide range of solution providers to choose from. The Best Identity Verification Software category on any listing platform includes over 200 products. Add to it all sorts of hardware devices, and you’ll get the impression that the market is over-saturated, and there are plenty of go-to solutions for handling ID documents and Biometric verification for any business use case. Identity Verification Below is a 9-step guide that will empower you with insights on how to choose the best Identity Verification (IDV) software provider effectively, guiding your selection towards ones that excel in delivering precision, security, and the utmost trustworthiness in the field of identity verification. Step 1: Make sure that document capture is fully automated If you ask any forensic expert how to verify ID documents reliably, they will likely give you the following answer: first of all, make sure you have a physical document at hand. The reality, however, is often far from such ideals. In the face of digital transformation, the capture process largely determines the check’s credibility. At the capture stage, an identity verification solution should cope with two things: recognize the submitted document and assess its quality. The thing is that software solutions use the capabilities of the user’s device, and rely on the user’s accuracy in taking photos. A bad camera, poor lighting, glares, or a skewed image make it difficult to read the information. This may lead to either errors or asking users to re-take photos over and over again until they meet the quality requirements. The best solutions can assess an image's quality and fine-tune it if necessary: adjust brightness or contrast, remove glares, or deskew the photo if it was shot at an angle. As for document type recognition, it’s mostly a matter of Unfortunately, many identity verification solutions still add two extra steps in the user journey: “choose your document type” and “choose your country.” UX. You can avoid making your users do extra clicks and typing by choosing a solution that identifies the document type and its origin country. As simple as it sounds, such capabilities require the vendor to invest a lot of time, money, and effort into creating a comprehensive document template database. For now, let’s just make a mental note that, ideally, your identity verification solution should do most of the work automatically. Questions to ask a vendor: Does your solution identify the document type automatically? Does it assess the document image quality? What are the criteria? Can it enhance the image’s quality to improve the data's readability? Step 2: Test-drive document checks Identity verification solutions must comply with ICAO and BSI guidelines for checking international documents, visas, etc. For domestic ID documents, though, there may be their own variants of execution of security elements of protection that require specific checks. Ideally, an identity verification solution should offer a check for each aspect: both standardized international features and local characteristics. The denser the coverage with checks, the more credible the conclusion about the authenticity and validity of the document. You can judge a vendor to provide first-class performance when their solution performs a long list of checks, plus runs massive cross-checks comparing the data from different elements to make sure they match. Document checks to ask for: Automatic document type identification Visual inspection zone verification MRZ validation Barcode type and format reading and validation Document liveness check (includes checks of holograms and screenshot detection) RFID chip detection and reading Cross-checks of the data fetched from different elements Step 3: Ensure that NFC-based RFID chip verification is in place A radio-frequency identification—or simply —chip is a tiny device that modern electronic ID documents contain. RFID chips store document holders’ data. The ICAO (International Civil Aviation Organization) strictly regulates the exact list of the data encoded in the chip. RFID Usually, reading this information entails using special passport readers or NFC technology built into mobile phones. You should be especially careful if it’s the latter. In the case of a remote process, there’s a risk that fraudsters can intercept and modify verification results. That’s why a good vendor needs to practice a zero-trust-to-mobile approach. In practice, the data needs to be re-verified to ensure the chip is not cloned or its data manipulated. Here’s an example of how Regula tackles this issue. RFID sessions can be re-verified on the server side in addition to the checks conducted on mobiles. Thanks to this, the data of the check is impossible to compromise. All this is done strictly on-premises: the results of the verification session are stored on a customer’s server, so it’s possible to get back to them afterward. This approach is considered the most reliable and secure way to verify a document with an NFC chip. Questions to ask a vendor: Does your identity verification solution read and verify an RFID chip? Can it read the information encoded in the chip? Do you apply a zero-trust-to-mobile approach? If so, how exactly does it work in your solution? Step 4: Explore capabilities for Biometric checks Validating the ID document is just a part of identity verification. Another part is ensuring that the person submitting it is the legitimate holder. That’s done with the help of biometric checks. Biometrics is a broad term that may imply a wide range of checks: face, fingerprints, voice, or even iris recognition. Most of them are overkill for regular business’ digital onboarding, but state authorities can require them, for example, when you apply for a visa. The most common way to run a biometric check for business is by using face biometrics. The process is simple: the user takes a selfie, and then the technology matches that photo with the photos in the submitted ID document (e.g., in the visual inspection zone and the RFID chip), and flags if there’s anything wrong. The essential feature of a proper face recognition module is liveness detection. That means you need not only to determine that it’s the same person in the photographs but also to make sure that the selfie is real. Sometimes attackers try to pass off an edited photo as a real selfie. Sometimes, they might use various masks and even deepfake technology. A good identity verification solution should be able to detect such issues. There are two types of liveness verification: passive and active. A passive liveness check implies the use of a static selfie, while active liveness requires the user to perform a series of simple movements: rotating the head, moving closer or further away, etc. Questions to ask a vendor: Do you perform biometric verification? What components does it include (face detection, face matching, face search, age verification, etc.)? Do you provide active or passive liveness checks? What PAD (Presentation Attack Detection) do you protect from? Step 5: Investigate the database of supported documents Cutting-edge technologies on their own aren’t enough for thorough identity verification. The most reliable IDV requires a deep knowledge of identity documents. This usually comes in the form of a document template database containing detailed information for every document: layout, fields, data recording methods, security elements, and more. Gathering such a collection requires tremendous resources and a solid background in document forensics. Ideally, your vendor should have seasoned forensic experts on their team to guarantee the comprehensiveness of templates and competent maintenance of the database. The importance of this cannot be overstated. If the vendor has thoroughly examined thousands of document types, its software solutions can easily distinguish one from another, fully read them, and extract the required data, therefore effectively verifying them as a result. The database needs to be extensive, as even within one country, there can be multiple versions of a document based on the date of issuance. Considering the frequent renewal and re-issuance of identity documents, the database should be regularly updated too. Cooperation with international organizations and security agencies is also a point in favor of the provider. Established relationships help them regularly expand the database with new document types and share the best practices in detecting document fraud. Questions to ask a vendor: How long have you been operating in the identity verification industry? Are there experts on your team with hands-on experience as forensic analysts? Do you have a document template database? How many items does it include? If there is no document in the database, how long does it take to add? Is it proprietary or bought from third parties? Step 6: Evaluate the time to value Reliability and completeness of checks are all good, but they won’t be of help if the integration process takes ages and blows up your budget. The pressure comes from multiple sides. On one side, there are regulators whose requirements are better met sooner than later. Then, there’s also your product and workflows, which will likely demand some customization from the identity verification solution. Add to it a learning curve that, for some solutions, can be rather steep, and lengthy. Questions to ask a vendor: Is it possible to build your solution into a web app? How fast will we be able to get up and running? What does the integration process look like? What customization options does your solution offer? Are they available out of the box, or will they require additional configuration/costs? Step 7: Check out the compliance of the workflows Regardless of the industry, safeguarding customer data is of the utmost importance. Therefore, it’s crucial to find out how your identity verification provider addresses the matter. When it comes to working with customers’ personally identifiable information (PII) during identity verification, there are two primary approaches: where vendors handle the storage and processing of customer data on their own infrastructure. A Software-as-a-Service (SaaS) scenario, where you store and process your customers' PII within your own secured perimeter. An On-premises approach, While the SaaS use case can save you time and the expenses associated with securing the data on your own, it’s even more critical to thoroughly examine the vendor’s procedures for customer data security and protection. Data breaches aren’t uncommon. An on-premises option places a higher emphasis on data privacy and security. Since no third parties have access to PII, it reduces the risk of breaches. However, extra protection comes at a price. The maintenance of infrastructure and reliability for following security policies is on you, so you’ll need to budget resources for engineering and security training for employees. Questions to ask a vendor: What type of solution do you offer: cloud or on-prem? Do you collect and store the personal data of end users? If it’s cloud… What certifications does your service have? (e.g., SOC1 type 2, HIPAA, etc.) Where can I read your security policy? What’s your service availability time? Step 8: Try to contact tech support Salespeople might be responsive before making the sale, but a company’s true colors show when a customer faces difficulties or has a request. If a vendor can provide prompt and professional technical support, it saves you a lot of time, effort, and headaches. Ideally, it’s good to have more than one way to contact support: by phone, email, live chat, or messengers. In most companies, additional communication channels come with the premium support option. Such support may include a dedicated manager, and the ability to communicate with the vendor’s team in real-time, 24/7. If this option is not available, pay attention to the response time SLA. This is the time in which the vendor commits to respond to a client’s issue. It’s usually defined by the support policy. As for cases when a client needs off-hours assistance, the vendor’s website should provide all essential instructions, guides, and FAQs. Comprehensive documentation should answer 90% of the questions, so pay attention to how thoroughly it is made, if it includes demo examples, how often it is updated, and if it’s localized to your language. Questions to ask a vendor: What’s your response time for support requests? Do you have a premium support option? Is support documentation comprehensive and easy to navigate? Step 9: Find customer references If you made it through the steps above, you can confidently trust the results. Still, it’s always a good idea to look for additional social proof from companies that have already succeeded in what you’re trying to achieve. Here, we advise you to conduct your own research and find as many reviews about the vendor as possible. The first go-to source of information is the case study section on the vendor’s website. There, you can look for stories from customers from your niche and find relevant use cases. For example, if you’re a crypto platform on a quest to deploy KYC, then KYC-related cases from banks will also give you valuable insights. Also, pay attention to who a vendor’s existing customers are. The presence of well-known names is a good sign. If the vendor has state-level organizations and security agencies, such as border controls, among their customers, it’s also great, because they have the strictest requirements for identity verification. Also, you may want to research what the media say about different vendors. Consider exploring not only so-called tier-one resources, such as The New York Times or Business Insider but focus more on industry-specific publications that specialize in covering identity verification and cybersecurity. This approach will broaden your research and provide a more comprehensive understanding of different vendors' reputations. Among such media outlets are: Biometric Update, Help Net Security, Infosecurity Magazine, Source Security, Identity Week, Find Biometrics, and others. Try to find out: Are there any references available in open sources from customers in my industry? Can the vendor provide those references on request? Are there relevant use cases in the vendors’ customer success stories? Are there any mentions of the vendor in the industry and/or tier-one media? In conclusion, this 9-step guide empowers you to choose Identity Verification software that aligns with your specific needs while excelling in precision, security, and trustworthiness, ensuring your data and operations are in safe hands. With the right software, you can confidently navigate the evolving landscape of identity verification for a secure and seamless future. If you’re struggling with finding the right solution for your business use case – feel free to get in touch. Also published . here

How to Choose the Best Identity Verification Software

In programming, a lot of the challenge is related to organizing data in a structure that makes sense and which helps to solve the problems and facilitates understanding of the . In this article, I’ll show you what tools you have at your disposal and how to use them. code Building blocks To simplify our discussion, let’s focus on what we have available in JSON—the data format that was created based on how data can be . In this way, we don’t need to think about the performance implications of different data structures—we only focus on the usefulness of the data model. hard-coded in JavaScript Primitive values These are the most basic value types we can have in our program. In JSON, those values are: boolean—a true or false value, a number—a floating-point value, string—a sequence of letters, and —a missing value, which is still useful for expressing certain states. null Already, with those basic types, we have the possibility of addressing the same thing in different ways. For example, for customer type, you could use some of the following: —which could accept any other string value, both valid or invalid ones const customerType = “retail”; —a more rigid approach: its two options are obvious and clear, but it has no way of allowing more possibilities const isRetail = true; —can be expanded but requires documentation for understanding what each number means const customerType = 2 Anything you decide will impact how easy it will be to change code later on. As you are picking the value type for your variable, it’s good practice to try to think about possible changes and pick a type that fits well for both current and future needs. Arrays Arrays are ordered lists of values. The JavaScript arrays can mix any type of values, for example: const someArray = [ 1, 2, “test”, [“another”, “array]]; Although it is a valid array, mixed types arrays are confusing to work with. It’s way easier to stick to keeping the same types of values in an array. An important feature of arrays is that they are ordered. This makes them a great match for places where you care about similar data points and where the order is meaningful. For example: Products in cart—you definitely want to display items from a cart in the same order and students in a class—each one of them has their list number. Arrays work well when you access many values in sequence. It’s less useful when you want to find whether a given value is in an array. When I find myself often using to see if an array has some object I’m looking for, I take it as a potential sign that I should reorganize the data structure. includes() Objects The object is a structure that consists of a set of key–value pairs. The structure is unordered, so it makes it a suitable choice for many places that don’t fit the array. Each value on an object can be understood independently of all the rest, so you can mix different types of values without worry. Example of an object: const user = {
  login: “lorem.ipsum”,
  email: “sin@dolor.com”,
  age: 24,
} Objects are a good match for collections of named, unordered properties. For example, a set of permissions: const hasAccessTo = {
  blog: true,
  blogAdmin: true,
  userAdmin: false
} Built-in objects JavaScript provides many object types that you can use on the code side to keep your values in a more suitable form—and with methods that make them more usable. For example: —for date-time values, Date —for unordered collections of unnamed values that don’t repeat and Set —a non-string identifier that will not collide with other values. Symbol Those types are very useful, but in communication that goes through a web API that uses JSON to represent data, they all will be serialized to the value types described above. In my daily job, where I program business applications, the only types that I use regularly are dates. Pick the appropriate structure The way we organize data impacts what can be easily done in our code. Let’s take a look at the permission example from above: const hasAccessTo = {
  blog: true,
  …
} This value can be used easily in conditional cases such as: . if (hasAccessTo.blog) The same requirement for a permission list could be met with another data structure: const permissions: [{
    resource: “blog”,
    access: true,
  },{
    resource: “blogAdmin”,
    access: true,
  },{
    resource: “userAdmin”,
    access: false,
  },
] The same condition from above would need to be replaced by: const blogPermission = permissions.find(x => x.resource === ”blog”);

if (blogPermissions?.value) {
… This is much harder to read, and it’s easier to write code that will throw errors in some specific cases. By using arrays, the only thing we “gain” is the order of permissions, which shouldn’t matter. Create a draft There are plenty of things to consider as we organize data in our codebase, and calls can get complicated pretty quickly. To avoid rewriting code, it makes sense to create some form of draft for the structures. In my case, I often describe them with the names I plan to use in the tickets I’m going to work on. Often, when you see your structure design written down, you’ll notice some issues with it. Keep on improving The data structure for your code will never be finished—it should keep evolving with your application. Changing names they appear in many places in code and in databases be painful. However, it is necessary because otherwise, the mismatch between names and their current understanding will only grow as things change over time. This is a common source of code debt that often slowly piles up until it makes working with the code an unpleasant and unproductive experience. after will Also published . here

Tips For Writing Maintainable Code: Data Models

Test Faster, Test Smarter: Unlocking Selenium's Full Potential In today's fast-paced software development landscape, speed is of the essence. Testing teams strive to deliver high-quality software at a rapid pace, and plays a crucial role in achieving that goal. the execution time of UI test suites Selenium, the widely adopted UI testing framework, provides powerful capabilities for automating browser actions and validating web applications. However, is essential to ensure efficient testing processes and timely feedback. optimizing the execution time of Selenium test cases Explore five key strategies for accelerating Selenium test execution, along with the integration of Launchable's intelligent test optimization tools to further enhance speed and efficiency. 1. Efficient Selenium Locator Strategies Selenium Locators identify the correct UI element on a page, making it possible to create successful automation scripts. Optimizing your locator performance significantly reduces test execution time. Consider the following strategies: Utilize CSS selectors instead of XPath, as CSS selectors are generally faster and more efficient. driver.find_element(By.CSS_SELECTOR, "input#username") Employ unique and stable locators, such as IDs, names, or CSS classes, to ensure consistent test case execution. driver.find_element(By.ID, "username") Avoid excessive nesting of locators, as it can lead to slower element identification. driver.find_element(By.CSS_SELECTOR, "div.container input#username") For example, let's say you're automating a login functionality. Instead of using a complex XPath expression to locate the username input field, you can use a unique CSS selector, such as , resulting in faster test execution. input#username` 2. Test Case Prioritization and Parallel Execution To maximize test execution speed, it's crucial to prioritize test cases based on their business impact and implement parallel execution. Consider the following techniques: Identify high-impact test cases that cover critical functionality and prioritize their execution. Implement parallel execution using frameworks like TestNG or JUnit to execute multiple test cases concurrently. If you have a test suite containing multiple independent test cases, , reducing overall execution time. parallel execution can allow you to execute them simultaneously Get faster feedback without spending more on running parallel tests Launchable leverages AI to , generating dynamic test subsets. This approach enables teams to identify issues earlier while simultaneously minimizing compute costs. by adopting dynamic parallelization. predict which tests may fail 3. Make Your Selenium Testing Data-Driven Data-driven testing uses your test data to make your test execution more efficient and allows you to validate application behavior without running every test every time. Consider the following practices: Implement data-driven testing using Selenium and related frameworks, such as TestNG's data provider or JUnit's parameterized tests. Manage test data efficiently by using external data sources, such as CSV or Excel files. Make by leveraging Predictive Test Selection to optimize tests based on historical data patterns. Launchable builds a machine learning model based on your git data to get deep insights into how your tests run and how efficient they are. Selenium truly data-driven Then, it identifies what tests you should run, recommending the tests that directly test what was changed, . saving you time and speeding up your releases 4. Selenium Test Suite Optimization Optimizing the structure and content of your Selenium test suite can significantly reduce execution time. Consider the following optimization techniques: Identify and remove redundant test cases that cover similar scenarios. Reduce setup and teardown time by utilizing test fixtures and leveraging browser session reuse. Isolate and parallelize independent test cases to execute them concurrently, further speeding up the test suite execution. If your test suite includes multiple tests for user authentication, consider consolidating them into a single test case with parameterized inputs. To fully optimize your Selenium test suite, you need insights into the health of your tests and trends over time. like test runtime is crucial for improving development cycles, enabling faster feedback loops, and better test maintainability. Test runtime measurement brings various advantages, such as optimizing parallelization, managing resource allocation, prioritizing tests, and facilitating continuous improvement in your CI/CD pipelines. Measuring critical test health metrics Launchable's , enabling you to identify and optimize the test suite execution for maximum efficiency. Launchable provides automated tracking of test runtime, giving you in-depth visibility into the health of your test suite. This includes the ability to identify flaky tests, track the duration of test sessions, optimize the frequency of test sessions, and analyze the ratio of test session failures. Test Suite Insights provides valuable insights into test suite health KPIs 5. Selenium Test Environment Optimization Optimizing the test environment configuration contributes to faster test execution by making your environment run as smoothly as it possibly can. Consider the following strategies: Use lightweight browsers or headless modes, such as Headless Chrome or Firefox, to reduce resource consumption and improve performance. Using a headless browser eliminates the need to open a physical browser window, saving system resources and reducing overall execution time. from selenium.webdriver.chrome.options import Options

options = Options()

options.add_argument("--headless")

driver = webdriver.Chrome(options=options) Leverage cloud-based testing platforms, such as Selenium Grid or Sauce Labs, to distribute tests across multiple machines and browsers, increasing parallel execution capabilities. Ensure proper resource allocation and management, including CPU, memory, and network, to avoid performance bottlenecks. Reduce the need for context switching and streamline your test notifications where your developers are already working with Slack Test Notifications. Eliminate the need to hunt through inboxes or check in your CI; Launchable's and failures in the test environment, enabling swift action without disturbing your workday. personalized Slack Test Notifications provide real-time alerts about test execution performance Takeaways Optimizing Selenium test execution is paramount for efficient testing processes and timely feedback. By implementing these five strategies and , UI testing can significantly accelerate test execution, reduce development cycles, and deliver high-quality software with confidence. Achieve faster test execution times, optimize your testing workflow, and ensure high-quality releases in a time-efficient manner. integrating Launchable Also published . here

Speeding Up Selenium Test Execution: Five Ways for Quicker Feedback

Let's learn about via these 41 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Test Driven Development /Learn Repo trial and errors are key to building anything new and innovative. 1. Why You Need to Stop Writing Unit Tests The way you test has a huge impact on your codebase maintainability. There is  a simple way to improve it 2. Test Driven Development (TDD): Killing Bugs Before Day Zero Here's why you need to know TDD craft. 3. 5 Ways to Prevent Server Hardware Failure Use mature tools to make mature software 4. Test-driven Development: How to Write Unit Tests Tests are a way to explicitly set expectations about code. You establish them to allow the machine to check whether your code meets the expectations. 5. Best Practices to Write Unit Tests the Right Way In this article we will review some best practices and must-have libraries which will get your unit tests to the next level. 6. Just Enough Testing: How Many Automated Tests Should We Write? How many automated tests should you write? Just Enough. Just Enough Testing balances the need to ship with the quality your customers deserve. 7. TDD Conference 2021: Opening Ceremony by Alex Bunardzic Alex is a software engineering veteran with over 30 years of hands-on professional experience. He is a staunch advocate of Extreme Programming and TDD. 8. Best Practices to Write Unit Tests the Right Way (Part 2) All you need to know about unit testing LINQ and mappings in .NET to make more reliable code faster as you develop software using systems like AutoMapper 9. Build a CLI App with Oclif and Node.js Using Test-Driven Development: Part 2 TDD for building a CLI application using the oclif framework. 10. Why I Don't Think TDD Is Essential Test Driven Development puts emphasis on unit over integration tests. The result can be lower quality featuring bugs that are baked into the product. 11. A Closer Look at the Most Used Tools for Software Testing Using automation testing tools is an excellent way to go about verification of your product. 12. Choosing Test Cases for Automation That Can Increase Your ROI In this article, we will explore the key considerations when choosing test cases for Automation Testing. 13. Wordles, Centaur and Code: Building a Game The Conquest and Conquer is an analogy for serious software development. I will combine the TDD solution with an automated code generated using Artificial Intel 14. Test Your APIs in an Easy Way By Using Python Module PyHTTPTest I've struggled and spent a lot of hours writing manual Python scripts to be able to test a product involving many microservices. All of them following the REST conventions. Overwhelmed with too many scripts, each one test different endpoints, how they respond also what they respond, I've decided to create a Python package to easily do this, without needing to create a manual Python script, import libraries to send an HTTP Request, copy and paste code from other scripts to test fast the new endpoints, etc. 15. Unit Testing Made Easier With Pure Functions By getting used to unit tests, they can start to familiarize themselves with the codebase and make significant progress while facing no risk or stress. 16. Build a CLI App with Oclif and Node.js Using Test-Driven Development While writing a CLI tool can be a lot of fun, the initial setup and boilerplate—parsing arguments and flags, validation, subcommands—is generally the same. 17. How To Adopt TDD In Your Team This is the first in a series of blog posts in which we outline the benefits of TDD for your organization and what are the projects to consider while adopting this agile process. Link of Part Two. 18. Using the Software Testing Maturity Model to Refine Your Testing Strategy Want to improve your software quality engineering processes but don't know where to start? This software testing maturity model and assessment may help! 19. Road to Simplicity: Tests Are Not Tests [Part Two] This is the second story of Road to Simplicity. And it’s about the role of tests in software writing. 20. Creating a Wordle with TDD in Javascript We keep practicing this amazing Kata and learning. You can follow the steps! 21. We Should Get Rid of "Hello, World!" Forever💩 Every tutorial I've read for the last 30 years starts with the infamous 'Hello World' example. This could be one of the reasons we write crappy software. 22. Dependency Injection to Make Your Code Testable [A How-To Guide] Have you ever wanted to write unit tests for your code, but you’ve found that it’s difficult to do so? Often this is the result of not writing code with testing in mind. An easy way to solve this is through utilizing test-driven development, a development process in which you write your tests before your app code. 23. An Example of True Test Driven Development in Vue In true TDD, tests must fail, tests must pass, and the code must be refactored. 24. The Complex Relationship Between Software Developers and Test Code I was preparing yet another session about Test Driven Development (TDD) (I do quite a lot of these as part of my position) and I thought it would be a good idea to allure the audience at the beginning by showing some statistics. 25. Developers Should Drive Their E2E Tests to Cypress All software companies strive to produce quality products. The journey begins with a product that works. Software correctness is usually defined as the adherence of software to its specification. In most cases, formal verification is not possible. As such, we typically impose functional correctness with test coverage. 26. Don't Get Bitten by Hidden Bugs: Test Driven Development (TDD) and AAA Testing via xUnit Test Driven Development (TDD) is a software development cycle that focusses on describing the behaviour of your code first using tests 27. The Untold Truth of Test Driven Development Your Practical Introduction to Test Driven Development 28. Why Software Testing is So Important It isn't about being diligent.  It's because we've forgotten what we knew. 29. Software Development is Not About Unit Tests Photo by ShareGrid on Unsplash 30. Effective Testing in Complex Software Development Projects Manual Or Automatic Testing? 31. Writing Software from the Outside In Whenever you’re developing software, a certain amount of refactoring and rewriting is inevitable. This is sometimes due to a new idea that will simplify the design or a change to the project requirements. But unfortunately, it often also happens because of a misunderstanding about how the software will connect and interact with its external environment. While it’s impossible to completely avoid such miscommunication, you can minimize it by using a principle I call writing software from the outside in. 32. Test-Driven Development with RSpec in Rails When we start developing an app we usually do it the old fashion way: we get an idea, we elaborate on it, we make a plan to develop an app then we start coding and in the end, we test if our app is working properly. 33. Behavior-Driven Development From Scratch [101] Behavior-Driven Development (BDD) is a really powerful tool that helps us build robust, value-based software. You can sometimes hear detractors say that it brings a lot of complexity and leads to long tests difficult to maintain. Let's take stock of what BDD really is, by determining anti-patterns and best practices. 34. Test-Driven Development is Fundamentally Wrong It sounds backwards because it really is 35. How to Write Tests for React - Part 1 [Beginner's Guide] Writing React Test with react recommended libraries — Jest & React Testing Library for complete beginners. 36. Building a Wordle Game Using Test-Driven Development in 25 Minutes Developing a complete word game is very easy with TDD 37. How to Use RSpec From Basics to Testing User Input On my journey on Microverse, one of the things that troubled me the most to understand was RSpec, a Ruby gem for Test-Driven Development. The concept is quite simple. You create tests as you code along, so in the future, if an update breaks something, it will be easy to notice because one or more tests will fail. 38. An Introduction to Test-Driven Development Test-driven development (TDD) is all the rage these days and has been a discussion topic for quite some time. If you are brand new to TDD, this article should serve as a good introduction to what TDD is, why it’s useful, what a typical TDD workflow looks like, and when to use TDD. 39. How to Squeeze Test Driven Development on Legacy Systems We all love T.D.D. We know its benefits, we have read a thousand tutorials on how to build a system using this technique. But this not feasible for currently legacy systems. 40. 6 Lessons I Learned with Software Testing For the last few of months I started digging into the software testing world. I really wanted to learn more about how to create more effective tests, refactor code with more confidence and feel safe about adding new features. However, I felt it is little difficult to dive right in this area, that in my point of view, is underestimated. 41. Zombie Testing: One Behavior at a Time Selecting great test cases is very hard. Unless you summon the undead. Thank you for checking out the 41 most read stories about Test Driven Development on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

41 Stories To Learn About Test Driven Development

Global Retries Configuring Global Retries provides a built-in global retry mechanism for test cases. This means that when a test fails, Playwright automatically retries the test up to the configured number of times before marking it as a failure. To set the global retry ability, you can use the option in the Playwright config file ( ): Playwright retries playwright.config.ts import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: process.env.CI ? 2 : 0,
}); This code snippet configures retries only when running tests in a ) environment. You can override this setting by using the flag when running tests from the command line: continuous integration (CI --retries npx playwright test --retries=1 Configuring Retries per Test Block If you need more granular control over retries, you can configure them for individual test blocks or groups of tests. To do this, use the function: test.describe.configure() import { test } from '@playwright/test';

test.describe('Playwright Test', () => {
  test.describe.configure({ retries: 5 });

  test('should work', async ({ page }) => {
    // Your test code here
  });
}); This configuration allows the specified test block to be retried up to 5 times before being marked as a failure. Auto-waiting and Retrying Built-in Auto-waiting Mechanism Playwright has a built-in auto-waiting and retry mechanism for locators (e.g., ) and matchers (e.g., ). This mechanism continuously runs the specified logic until the condition is met or the timeout limit is reached, helping to reduce or eliminate flakiness in your tests. For instance, you don't need to manually specify a wait time before running some code, such as waiting for a network request to complete. page.getByRole() toBeVisible() To learn more about the specific timeout limits, refer to the . Playwright timeout documentation Custom Conditions with Retrying and Polling APIs Sometimes, you might need to wait for a condition unrelated to the UI, such as asynchronous processes or browser storage updates. In these cases, you can use Playwright's Retrying and Polling APIs to explicitly specify a condition that is awaited until it is met. Using the Retry API The API uses a standard method along with the method to retry an assertion within the block. If the assertion fails, the block is retried until the timeout limit is reached or the condition passes. Retry expect toPass(options) expect expect The example below demonstrates waiting for a value to be written to local storage: import { test } from '@playwright/test';

test('runs toPass() until the condition is met or the timeout is reached', async ({ page }) => {
  await expect(async () => {
    const localStorage = await page.evaluate(() => JSON.stringify(window.localStorage.getItem('user')));
    expect(localStorage).toContain('Tim Deschryver');
  }).toPass();
}); Using the Poll API The API is similar to the API, but it uses the method instead of a standard block. The method also returns a result, which is used to invoke the matcher. Poll Retry expect.poll() expect expect.poll() The example below demonstrates waiting for a process state to be completed: import { test } from '@playwright/test';

test('runs expect.poll() until the condition is met or the timeout is reached', async ({ page }) => {
  await expect
    .poll(async () => {
      const response = await page.request.get('https://my.api.com/process-state');
      const json = await response.json();
      return json.state;
    })
    .toBe('completed');
}); Both the and APIs can be configured with custom timeout and interval durations: Retry Poll import { test } from '@playwright/test';

test('runs toPass() until the condition is met or the timeout is reached', async ({ page }) => {
  await expect(async () => {
    // Your test code here
  }).toPass({ intervals: [1000, 1500, 2500], timeout: 5000 });
});

test('runs expect.poll() until the condition is met or the timeout is reached', async ({ page }) => {
  await expect
    .poll(async () => {
      // Your test code here
    }, { intervals: [1000, 1500, 2500], timeout: 5000 })
    .toBe('completed');
}); Test Retries in Worker Processes How Worker Processes Work Playwright Test runs tests in worker processes, which are independent OS processes orchestrated by the test runner. These workers have identical environments and start their own browsers. When all tests pass, they run in order in the same worker process. However, if any test fails, Playwright Test discards the entire worker process along with the browser and starts a new one. Testing continues in the new worker process, beginning with the next test. Enabling Retries in Worker Processes When you enable retries, the second worker process starts by retrying the failed test and continues from there. This approach works well for independent tests and guarantees that failing tests can't affect healthy ones. To enable retries, you can use the flag or configure them in the configuration file: --retries npx playwright test --retries=3 import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: 3,
}); Playwright Test categorizes tests as follows: "passed" - tests that passed on the first run; "flaky" - tests that failed on the first run but passed when retried; "failed" - tests that failed on the first run and all retries. Detecting Retries at Runtime You can detect retries at runtime using the property, which is accessible to any test, hook, or fixture. testInfo.retry The example below demonstrates clearing the server-side state before retrying a test: import { test, expect } from '@playwright/test';

test('my test', async ({ page }, testInfo) => {
  if (testInfo.retry) {
    await cleanSomeCachesOnTheServer();
  }
  // Your test code here
}); Configuring Retries for Specific Groups or Files You can specify retries for a specific group of tests or a single file using the function: test.describe.configure() import { test, expect } from '@playwright/test';

test.describe(() => {
  test.describe.configure({ retries: 2 });

  test('test 1', async ({ page }) => {
    // Your test code here
  });

  test('test 2', async ({ page }) => {
    // Your test code here
  });
}); Grouping Dependent Tests with test.describe.serial() For dependent tests, you can use to group them together, ensuring they always run together and in order. If one test fails, all subsequent tests are skipped. All tests in the group are retried together. While it's usually better to make your tests isolated, this technique can be useful when you need to run tests in a specific order. test.describe.serial() import { test } from '@playwright/test';

test.describe.serial.configure({ mode: 'serial' });

test('first good', async ({ page }) => {
  // Your test code here
});

test('second flaky', async ({ page }) => {
  // Your test code here
});

test('third good', async ({ page }) => {
  // Your test code here
}); Reusing a Single-Page Object Between Tests By default, Playwright Test creates an isolated object for each test. However, if you'd like to reuse a single object between multiple tests, you can create your own in the hook and close it in the hook: Page Page test.beforeAll() test.afterAll() import { test, Page } from '@playwright/test';

test.describe.configure({ mode: 'serial' });

let page: Page;
test.beforeAll(async ({ browser }) => {
  page = await browser.newPage();
});

test.afterAll(async () => {
  await page.close();
});

test('runs first', async () => {
  await page.goto('https://playwright.dev/');
});

test('runs second', async () => {
  await page.getByText('Get Started').click();
}); Conclusion In summary, Playwright offers various retry APIs to make your tests more resilient and less flaky. The built-in retry mechanism for locators and matchers covers most daily use cases. However, for assertions that need to wait for external conditions, you can use the explicit retry and polling APIs. Additionally, you can utilize the global retry mechanism for test cases to handle inconsistencies caused by conditions beyond your control. By incorporating these retry strategies into your testing workflow, you can ensure a more robust and reliable testing experience, leading to higher-quality software and happier end-users. Also published here. The lead image for this article was generated by HackerNoon's AI Image Generator via the prompt "A computer screen with a real bug on it."

Enhancing Test Reliability: Exploring Playwright's Retry APIs for Robust Testing

Setting the Stage: Unleashing the Data Dragon How often have you struggled to generate test data for you tests? ! Join the dark side and start using AI to generate it. : you’re a regular software tester armed with your trusty test cases and ready to find all the bugs… But, you spend hours generating data which you need. Stop your struggles Imagine this Test data generation is the process of creating meaningful and representative data that reflects the real-world scenarios in the software. It’s the main entry point for validating the functionality, performance, and security of applications. Also, ensuring data diversity and coverage is not an easy feat. Testers must include a wide range of data variations, edge cases, and boundary conditions to thoroughly check software. Another challenge is a limited availability of quality test data. Sensitive or realistic data cannot be used for testing due to privacy concerns or data protection regulations, while it’s safe for the user, testers spend significant amount of time generating this data manually. In this article we will discuss how AI can help us with test data generation and what the best approaches are. Test data is generated by chatGPT and Google Bard JSON file generation we have an API test and we need to send a request with JSON for two cities in EU along with the correct phone number for each city. Let’s imagine: This JSON file has to have: City Country Country code Phone number Let’s try to create a prompt for this task: Dear Chat GPT, I need you to generate JSON file with 2 the most popular cities in EU. JSON should have following variables: city, country, country code and correct phone number according to the city. : Result [
  {
    "city": "Berlin",
    "country": "Germany",
    "country_code": "DE",
    "phone_number": "+49 30 12345678"
  },
  {
    "city": "Paris",
    "country": "France",
    "country_code": "FR",
    "phone_number": "+33 1 23456789"
  }
] We saved so much time! Now instead of browsing and searching for country code and phone number we can use AI to generate data. Additional data generation Let’s take example with JSON and ask it to generate few more sets of data based on the JSON which we will provide. My lovely Chat GPT, based on the JSON which I’m providing to you, could generate more data to this? <JSON> : Result [
  {
    "city": "Berlin",
    "country": "Germany",
    "country_code": "DE",
    "phone_number": "+49 30 12345678"
  },
  {
    "city": "Paris",
    "country": "France",
    "country_code": "FR",
    "phone_number": "+33 1 23456789"
  }, 
  {
    "city": "New York",
    "country": "United States",
    "country_code": "US",
    "phone_number": "+1 212 3456789"
   },
   {
    "city": "London",
    "country": "United Kingdom",
    "country_code": "UK",
    "phone_number": "+44 20 12345678"
    }
] YAML file generation we have to test e-commerce shop integration, and for this we would need YAML file with products inside. Let’s imagine: Let’s try to create a prompt for this task: Dearest Chat GPT, I need you to generate YAML file with 3 random products inside. Can you add as much complexity as possible? (we want this to be as complex as possible to test how our e-commerce integration works with unusual attributes) Result: - name: Product 1
  description: High-end smartphone with advanced features and powerful processor.
  price:
    amount: 999.99
    currency: USD
  quantity:
    available: 50
    minimum_order: 1
    maximum_order: 5
  category: Electronics
  specifications:
    - display: OLED
    - processor: Qualcomm Snapdragon 855
    - camera: Dual rear cameras (12MP + 16MP)
    - battery: 4000mAh
    - features:
      - Wireless charging
      - Water and dust resistance (IP68)
- name: Product 2
  description: Stylish and durable stainless steel watch with a classic design.
  price:
    amount: 199.99
    currency: USD
  quantity:
    available: 20
    minimum_order: 1
    maximum_order: 3
  category: Fashion
  specifications:
    - movement: Quartz
    - crystal: Sapphire
    - water_resistance: 100 meters
    - strap: Stainless steel bracelet
- name: Product 3
  description: Versatile laptop designed for work and entertainment.
  price:
    amount: 1499.99
    currency: USD
  quantity:
    available: 30
    minimum_order: 1
    maximum_order: 2
  category: Electronics
  specifications:
    - display: 15.6-inch high-resolution screen
    - processor: Intel Core i7
    - storage: 1TB SSD
    - keyboard: Backlit
    - security: Fingerprint scanner
    - battery: 8 hours That looks awesome! XML file generation we need to generate an XML file with catalogs of product inside for testing e-commerce software. Let’s imagine: Dear Chat GPT, I need you to generate an XML file with 3 catalogs inside. Be as creative as you can. : Result <?xml version="1.0" encoding="UTF-8"?>
<catalogs>
  <catalog>
    <id>101</id>
    <name>Enchanted Garden</name>
    <description>A magical collection of plants and garden accessories that bring your outdoor spaces to life.</description>
  </catalog>
  <catalog>
    <id>102</id>
    <name>Adventure Gear</name>
    <description>Equip yourself for thrilling adventures with our wide range of gear, from hiking essentials to camping equipment.</description>
  </catalog>
  <catalog>
    <id>103</id>
    <name>Mystical Potions</name>
    <description>Discover the secrets of ancient alchemy with our extraordinary collection of potions and elixirs.</description>
  </catalog>
</catalogs> SQL data generation we’ve got some SQL automation tests to write, and we need to generate some data in these auto tests Let’s imagine: Dear ChatGPT I need you to generate SQL script where you put some data to XXX table. It has username, order number, price for the order and current time. : Result INSERT INTO XXX (username, order_number, price, current_time)
VALUES ('JohnDoe', 'ORD001', 19.99, CURRENT_TIMESTAMP),
       ('JaneSmith', 'ORD002', 24.99, CURRENT_TIMESTAMP),
       ('RobertJohnson', 'ORD003', 14.99, CURRENT_TIMESTAMP); We got script ready to use and some generated data for this script. But. Let’s try to ask something more complex: Dear ChatGPT I need you to generate SQL script where you put some data to XXX table. It has id, username, order number, price for the order and current time. Second table should have id, username, phone number and address. I want you to join these two tables and write me the result. : Result -- Create XXX table
CREATE TABLE XXX (
    id INT,
    username VARCHAR(50),
    order_number VARCHAR(50),
    price DECIMAL(10, 2),
    current_time TIMESTAMP
);

-- Insert data into XXX table
INSERT INTO XXX (id, username, order_number, price, current_time)
VALUES (1, 'JohnDoe', 'ORD001', 19.99, CURRENT_TIMESTAMP),
       (2, 'JaneSmith', 'ORD002', 24.99, CURRENT_TIMESTAMP),
       (3, 'RobertJohnson', 'ORD003', 14.99, CURRENT_TIMESTAMP);

-- Create YYY table
CREATE TABLE YYY (
    id INT,
    username VARCHAR(50),
    phone_number VARCHAR(15),
    address VARCHAR(100)
);

-- Insert data into YYY table
INSERT INTO YYY (id, username, phone_number, address)
VALUES (1, 'JohnDoe', '1234567890', '123 Main St'),
       (2, 'JaneSmith', '9876543210', '456 Elm St'),
       (3, 'RobertJohnson', '5555555555', '789 Oak St');

-- Perform a join between XXX and YYY tables
SELECT XXX.id, XXX.username, XXX.order_number, XXX.price, YYY.phone_number, YYY.address
FROM XXX
JOIN YYY ON XXX.username = YYY.username; Data generation for testing Sometimes you have a task to test an input, and that’s a lot of test cases and loads of data. But what if you don’t have time to write all these test cases. How do you check that you’ve tested everything? How do you generate random data for every case which you’re going to do? The answer is easy! Let’s use ChatGPT 😀 Dear ChatGPT, can you generate me test data for LinkedIn search field with positive, negative scenarios, and with edge cases : Result Positive scenarios:

Search for a job title: "Software Engineer"
Search for a company name: "Google"
Search for a location: "San Francisco, CA"
Search for a skill: "Python"
Search for a keyword: "machine learning"

Negative scenarios:

Search for a job title that does not exist: "Unicorn Hunter"
Search for a company name that does not exist: "Acme Corporation"
Search for a location that does not exist: "Hogwarts, UK"
Search for a skill that does not exist: "Quantum Computing"
Search for a keyword that is too common: "the"

Edge cases:

Search for a job title with multiple words: "Senior Software Engineer"
Search for a company name with special characters: "Acme, Inc."
Search for a location with multiple words: "San Francisco Bay Area"
Search for a skill with multiple words: "Data Science"
Search for a keyword with special characters: "@" It needs some more cases, but it’s a perfect start! You can use it on your own, you can send it to the office intern or write test cases from this. SQL injection we also need to check the input on some security protection. Let’s imagine: Dear ChatGPT, can you generate me SQL injection LinkedIn search field Result: 1. SQL Injection Attack:
Input: ' OR '1'='1
Expected Result: The search query is manipulated to retrieve all records from the database, potentially exposing sensitive information.

2. Union-based SQL Injection:
Input: ' UNION SELECT username, password FROM users --
Expected Result: The search query is manipulated to perform a UNION SELECT operation, potentially retrieving usernames and passwords from the database.

3. Boolean-based Blind SQL Injection:
Input: ' OR (SELECT COUNT(*) FROM users) > 0 --
Expected Result: The search query is manipulated to perform a Boolean-based SQL injection, exploiting a vulnerability in the application's response to different conditions.

4. Time-based Blind SQL Injection:
Input: ' OR SLEEP(5) --
Expected Result: The search query is manipulated to introduce a delay (in this case, 5 seconds) to test for time-based blind SQL injection vulnerabilities.

5. Error-based SQL Injection:
Input: ' OR 1/0 --
Expected Result: The search query is manipulated to trigger a SQL error, potentially revealing information about the database structure or sensitive data. In conclusion, test data generation with AI presents a powerful and efficient solution to one of the challenges during testing. Now we can automate the process of generating any test data sets and spending seconds on this. However, it's important to remember that AI-generated test data is not a panacea. Human expertise is still crucial part in defining appropriate test data sets, validating results, and more.

Test Data That Thinks for Itself: 
AI-Powered Test Data Generation

Writing test cases is the most underestimated and yet most effective requirements analysis tool. Only the question “How are we going to test this?” can reveal all hidden risks and problems with the requirements. Test cases are software development safety nets. Not if they are poorly written, though. Then they can only contribute to the misconception that writing test cases is a waste and a bottleneck. Here are my 13 tips on writing powerful test cases that build trust in testing. #1 Test cases must be easy to understand The first rule of thumb about writing a test case is that (almost) everyone can understand its purpose, its importance, how to prepare for it, how to execute the test case, and what the expected result is. When writing a test case, always imagine yourself on your first day at work. Could you execute this? Always add all the necessary information about the test execution and the verification of the expected results. For example, if a log record must be checked, add instructions on how to access the log. If those instructions are not written, write them and add the link to the article. Or if the expected result must be checked in the database, attach the query to the test case, so the next person doesn’t have to write it again. #2 Test cases must be easy to execute – The test case must be written in a clear and straightforward manner so the execution, when manual, is not slow, error-prone, and tedious. Efficient – A test case must enable quick execution and traceability of the performed steps and the used test data so that the reason for it can be easily deducted from the test when a bug is found. Effective – The test case must always provide the same results when there are no regressions or modified behavior requiring modification of the test itself. Reliable – Also, always have in mind how this test case can be automated. Will it change the way it’s written? Automatable #3 Test cases must be independent from each other Independent test cases are a tricky practice, but it’s rule number one when it comes to automated tests. For manual tests, though, here’s how to avoid the confusion: Use preconditions and shared steps. If the result of a test case can be used as input to another test case, mark the last step (or the entire test case) as a “shared step”. This way, instead of sequencing the test cases, you can indicate that the precondition to Test Case A is the X, which can be achieved if you execute Test Case B. The automation rule mentioned above says that each test must create its test data and then destroy it after, leaving the system in a neutral state not affected by the test execution. The second automation rule is that tests must be possible to execute in parallel. Imagine the 2104 tests you need to run on every push to the main branch, most of which are unit tests. Some need 10-20 seconds to run. That’s an hour. While typically, that’s an insanely good test performance, imagine that they depend on each other, and the 486th one fails… You’ve wasted 20 minutes, not knowing what’s wrong, because all the rest failed too. Now imagine you can run them independently in parallel, and instead of an hour, the test run takes 20 minutes. All pass – three fail. This will ensure a very narrowed-down reasoning and effective troubleshooting, not to mention the emotion you’d feel when you see 3 tests failed by that exact change vs 1618 RED. #4 Test case must be literate, thorough, but compact The quality of the test cases is the second factor that demonstrates the tester’s skills after the bug description. In whatever language you write, make sure the text is well-written. Appropriate for the test case. Keep it focused, straightforward, and compact. Do not use filler words. Think about the test case like it is a diagram or a flowchart written in text. Yet, it should be thorough and clear, and the reader shouldn’t have to guess anything. #6 If a step doesn’t make a difference, don’t write it. Review and optimize your test cases as often as necessary to ensure there are no steps that might change the way the test case is executed or distract from the original intent without being relevant to the expected result. #7 If a step is the same for every case, extract it. Do not repeat. Every time you need to do a copy/paste, it’s a red alarm that you’re accumulating waste. If there’s a need to copy and paste and have the same information in more than one place, then when the time comes to update them, you’ll have to update it in multiple places. Nothing is more certain to cause errors. This rule applies to everything in life and work. Extract the repeating information to one single place and link or associate its consumers. #8 Always keep in mind the expected result. A test is a waste of time if you don’t have expectations for it. You also have expectations when you do exploratory testing. You want to find issues, problems, failures, inconsistencies, non-conformities, and anything else that bothers you as a tester and that might bother the user. The expected result is your testing North Star. Why are you testing this? What do you expect to find? You don’t expect that it will work. Nothing works until proven with testing. If you enter the testing with “everything seems to work”, you’re sabotaging yourself. So when you write the test case, always ask yourself what the expected result of each step is and out of the entire test case. Each step and test data must exploit various ways that might cause the system to fail to meet expectations. The relation between the positive and negative test cases is one to many. Do not get satisfied with only the positive ones. Prove the system is robust. #9 There is more than one expected result. Only one expected result is a rare occasion. In a typical web application, you might have to check the result in the UI, check if the value is properly sent through the API, and if it’s stored correctly in the database. You need to verify the expected result at 3 different places. Write in the test case and the additional info on how to access those places and how to verify the expected result. #10 Always write in Tables instead of Lists What we call a test case can be a single value or a combination of values. It can be a single action or a combination of actions. But most of all, it’s a writing concept depicting a combination of input values, actions, and expected results. A table is the best format for a test case. Even if you need to execute a sequence of actions, do not write it as a list of actions. When you start writing a test case, regardless if you use a test case management system or not, always start with a table. – that’s your test case table, all you need. Add a simple section about the purpose, the preconditions, configurations, etc. #, Step, Input, Expected Result #11 Use domain and UI language. The wording in the test case must follow your conventions. If there aren’t any, the test cases are a good starting point to build your product vocabulary. Very often, the engineers use different names for features and components than the product or the sales and marketing teams or the customers. Testers are the ones who need to speak all the languages, and the test cases are the source of truth for names and behavior. #12 Follow the test case naming convention: Whatever content you create, it should be clear at a glance what information it provides and how to navigate through it. The best naming convention for a test case is: “Module – Functionality – What case do you verify with it.” Do not write test case titles like “Verify the reporting works as expected”. There are two major advantages of writing title schemes: First, it’s unambiguously clear what the purpose of the test is and what the feature under the test is. Second, it makes the test cases searchable, filterable and sortable, easy to group, organise and prioritise. #13 Follow test data styling convention: If you’ve read my previous threads on test case writing, you’ll notice a major theme – clarity. And it’s not only about test cases. What is known must be clearly expressed to the target audience. Not surprising at all, the style makes the difference. Do not go the other extreme wasting too much time and effort into design-first text that is even more unclear. Use font style clever to distinguish test case text, instructions, steps, expected results from test data, configurations, feature and component names, etc. The easiest thing to do is use ALL CAPS for anything that is not plain text – any UI components or feature names. That’s the fastest to write too. The other examples of test data styling conventions Save, SAVE, button. Save, SAVE File, EDIT, , menu View HISTORY “Save” button; “File” menu. Styling conventions also apply to input test data, but the smartest practice is using test parameters and extracting the test data outside the test. @firstName, @first_name, @FIRST_NAME @firstName, <firstName>, [first_name] [firstName], “first_name” Start writing and see for yourself what’s the easiest and fastest to write. For me, the best format for test parameters is @firstName Last but not least, parameterised test cases are much faster to automate. Keep writing These are my 13 rules on writing test cases that have proven to build great relationships between product, design, testing, engineering and support teams through the years, as they unify the product language and streamline the communication between the teams. Test cases are a knowledge base. They are the living history of the software and are there to reply to exclamations like “This has always worked like that!”, “It’s by design!”, “How is this supposed to work?”, “Why did that bug escape?”. Test cases are also a great analysis and risk management tool. While writing them, the team could immediately spots omissions, contradictions, ambiguities, scope creep and scope cuts in the requirements and thus ensure the team enters development with much more clarity about the customer expectations. If “We should’ve refined the story better” is one of the regular items under the “Improve” section of your retrospectives, don’t miss my “Mastering User Story Analysis” workshop with the state-of-the-art ! User Story Analysis Miro board Also published . here

13 Tips on Writing Powerful Test Cases That You Need to Know

Hi there, my name is and I am 18 years old. My goal is to become a successful developer, and I am interested in building web apps with frontend technologies. Rahul Today, I'd like to share some useful JavaScript stuff I have saved and I think it can help make your life as a developer easier. Let's get started! Generate a random number between two values: const randomNumber = Math.random() * (max - min) + min Check if a number is an integer: const isInteger = (num) => num % 1 === 0 Check if a value is or : null undefined const isNil = (value) => value === null || value === undefined Check if a value is a truthy value: const isTruthy = (value) => !!value Check if a value is a falsy value: const isFalsy = (value) => !value Check if a value is a valid credit card number: const isCreditCard = (cc) => {
  const regex = /(?:4[0-9]{12}(?:[0-9]{3})?|[25][1-7][0-9]{14}|6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})/;
  return regex.test(cc);
} Check if a value is an object: const isObject = (obj) => obj === Object(obj) Check if a value is a function: const isFunction = (fn) => typeof fn === 'function' Remove Duplicated from Array const removeDuplicates = (arr) => [...new Set(arr)]; Check if a value is a promise: const isPromise = (promise) => promise instanceof Promise Check if a value is a valid email address: const isEmail = (email) => {
  const regex = /(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))/;
  return regex.test(email);
} Check if a string ends with a given suffix: const endsWith = (str, suffix) => str.endsWith(suffix) Check if a string starts with a given prefix: const startsWith = (str, prefix) => str.startsWith(prefix) Check if a value is a valid URL: const isURL = (url) => {
  const regex = /(?:http(s)?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~:/?#[\]@!\$&'\*\+,;=.]+/;
  return regex.test(url);
} Check if a value is a valid hexadecimal color code: const isHexColor = (hex) => {
  const regex = /#?([0-9A-Fa-f]{6}|[0-9A-Fa-f]{3})/;
  return regex.test(hex);
} Check if a value is a valid postal code: const isPostalCode = (postalCode, countryCode) => {
  if (countryCode === 'US') {
    const regex = /[0-9]{5}(?:-[0-9]{4})?/;
    return regex.test(postalCode);
  } else if (countryCode === 'CA') {
    const regex = /[ABCEGHJKLMNPRSTVXY][0-9][ABCEGHJKLMNPRSTVWXYZ] [0-9][ABCEGHJKLMNPRSTVWXYZ][0-9]/;
    return regex.test(postalCode.toUpperCase());
  } else {
    // Add regex for other country codes as needed
    return false;
  }
} Check if a value is a DOM element: const isDOMElement = (value) => typeof value === 'object' && value.nodeType === 1 && typeof value.style === 'object' && typeof value.ownerDocument === 'object'; Check if a value is a valid CSS length (e.g. , , ): 10px 1em 50% const isCSSLength = (value) => /([-+]?[\d.]+)(%|[a-z]{1,2})/.test(String(value)); Check if a value is a valid date string (e.g. , , ): 2022-09-01 September 1, 2022 9/1/2022 const isDateString = (value) => !isNaN(Date.parse(value)); Check if a value is a number representing a safe integer (those integers that can be accurately represented in JavaScript): const isSafeInteger = (num) => Number.isSafeInteger(num) Check if a value is a valid Crypto address: //Ethereum
const isEthereumAddress = (address) => {
  const regex = /0x[a-fA-F0-9]{40}/;
  return regex.test(address);
}

//bitcoin
const isBitcoinAddress = (address) => {
  const regex = /[13][a-km-zA-HJ-NP-Z0-9]{25,34}/;
  return regex.test(address);
}

// ripple
const isRippleAddress = (address) => {
  const regex = /r[0-9a-zA-Z]{33}/;
  return regex.test(address);
} Check if a value is a valid RGB color code const isRGBColor = (rgb) => {
  const regex = /rgb$\s*([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\s*,\s*([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\s*,\s*([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\s*$/;
  return regex.test(rgb);
} Quickly create an array of characters from a string: const string = "abcdefg";
const array = [...string]; Quickly create an object with all the properties and values of another object, but with a different key for each property const original = {a: 1, b: 2, c: 3};
const mapped = {...original, ...Object.keys(original).reduce((obj, key) => ({...obj, [key.toUpperCase()]: original[key]}), {})}; Quickly create an array of numbers from 1 to 10 const array = [...Array(10).keys()].map(i => i + 1); Quickly shuffle an array const shuffle = (array) => array.sort(() => Math.random() - 0.5); Convert an array-like object (such as a NodeList) to an array const toArray = (arrayLike) => Array.prototype.slice.call(arrayLike); Sort Arrays //Ascending
const sortAscending = (array) => array.sort((a, b) => a - b);

//Descending
const sortDescending = (array) => array.sort((a, b) => b - a); Debounce a function const debounce = (fn, time) => {
  let timeout;
  return function(...args) {
    clearTimeout(timeout);
    timeout = setTimeout(() => fn.apply(this, args), time);
  };
}; Open a new tab with a given URL const openTab = (url) => {
  window.open(url, "_blank");
}; Get the difference between two dates const dateDiff = (date1, date2) => Math.abs(new Date(date1) - new Date(date2)); Generate a random string of a given length const randomString = (length) => {
  let result = "";
  const characters = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
  for (let i = 0; i < length; i++) {
    result += characters.charAt(Math.floor(Math.random() * characters.length));
  }
  return result;
}; Get value of cookie const getCookie = (name) => {
  const value = `; ${document.cookie}`;
  const parts = value.split(`; ${name}=`);
  if (parts.length === 2) return parts.pop().split(";").shift();
}; Thank you for Reading. It is important to note that simply copying and pasting code without understanding how it works can lead to problems down the line. It is always a good idea to test the code and ensure that it functions properly in the context of your project. Additionally, don't be afraid to customize the code to fit your specific needs. As a helpful tip, consider saving a collection of useful code snippets for quick reference in the future. I am also learning if I am going wrong somewhere, let me know. Also Published Here

32 Awesome JavaScript Snippets (one-liners) You Should Use in 2023 

Hey there, Today I want to speak about Fixtures, a vital friend if you're going to work with Playwright and you want to share data between your tests or create custom executions. Let's start with a simple example. test('should win the player "X"', async ({ page }) => {
  await page.goto("/");
  await page.locator("button:nth-child(1)").click();
  await page.locator("button:nth-child(5)").click();
  await page.locator("button:nth-child(6)").click();
  await page.locator("button:nth-child(7)").click();
  await page.locator("button:nth-child(3)").click();
  await page.locator("button:nth-child(9)").click();
  await page.locator("button:nth-child(2)").click();

  const winnerParagraph = await page.getByText(/winner/i);
  await expect(winnerParagraph).toContainText("X");
}); Imagine you want to extract all the indexes of the buttons in an array to replicate this test and reduce all these click commands to have a shorter test. So, the first refactor before introducing the fixture feature could be this. test('should win the player "X"', async ({ page }) => {
  const playerXWinMoves = [1, 5, 6, 7, 3, 9, 2];

  for (const move of playerXWinMoves) {
    await page.locator(`button:nth-child(${move})`).click();
  }

  const winnerParagraph = await page.getByText(/winner/i);
  await expect(winnerParagraph).toContainText("X");
}); As you can notice, all the steps to reproduce the players' moves are contained in the variables. The test is concise, but most important, all the actions are in a single variable. Now it's time to introduce fixtures. playerXWinMoves Before carrying on, let me spend some words describing what fixtures are. What fixtures are Test fixtures are used to establish an environment for each test, giving the test everything it needs and nothing else. Test fixtures are isolated between tests. With fixtures, you can group tests based on their meaning instead of their standard setup. After this definition, you can think fixtures could be replaced by before/after hooks, but fixtures have several advantages over these hooks: Fixtures setup and teardown in the same place, making it easier to write. encapsulate Fixtures are between test files - you can define them once and use them in all your tests. That's how Playwright's built-in fixture works. reusable page Fixtures are - you can define as many fixtures as you'd like, and Playwright Test will set up only the ones needed by your test and nothing else. on-demand Fixtures are - they can depend on each other to provide complex behaviours. composable Fixtures are . Tests can use any combination of fixtures to tailor the precise environment they need without affecting other tests. flexible Fixtures simplify . You no longer need to wrap tests in s that set up an environment and are free to group your tests by their meaning instead. grouping describe Perfect, after this boring theoric part, it's time to get your hands dirty with fixtures in Playwright. To start the explanation, I want to begin with the final result. import { expect, test as base } from "@playwright/test";

type TestFixtures = {
  playerXWinMoves: [number, number, number, number, number, number, number];
};

const test = base.extend<TestFixtures>({
  playerXWinMoves: async ({}, use) => {
    await use([1, 5, 6, 7, 3, 9, 2]);
  },
});

test('should win the player "X"', async ({ page, playerXWinMoves }) => {
  for (const move of playerXWinMoves) {
    await page.locator(`button:nth-child(${move})`).click();
  }

  const winnerParagraph = await page.getByText(/winner/i);
  await expect(winnerParagraph).toContainText("X");
}); As you can see, to build our fixture, you have to extend the test object exposed by Playwright. Inside this function, you can create an object that represents your fixtures. In this case, I added the property to share the steps to reproduce the Player X win. In addition, just because I love working with Typescript, I added the type to describe my fixtures. It's essential to notice that inside the fixture, you must use the method to set the value for your fixtures. playerXWinMoves TestFixtures use Ok, carry on, and let's take a look at the test now. You can notice that you can get the fixture directly from the object passed to your test. And now, if you remove the previous variable and use the new one from the params, you will create your first test with Playwright using fixture. playerXWinMoves Great, I hope you can understand the incredible power of this feature and how it can help you to make your test more readable or to bring together some data useful in many tests. But before closing, let me leave you another bit. With fixtures, you can also override an object exposed directly by Playwright. For instance, you can override the page object, and instead of creating a hook to navigate to the home page, you can override the page and add this step in a fixture, so every test, before its execution, runs the steps to navigate to the home page. beforeEach And how can we do that? Simply in this way. const test = base.extend<TestFixtures>({
  page: async ({ baseURL, page }, use) => {
    baseURL && (await page.goto(baseURL));
    await use(page);
  },
}); Doing that, now each test, when it starts its execution, goes to the baseURL and then runs all the code. Incredible no? Now, you can remove the hook in the test file and rerun your tests to check the result. beforeEach Ok, that's all folk! Today you have learned how Playwright fixtures work and how to build one. I hope you enjoyed this post, and if you have any doubts, don't hesitate to reach out to me; I will be happy to help you. See you soon Bye Bye 👋 p.s. you can check out the code of this article here

An Essential Guide to Fixtures in Playwright

Una guía esencial para los accesorios en Dramaturgia

Un guide essentiel des luminaires dans Playwright

नाटककार में फिक्स्चर के लिए एक आवश्यक गाइड

Playwright のフィクスチャの基本ガイド

Um guia essencial para acessórios em dramaturgos

Hướng dẫn cần thiết về đồ đạc trong nhà viết kịch

剧作家中固定装置的基本指南

In one of my previous articles, I already talked about how sometimes we don’t even notice the algorithms around us. When using a tool or library, we consume it as a given without even understanding how it works behind the scenes. Today I am going to reverse-engineer the algorithm Jest uses to find related tests when we run . jest --findRelatedTests What is Jest and how I can use it? is a popular JavaScript testing framework used in almost every project nowadays. You can start with it simply by creating any test file with a extension and running the jest command in your terminal. Jest also provides extensive configuration options that developers can use to define what tests to include/exclude, what coverage output to generate in which format, and how to transpile source files, etc. - you can find all of this in the . Jest test.js official docs The most exciting option is to run jest with and passing list of the files you want to test: --findRelatedTests jest --findRelatedTests /path/to/file1, /path/to/file2 This way, Jest will run not only direct tests associated with these files but also the test for transitive dependencies. It becomes super handy when it comes to optimizing git pre-commit command execution. Instead of running an entire set of tests every time you are about to commit a new change, Jest will run only a set of tests that are actually impacted by corresponding change saving developers a great amount of time while being accurate with results. Jest file system Jest uses a custom for building metadata about the files and dependencies between them in an optimized way. This component definitely deserves its own “Under the hood” article. For now, let’s simplify that it provides API to get a collection of all files in the project and its dependencies in the following shape: haste module system Array<{ 
// absolute file path 
file: string; 

// list of depedencies (modules imported in current file)
dependencies: Array<string>; 
}> Looking under the hood What do we know: set of files that have been added/modified/deleted and staged in Git and entire file system metadata Now we need to find an algorithm to detect and run only test files that have been impacted by this change. And Jest provides API to solve this problem: resolveInverse resolveInverse(
paths: Set<string>, 
filter: (file: string) => boolean, 
options?: ResolveModuleConfig, ): Array<string> { 
   return this.resolveInverseModuleMap(paths, filter, options)
          .map( module => module.file, ); 
} where: is a set of file paths that have changed paths is a predicate to detect whether the current file is a test file filter function object (not relevant for the algorithm, ignore it for now) options uses a very straightforward approach to solve this problem that is based on (BFS). First of all, it initializes the following sets: resolveInverseModuleMap Breadth First Search - set of files that have been affected. Basically, it is a temporary set that is mutated after every iteration of BFS, and serves as a termination indicator - the algorithm stops when it's empty. On the first iteration, this set is populated from provided as input. changed paths - set of files that have been affected. By default, this set is populated from input but with the condition that the file is actually a test file. On every iteration of BFS, every matched test file is removed from this file. At the end of the algorithm, the remaining file paths in this set are concatenated with the results of BFS execution. This is to handle a simple use case when the change was made in the test file and not in the source. relatedPaths test paths - files that have been already traversed and can be skipped from further processing. visitedModules So let’s visualize our algorithm. First of all, let’s define our example file system: Blocks in red represent changed modules (rectangles are test files, and circles are source files). Arrows start from parent to child (meaning that module is imported by and ). Initially, our data structures will look like this: a a.test a1 Our set is not empty. changed That means we need to traverse all files in our filesystem to get a list of dependencies for every module in set. After the first iteration, we already discovered that at least and needs to be run as dependencies of and respectively: changed b.test a1.test a.test a1 Carrying out another traversal. Now we see that is imported into both and but we already have in our originally changed files. In this case, we need to remove it from set and add it to the results. In case if didn't have any dependency, Jest would simply merge it into results when returning it to the upstream caller. b2 a.test b2.test a.test relatedPaths a.test In the end, we have only 2 test modules in our set and we already see that is in our visited set and does not have any dependents. Basically, no-op here - after this iteration changed set will become empty, and we will break out of our BFS loop. changed a.test b2.test As a result, we can narrow down our test scope from 5 test files down to 4. It’s not so impressive but keep in mind that I have picked a very small dataset for this example for simplicity — in real-world projects, there would be thousands of files, and improvement can be really drastic. You can find the original implementation . here Final thoughts Do you think this algorithm can be improved? What if we could have inverse links to the dependent modules as part of file metadata (similar to DOM elements that have reference to the parent)? In this case, we would not have to traverse the entire file system and focus only on a subset of it.

Taking a Look under the Hood to See How Jest Finds Related Tests

Nowadays, End-to-End testing of functional and non-functional requirements is a must but is time-consuming and can be solved using multiple tools and approaches. This article explores how to test the performance of an IoT system using Locust and Azure. Defining the test scope and requirements is essential to maximize the value of the obtained results. Focusing on the metrics with a higher ROI is crucial (Those that are easily measurable and have a high value for the development team). The metrics are objective measurements of the system performance and dependent variables of: Infrastructure configuration (Cluster Nodes, CPU, Memory, Replicas, etc.) EventHub configuration (TU, nº of partitions, consumer groups, etc.) Solution architecture scalability and endurance The performance tests help us to understand how the systems behave under a defined load: telemetries sent by a device per second multiplied by the number of total devices. The testing scope is the IoT platform and the integration (It's not a goal testing the IoT device). The key metrics to measure the solution performance are: Azure EventHub EventHub events ingestion delay time (ms) Events loss rate (%) Tools and Infrastructure Locust is a Python performance tool that officially supports the HTTP/HTTPS protocol, but the developer can extend it to test any protocol or system. Users can wrap the protocol or system library and trigger Locust events after each request to inform Locust about the status of the test. Locust Wrapping the library is one of the solutions to test the and IoT systems. Locust has some plugins which extend the supported protocols and systems : Paho-MQTT MQTT protocol https://github.com/SvenskaSpel/locust-plugins Azure Container Instance offers an easy way to configure, deploy and run multiple auto-managed containers. It’s an on-demand service that allows you to run any container using the cloud. Azure Container Instances Azure EventHub An is a scalable data ingestion service managed by Azure. Azure products can consume the outgoing messages to analyze them and then report the analysis on a dashboard (Streaming Analytics and PowerBI). Moreover, Azure developed multiple EventHub SDKs to consume the outgoing events and implement your business logic. Azure EventHub Technical Approach The article's goal is to have an overview of the overall solution implementation and architecture. A detailed technical explanation is out of the scope. To know more about the implementation, use the following link to review the source code of the solution: . https://github.com/joan-mido-qa/mqtt-locust-on-azure The repository contains an example of the framework to test the IoT system performance using . Locust Deploy Each container (Locust master/workers and Consumer) can be easily deployed using the Azure GitHub Action: . An alternative to GitHub Actions is Terraform. You can define your testing infrastructure using a Terraform plan and apply it using a GitHub workflow. aci-deploy The master node has attached a file share (Azure Storage Accounts) to save the test tun logs and HTML reports. The action can be triggered automatically by any GitHub event or manually by a user. Using the GitHub UI the user can define the test inputs: run time, spawn rate, number of users, etc. The deployed number of worker nodes depends on the number of devices (~500 devices per worker) The Consumer also uses the action. A user can deploy the Consumer defining the container name, azure region, EventHub name, connection string, and consumer group. aci-deploy Performance Test IoT systems use the . Devices can publish telemetries about a topic using the MQTT broker. Then subscribers can choose a topic to subscribe to and receive telemetries from the subscription. In this case, the IoT platform receives and processes the telemetries sent by the devices to ingest them into the EventHub. To End-to-End test the system, the approach uses to connect multiple IoT devices to the MQTT broker and publish telemetries with a constant throughput. The EventHub receives the telemetries from the IoT platformer once processed. Finally, the EventHub Consumer reads the outgoing telemetries to aggregate the received events count and the ingestion delay time. MQTT protocol Locust Host CPU and Memory limits the maximum number of users per Locust instance (~500-750 users). Locust can run in distributed mode to overcome these limitations and spawn thousands of users. The test uses to run the Locust master/workers and the events Consumer. The events Consumer reads the EventHub outgoing messages and dumps the aggregated measurements into the InfluxDB. Azure Container Instances Last, Grafana plots and displays the performance results using InfluxDB data. InfluxDB calculates the total telemetries count, telemetries per minute, and nth% percentile delay time using the aggregated data from the Consumer. Metrics 1. Event Ingestion Delay Time (ms) The ingestion delay time is measured as the event enqueue time minus the event sent time. Locust adds a timestamt to each telemetry sending it. 2. % of Lost Telemetries: Received Events / Sent Telemetries Report Locust uses a self-developed InfluxDB plug-in to report the number of devices and telemetries (Explaining the plug-in implementation is out of the scope of this article). InfluxDB calculates each metric using Flux, INfluxDB query language. Grafana is not only used to visualize and analyze the performance results but to also monitor the system infrastructure: Cluster CPU, Memory, Node, Pods, etc. Implemented Solutions Documenting the overall solution and the problems faced during the development phase helps to understand the solution architecture and the technical decisions. The section goal is discussing the issues faced during the development and the implemented solution. Number of Events Consuming a certain number of events per minute, aggregate the statistics, and dumping them into the database is not a straightforward task. One of the requirements of the test was to support a load of 340k events per minute. It’s not only the events number but also the events size. To overcome the vast amount of outgoing data the following solutions were implemented: Use multiple consumers to read from different EventHub partitions. Avoid dumping all the events into the database. The consumer aggregates the data by the number of events received per delay time. Each row has a UUID to avoid overriding the data rows when dumped into the InfluxDB. Enqueue Time Delay Time Count uuid 20:00:00 1000 5 704b496b-… 20:00:00 1050 2 704b496b-… 20:01:00 1000 10 704b496b-… 20:01:00 950 7 704b496b-… InfluxDB aggregates the data before displaying it on Grafana: # 95% Delay Time Percentile

getFieldValue = (tables=<-, field) => {
    extract = tables
        |> map(fn: (r) => ({ r with _value: r._value * 95 / 100 }))
        |> findColumn(fn: (key) => key._field == field, column: "_value")

    return extract[0] 
}

total = from(bucket: "")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "data")
  |> filter(fn: (r) => r["run_id"] == "$run_id")
  |> filter(fn: (r) => r["_field"] == "count")
  |> drop(columns: ["uuid", "delay_ms"])
  |> sum()
  |> getFieldValue(field: "count")

from(bucket: "")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "data")
  |> filter(fn: (r) => r["run_id"] == "$run_id")
  |> filter(fn: (r) => r["_field"] == "count")
  |> keep(columns: ["delay_ms", "_value"])
  |> sum()
  |> group()
  |> map(fn: (r) => ({ r with delay_ms: int(v: r.delay_ms) }))
  |> sort(columns: ["delay_ms"])
  |> cumulativeSum(columns: ["_value"])
  |> filter(fn: (r) => r._value < total)
  |> last()
  |> keep(columns: ["delay_ms"])

# Number of Telemetries per Minute

from(bucket: "")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "data")
  |> filter(fn: (r) => r["run_id"] == "$run_id")
  |> filter(fn: (r) => r["_field"] == "count")
  |> drop(columns: ["uuid", "delay_ms"])
  |> window(every: 1m)
  |> sum()
  |> group()

# Number of Telemetries

from(bucket: "")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "data")
  |> filter(fn: (r) => r["run_id"] == "$run_id")
  |> filter(fn: (r) => r["_field"] == "count")
  |> drop(columns: ["uuid", "delay_ms"])
  |> sum() Number of Devices per Worker Paho-MQTT uses the `select(..)` method, which depends on the FD_SETSIZE variable (FD_SETSIZE limits the number of open files). Python compilation uses a fixed FD_SETSIZE=1024. Each MQTT client opens three files per connection: 3 * 340 = 1020. Each worker has a limit of ~250 - 300 devices because Locust also opens a file per user. To spawn 6k devices, Azure needs to run 24 containers. Running a high number of workers increases the error probability. In rare cases, the workers can’t successfully set up the environment or connect to the master making the test fail on start. To increase the number of devices per worker and make the test error-prone the following solution was implemented: Use a shell script to increase the open files soft limit and run locust as the container entry point: ulimit -Sn 10000 Override the MQTT client  `_loop(..)` to use Python instead of . Selectors use the method that best fits your OS: selectors select import selectors

class MqttClient(Client):
    [...]
    def _loop(self, timeout: float = 1.0) -> int:
        if timeout < 0.0:
            raise ValueError("Invalid timeout.")

        sel = selectors.DefaultSelector()

        eventmask = selectors.EVENT_READ

        with suppress(IndexError):
            packet = self._out_packet.popleft()
            self._out_packet.appendleft(packet)
            eventmask = selectors.EVENT_WRITE | eventmask

        pending_bytes = 0
        if hasattr(self._sock, "pending"):
            pending_bytes = self._sock.pending()

        # if bytes are pending do not wait in select
        if pending_bytes > 0:
            timeout = 0.0

        try:
            if self._sockpairR is None:
                sel.register(self._sock, eventmask)
            else:
                sel.register(self._sock, eventmask)
                sel.register(self._sockpairR, selectors.EVENT_READ)

            events = sel.select(timeout)
        except TypeError:
            return int(MQTT_ERR_CONN_LOST)
        except ValueError:
            return int(MQTT_ERR_CONN_LOST)
        except Exception:
            return int(MQTT_ERR_UNKNOWN)

        socklist: list[list] = [[], []]

        for key, _event in events:
            if key.events & selectors.EVENT_READ:
                socklist[0].append(key.fileobj)

            if key.events & selectors.EVENT_WRITE:
                socklist[1].append(key.fileobj)

        if self._sock in socklist[0] or pending_bytes > 0:
            rc = self.loop_read()
            if rc or self._sock is None:
                return int(rc)

        if self._sockpairR and self._sockpairR in socklist[0]:
            socklist[1].insert(0, self._sock)
            with suppress(BlockingIOError):
                # Read many bytes at once - this allows up to 10000 calls to
                # publish() inbetween calls to loop().
                self._sockpairR.recv(10000)

        if self._sock in socklist[1]:
            rc = self.loop_write()
            if rc or self._sock is None:
                return int(rc)

        sel.close()

        return int(self.loop_misc()) Conclusions Data is crucial to make informed decisions. That’s the reason why testing is essential to solve a known problem. Testing allows us to understand the current status of the product regarding a specific requirement and helps us to decide which are the following steps. Define the output metrics depending on the resulting technical value. Avoid measuring costly and low-technical value metrics to not add noise to your performance analysis. Avoid developing an overengineered solution by setting a clear test scope and correctly gathering the performance requirements from the development and product teams. It’s better to test a specific requirement and have fast feedback loops than trying to test all the possible scenarios and not having enough feedback. The system performance depends on the product infrastructure design and architecture. Different infrastructure configurations can lead to totally different results. Testing the system helped us to understand how it behaves depending on the incoming events load and to focus on the bottlenecks. The analysis of the results served as guidance to have well-defined infrastructure requirements to optimally run the system under a specific load.

IoT Performance Load Testing with Locust and Azure

Arrays are one of the most important concepts in JavaScript. Arrays are objects that enable storing a collection of items and data under a single variable name and have the capability to perform a certain operation. In this article, we’ll go through 20 methods we can use with an array to manage the data we have. 1)- pop() One of the array methods is the method. pop() The method removes the last element of an array. pop() var phones=["Nokia","Iphone","Samsung","Xiaomi"];
        phones.pop();
        console.log(phones); If we console.log the array, it becomes: 2)- push() Unlike method, method adds an element to the end of an array. pop() push() var phones=["Nokia","Iphone","Samsung","Xiaomi"];
        phones.push("Oppo");
        console.log(phones); The array now becomes: 3)- shift() method removes the first element of an array. shift() var phones=["Nokia","Iphone","Samsung","Xiaomi"];
        phones.shift();
        console.log(phones); The array becomes then: 4)- unshift() method adds an element at the beginning of an array. unshift() var phones=["Nokia","Iphone","Samsung","Xiaomi"];
        phones.unshift("Sony");
        console.log(phones); The array becomes: 5)- splice() To add or remove elements to/from a specific position in the array the method allows us to do that. splice() a)- Add an element to a specific position var phones=["Nokia","Iphone","Samsung","Xiaomi"];
        phones.splice(2,0,"Sony","Oppo");
        console.log(phones); To add an element to a specific position we use method. It takes 3 parameters at least: splice() The first parameter is where, or at what position we want to add our elements. In this example we want to add them at position 2, which means after “Iphone”, hence “2” is added as first parameter of . splice() The second parameter reflects how many elements we want to delete. In this example we don’t want to delete any element, so we added “0” as second parameter of . splice() The third parameter and beyond are the elements that we want to add. So the array becomes: b)- Remove a specific element method, unlike and methods, allows us to remove any element from an array. All we have to do is to specify its position. splice() pop() shift() var phones=["Nokia","Iphone","Samsung","Xiaomi"];
        phones.splice(2,1);
        console.log(phones); To remove a specific element from the array we use method. It takes 2 parameters: splice() The first parameter is where, or at what position we want to add our elements. But here we will use the first parameter to specify from where the “delete” operation should start. In this example it should start from position 2. The second parameter is to specify how many elements should be removed. In this example we want to remove one element. At the end we have the following array: 6)- map() method generally accepts a function as argument. This function runs through each element of the array and returns a new array based on these elements. map() var numbers=[10, 20, 30, 40];
       var newArray= numbers.map(function(element){
           return element*2;
        })
        console.log(newArray); So here the function took every element of the array “ ” and multiplied it by 2, and at the end returned a new array with the new results. numbers 7)- filter() method returns a new array that contains all the elements that pass the test. filter() var age=[70,45,15,39];
       var newArray= age.filter(function(ages){
        return ages>30; 
       })
        console.log(newArray); Here the function returns the age if the age is greater than 30 and stores it in the new array. 8)- concat() method helps us merge arrays to get one array. concat() var vegetables=["broccoli","eggplants","potatoes"];
       var fruit=["banana","kiwi","apple"];
       var mixed=vegetables.concat(fruit);
       console.log(mixed); The new array is: 9)- fill() To fill an array with a static value we use method. fill() var numbers=[100,10,20,25,15];
var static=numbers.fill("kiwi");
   console.log(static); The array becomes: We can also add the static value at a specific position of the array. var numbers=[100,10,20,25,15];
var static=numbers.fill("kiwi", 2, 4);
   console.log(static); The array becomes: To add static values to specific positions of the array we pass the method some parameters: fill() The first parameter is the static value, in this example it is “Kiwi”. The second parameter is starting from which position this value should be added, in this example it should be added starting from position 2. The third value is at what position we should stop adding the static value. In this example it is at position 4. 10)- join() method joins the elements of the array and returns a string. join() var fruit=["banana","orange","kiwi","apple"];
       var elements=fruit.join("/");    
       console.log(elements); In the browser we have: method accepts a separator that is added between the parenthesis. In this example we chose the backslash “/” as separator, we can choose any separator we want. join() If we choose a hyphen as separator the code will be: var fruit=["banana","orange","kiwi","apple"];
       var elements=fruit.join("-");   
       console.log(elements); And so the results will be: 11)- IndexOf() method returns the first index at which a given element can be found in the array. If the element is not in the array, then the method returns -1. indexOf() var fruit=["banana","orange","kiwi","apple","mango","kiwi"];
       var element=fruit.indexOf("kiwi");
       console.log("the index of this fruit is " + element); In the browser we have: Here the index of “Kiwi” is 2. Because returns the index of the first occurrence of the element. If we try this with “apple” we’ll have 3 as an index. indexOf() However, what about if we have the same element repeated twice or 3 times or more in the array, and we want to know the index of the second occurrence or third or fourth occurrence? Let’s see that with “Kiwi”. “Kiwi” is repeated twice. So the index of the first “Kiwi” is 2. To know the index of the second “Kiwi” we’ll add: var fruit=["banana","orange","kiwi","apple","mango","kiwi"];
       var element=fruit.indexOf("kiwi",3);    
       console.log("the index of this fruit is " + element); 3 represents the start point from where the array should start looking. And here it should start after the first “Kiwi”, which means at position 3. In the browser we have: 12)- includes helps to check if an element is in the array. If so, it returns true. includes() var fruit=["banana","orange","kiwi","apple","mango"];
       var element=fruit.includes("orange");  
       console.log(element);
       //true 13)- reverse() method reverses the order of the array elements. reverse() var fruit=["banana","orange","kiwi","apple","mango"];
        fruit.reverse();      
       console.log(fruit); In the browser we have: 14)- every() method returns true if all the elements of the array pass the test. This method accepts a function that runs through all the array’s elements. every() var ages=[70,54,36,19,33];
        var check= ages.every(function(age){    
            return age>18;
        })
        console.log(check);     
        //true 15)- some() is a method that returns true if at least one element in the array passes the test given by the function passed as argument. some() var ages=[70,54,36,19,33];
        var check= ages.some(function(age){  
            return age>60;
        })
        console.log(check);      
        //true 16)- at() returns the element that is in the index specified in the parenthesis. at() var cars=["BMW","Audi","Ford","Tesla"];
       var carName=cars.at(2);
       console.log(carName + " is at index 2"); In the browser we have: 17)- of() This method creates a new array from a variable number of arguments, regardless of number or type of the arguments. var newArray=Array.of("Thomas","John","Liam","Michael");
       console.log(newArray); In the browser we have an array that is created: 18)- slice() method returns a slice or a part of the array. slice() var names=["Thomas","John","Liam","Michael"];
       var slicedArray=names.slice(1,3);
       console.log(slicedArray); The new portion of the array will be: 19)- Array.isArray() This method returns true if the declared variable is an array. var names=["Thomas","John","Liam","Michael"];
       console.log(Array.isArray(names));    
       //true 20)- delete() This methods deletes an element from the array by specifying the index we want to be deleted. However, the method leaves in the array after the element is deleted. delete() undefined or empty holes Which means even if the element is deleted, the length of the array does not change. If for example the length of the array is 5, then after an element is being deleted the length will stay 5. var names=["Thomas","John","Liam","Michael"];
       delete names[2];
       console.log(names); In the browser we have: It is not advisable to use this method to delete elements from an array. To delete an element from an array use method instead. splice() Also published . here

CheatSheet: 20 JavaScript Array Methods

A Humble Note to the Audience I am a beginner and an avid learner. In this article, I will share my journey of becoming a Software Quality Analyst by translating my previous experience in Journalism and how I am presently combining my Journalism and SQA expertise to access the world of Cybersecurity. There are many seemingly ‘unrelated’ skill sets I have ‘hacked’ to tailor to my SQA job responsibilities alongside basic application security testing. So, the terms Testing and SQA will often overlap with my ongoing Cybersecurity learning journey. First, I would like to brief the transitions of my job responsibilities before getting into further details. The Transitional Periods: Journalism to Software Testing I started my career at the print version as a city desk sub-editor after completing my graduation in English Literature and Linguistics in 2014. The newspaper was a hub of veteran senior journalists from the ‘typewriter age,’ and the year itself was a transitional period to digital journalism. The journalists were interestingly either indifferent or facing a bit of hardship in adopting new technologies like editing copies in text editors, utilising news agencies and emails from correspondents on the internet as news sources. Daily Observer Being the junior most staff in the house who had some handy technical knowledge, I was unofficially appointed to another post as a communicator between these journalists and seemingly 'new technologies. So, apart from my regular journalism, I became an unofficial tech support for them. I spent countless nights listening to their technical hardships, analysing their problems, and finding out solutions for them. I took full advantage of this opportunity by working closely with technology bit reporters, reading technical books, blogs, and articles on programming, problem-solving, computer systems, operating systems, cybersecurity… anything related to computer systems and the internet. In 2018, I joined the print version as a sub-editor. A few months later, I was shifted to its online version. New Age I had an opportunity to interact directly with journalists from the spot, collecting information from them, cross-checking facts, preparing to-the-point stories, and publishing them on the portal in time. The fun parts in this section were analysing reader behaviour from google analytics. After primary analysis, we used to fix what type of stories will be published at which time. I also had to work closely with our technical team if there were any unusual behaviour like 502 errors, any story or photo missing, news item format issues, etc., in the web application. Gradually I got an additional role of ‘Website Quality Analyst’ here. During this period, I wrote after a month’s effort. This piece was published in the now defunct youth segment of the newspaper, ‘New Age Youth’ — one of the sweetest memories of my career as a journalist. Myths and reality of a parallel world’s defence mechanism Later in 2021, I finally joined Penta Global Limited as an SQA — my real-life entry into the Information Technology industry. Cordial thanks to my childhood friend Farhad Naeem who first taught me programming and recommended me as an SQA before my CTO. So far, this is my story. Remember that I am not an expert Journalist, SQA or Cybersecurity professional. The below writing is my personal opinion on the most important skill sets and how I translate them into my daily job responsibilities. Communication, Communication and Communication The way humans communicate makes them stand out from other animals. So, it does not matter if a person is an introvert or a ‘geektrovert,’ s/he needs to maintain some level of communication; however, the audience will vary in size. To be precise, an ‘introvert’ most of the time communicates with the self, while the ‘geektrovert’ communicates with one or two specific cliques. A good problem solver knows different ways to communicate with problems, and their solution, utilising required tools and translating the solution for specific purposes. , I would advocate for a ‘weak’ student as a person who is having a sort of hardship with communicating the core concepts of a specific subject e.g. Math or a Foreign Language, as well as the classroom and instructor. It is NOT that the individual lacks ‘intellectual abilities’ like toppers. So, as a side note As an SQA, tester, or cybersecurity professional, a person must have good communication skills to access the process of , Business and Development Technologies The very basic triad can be explained like this — A firm employs developers if a concept has business values. Developers then use different technology stacks to realise the concept into a software application and deliver it for testing. So, an SQA must have some solid communication skills to understand these scenarios for a better test plan and execution to ensure product quality and security. I had to frequently put on the shoes of readers, reporters, and sub-editors to publish timely stories in my past career. I had to train my brain for any inconsistency in information in the shortest possible time. Let me share a recent experience as an SQA. We have deployed a stable one-stop service for the national export/import authorities for around three months. This system consists of different forms to be filled out by investors to obtain different certificates from the authorities concerned. One of the forms had the following feature (I’m using dummy labels) On a good sunny Sunday morning, my project lead, who was then busy with another project of our firm, sent me a short message to immediately deal with a client requirement that the Sum field would be . At first look, it seems insane, naturally. But my journalist instinct gave me a hint that there might be some communication or information gap. editable So, I quickly contacted the client-side IT Systems Expert. After analysing the requirement, the fact was clear finally. They wanted the third field to be an individual editable numeric input field, and there would be no summed-up value of Num Input 1 and Num Input 2 in this field. To sum up, the calculation was NO longer needed. Good communication skill lets a tester make authorities concerned to understand the weight of a bug and the values it adds to a product. To be precise, if an SQA or Tester is good at communicating with Problems, Machines, and Humans, it will build a concrete base of any skill set. Among them, a few are Understanding and getting involved in Software Development Life-cycle Simultaneously interacting with development, management, and client teams Mastering problem-solving skills, required programming languages, and tech stacks Getting a clear picture of a system under test and discovering bugs Understanding the weight of a bug, clarify development and management teams the value it can add to the product The list goes on …. Not all Reports are Published: Some Bugs are NOT Bugs Those who have some in-house experience in media or submitted contributions to one are aware that all the stories are not published because it is a waste of time and sometimes damages the reputation of a house. A seemingly quiet newsroom is a place where the storm takes place in the brain of journalists, like the development room in the Tech Industry. They are busy with problems worth solving. For example, a development team is busy implementing a new feature on a tight timeline. There are one or two typos in the feature. As well as, they did not tokenize some sensitive API calls due to test purposes. Should a tester make reports on the aforementioned issues? No, it is a waste of time and effort. The tester should rather be aligned with the development team and keep track of those issues. S/he should ensure that the issues are implemented before the version goes to production. Personally, I define a bug as a BUG if it system or features Breaks security issues and Uncovers bad reputation of a team or firm Generates Yes, when there are no bugs, a typo can question the language efficiency of a team. A simple text box alignment issue can also generate a bad reputation of an Omega SQA as a cherry on the top, especially when s/he is under a close-door ‘interrogation’. Adopt OWASP It all works. The reported bugs have been fixed. Now we are prepared to seal it off ‘QA Passed’. Hold on!! During the dawn of Web3, almost every software project interacts with the Internet and API calls to different extents. This makes software applications vulnerable to various types of threats in the ‘ Wild Wild Web.’ So, adopting OWASP is not only a good choice for further ensuring better software quality, but I have found a solid platform to start my journey towards cybersecurity practically. The project OWASP contains many Top 10 vulnerability lists on Web, Mobile, API, and IoT software applications that are good starting points for beginners and references for experts. I assume the steadily growing project will soon cover the Top 10 vulnerabilities of AI, ML, Blockchain, and W3 applications. I would share a bit of my experience here. Our production deployment was delayed for about a few weeks, and I had a golden opportunity to conduct penetration testing on our web application. I found three textbook vulnerabilities from OWASP. The finding, exploiting, PoC, and reporting phases were equally enjoyable to me, where I had another golden opportunity to utilise my technical and journalistic skill sets successfully. Among them, two critical ones were fixed, and the rest had been put in the backlog. Why?!! Keep reading … Discovering Vulnerability: It is the Beginning of a Long Journey If we find and can reproduce a vulnerability, e.g. an injection point somewhere in an application, it is not the ending and often not worth a formal bug report. It is often the tip of an iceberg. Because a vulnerability is worthless sans a proper exploit. In my words, an exploit is a procedure to take advantage of a vulnerability to modify a software application in such a way that it was not meant to be. So, in this case, the better approach is to inform the development team and keep track of if a new feature or any modification in the system can open a risk of exploiting the vulnerability. For an oversimplified journalistic example, a news reporter submits a 500-word story about bumper production of main staples in areas where it has been growing constantly for the past 40 years, contributing to a country's economy. This ‘news’ is worth mentioning once in a news story as a piece of information. This piece of information might hit the headlines if the country faces a food crisis despite the bumper production of that staple. So is discovering a vulnerability. It takes effort and time to find out one. But without a proper exploit and a proper workflow to utilise it, the discovery is often not worth a detailed report. So, we must develop a way to exploit a vulnerability when we find one. Sometimes we have to wait patiently, like a real-life cyber criminal to develop a workflow to use the vulnerability to produce PoC that brings the true weight and relevance to our report. Tooling: The Samurai Dilemma “They are my slaves [ books and papers ] and they must serve me as I please.” - Karl Marx Tooling does not give birth to an artist, artiste, tester, or person who stands out in her/his arena. I cannot remember any argument on Albert Einstein’s efficiency in using calculators, handwriting, or accounting. Have you ever heard or read art critics arguing over which tools Leonardo Da Vinci or Michelangelo were efficient on? Do we expect a person efficient in the grammar of a certain language would contribute to the literature of that particular language? The answers are big, Big NOs. The person behind tools defines the shape of a piece of art. As any testing is an art, this is true for Quality Assurance and Cybersecurity. Hold on! There is a catch. In reality, some tools are so embedded in certain crafts. For example, is a nylon string guitar for classical music and a Nmap for network scanning. OK, here comes my ‘ ’ Samurai Dilemma Theory Anyone will want to be a Samurai after seeing their depiction in anime and movies. But it takes a long time, rigorous training, and extreme devotion to become a samurai soldier in reality. So, if a random person is given a samurai sword to show some acts, chances are high that he would injure himself. Moreover, he might lose his life on the real battlefield. On the other hand, the result will be the same if an expert Samurai guru wants to fight against modern artillery without ‘modifying’ her/his sword. So, the point is a Tester, or SQA, will choose, modify, and develop her toolbox utilising creativity and responding to needs. S/he would not stick to a specific testing method, OS distro, or tool set but rather modify existing ones or adopt new ones. In the end, for creative minds, Windows vs Linux, Python vs Ruby, C++ vs Rust, and VIM vs VS Code arguments don’t matter. They just pick or discard their tools to create a piece of art. Do you need some Proof of Concept about the classical guitar example I mentioned above? Just listen to Tim Henson and see how to play rock pieces with a custom nylon string classical guitar !! In future writeups, I will show you how Nmap can be used in a bash script on a raspberry pi box to build a basic 24/7 intrusion detection system. A Note on Automation This is the bonus section before I wrap the long story up. I was an automation freak when I started my career as an SQA in August 2021. A few months later, our tech lead and development lead held a meeting with me. They assured me that they will inform me when we REALLY need test automation suits. Until then, we should focus on ‘manual testing.’ I researched the internet and realised there is no such thing as ‘automated testing.’ It is yet another utility tool under the belt of a seasoned tester to ensure a software application's quality or security measures. Although, this point of view has criticism. I have never heard of ‘automation’ hypes in the world of cybersecurity or penetration testing other than vulnerability scanners like Nessus or frameworks like Social Engineering Toolkit (SET) or Metasploit. The only field where the term ‘automation’ has become a buzzword in the SQA world. Why? This answer needs in-depth research and an entire article on the topic. Dear learned readers, please pardon my ignorance if I am wrong. There are many AI tools in the wild for writing articles and making beautiful paintings. But, human intervention is a must for fine-tuned results as of writing this article on November 29, 2022. Yes, there are arguments that GitHub Copilot and AI will rule the world in the near future, which I am not taking into consideration in this article. Rather, I would reiterate that automation is merely a tool in the toolbox of a tester, as discussed in the above section. A seasoned SQA or Security Professional will miss certain important findings if they heavily depend on automation results, while beginners like me will OBVIOUSLY miss a lot of interesting things that will build a solid foundation of their understanding of the art of testing. Personally, I prefer automating a few regressive features with well-handled corner cases in a system. This will add good value to testing by saving some regression test time and effort given by testers. stable If the requirements of a product change too frequently in a cyclic order, test automation will NOT bring any value against the given time and effort. So, confirming a product’s stability and pin-pointing test scopes are inevitable parameters to be ensured before we build our automation test suites. Final Thoughts Dear reader, you deserve cordial thanks for reading this long article. Here are some final words for my fellow journalists, SQAs, Testers, and Cybersecurity professionals. The Reporter vs Desk and SQA vs Dev fighting is a long-existing tradition worth mentioning. When we report a bug, a developer first tries to identify whether it is a feature we misunderstood. When developers successfully deliver a new feature, an SQA suspects there must be one or more bugs, while security professionals are one step ahead to hack into their implementation logic.  The internet is full of humorous memes in this regard. This is a healthy creative practice because our ‘confrontations’ keep the information intact or deliver quality software applications to serve humanity in the end. So, there is nothing personal if the concerned authorities ask us why a bug that is actually stashed under a 20-day-old backlog was not reported. And this will often happen if you are an and testing multiple projects. Just put on their shoes. They have critical problems, like us, to solve. The piece by James Marcus Bach is a must-read for  every Tester, SQA and Cybersecurity professional. Omega Tester Omega Tester Please feel free to share your opinion, constructive criticism, and professional advice with me. Note: All the images are captured by the author.

Stories, News & Mentions about 2TEST

2test’s Stories on HackerNoon

2test's News & Mentions