Exploring Quality in Pentesting

Quality in pentesting can mean different things for different groups of people--from the prospective buyer to an existing customer, to the pentester delivering the test, to practitioners in our industry. There are several factors that make up a good pentest; in this series, I want to uncover which of these factors is important to whom and when.

First, it is important for us to consider quality as an overall measure; not just at each step of a process, but rather as an overall measure and a sum of many parts. While quality may be perceived or defined one way by one participant, it can look very different to others. For example, a pentester might feel they did a great job and uncovered worthwhile findings--meanwhile, a customer may not feel the same way if their environment was negatively impacted during the test. As such, there are multiple areas where quality can be explored and dissected.

Using pentesting as an example, we can discuss how quality impacts every step of what we call the pentesting lifecycle. This will also allow us to bring in different perspectives from the different people involved at each stage, and enable us to explore how each of these perspectives defines quality.

For reference, here is the pentesting lifecycle as we see it at Cobalt:

1. Preparation - This includes the sales cycle, scope agreement, and credential sharing; in short, everything that is needed to perform the test is gathered during this step.

   
   

2. Kickoff - This is where stakeholders and testers meet to align their expectations and approach.

   
   

3. Testing - This is the main activity for the penetration testers; it includes discovery, threat modeling, exploitation, post-exploitation - all of which we will touch on in more detail in future installments of this conversation."

   
   

4. Reporting - Here, we document and share our testing and findings with customers and/or stakeholders.

   
   

5. Re-testing - This is where the recommended remediation steps are implemented by our customer and we double-check their efforts to ensure the vulnerabilities are no longer present and the exploits are no longer viable.

   
   

6. Feedback - Here, we gather information about how the test went, how pentesters performed, how engaged the customer was, etc.

Each of these stages presents opportunities to align all teams involved, which will provide a better understanding of the needs, expectations, scope, and requirements that each group has. Gaining this close alignment gives us the ability to see how to attain quality in the overall process.

Every one of these stages has different stakeholders and teammates working on them. Each area is required to fully explore the scope of the engagement, and each is critical to the definition of and successful delivery of quality. With so many voices in play, early alignment and constant communication is not only important but critical to have a sense of quality.

In this series, I will be exploring the topic of quality across the pentesting lifecycle to better understand the drivers, hurdles, and expectations of everyone involved. This exploration will be over a number of mediums: from articles to webcasts, and even speaking engagements. I will chat with customers, colleagues, and practitioners on the front lines to include their varying perspectives into the narrative to help define what quality in pentesting means.