Secure Development Lifecycle (SDL) testing and Penetration testing

Table of Links

Abstract and 1 Introduction

2. Current Security Testing Platforms

2.1. Recent progress

3. A New Testing Platform and 3.1. Testing platform roles

3.2. Web-based remote access

3.3. Testbed setup

4. Enabled Testing Methodologies

4.1. Secure Development Lifecycle (SDL) testing and 4.2. Penetration testing

4.3. Research testing

5. Conclusion & Outlook, and References

4. Enabled Testing Methodologies

4.1. Secure Development Lifecycle (SDL) testing

VSEC Test aims to integrate all methods of vehicle cybersecurity testing into a single easy-to-use platform. As discussed above, a portion of cybersecurity verification, as required by the ISO/SAE standard, is no different than software and systems verification required by other industry standards. The remote capabilities and test management features offered by VSEC Test allows test engineers to design and set up the test procedures one time in order to run on a schedule or triggered by other tools as a part of CI/CD frameworks. This centralized platform for test procedures and results can potentially be used to exchange data throughout the supply chain and ensure consistent verification test coverage for the development life cycle of a vehicle.

For example, we perform continuous cybersecurity functional testing on components with over-the-air (OTA) update functionality as a service. We set up the test rack and develop the test procedures once on VSEC Test, and it is performed on a periodic schedule and also whenever a new software update is applied. Continuous functional testing against cybersecurity requirements effectively identified vulnerabilities that were overlooked during the development cycle. In addition, VSEC Test provides a set of built-in tests as a preliminary scan for known weaknesses and vulnerabilities, which can be used directly against a target vehicle or ECU. This scan can allow users to easily and quickly determine if there are large gaps in measures protecting the interface or if there are known exploits that will work on the target vehicle or ECU.

4.2. Penetration testing

Pentesting approaches. Most pentest engagements start with an exploratory phase, where the tester aims to discover as much information about the target as possible. The duration and effort in this phase is largely determined by the amount of information available to the tester. VSEC Test allows users to build a pool of their own test scripts or use built-in discovery scans specifically targeting automotive networks, including protocols such as UDS and XCP. These tests can be queued up and ran in the background or over the weekend, saving time and allowing a team to focus on more technically challenging tasks or other projects. The web interface allows testers to access and check on test status at any time from anywhere to determine the next step of action.

Partner Pentesting. A major difficulty in accessing the talent pool of world-class vehicle penetration testers is physical location. The VSEC Test platform enables any remote engineer with credentials to connect to the bench and handle any physical interactions with the component and remotely script and execute tests from VSEC Test. We call this Partner Pentesting.

To evaluate the Partner Pentesting concept, we set up a scenario as depicted in Figure 2, where a remote wireless penetration testing specialist is working with a local test engineer to control a software-defined radio connected to the cloud and capture communication between the key fob and the car in addition to any vehicle network messages. The specialist then proceeds to study and decode the traffic, create scripts to attack the communication, and work together with the local test engineer to validate findings. In this case, without physical access to the vehicle and a local partner to perform tasks, it would have been very difficult to perform a meaningful penetration test against the vehicle’s wireless entry system.

Side-channel analysis. On the other hand, we have other benches in the lab can be tested with little to no user interactions. For these targets, fully remote networklevel penetration test with a test bench setup can achieve results on-par with on-site engagements. However, we found that more hands-on penetration testing procedures such as hardware analysis and side channel attacks are still very difficult to perform without having an onsite specialist. As a result, while acceptable for many scenarios, fully remote penetration testing remains very limited for certain configurations due to manual activation required for physical interfaces and required hardware and tooling interaction for certain procedures. However, we are able to make progress with a Partner Pentesting setup that provides assistance from onsite personnel and achieves comparable outcomes as a fully onsite engagement.