The Efforts Being Made Recently to Address the Shortcomings of Cybersecurity Testing

Table of Links

Abstract and 1 Introduction

2. Current Security Testing Platforms

2.1. Recent progress

3. A New Testing Platform and 3.1. Testing platform roles

3.2. Web-based remote access

3.3. Testbed setup

4. Enabled Testing Methodologies

4.1. Secure Development Lifecycle (SDL) testing and 4.2. Penetration testing

4.3. Research testing

5. Conclusion & Outlook, and References

2.1. Recent progress

Despite the challenges faced by the industry, we have seen considerable effort being made recently to address the shortcomings of cybersecurity testing. Table 1 compares our platform against several other modern testing platforms and highlights the key attributes that we offer and also explore in this paper.

Testing hardware. Electronics test and measurement equipment and software manufacturer, Keysight Technologies, launched the SA8710A Automotive Cybersecurity Test Platform [8] a few years back with hardware that connects to all relevant interfaces on a modern vehicle and built-in security scanners and fuzzing tools. Built with its product line of connectivity and measurement instruments, the Keysight platform offers hardware that is capable of capturing small variations and disturbances in communication signals. Last year, we saw the launch of the BG Networks CRATE [13], an all-in-one hardware box capable of connecting to most of the vehicle interfaces, with options to expand using USB. The CRATE™ also offers some software tools to facilitate remote access and fuzz testing. Other traditional hardware test interface providers do not offer specific hardware for security testing, but they offer software plug-ins to support limited security testing [18].

Testing frameworks. The Eclipse openDuT [4] framework focuses on automated and repeatable test and validation executions in the automotive space. This work is still in an incubating state and eventually aims to support a number of use cases, such as fully automated gray-box tests, tests across distributed test benches, and other functional tests. This framework provides end-to-end encryption of a private network between edge devices (ECUs, Restbus simulations, etc.) with an underlying control and registration logic. Ideally, network traffic from different devices under test can be linked together to create a more unified approach for testing. Additionally, CANlay offers virtual configurable networks where isolated ECUs can be bridged with sensor simulators for J1939 testing. A subset of ECUs here called overlays can be tested by carrying network traffic between these isolated ECUs.

The ideal solution. In contrast to prior work, we aim to build a platform that is easy to use by many users without deep hardware knowledge and preferably provide remote access to enable global teams to work on the same project at the same time. The platform should be capable of offering multiple test beds and, if required, the test bed should be configurable so that only a subset of ECUs can be operated on-demand. This hardware should be located in a central location with staff on site that specialize in hardware to support remote users that access the system. It should also be easy for a remote test engineer to launch their own custom security tests using remotely-programmable hardware and measurement tools.