Testbed setup

Table of Links

Abstract and 1 Introduction

2. Current Security Testing Platforms

2.1. Recent progress

3. A New Testing Platform and 3.1. Testing platform roles

3.2. Web-based remote access

3.3. Testbed setup

4. Enabled Testing Methodologies

4.1. Secure Development Lifecycle (SDL) testing and 4.2. Penetration testing

4.3. Research testing

5. Conclusion & Outlook, and References

3.3. Testbed setup

Sourcing ECUs. In the context of this work, we define ECUs as production ECUs from the field (i.e., real vehicles), engineering samples (e.g., A-samples, Bsamples), or even off-the-shelf automotive-grade microcontrollers (MCUs). Depending on the user, obtaining ECUs for the testing platform will require different approaches. Where an engineer and pentester would get hardware from their own supply chain or from the contracted customer, respectively, a researcher would likely obtain ECUs from a real vehicle from either a salvage yard or buying ECUs directly from OEM parts shops. Testing a single ECU can be sufficient if the tests are focused on purely CAN physical layer specifics or other singleECU session-based protocols, such as Unified Diagnostic Services (UDS) protocol and Universal Measurement and Calibration Protocol (XCP). In some cases, multiple ECUs are needed to wake up certain ECUs (e.g., a powertrain ECU may require a body ECU and vice versa), although we have reverse-engineered some of these wake-up messages. For our platform, we source ECUs across three different OEMs from a local salvage yard and from a real vehicle: five powertrain ECUs (two pairs of duplicates from different model years), a gateway ECU, a power steering ECU, and two instrument panel clusters (IPCs). We also acquire a C-sample gateway ECU from one of the aforementioned OEMs, which matches the OEM of one of the IPCs. Additionally, we also obtain a CANenabled automotive-grade MCU with full programming capabilities.

Configurable network. Most users will often work on projects that require ECUs with different bus speeds, different OEMs, and different attached hardware (e.g., measurement tools, actuators, etc.) As such, it is necessary for a testing platform to provide the ability to configure the platform remotely. The alternative here is having access to multiple real vehicles or (re)building separate test beds, which is expensive and harder to manage. We build a prototype of a configurable CAN bus network system that supports multiple bus speeds, multiple ECUs from multiple OEMs, and the ability to switch on/off the power of each ECU on-demand via VSEC Test.

In Figure 1, we depict an example of of a single platform that: (1) hosts multiple buses with different bus speeds, (2) multiple OEM platforms on one bench, and (3) the ability to control the network topology. The figure demonstrates what the entire bench looks like with ECUs from three different OEMs as well as how our testing platform would bring up a network consisting of just ECUs from OEM #1 and OEM #2. Using a relay board that powers on/off each ECU on command, a tester could also isolate a single ECU or subset of ECUs from the entire set. The ability to slowly ramp up the number of ECUs under test is useful when developing a technique (e.g., a new attack), and the ability to test a technique against multiple OEM implementations can determine the applicability of the technique. For our platform, we construct a configurable network by attaching the power supply of each ECU to a 16-port relay board,[1] which is controlled by an Arduino Due MCU. A script on the tester PC communicates with this MCU over Serial, and the code on the Arduino toggles the user-specified ECU to activate/deactivate. In this work, we focus on CAN buses, and adding a new bus protocol such as Automotive Ethernet or CAN-FD would be similar.

Network simulation. Real CAN traffic from road tests is often only available after a vehicle is in production. This traffic is usually the only information a researcher can access to simulate the bus network. Reverse-engineering CAN traffic can successfully uncover which CAN messages control which vehicle features/sensors [14], [20]. Engineers and some pentesters (depending on the contract) have access to proprietary CAN database (DBC) files, which identify the producer/consumer of each signal for all CAN messages. An alternative source of information for engineers and pentesters are Restbus simulations, which provides sufficient CAN traffic for the ECU under test to function as expected. For our platform, we investigated another method of obtaining a larger network of ECUs by combining the traffic from multiple ECUs that belong to the same make and model but are physically-distributed. We implemented a test bed with an IPC located in Pennsylvania and remotely connected to a powertrain ECU hosted in Michigan. While there is some latency involved with forwarding CAN traffic between these two locations, the latency is small enough to enable access to most UDS commands and exhibit control of the IPC without any obvious delays.

Environment simulation. While reverse-engineering and proprietary DBC files can enable users to interpret CAN signals, it is still challenging to mimic traffic from road tests without spending hours driving a vehicle. Users can utilize driving simulators, such as CARLA [15], to provide hours and miles of

Figure 1. The entire physical network layout (left) can contain ECUs from a number of different original equipment manufacturers (OEMs). These ECUs may connect to CAN buses with different speeds, which ultimately connect to a PC via a CAN-USB interface. By powering off all other ECUs except those from OEM #1, we can construct a network consisting of ECUs from just OEM #1 (middle). Likewise, we can construct a network of ECUs from just OEM #2 (right) by powering off all other ECUs.

simulated road traffic. Combining these simulators with either a reverse-engineered or original DBC file can generate realistic CAN bus traffic to the ECUs under test. Engineers often have access to advanced sensor and actuator models that can significantly enhance the realism of the CAN bus traffic. While these models are typically deployed in software-only simulations, they could be connected to our testing platform to capture results only observed on real hardware. For our testing platform, we utilize CARLA in combination with a DBC file to provide realistic speed values to an IPC and then deploy an intrusion detection system (IDS) from prior work [5] as if it were in a real car.

Measurement tools. We expand the capabilities of the platform by connecting measurement tools, such as oscilloscopes and logic analyzers, and offering software control of these tools. As these tools are often expensive, a platform with centralized hardware can reduce overall hardware costs by permitting shared access to these tools. For our platform, we connect a Picoscope oscilloscope to the power pins of an ECU and a Saleae logic analyzer to all of our CAN buses. Via command-line scripts, the Picoscope can capture power profiles as input to the aforementioned IDS. Additionally, the logic analyzer can observe bit-level CAN bus traffic for the purpose of building advanced CAN bus attacks [9]–[11].

[1] The ignition pin can be powered by a separate relay if needed, thus having up to 16 ports can help control both main power and ignition while maximizing the number of ECUs under test.