The recent news of the Verkada hack that breached 150,000 surveillance cameras sent shockwaves through the security world. For those of us in the edge computing industry, it simply underscored what we already knew — securing distributed devices is hard. As intelligent systems increasingly expand into the field, we’ll see more and more attacks of this sort if we continue to leverage the same security stance and tools that we’ve used for decades within perimetered locations like data centers and secure telecommunication facilities.
In this article, I will go through a few of the widely-cited edge breaches in the industry and highlight how a zero trust security model optimized for the unique challenges of the edge would have helped prevent them from happening.
Lessons Learned
In a hack reminiscent of Verkada, the 2016 Mirai virus infected millions of cameras and turned them into bots that together launched a massive DDOS attack on upstream networks, briefly taking down the internet in the northeastern United States. Something on the order of under twenty combinations of username and password got into all those cameras, because the developers made it too easy to change, or not even possible to change, these security credentials. Often this was due to prioritizing usability and instant gratification for users over security.
Another commonly-cited example is the massive Target data breach in 2014 that was a result of attackers accessing millions of customers’ credit card information by way of a networked HVAC system. The hackers stole credentials from an HVAC contractor and were able to access the payment system because the operations network the HVAC was on wasn’t properly segmented from the IT network.
In a final example, the 2010 Stuxnet breach involved malware that was loaded into process control systems by using a USB flash drive to bypass the network air gap. The worm then propagated across the internal process control network, scanning for Siemens S7 software on industrial PCs. When successful, the virus would send unexpected commands to PLCs controlling industrial processes while giving the operators a view of normal operation.
Viruses like Stuxnet that focus on compromising industrial systems are especially concerning because attacks can lead to immediate loss of production, or worse, life. This is compared to breaches of IT systems which typically play out over long periods of time, with compromises to privacy, financial data and IP.
Challenges of Securing Distributed Edge Computing
With these examples in mind, what is unique about the proverbial edge that makes security such a challenge?
- Scale: Part of the value of IoT and edge computing comes from having devices connected across the organization, providing a holistic view of operations. Over time we will see edge device deployments grow into the trillions, and traditional data center solutions for security and management are not designed for this kind of scale.
- Lack of Physical and Network Perimeters: Distributed edge computing resources rarely have the defense of four physical walls or a traditional network perimeter. This requires a security approach that assumes that these resources can be physically tampered with and doesn’t depend upon an owned network for defense.
- Heterogeneity: The edge is at the convergence of the physical and digital worlds. In addition to a highly heterogeneous landscape of technologies, we also have to account for diverse skill sets spanning IT and OT (e.g. network and security admins, DevOps, production, quality and maintenance engineers, data scientists).
- Varying Priorities: As OT and IT converge at the edge, each organization’s often conflicting priorities must be considered. While OT typically cares about uptime and safety, IT prioritizes data security, privacy and governance. Security solutions must balance these priorities in order to be successful.
- Constrained Devices and Legacy Systems: Many IoT devices are too resource-constrained to host security measures such as encryption, plus the broad install base of legacy systems in the field was never intended to be connected to broader networks, let alone the internet. Because of these limitations, these devices and systems need to rely on more capable compute nodes immediately upstream to serve as the first line of defense, providing functions such as root of trust and encryption.
EVE-OS: Zero Trust Foundation Built for Distributed Edge Computing
The Verkada hack and its predecessors make it clear that edge computing requires a zero trust architecture that addresses the unique security requirements of the edge. Zero trust begins with a basic tenant — trust no one and verify everything.
Project EVE started in 2019 with the Linux Foundation’s LF Edge organization with the goal of delivering an open-source, vendor-agnostic and standardized foundation for hosting distributed edge computing workloads. Project EVE is driven by EVE-OS, a lightweight, secure, open, universal and Linux-based distributed edge operating system with open, vendor-neutral APIs for remote lifecycle management. The solution can run on any hardware (e.g., x86, Arm, GPU) and leverages different hypervisors and container runtimes to ensure policy-based isolation between applications, host hardware, and networks. The Project EVE community is now over 60 unique developers.
EVE-OS Zero Trust Components
Let’s take a look at the individual components of the EVE-OS zero trust security framework.
- Hardware Root of Trust: EVE-OS leverages the cryptographic identity created in the factory or supply chain in the form of a private key generated in a hardware security model (e.g., TPM chip). This identity never leaves that chip and the root of trust is also used to store additional keys (e.g., for an application stack such as Azure IoT Edge). In turn, the public key is stored in the remote console.
- No Usernames and Passwords: An edge compute node running EVE-OS leverages its silicon-based trust anchor (e.g., TPM) for identity and communicates directly with the remote controller to verify itself. This eliminates having a username and password for each edge device in the field, instead, all access is governed through role-based access control (RBAC) in a centralized console. Hackers with physical access to an edge computing node have no way of logging into the device locally.
- Distributed Firewall: EVE-OS has granular, software-defined networking controls built in, enabling admins to govern traffic between applications, compute resources, and other network resources based on policy. The distributed firewall can be used to govern communication between applications on an edge node and on-prem and cloud systems, and detect any abnormal patterns in network traffic. As a bare metal solution, EVE-OS also provides admins with the ability to remotely block unused I/O ports on edge devices such as USB, Ethernet and serial. Combined with there being no local login credentials, this physical port blocking provides an effective measure against insider attacks leveraging USB sticks.
- Layered Security Model: All of these tools are implemented in a curated, layered fashion to establish defense in depth with considerations for people, process, and technology.
- Centralized Management: All features within EVE-OS are exposed through an open, vendor-neutral API that is accessed remotely through the user’s console of choice. Edge nodes block unsolicited inbound instruction, instead reaching out to their centralized management console at scheduled intervals and establishing a secure connection before implementing any updates.
Ounce of Prevention > Pound of Cure
Returning to the above examples of security breaches, what would the impact of these attacks have looked like if the systems were running on top of EVE-OS? In short, there would have been multiple opportunities for the breaches to be prevented, or at least discovered and mitigated immediately.
- No Physical Breach: In the Verkada and Mirai examples, the entry point would have had to be the camera operating system itself, running in isolation on top of top EVE-OS. However, this would not have been possible because EVE-OS itself has no direct login capabilities. The same benefit would have applied in the Target example, and in the case of Stuxnet, admins could have locked down USB ports on local industrial PCs to prevent a physical insider attack.
- Network Communications Intercepted: In all of these example attacks, the distributed firewall within EVE-OS would have limited the communications of applications, intercepting any attempts of compromised devices to communicate with any systems not explicitly allowed. Further, edge computing nodes running EVE-OS deployed immediately upstream of the target devices would have provided additional segmentation and protection.
- Activities Monitored: EVE-OS would have provided detailed logs of all of the hackers’ activities. It’s unlikely that the hackers would have realized that the operating system they breached was actually virtualized on top of EVE-OS.
- Automatic Quarantine: Security policies established within the controller and enforced locally by EVE-OS would have detected unusual behavior by each of these devices at the source and immediately cordoned them off from the rest of the network, preventing hackers from inflicting further damage.
- One-click Mitigation: Centralized management from any dashboard means that updates to applications and their operating systems (e.g. a camera OS) could have been deployed to an entire fleet of edge hardware powered by EVE-OS with a single click. Meanwhile, any hacked application operating system running above EVE-OS would be preserved for subsequent forensic analysis by the developer.
- Reputation Preserved: The zero-trust approach, comprehensive security policies, and immediate notifications would have drastically limited the scope and damage of each of these breaches, preserving the company’s brand in addition to mitigating direct effects of the attack.
We’re in this Together
The diverse technologies and expertise required to deploy and maintain edge computing solutions can make security especially daunting. The shared technology investment of developing EVE-OS through vendor-neutral open source collaboration is important because it provides a high level of transparency and creates an open anchor point around which to build an ecosystem of hardware, software and services experts. The open, vendor neutral API within EVE-OS prevents lock-in and enables anyone to build their own controller. In this regard, you can think of EVE-OS as the “Android of the Edge”.
Simplifying the Complicated
Finally, in the process of addressing all potential threat vectors, it’s important to not make security procedures so cumbersome that users try to bypass key protections, or refuse to use the connected solution at all. Security usability is especially critical in IoT and edge computing due to the highly diverse skill sets in the field. In one example, while developers of the many smart cameras that have been hacked in the field made it easy for users to bypass the password change for instant gratification, EVE-OS provides similar zero touch usability without security compromise by automating the creation of a silicon-based digital ID during onboarding.
The ideal solution is architected to streamline usability throughout the lifecycle of deploying and orchestrating distributed edge computing solutions so users have the confidence to connect and can focus on their business. While the attack surface for the massive 2020 SolarWinds hack was the centralized IT data center vs. the edge, it’s a clear example of the importance of having an open, transparent foundation that enables you to understand how a complex supply chain is accessing your network.
In Closing
Security at the distributed edge begins with a zero-trust foundation, a high degree of usability, and open collaboration. The industry is working together to take the guesswork out so customers can securely orchestrate their edge computing deployments with a choice of hardware, applications, and clouds, with limited IT knowledge required. The goal is to enable users to adopt distributed edge computing to drive new experiences and improve business outcomes, without taking on unnecessary risk.