You can use on your servers. Important: Lightrun for free I’m not a security expert. I’d like to think of myself as a security-conscious developer, but this is a vast subject with depth and breadth. What I understand is . In that capacity, I can show some creative ways you can use it as a security tool. A “proper” security expert could take this to the next level. Lightrun and Debugging What is Lightrun? Lightrun is a developer-oriented observability tool. Like a debugger in your without the security risks. Lightrun is a tool that’s flexible enough to fit into multiple molds, just like the debuggers that birthed it. production environment With Lightrun, you can inject logs without code changes. Add snapshots (breakpoints that don’t stop the code execution) and use metrics to get observable insight at the code level. Security Tool Use Cases There are several reasons I would reach for Lightrun as a security tool. Here I’ll focus on: Verifying that a security vulnerability exists Checking if someone actively exploited a security vulnerability Verifying that we deployed a fix correctly There’s a lot more that needs to be done in order to secure your application. Lightrun is a generic tool, it isn’t a replacement for existing security tools like Snyk, etc. It’s complimentary, filling in the gaps at the code level. Finally, I will discuss how Lightrun secures itself. We can’t have a vulnerable security tool… We can’t consider Lightrun as a security tool if it isn’t inherently secure… Enough with the high-level theory. Let’s show the code! Verify a Security Vulnerability Security tools are like . They provide high-level alerts of potential risks. But they rarely communicate at the code level. As a result, a developer might have a hard time with actionable security tasks and validation. If the security issue reproduces locally, that’s great. You can often fill in the gap with a debugger. observability tools But some security issues are tough to reproduce outside of a production environment. Lightrun won’t find a vulnerability out of thin air, for that you need a dedicated security tool. However, if you have a suspicion, Lightrun can help in the investigation and prove the vulnerability. E.g. let’s take this obvious bug: This is an obvious SQL injection bug. But is it exploitable? Do we need to be hysterical, or can we take our time adapting the code? BTW, notice I’m using because that’s the platform I’m most comfortable with. This applies to all Lightrun supported platforms/languages equally. So everything here easily applies to , JavaScript/TypeScript, , , Scala, etc. Java NodeJS Python Kotlin This is trivial to test in Lightrun. We can just add a log or a snapshot that will be triggered when an invalid request happens. Then we can try sending invalid values via a curl command to see if our log is triggered. Notice that we use a regular expression to validate the name value. If we receive a log, it means the problematic value is exploitable. This also means the risk of the security vulnerability is high. Is it Actively Exploited? So we found a security vulnerability like the one above. Should we panic? Are there hackers already in the system? What do we do? Well, we can do something similar to what we did above and add a snapshot with a similar condition and a few “tune ups”: This image contains a lot, so let’s try to unpack it. Why Snapshot and not Log? Logs are great to see if something happened. They’re quick and they handle high volume well. But if someone is actively breaking into our system, we want to get all the information that’s available. Possibly even things we haven’t thought about. We want to know the vector of attack, which means knowing the call stack etc. Snapshots are an ideal security tool. Targeting a Tag Notice that the “Agent” entry points at “Production”. We can apply the snapshot to a group of machines based on tagging. So in this case, we can target all potentially vulnerable machines with one swoop. Max Hit Count Unlike a log, snapshots fill up the UI and storage. So we have a default limit of snapshots we can take before expiring the snapshot. This defaults to one normally. Here I raised it to 20 but we can probably go even higher if we’re willing. Notice that if we see this happen and exploits are happening, we might want to switch to logs since they don’t have a hit count. Ignore Quota This option might not be available to you since it requires special permissions. If you’re in this situation, ask your manager for this permission. This is a risky feature, which is why it’s guarded. But with an exploitable hack, it might be a risk worth taking. The quota limits the amount of CPU a condition, or expression can take per Lightrun action. The risk here is that an exploit might happen and some information would be “dropped” because of CPU usage. This will mean the snapshot won’t be paused at any point and we won’t “miss” a potential exploit. This might affect your server performance though, so it isn’t without risk. Expiry Lightrun actions default for one hour of expiry. We want to keep your servers fast and nimble so we expire actions that aren’t needed. In this case, we want the action there until we get the fix out. So I set the expiry value to 60 hours. With these in place, we will get actionable information on any exploit coming our way. Verify the Fix Verifying the fix is pretty similar. We can place a log or a snapshot in the problematic area of the code and see if that code is reached with problematic values. You can also add additional logging to verify that attempted exploits reach the area they’re expected to reach and are handled as you would expect. Lightrun Security A security tool that’s vulnerable defeats its purpose. So understanding the security measures in Lightrun is an important part of this post. Following are the high level features in Lightrun that make it so secure. Architecture Lightrun made several architectural decisions that significantly reduced attack vectors. Agents only connect to the Lightrun server to fetch actions. Not the other way around. That means they are hidden completely from end users and even from the organization. If the Lightrun server fails, an agent just does nothing. This means that even a DDoS attack that would bring down Lightrun won’t affect your servers. You won’t be able to use Lightrun, but the servers will work just fine. Certificate pinning & OIDC Agents and clients of the Lightrun server use certificate pinning to prevent elaborate man-in-the-middle attacks. Lightrun uses for secure proven authorization across its tools. OpenID Connect (OIDC) The Lightrun server limits user privileges based on assigned roles. Most importantly, every operation is written to an administration log. This means that a “bad actor” can’t be abusive without leaving a footprint. Sandbox All operations within an agent are sandboxed and have limited access. All actions are “read only” and can’t use too much CPU as we saw in the article above. There are exceptions to these rules, but they need higher privileges to circumvent. Block List A malicious developer in the organization can use a snapshot or a log to get information from a running application. E.g. a snapshot can be placed in the authorization logic to steal user data before encoding. A blocklist can define files that are blocked inside Lightrun agents. These files won’t let a developer place an action within them. PII Reduction Personal Identifiable Information, such as credit card numbers, can be logged intentionally or unintentionally. PII reduction lets us define patterns that are risky and those will be implicitly erased from the logs. As a result, you won’t need to purge such logs and won’t expose yourself to potential regulatory liability. TL;DR We did not design Lightrun as a security tool. It shouldn’t replace existing security tools. But it’s to the tools you already have. It plays to their strengths and pushes the envelope of fast response to vulnerabilities/hacks. a perfect sidekick Lightrun’s low-level deep code observability lets us respond to potential threats faster and mitigate vulnerabilities sooner. I’m not a security expert. I’m sure that if you are, you can come up with even more amazing security-related use cases for Lightrun. This is very exciting, I can’t wait to hear about them! Also Published Here

Fetch

Target

The Basics of Breakpoints you Might not Know

Exploring an Open Source Toolkit for Debugging Kubernetes - KoolKits

Debug Agent

Buy My Book

Learn on my YouTube Channel

Read My Stories

Newsletter

Follow me on Twitter

Too Long; Didn't Read

Make resilience your competitive advantage

Detecting, Investigating, Verifying Fixes for Security Incidents, Zero Day Issues Using Lightrun

Detecting, Investigating, Verifying Fixes for Security Incidents, Zero Day Issues Using Lightrun

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Deep Insight on Memory Debugging

108 Stories To Learn About Cybersecurity Tips

10 Ways to Mitigate Cybersecurity Risks and Prevent Data Theft

10 Tips For Securing Zoom Meetings

10 Things I Did To Increase CloudTrail Logs Security

A Deep Insight on Memory Debugging

108 Stories To Learn About Cybersecurity Tips

10 Ways to Mitigate Cybersecurity Risks and Prevent Data Theft

10 Tips For Securing Zoom Meetings

10 Things I Did To Increase CloudTrail Logs Security

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps