Sensitive data is spread across all areas of your business. Learn how to discover where it is and bring your data practices under control. Before you start Before scanning for personal data in your systems, you need to understand the technical aspects of your system components, the types of files you need to scan, and which services you have direct control over. Aside from scanning and detecting data, you also need to be ready to interpret what you find and understand its importance. Types of services The storage of data systems will be split across the different services you use. The types of services your company uses come in two kinds: ones you can control (controlled) and ones you can’t (uncontrolled). Let’s take a look at them in more detail. Controlled There are two types of services that you are able to control: services that don’t store data, but just process it — ​​they are for functionality. Stateless services that store and process data — they are for storageStateless Stateful Stateless Stateless is the most popular type of service. This is because you want to split your data storage and data processing. This will allow you to scale because stateless services just need access to data to function, meaning you can just link them up to the databases without creating additional storage. These types of services process the most amount of data. When dealing with stateless services you need to be able to analyze incoming and outgoing data on the fly. With these services, you can only scan the traffic, but by doing so there are limitations. For example, if you use a data loss prevention (DLP) service like the one provided by Google (stateless because it doesn’t store any data) and want to , you will need to create your own additional services (e.g. proxy) for traffic management with the DLP. discover personal data in the cloud It’s also worth noting that these DLPs are also third-party services. If the DLP provides you with reports on specific users (which most likely is the case), you will have to state this usage in your privacy policy. Stateful Stateful services are databases and include the likes of Postgres, MySQL, and sometimes cloud storage too. For stateful services, you also need to be able to analyze the data stored as it flows in and out. We can use traffic analysis here and it will work just as does for stateless, but if you want to analyze historical data: you will need to write a script or use a special function to scan (on top of your traffic analysis) and how this function works and is written will depend on the type of storage you use. Moreover, in this instance, you are just carrying out traffic analysis, and this doesn’t tell you anything about what data is there; just about what changes have been made to it. Some types of storage make analyzing traffic easy but others are more difficult. Different storages have different interfaces and some are made for ease of use, but not all of them. For example, in MySQL, you need to create your own data structure - meaning that every time you add new storage you will need to create a non-generic script to analyze it. Uncontrolled On top of controllable services, there are also uncontrollable services, also known as third-party services. Third-party services are the external SaaS tools you purchase. Examples include Salesforce, Hubspot, Zendesk, etc. You cannot control these services. For example, if you employ Zendesk and a user wants to ask a question, they write in a special form for Zendesk, and then you will check this support issue from Zendesk’s interface. You cannot control how Zendesk stores or processes this data. When sharing data with third-party services (which is what you do when you use them), you generally can’t scan for personal data because the interface won’t always provide the functionality to scan for personal data. This means you need to be sure about the safeguards that they have in place to protect data. So now we know what we need to be scanning, let's take a look at how it is done. How to scan for personal data First of all, we need to bear in mind that there are requests (get requests) that don’t contain personal data (because get requests have no body). Then there are responses that could contain personal data and post requests that could contain personal data as a request and as a response. The code below is an example of how you can analyze personal data in your stateless services: http.server BaseHTTPRequestHandler, HTTPServer = get_response()
  	pii_analyzer(response)
  	self.wfile.write(response.encode( ))
  
  def do_POST(self):
  	content_length = int(self.headers[ ])
  	request = self.rfile.read(content_length)
  	pii_analyzer(request.decode( ))
  	self._set_response()
  	response = get_response()
  	pii_analyzer(response)
  	self.wfile.write(response.encode( ))
  
  def pii_analyzer(data):
  	pass
  	# post data to analyzer service and get result
  	# publish result to database
  
  def get_response(): server_address = ( , )
httpd = HTTPServer(server_address, handler)
httpd.serve_forever() #!/usr/bin/env python3 from import ( ): ( ): . (200) . () ( ): . () class handler BaseHTTPRequestHandler def _set_response self self send_response self end_headers def do_GET self self _set_response response 'utf-8' 'Content-Length' 'utf-8' 'utf-8' return "response with pii data" '' 8000 In this example, we analyze personal data without middleware. Middleware is a specific pattern in an HTTP server that can be used to analyze the responses irrespective of whether you are sending responses or receiving requests. With middleware, you can analyze the responses/requests after all handlers. ​​However, this approach isn’t great because if you want to create new stateless services you will need to write new code for data analysis and this new code could affect the throughput of the service latency and even CPU usage because it is a different load for the server. Basically, it’s better to use a traffic analyzer for the network layer; not scan data on your application layer. Traffic analysis Analyzing traffic is the best way to understand what is going on in your systems. Traffic analysis allows you to see what data is coming in and out, instead of just seeing what is stored. However, traffic analysis also has its issues: you need to push the traffic through an additional analyzer or monitoring tool. To keep your analysis and your systems working well, you need to select an instrument that: Doesn’t pass data on to third parties Is easy to set up without involving programmers Doesn’t have a critical effect on service latency Analyzes all the data passing through systems from one place Keeps usage charges manageable/predictable One of the most important points above is the latency factor. Your business will have a maximum latency factor and you will probably have some functionality that requires execution as fast as possible. This means that things should be executed in milliseconds and you just won’t be able to place a traffic analyzer as an extra hurdle. That’s not to say there is no way around this though. You could just build extra infrastructure to accommodate the analyzer, but that is easier said than done and will involve resources that you just may not have. On that note, it is worth mentioning that your infrastructure is changing all the time anyway as you scale or add new functionality to your product. In this case, you will need to increase the size of the stateless structures so that they can cope with increased traffic. Otherwise: if you are scaling up and you need your traffic to pass through an analyzer, you will find that the traffic will pile up at this bottleneck, slowing your system down. Your traffic can be pretty unpredictable. Finding yourself in the news, a new marketing campaign you weren’t told about, or an explosion of users for whatever reason will lead to a spike in traffic. You need to be able to deal with this traffic; not have something throttling it. The other thing is that the end price that you sell your product to users depends on your baseline costs. If they go up because you are having to implement a proxy, which means you have introduced some complex architecture or other: either you will have to put your prices up or you will have to tolerate a drop in profits. Plus, if you are using tools that process data at variable costs depending on the data volume (like DLP), costs may run out of control. Storage analysis There are 2 approaches to storage analysis: Analyze new data Analyze historical data For new data, we can use a traffic analyzer. Historical data refers to data that is already in your database and to scan this you will need to create scripts for data analysis. Services like databases, relational and nonrelational databases, and other storage usually offer you an interface through which you can carry out a full scan of the storage, analyze new files, and update files. However, each database has its own approach to scanning, analyzing, and updating, meaning you often have to create custom logic to carry out actions with the storage. This type of scanning can also have a negative effect on the cost and throughput of your databases. Moreover, the analysis only really makes sense when you have a specific task at hand, for example, scanning for and analyzing . personal data Conclusions Traffic analysis is the best way to because you can create a process for analyzing personal data without having to do it by hand. Moreover, you don’t need to get developers involved to write scripts so that you can understand where the data actually is. scan personal data However, like everything: it depends on your needs. If you need to see what data you store (historical) and not just hot data, you will need to employ a scanning tool. Both techniques allow for automation, meaning you remove the human factor and don’t need to educate your engineers on what is or isn't personal data so that they can write the programs to search for it.

How to Scan Your Systems for Personal Data

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Windows Sticky Keys Exploit: The War Veteran That Never Dies

06/02/2018: Biggest Stories in the Cryptosphere

0-Days are on the Rise and that Means a Lot More Work for SOC Teams

Time Bombs Inside Software: 0-Day Log4Shell is Just the Tip of The Iceberg

The Noonification: How the Liver Transplant System Changed Forever (11/25/2023)

The Noonification: Panda Power (11/28/2023)

Windows Sticky Keys Exploit: The War Veteran That Never Dies

06/02/2018: Biggest Stories in the Cryptosphere

0-Days are on the Rise and that Means a Lot More Work for SOC Teams

Time Bombs Inside Software: 0-Day Log4Shell is Just the Tip of The Iceberg

The Noonification: How the Liver Transplant System Changed Forever (11/25/2023)

The Noonification: Panda Power (11/28/2023)

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps