Reasonable security through architecture As a web developer, I am often asked to build pages to collect basic users’ data. The first goal of every advertising campaign, for example, is to collect a large number of emails or phone numbers from people who could be interested in buying some service / product, in order to call them back, contact them, whatever. For many years, sadly but truly, nobody has really cared about how those data were stored in databases. They haven’t been considered “too sensitive” data, still their privacy should have been granted in some way. It was a grey area nobody has ever paid too much attention to. For many years Earth biggest companies haven’t even cared about encrypting , but this is another, possibly even worse, story. If you are interested in some speculation about password encryption and PHP, here’s . passwords of the users another article So guess those (and smaller ones) less sensitive information such as . how much attention companies have been paying to emails or phone numbers Today, after many leaks from famous and less famous companies, and after that kicked in here in Europe, some companies (most of them, to be honest) are starting to be aware of the risks related to storing a large amount of users’ data without any encryption. GDPR A typical , which I found in many old-school projects, is to collect names, mails and phone numbers and store them, , . bad scenario unencrypted in a database located on the same host of the web server This configuration exposes the project to many threats. Especially, if some attackers gain access to database, they can freely retrieve any data of all the users collected by the application. A would be to collect names, mails and phone numbers, them . better scenario encrypt and store in a database hosted on a server different than the web server The should be stored at least on the web server, . encryption keys never on the database The web server should be running a decent Web Application Firewall to keep it as safe as possible. The prototype that I am sharing with you stores the keys in a configuration file inside the public folder of your web server, to keep things simple and let almost everybody install and test the prototype in a super basic LAMP environment hosting. A safer approach would be to , for example. This goal can be reached with little effort for an average experienced developer, and it’s not part of this article. save the keys in a file outside your public folder if you can spend a little bit more on your project, but is to show how to , collecting data. Other, even safer, options are available this article’s goal provide a reasonable safe environment for your small low budget projects “not too sensitive” Obviously, if you are going to store credit cards, medical information, military intelligence and so on, you shouldn’t rely on this simple architecture. A basic working prototype made with PHP and MySql The main problem, when developing an application that encrypts data, is to check or search for those data, once they have been encrypted. We will follow the solution proposed by ParagonIE team in , trying to keep things as simple as possible. this article I will describe how to build a typical landing page that , but still allows us to with the same email or phone , before saving a new user. encrypts private data check if a user has already been stored Moreover, we will build a very basic admin panel (for testing purposes only), with a feature that allows us to in the database, . search for an email or a phone even if they are encrypted Before starting, I am assuming here that you are not using or any other PHP Framework for your small project, in order to keep this article as generic as possible. Laravel pretty that works just out of the box. You can find some information about it in its and in some interesting article . Laravel has a built-in encrypter/decrypter official documentation here This prototype relies on Sodium crypto libraries, which are included as PHP extensions in currently supported versions of PHP (7.1, 7.2, 7.3). To be sure, check your and look for this part. If you can see it, Sodium library is enabled on your server. phpinfo() You can download a on , and preview the . Do not use this prototype as it is in production environments. Use it freely to build your own encryption system and deepen your encryption knowledge. demo source code of my prototype github final result here Basic setup First of all we are going to build a simple html form that should look like this one below. We set all form fields as compulsory, using Bootstrap built-in validation. This form will post data to a script. And that’s it. save.php In folder you can find a , consisting of a single table named Here’s its structure. /_sql dump of the database “subscribers”. subscribers ( ( ) , first_name ( ) , last_name ( ) , email ( ) , phone ( ) , privacy ( ) , ( ) , ip ( ) , email_hash ( ) , phone_hash ( ) , creation_date datetime ) = =utf8; subscribers PRIMARY ( ), email_hash (email_hash), phone_hash (phone_hash); CREATE TABLE id int 11 NOT NULL varchar 255 DEFAULT NULL varchar 255 DEFAULT NULL varchar 255 DEFAULT NULL varchar 255 DEFAULT NULL char 1 DEFAULT NULL source varchar 255 DEFAULT NULL varchar 255 DEFAULT NULL varchar 255 DEFAULT NULL varchar 255 DEFAULT NULL DEFAULT NULL ENGINE InnoDB DEFAULT CHARSET ALTER TABLE ADD KEY id ADD KEY ADD KEY To save those keys as a text, we need to encode them using function, as below. base64_encode() The table fields named and will store the encrypted data, while and are that we will use later to implement search functions on encrypted data. These two fields will store blind indexes, which are basically a keyed hash (HMAC) of the plain text email and phone number. email phone email_hash phone_hash two indexed fields We can import the file to our MySql/MariaDB database. In root folder we can find a little script called . Its task is to generate two random keys for our project. We only need to run this script once, take note of the keys and get rid of the script from our server. generate_key.php The scripts uses function to create the that we will use to encrypt data, and a used to generate a unique hash (the blind index) for email and phone number, in order to make those data searchable in our application. The lengths passed to random_bytes() as parameters are two constants needed by LibSodium to work properly later on. random_bytes() private key blind index key $private_key = random_bytes(SODIUM_CRYPTO_SECRETBOX_KEYBYTES); $index_key = random_bytes(SODIUM_CRYPTO_PWHASH_SALTBYTES); <?php // Generating your encryption key // Generating your blind index key To save those keys as a text, we need to encode them using base64_encode() function, as below. base64_encode($private_key); base64_encode($index_key); <?php echo ?> <?php echo ?> Once we created our , we can write them down in our configuration file, located in folder. If you downloaded the source code from git, you can edit line 50 and 51 of and rename the file as . Don’t forget to data on line 20–23, too. keys for the project /includes config-sample.php config.php add your database connection $private_key = ; $index_key = ; <?php // generate values using /generate_key.php script "" // INSERT PRIVATE KEY HERE "" // INSERT BLIND INDEX KEY HERE We can now publish the project to our web server and check that everything is working, by trying to submit a new subscription to the system. We should see a new record in the database table, as well as on the backoffice demo page. The backoffice page shows the encrypted value actually stored in the database for email and phone number on mouse over. Moreover, if we try to search for an encrypted email among the subscribers, we can find it quickly: A deeper look at the code To understand how this prototype works, we need to have a deeper look at our Data Access Object, in /includes/ dao.php function gets a string as a parameter, in our case email and phone number, and it creates a blind index using our previously generated blind index key and . This will be used of encrypted values in our database. Keeping encrypted values searchable is an interesting matter, well covered by by , where you will also find more advanced solutions. getBlindIndex() sodium_crypto_pwhash() blind index to perform a search this article ParagonIE Here’s the snippet of the function that deals with the creation of blind indexes in our database, to later store the keyed hashes in and table fields email_hash phone_hash { $index_key = base64_decode( ->index_key); bin2hex( sodium_crypto_pwhash( , $string, $index_key, SODIUM_CRYPTO_PWHASH_OPSLIMIT_MODERATE, SODIUM_CRYPTO_PWHASH_MEMLIMIT_MODERATE ) ); } <?php public function getBlindIndex ($string) $this return 32 function writes a new record in subscribers table, , and to enable searches on them. Here’s where the encryption happens. The function retrieves the encryption key (l. 47 in ), creates two separate for encrypting email and phone number (l. 50–51), encrypts email and phone number using (l. 53–54) and encode them for safe saving to database (l. 56–57). Here’s a snippet of the funciton. insert() encrypting email and phone number creating two blind indexes dao.php nonces sodium_crypto_secretbox() { ($first_name. == || $last_name. == || $email. == || $phone. == || $privacy. == ) { ; } { $email_hash = ->getBlindIndex($email); $phone_hash = ->getBlindIndex($phone); $key = ->key; $key = base64_decode($key); $nonce_email = random_bytes(SODIUM_CRYPTO_SECRETBOX_NONCEBYTES); $nonce_phone = random_bytes(SODIUM_CRYPTO_SECRETBOX_NONCEBYTES); $email = sodium_crypto_secretbox($email, $nonce_email, $key); $phone = sodium_crypto_secretbox($phone, $nonce_phone, $key); $email = base64_encode($nonce_email . $email); $phone = base64_encode($nonce_phone . $phone); { $query = ->db->prepare( ); $query->bindParam( , $first_name); $query->bindParam( , $last_name); $query->bindParam( , $email); $query->bindParam( , $phone); $query->bindParam( , $privacy); $query->bindParam( , $email_hash); $query->bindParam( , $phone_hash); $query->bindParam( , date( )); $query->bindParam( , $source); $query->bindParam( , $ip); $query->execute(); $id = ->db->lastInsertId(); $id; } (PDOException $e) { ; } } } <?php function insert ($first_name,$last_name,$email,$phone,$privacy,$source,$ip) if '' '' '' '' '' '' '' '' '' '' return false else // create blind index for encrypted searchable fields $this $this // get key for encryption $this // get nonce for encryption // encrypt data // encode data for saving into db try $this 'INSERT INTO subscribers (first_name, last_name, email, phone, privacy, email_hash, phone_hash, creation_date, source, ip) VALUES (:first_name,:last_name,:email,:phone,:privacy,:email_hash,:phone_hash,:creation_date,:source,:ip)' ':first_name' ':last_name' ':email' ':phone' ':privacy' ':email_hash' ':phone_hash' ':creation_date' 'Y-m-d H:i:s' ':source' ':ip' $this return catch //echo "Error: " . $e->getMessage(); //exit(); return false ?> Our data access object in has other methods for listing the records in subscribers and count them, but I will focus on two more methods. dao.php The first one is , the function that we need to encrypted . Here’s a snippet dec() decrypt data { $decoded = base64_decode($string); $key = base64_decode( ->key); $nonce = mb_substr($decoded, , SODIUM_CRYPTO_SECRETBOX_NONCEBYTES, ); $ciphertext = mb_substr($decoded, SODIUM_CRYPTO_SECRETBOX_NONCEBYTES, , ); { $plaintext = sodium_crypto_secretbox_open($ciphertext, $nonce, $key); $plaintext; } (Error $ex) { ; } ( $ex) { ; } } <?php public function dec ($string) $this 0 '8bit' null '8bit' try return catch return "Cannot decrypt data." catch Exception return "Cannot decrypt data." ?> function gets an encrypted string as input, retrieve the encryption key, separate the nonce from the encrypted text and then decrypt it. dec() This function can be used while we loop our records from table, to decrypt data in our , (l. 125–126). Here’s a snippet that shows its (paginate your rows or your server will start sweating). subscribers backoffice demo page very basic usage <td> <span =" " - =" " =" :<? $ [' '];?>"> <? $ -> ($ [' ']);?> </ > </ > < > < =" " - =" " =" : <? $ [' '];?>"> <? $ -> ($ [' ']);?> </ > </ > class moreinfo data toggle tooltip title string stored in database php echo row email php echo dao dec row email span td td span class moreinfo data toggle tooltip title string stored in database php echo row phone php echo dao dec row phone span td Looking at script, we can notice that before saving a new subscriber to database, we save.php check if a user with the same email or phone has already subscribed $email_hash = $dao->getBlindIndex($email); ($dao->exists_email($email_hash)) { header( ); (); } $phone_hash = $dao->getBlindIndex($phone); ($dao->exists_phone($phone_hash)) { header( ); (); } $new_subscriber = $dao->insert($first_name,$last_name,$email,$phone,$privacy,$source,$ip); (!$new_subscriber) { header( ); (); } { header( .$new_subscriber); (); } // check if mail already exists if "location: index.php?e=email_exists" exit // check if phone already exists if "location: index.php?e=phone_exists" exit if "location: error.php" exit else "location: thankyou.php?id=" exit Function in check the existence of the same email by simply checking if the same blind index exists in email_hash field of the table. For better performance, we set an index on this email_hash field. Same logic occurs for function exists_mail() dao.php exists_phone() { (! ($email)) { ; } { $query = ->db->prepare( ); $data = ($email); $query->execute($data); $all_mails = $query->fetchAll(); ($all_mails $row) { ; } } (PDOException $e) { ->logger->writeDbError($e->getMessage(), ); } ; } { (! ($phone)) { ; (); } { $query = ->db->prepare( ); $data = ($phone); $query->execute($data); $all_numbers = $query->fetchAll(); ($all_numbers $row) { ; } } (PDOException $e) { ->logger->writeDbError($e->getMessage(), ); } ; } function exists_email ($email) if isset return true try $this "SELECT id FROM subscribers WHERE email_hash=?" Array foreach as return true catch $this $this return false function exists_phone ($phone) if isset return false exit try $this "SELECT id FROM subscribers WHERE phone_hash=?" Array foreach as return true catch $this $this return false Once this function is clear, it is pretty straightforward to implement a in our demo backoffice page. See function in for implementation details. Obviously, the search will give you results only if the full email or full phone number is inputed, while are allowed. basic search by email or phone number listItems() dao.php no partial searches Final thoughts and warnings This article aims at showing how little effort is needed to meaningfully strengthen the privacy of your data in small / medium sized projects. The prototype described here must be taken as an opportunity to deepen our knowledge of encryption techniques and make it accessible to unexperienced developers and teams. As stated before, do not use this prototype in production environment, but play with it as much as you want.
