Autonomous Vehicles and the End of Privacy

Everything, everywhere, will soon be continuously recorded and uploaded to the internet. This will start with dense, urban areas, but over time every single square meter of every part of the globe will be recorded. Advances in computer vision & AI mean this data will be usable at scale, which will revolutionise advertising, law enforcement, and bring us back to a pre-privacy world. (and yes, you guessed it, the movie to pair this post with is Minority Report) Photo Courtesy of Tesla Video Everywhere Here is where we are today: record videos of anywhere on earth. This is what enables for instance (a Google subsidiary) to sell HD video feeds of any place on earth for commercial applications. In most cases the resolution is still quite coarse — for instance you can monitor whether a truck is on a building site from space, but not see its license plate. Satellites & Drones Terra Bella have proliferated as sensor costs have dropped and surveillance requirements increase both at the national security level and the individual level (home security, babycams, etc). There are more than 6 Million CCTVs in the UK ( ). CCTVs & Home Camera systems source recordings have increased ridiculously. For an order of magnitude: more than 4 Hours of video are uploaded to Youtube each day ( ). This isn’t slowing down as the 50% of the planet who do not yet have smartphones get access to them, and products such as Facebook Live, Twitter’s Periscope, and Snapchat drive current smartphone users to take more (now ‘live’) video. Consumer Video Million source But the real step change is around the corner: will become mass market, and will need to record a constant video feed of what the user is seeing to be able to augment the view with software.. Today Microsoft Hololens and Snapchat Spectacles are leading, and have very small penetration. But many smart, well funded people are working on making AR work, spearheaded by $5bn in funding and 700+ employee stealth startup . Augmented Reality “googles” Magic Leap Photo courtesy of Snap Inc This is huge when you consider the scale of this AV revolution as a , but even more mind-blowing when you consider the role AVs in an even bigger market — logistics. As the market demands increasingly “on-demand” goods ( ) AVs (including drone) will be the cost effective path for companies to satiate this need. If driving the vehicle is free and powering it becomes practically free (with the advances in electric engines efficiency and Solar Power) — why not get 10 shirts shipped to your door for you to try on and return the 9 you don’t like? (For more on this: my friend Alex Flamant gave on this topic at the London Futurists meetup.) Autonomous Vehicles (AVs) will become mass market and will capture continuous, high definition, high frame rate, 360° video. replacement for all cars today Amazon now offer 1 hour deliveries a prescient talk Source: Google Self-Driving Car Project Monthly Report As these technological evolutions march on, we are getting closer to continuous HD video recording of every square meter of every part of the globe. Now. As awesome as some videos are ( ), no one is going to sit through and watch all this stuff… here’s a funny one From Video to Insight Fuelled by this explosion of video data, there have been incredible advances in Computer Vision over the last 20 years. Let’s start with something you may not be aware of: Very high definition video feeds can now be processed by algorithms to reconstruct the sound as it was in the area filmed. This works by being able to recognise tiny vibration motions caused by sound on surrounding objects (some of the same technology that enables us to get someone’s heart-rate from a video feed). In a striking example in , . The recovery is good enough to understand what people were saying, While there is no reason for AVs, CCTVs, etc to record sound, these techniques may help get better sound definition, especially at long distances and in noisy places… Videos as Microphones: silent this video researchers are able to recover a conversation by filming a packet of crisps from behind sound-proof glass and there was no sound recorded at all! not Hearing through a pack of crisps. The Visual Microphone: Passive Recovery of Sound from Video : Your clumsy Google Now or Siri may betray this today, but in the research lab we’re there in terms of reliably going from sound to text. As with Facial Recognition, the explosion of data on which to train NLP algorithms (from conversational interfaces in smartphones, bots, and Amazon Alexa/Google Home speakers) has spurred this innovation. In 2016 Microsoft beat human benchmarks for speech-recognition. Shortly after, Google showed off new ways to do live translation for 8 language pairs, showing that the innovation is perfectly comfortable whizzing past what (most) humans can do! Natural Language Processing Finding a Voice — The Economist can be considered solved. In June 2015, Yann LeCun, head of artificial intelligence at Facebook, that on a set of 40 000 from Flickr, Facebook’s algorithms were able to correctly identify faces in 83% of cases. Interestingly people’s faces didn’t have to always be in the picture — with enough data about a user the algorithm could use other cues such as clothing to identify them (“Mark seems to wear a lot of grey t-shirts…”) Photo-based recognition is continuously improved with the explosion of photos and the continuous feedback Facebook and Google get as users tag their friends in pictures. With video, and a of each user’s habits, clothing style, and locations, . (remember the last time you struggled to recognise someone you hadn’t seen in a few years?). They might also take visual cues better than you: Facial Recognition announced pictures history the algorithms will easily surpass people in recognising who is who Extracting the needle from the haystack AI systems are capable of inferring higher level concepts (‘features’) from raw data. For example, faces are a feature that is inferred from raw pixels forming the video feed. This effectively makes all video, and all sound, searchable. Clarify.io as an example of a company providing a service to do this: Clarify.io (seems to work — the top result was an Elon Musk video!) But AI goes much further than enabling humans to search. It can effectively perform all possible searches for you and return higher level ‘features’: interesting patterns, anomalous events. It also does so across as many data sources as are available, joining the data sets on common identifiers across them. ( , all browsing activity, all messaging, all social network opinions, etc) So it’s not “there is a face in this video”, it’s also the name behind the face, the internet ‘breadcrumb’ of this person including all other locations where this face was seen ever and any other data set that might be available. These AI systems will continue to improve, and . Say the UK police runs an algorithm through all public CCTV recordings from the last week, looking for a criminal’s face. Today’s algorithm may not find it, but a future version will. each improved version will reprocess the raw data So, what are the implications of these advances? Commercial Implications The aim remains to build a detailed profile of you to sell you more stuff and quantify the risk/liability you represent. Advertisers, insurers, retailers, banks, will feast on this new data like they have d0ne with all other internet data so far. Ubiquitous video combined with ever improving AI systems provide: where you live, where you work and how much, whether you go drop off kids at school, who you hang out with, where, how frequently Your socioeconomic status: where you shop, what you wear, what your friends wear, what you have to wear at work Your shopping habits: : how active you are, whether you smoke/drink, which kinds of restaurants you visit or order from, how many doctor visits you have been to Your health truly 121 advertising with a side of leather jackets and guns (Minority report) It also brings up some interesting forecasting opportunities at a more macro level: What happens to the fashion industry when half a dozen static $100 cameras can tell you everything that anyone in Shoreditch wore this year — when you can trace a trend through social and street photography from start to the mass-market, and then look for the next emerging patterns? — Benedict Evans, Cameras, ecommerce and machine learning It seems inevitable that these video-sourced insights will be resold by the AV companies, in the same way that Google and Facebook sell targeting based on browsing history and likes. The commercial opportunity is huge. Whether this happens officially will depend on the nature of the regulation surrounding the use of this video data. Unofficially, given febrility of security on the internet, this data will get used for commercial means. Privacy Implications “The Right To Privacy” ~ December 15, 1890, Harvard Law Review As brilliantly lays out in , Gregory Ferenstein The Birth And Death Of Privacy full transparency is humanity’s natural state. Privacy as a concept is only about 150 years old. For a long time, it was not materially possible to have any form of privacy as we’d think about it today. Tribal communities all lived together in caves, Romans all lived in one room houses, medieval families and servants all slept in the same bed — there was no practical concept of private space or individual intimacy. As private life became possible — physically with separate houses, rooms, beds, and through communication with new technologies like writing and the telephone — the desire for privacy emerged. The first privacy-orientated law in the US was the 1710 Post Office Act, which banned sorting through the mail by postal employees. Nonetheless, privacy has always remained a secondary concern to convenience and cost. This explains the consistent, broad adoption of new technologies which encroach on our privacy but are deemed worth it. Today, Autonomous Vehicles is one of these new technologies to evaluate. Isn’t it crazy that today we let people drive a ton of steel at 100s of miles per hour, just with a few signs telling them not to press on a pedal too much? It is — and it causes 30 000 deaths per year in the US (that’s one every 17 minutes). With autonomous vehicles, this number will significantly decrease, which by itself makes this technology very attractive. But what are the privacy implications? This time, you can’t opt out The smartphone & internet revolution has also been a profound change, but you could choose not to have a smartphone or not to use Facebook, Google, or any other services that make a living collecting and selling your data. In the Video Everywhere world, tracking moves fully offline. The moment you step outside, all your movements are streamed to a database, timestamped, geolocated and added to your digital profile. You can’t opt out. (To delay this day as much as possible, you could (1) make sure your face is not on the internet, (2) live in a place that will have AVs last, e.g North Korea, (3) live in a cloudy place such that drones and satellites struggle to see you from far. On the plus side, I bet your rent will be cheaper there) 100% Enforceability As we all know, the USA-pioneered surveillance-without-warrant state is in vogue at the moment, for example with the recent , or the . UK ‘Snooper’s charter’ bill becoming law emergency state being preserved in France Video Everywhere means potentially one more source data from which to find anomalous citizen behaviour. This will be used by intelligence agencies for defence purposes, but it directly affect you and me in the hands of your average prosecutor. If the prosecutor can codify all infractions in a video processing algorithm — say jaywalking (to take a particularly ridiculous law) — and run it on all video ever taken of public life in the US, it could then immediately charge all jaywalkers. Law becomes 100% enforceable. I think the most important idea is to remember that there have been times throughout American history where what is right is not the same as what is legal — Edward Snowden Another factor to consider is the that results from more data collection, particularly video data. As tracking moves to the offline, physical world, so does the targeting — you could for instance imagine a drone being hijacked to go and hurt someone who was identified by hacking into an AV’s video feed. increased breadth and effectiveness of cybercrime 😬. What now? Today, the public debate about Autonomous Vehicles and AI focuses on the most pressing issue: the economic and societal impact of automating millions of jobs (such as truck driver which is the most popular job in America.) This is what the electorate is most worried about. However the privacy implications of Autonomous Vehicles also need a place in this public debate. Autonomous Vehicles should be a societal choice made after the pros and cons have been carefully weighed, including the privacy impact. deliberate The output of this debate should be clear regulation that addresses: Which data can be collected from AV video cameras, in which format, what should be anonymised and how? Where will this data be stored, shared, and secured, for how long, and what control will the public have on it? Who will control the algorithms for processing video information and how will we assess their effectiveness and regulatory compliance? (will judges need to learn to code? Will law become code…?) As the finishing touches are applied to the technology ( ) it is time for the policy debate to seriously consider the impact of privacy. Elon Musk expects the first fully autonomous Telsa by 2018 So far in my research, all I have seen in the US on this is one bill introduced by two Democrat senators called “Security and Privacy in Your Car Act of 2015” Introduced in July 2015, crickets since. ( ) Bill Tracker at Congress.gov This bill aims to: protect against unauthorized access to: (1) electronic controls or driving data, including information about the vehicle’s location, speed, owner, driver, or passengers; or (2) driving data collected by electronic systems built into a vehicle while that data is stored onboard the vehicle, in transit from the vehicle to another location, or subsequently stored or used off-board the vehicle. As reported , this bill was introduced in July 2015, and executives from Google, General Motors, Delphi, and Lyft appeared before a US Senate committee on March 15 2015 to discuss it. The executives were asked whether there should be minimum standards for privacy and cybersecurity, and all but one participant declined to say yes or no. Since then, the bill has not been discussed further. here Autonomous Vehicles may bring about a step change in public tracking and surveillance — everything, everywhere continuously recorded, this time with no way to opt out. I’d love to be proved wrong, but it seems that privacy is a complete afterthought in the push towards our driverless future. Thanks for reading — feel free to reach out with any thoughts by replying here or 👋 on my twitter Thanks to Christian Moroy , Alex Flamant , and Gabriel Oppetit for great conversations on all this mind blowing stuff (and patiently critiquing this post.)