How I hacked Google Daydream controller

Reversing and making it compatible with Apple iOS devices. ABOUT Hi all, I am a creative, curious and inspired software developer with hacking attitude and strong disposition toward reverse-engineering.I matured several years of experience in IoT, embedded systems development and in bridging the gap between the physical reality and the digital world. Matteo Pisani : matteo.pisani.91@gmail.com | : E-mail LinkedIn https://it.linkedin.com/in/matteopisani ABSTRACT Google Daydream controller and headset Mobile virtual reality is growing rapidly. The platform was launched just last month and it suggested that compelling VR experiences might become widely accessible to consumers sooner than expected. Today, solutions like are very appreciated by developers, media and entertainment companies, but… There’s one problem: . As announced, the binds only with a bunch of smartphones running . Moreover, as reported by , the . Google Daydream smartphone + headset + bluetooth controller compatibility Daydream controller Daydream-ready Android 7.0 Nougat Clay Bavor (VP, Virtual Reality at Google) Google Daydream ”It’s not currently compatible with iOS and won’t be for several years probably.” Original post: https://www.quora.com/Is-iPhone-compatible-with-Googles-Daydream-VR Since I like challenges I decided to hack the using , skills and some , to extend the compatibility also on devices: it was a success. Google Daydream controller code reverse-engineering math Apple iOS ANALYSIS works via but I wasn’t able to discover it in of my , so I used the which allows to easily implement and applications, serialize and deserialize messages exchanged with bluetooth devices and define reusable profile definitions. Google Daydream controller Bluetooth LE (Low Energy) Bluetooth settings iPhone 5 BlueCap ( github.com/troystribling/BlueCapapp ) Central Peripheral GATT I had a look at the data available for each : there were known services like and but I also found something interesting inside an unkown one, the : Service Device Information Battery FE55 As soon I explored inside the first with the and turning the , started showing . Waving the in the air, I could see the incoming changing in real-time. Same thing happened by touching the pad on top or randomly by pressing the buttons. Characteristic UUID 00000001–1000–1000–8000–00805f9b34fb On Notifications BlueCap BLE packets Daydream controller data According to standard each should weigh : Bluetooth LE packet 20 bytes 7be85b3ff13b48003bf1ffa00000000000000070 The anatomy revealed that they were encoded and represented into notation. Behind the laid the whole status of the controller, including , , , , and more. packets Hexadecimal masked data accelerometer gyroscope magnetometer touchpad buttons ENVIRONMENT The first step was to setup a to facilitate all the processes. I decided to start from scratch: I developed a with (working on a ) and an (with some ) that included the . Thanks to this, I could establish and manage communications and data flows over protocol. testing environment debug sandbox Apple XCode MacBook Pro iOS app Objective-C CoreBluetooth\CoreBluetooth.h framework ( developer.apple.com/reference/corebluetooth ) Bluetooth GATT After choosing the and requesting notifications for the I was able to get the data output flowing through the console: Service FE55 Characteristic 00000001–1000–1000–8000–00805f9b34fb Once the data was collected and opportunely decoded I decided to represent it into a . So, I migrated all the to a wrapping it all into a : thanks to this process, I was able to save time and perform several optimizations. 3D view iOS native code Hybrid environment Cordova plugin The use of reduced the overall complexity, speeded up the experiments and allowed me to improve the data visualization embedding also thanks to the amazing inside a view. JavaScript A-Frame WebGL framework ( aframe.io ) HTML5+CSS3 (the entire environment stack) With the use of , the open-source 3D creation suite, I was able to edit a bulky model found on the internet, making it suitable for my purpose. After the editing, I exported it to an compliant format ( ). Blender Google Daydream controller A-Frame .obj In few lines of code, I was able to finish the whole setup and this was the result: REVERSING Now for the hardest part: understanding the .Starting from an average knowledge about to conversion, I split up the in of then converted to : raw data Hexadecimal Decimal 40 chars 20 chunks 2 chars Binary 7b e8 5b 3f f1 3b 48 00 3b f1 ff a0 00 00 00 00 00 00 00 70 I just wanted to give it a try, so I tested an online to converter and this was the output Hexadecimal Decimal Later, I also tried the to converter. Decimal Binary Bringing everything to JavaScript The output expected was length chain (8 bits * 20 chunks) for each packet: 160 bits I got only instead of expected so I realized that something was wrong. After going deep into the issue, I found that the values converted in sometimes produced results shorter than , in other words, were not in groups of : the to solved all the problems. 94 160 bits hexadecimal bits 8 stuffed 8 zeropad 8 Once I added the method and changed the code in: zeropad this time the expected result was correct. After a couple of sleepless nights I started to give a shape to this mesmerizing bitchain: a comprehensive knowledge about and paired with a great patience and good observation skills helped me to figure out the sense of what was happening. IMU (Inertial Measurement Unit) MEMS (Micro Electro-Mechanical Systems) sensors The crucial points were: * observate all the oscillating ;* play a little more with . bits offests This allowed me to recognize, extract and categorize the values. I reported all of them below: (12 bits + 1 bit ) SENSORS for the value for the sign (8 bits ) TOUCHPAD for the value (1 bit ) BUTTONS for the value Once I achieved this goal, I tried to manipulate all these data to give a coherent orientation to the 3D model, through the e canvas: unfortunately the output on the screen resulted in a tilting controller with meaningless movements. Google Daydream controller A-Fram Reversing some of the of the (found inside the and that allows native communication with via ), I was able to get my hands on useful information. .apk Google VR Services Google Pixel OS Google Daydream controller BLE TOOLS Through reverse-engineering of app using , , to convert file to , it was possible to: Android Java apktool dex2jar jd-gui .apk .java * understand how a particular UI in an App is constructed* reading AndroidManifest.xml, permissions, activities, intents etc in the App * discover native libraries and images used in that App* find obsfucated code ( , by default, uses tool which shrinks, optimizes, and obfuscates the code by removing unused code and renaming classes, fields, and methods with semantically obscure names. Android SDK ProGuard The tools I used: ( ApkTool from http://code.google.com/p/android-apktool/ ) to extract and everything in res folder(layout xml files, images, htmls used on webview etc..), run the following command: AndroidManifest.xml It also extracts the file of all files, but which is difficult to read. .smali .class ( To generate file from file, we need to view the source code from this .Run the following command: Dex2jar from http://code.google.com/p/dex2jar/ ) .jar .apk JD-GUI .jar It decompiles the files (obsfucated, in case of app, but readable original code is obtained in case of other file). i.e., we get back from the application.Just Run the jd-gui executables on your and after, to view code from or file. JD-GUI (from http://java.decompiler.free.fr/?q=jdgui ) .class Android .jar .java OS File->Open Java .jar .class In particular, I found interesting information inside: * * com.google.android.vr.home.apk com.google.vr.vrcore.apk (magnetometer event) (accelerometer event) (gyroscope event) (compensation methods) Collecting all my developer-thoughts and making them fit together, I realized that the best solution was to use the calculation for . AHRS (Attitude Heading Reference Systems) JavaScript ( npmjs.com/package/ahrs ) This calculates the attitude and heading for a device with all of the following sensors: magnetometer, gyroscope and accelerometer. The or algorithms can be used to filter data in real time from these sensors, obtaining a great accuracy. Madgwick Mahony The method returns an object with the (heading/yaw, pitch, roll), in . getEulerAnglesDegrees Euler angles degrees The return contains: Object * is from north, going west (about ).* is from vertical, going forward (about ).* is from vertical, going right (about ). heading z-axis pitch y-axis roll x-axis Finally, it was possible for me to set the model orientation to the right coordinates The result was brilliant: as you can see in the video below that I recorded to show the potential of the entire hack YouTube Google Daydream controller running on iOS (on an iPhone 5) The responsiveness is extremely fluid, according to the parameter, are enough to cover a VR game or a 3D experience as well. PPS (packets per second) ~ 60 (Google Daydream controller running on an iPhone) CONCLUSION The scenarios that this hack opens are various. Now that the secret sauce has been exposed and the compatibility extended to devices, it is possible to replicate the job to include all the desktop platforms. This would help the developers debugging their own software in a , without passing through deploying an app on the smartphone every time. On the side, this hack will unleash the whole potential of the as it would be no longer restricted to the . iOS desktop environment Android Daydream controller OS Nougat 7.0 In this perspective, it is possible to see the working with older versions of . On the other hand, binding this controller with open source platforms like or , will extend the horizons of makers and creatives. Do you imagine using the to pilot your drone or your RC-car, playing a virtual drumset or maybe, making some sounds with a virtual synth? Daydream controller Android OS Raspberry PI Arduino Daydream controller Do you wanna see running on ? Check this out: Google Daydream controller Linux How I hacked Google Daydream controller (Part II) .