Building LeapDrop

Hi, there. I’ve been meaning to write a post about how we built LeapDrop at the InOut 3.0 Hackathon. We eventually won Best Hardware Hack sponsored by Hackster.io.

Team TheGeekBros has three bros in it: Siddharth Goswami, Soham Dodia, and I.

InOut had lots of tracks this time around, so we got to dabble around various ideas — eventually settling onto building something in the Virtual-Reality/Augmented-Reality track — something that’s been an interest of ours for a really long time.

We had a couple of ideas: a zombie-shooter FPS based on AR, a VR desktop experience (sort of like this). We had a LeapMotion, a borrowed Google Cardboard knock-off, and some two-way tape to piece it all together.

Now, none of us really knew how to proceed with any of the ideas. We didn’t have experience writing code to build VR/AR experiences — just some ideas in our heads — results of all those hours of sci-fi entertainment. So we sat there for two hours, cracking jokes and Googling stuff. It became evident after inefficiently sharing links back-and-forth for a while that it would be practically impossible for us to work on any of our ideas so far. Then it hit us.

We’d been sharing links really inefficiently. I don’t remember how, but maybe it was a Messenger group? We recalled this scene from the first Iron Man movie where Tony Stark movies stuff across computers (and onto a hologram) using just his hands:

1. C1 registers with the Server. An ID is assigned to C1.
2. C2 registers with the Server. An ID is assigned to C2.
3. PH registers with the Server.
4. LMC registers with the Server.
5. LM detects a pinch gesture.
6. LM notifies LMC about the detected pinch gesture.
7. LMC notifies the Server about the detected pinch gesture.
8. Server asks C1 to show its QR Code. This QR Code has encoded within itself the ID assigned to it in Step 1.
9. Server asks C2 to show its QR Code. This QR Code has encoded within itself the ID assigned to it in Step 2.
10. Server asks PH to detect the QR Code it sees, and send whatever the QR Code has encoded within itself. [This works because when the user makes a pinch gesture in front of a computer, the head-mounted mobile-phone camera will see the QR Code of the source computer.]
11. PH sends the decoded ID of the source computer to the Server. [C1 in this case.]
12. Server asks the computer to which the ID belongs [C1 in this case] to send it’s last visited URL.
13. C1 sends the last visited URL to the Server.
14. LM detects a release gesture.
15. LM notifies LMC about the detected release gesture.
16. LMC notifies the Server about the detected release gesture.
17. Server asks PH to detect the QR Code it sees, and send whatever the QR code has encoded within itself. [This works because when the user makes a release gesture in front of a computer, the head-mounted mobile-phone camera will the the QR Code of the target computer.]
18. PH sends the decoded ID of the target computer to the Server. [C2 in this case.]
19. Server asks C1 to stop showing its QR Code.
20. Server asks C2 to stop showing its QR Code.
21. Server asks the target computer [C2 in this case] to open the URL obtained in Step 13.

We got a bit of attention from the judges and fellow hackers around us, but unfortunately didn’t get selected to demo on-stage. This was a real bummer, since I can’t help but think to this date, how rad it would have been to grab a webpage from the projector-screen and put it on a laptop on a totally different side of the room.

Anyway, when the prizes were being announced for the Hardware category, I thought maybe we were deserving of the third prize. The third prize was announced and it wasn’t us. I thought maybe we were second since LeapDrop was really cool. The second prize wasn’t us either. I gave up on winning a Hackathon prize this time around. (Paging Imposter Syndrome!) TheGeekBros flashed on the screen as the first prize, and we took a jolly ride to Cloud 9. I had been to the first two versions of InOut as well, but like they say: Third time’s the charm.

Anyway, that’s it for this reeeeally long post. If you’re still reading, thanks for sticking around for this long. 😄

GitHub repo for LeapDrop.