How computer vision is changing everything
An irregular round-up of inspirations and disruptions in art, advertising, social media, urban play and human computer interaction.
The camera is fast becoming the the most important and powerful sensors we have, with face detection, image recognition and motion detection driving creativity in new directions and rapidly changing the world around us. From driving robots on mars, to shopping and war, to dance and art, interaction design, immersive realities, machine learning and surveillance, we’re only just getting started.
This is by no means a comprehensive breakdown of all the cool stuff that’s happening in computer vision, but here’s some of the cooler stuff that’s been happening to hopefully get you inspired:
The Computer Vision innovators:
The computer vision was, interestingly enough, also one of the first interactive artworks. Developed between 1969 and 1975, Myron Krueger’s legendary Videoplace, was motivated by his belief that the entire human body ought to have a role in our interactions with computers. Way way before it’s time, Videoplace is still beautifully simple and engaging:
Another seminal moment in computer vision was Loren Carpenter’s social experiment at SIGGRAPH in 1991. Carpenter, who is now co-founder and chief scientist of Pixar Animation Studios, distributed paddles to members of a theatre audience. Red on one side, green on the other. What happens next is amazing… (haha, yea this is very Buzfeedy)…
We also owe a great deal to the recent explosion of computer vision to the work of computer artist Kyle McDonald. I’m a big fan of his especially for his intervention/hack of Apple Store computers and as part of digital activists/pranksters F.A.T Lab.
McDonald pioneered face tracking by compiling a bunch of highly technical research papers into a set of open-source libraries that form the cornerstone of much of the software we see today. Paving the way for accessible face tracking, puppeteering and those silly face swapping apps have become all the rage.
Face swapping, silly and fun.
Face swapping can also be done in browser these days (though a little temperamental). Give it a go here: http://auduno.github.io/clmtrackr/examples/facesubstitution.html
Another great and simple project from McDonald and Golan Levin is the Augmented Hand series. Best explained by a 7-year old galley visitor: “It’s a box. You put your hand in it. You see your hand with an extra finger.” http://www.flong.com/projects/augmented-hand-series/
And from now on, don’t believe what you see, real-time augmented facial reenactment is a thing:
Microsoft and Pivothead’s awesome Seeing AI helps a blind person navigate life, and gives us another glimpse into just how much of a profound effect computer vision will have on our lives: http://thenextweb.com/microsoft/2016/03/30/watch-microsofts-seeing-ai-help-blind-person-navigate-life/
Face Mapping:
One of the more exciting emerging creative fields in computer vision is real-time face tracking and projection mapping.
Nobumichi Asai, Creative and Technical Director at WOW Inc. Japan takes projection mapping to the next level…
Here’s how it works:
Also worth checking out is Olga Bell’s Chase No Face by Zach Lieberman, Francisco Zamorano, Andy Wallace, and Michelle Calabro.
In New Video, Olga Bell's Face Controls A Sparkling Video Projection_Face painting is a simple pleasure most of us gave up when we turned 7 - unless we ran off to join the circus. But…_www.fastcodesign.com
The making of Chase No Face
Micro-movements:
A relatively new and extremely exciting and futuristic field in computer vision, micro-movements, is going to significantly change how we see and hear things.
Eulerian Video Magnification, is the magnification of motion, which enables you to see changes not visible to the human eye and (wait for it…because this is the most crazy and sci-fi thing you’ve seen in a long time) use vision to magically extract sound in video from objects…
Watch the brilliant Michael Rubinstein’s TED talk:
And you ca try this out for yourself with your own video: https://lambda.qrilab.com/site/geko/https://github.com/corbanbrook/heartrate
Augmented Reality (AR):
I was in two mind as to whether to put this section in. But though it’s one of the more mainstream Computer Vision innovations, we haven’t even begin to comprehend how AR is going to completely reshape our world. Sometimes called Augmented Innovation, showing us hidden things and suggesting new ideas. It’s not just gaming, toys, repair and maintenance, shopping, advertising that will be affected — almost every industry will be come to rely on this technology in the very near future, with use cases we’ve yet to comprehend. So hard not to give it a mention, though beyond the Hololens and Magic Leap hysteria it’s been hard to find some really cool stuff (let me know in the comments please that I’m totally wrong).
Body disassociation seems to be a major theme that’s been developing in the AR/VR arts space:
I’ve always loved Hiroo Iwata’s Floating Eye, that has a camera floating above you. Participants are presented with only a wide-angle image taken from the air. They can’t see the world around them directly. separating vision from the body
https://www.youtube.com/watch?v=In-M1eVAnYc&ab_channel=ikinamo
Similar is Singaporean Artist Eugene Soh’s The Overview, where wearing video goggles you navigate a maze from a CCTV viewpoint above you:
http://blog.dude.sg/eugenes-2-important-museum-exhibitions-of-2014/
An oldie but a goodie, Chris O’Shea’s Hand From Above, that’s influenced so many other AR-type experiences:
And Kyle McDonald’s Highsight is pretty fun:
No Ad is an AR app that allows you to see art where there once was advertising:
A similar project is The Advertiser, which I like more for it being a physical object: https://vimeo.com/9291451#at=50
Computer vision in art and beyond (aka other cool stuff):
Shadow Monsters:
Phillip Worthington’s fun and super engaging Shadow Monsters is essentially a digital version of a traditional shadow-puppet theatre, which turns your hand-shapes and body parts into monster-like shadows.
Would you report the jaywalker?
Media artist Dries Depoorter’s Jaywalking monitors live webcam footage of intersections in different countries and checks for jaywalkers, giving you then the choice whether to snitch on them or not: http://www.wired.com/2016/03/turning-live-surveillance-feeds-unsettling-works-art/
Rain Room
Random International uses computer vision at 3D cameras in the super cool Rain Room to make it rain everywhere except where you are:
Box:
Box a collaboration between Bot & Dolly and the brilliant GMunk it a beautiful piece of worth that explores projection mapping on moving objects.
Sniff:
Sniff is a storefront projected sentient 3D dog that reacts to passersby: http://www.gravitytrap.com/artwork/sniffhttp://www.creativeapplications.net/openframeworks/sniff-openframeworks/
Forms:
Forms is a beautiful project by the brilliant Memo Atken and Quayola is a study of human motion to sculpt abstract forms:
Here’s a video that show how the footage has been processed:
Googlefaces:
Googlefaces is a project that uses face detection to autonomously searches for faces on Google maps: http://www.onformative.com/lab/googlefaces/
AM:
Another body awareness project, Ali Phi’s beautiful AM installation projects onto yourself in front of a mirror:
Dance And Performance:
Another field that’s rapidly embracing computer vision, and there’s a lot of stuff out there. So I’d though I’d stick to the classics if you haven’t seen:
The first project I ever tell people about when talking computer vision, because it had such a profound effect on me is Messa di Voce by Golan Levin, Zachary Lieberman, Jaap Blonk, and Joan La Barbara done is 2003(!!!):
Klaus Obermaier & Ars Electronica Futurelab’s Apparition is another brilliant classic:
And I suppose I need to mention Microsoft Kinect at some time. So here’sa bit of fun with Amnon Owed’s Kinetic dance routine, for the Perfume Global project:
Music controllers:
The V Motion Project is a machine that turns motion into music:
Here’s a breakdown of how it was done: http://www.custom-logic.com/blog/v-motion-project-the-instrument/
Another simple music controller I’ve always loved is this simple alternative music controller from back in 2009 by Taemin Cho at Berklee College of Music in 2007:
Well that’s it. Hope you enjoyed this edition of The Computers Can See Me and was inspired to go play.
Btw. not sure what to call this series. Was going to call it Untitled Document. But now not so sure. Hit me up in the comments if you have a suggestion.
George Gally - Medium_Read writing from George Gally on Medium. Media artist, tinkerer, dreamer. Motion reaction, sound visualization…_medium.com
Follow me on Instagram here: https://www.instagram.com/radarboy3000/Follow me on Twitter here: https://twitter.com/radarboy_japanAnd like my Facebook Page here: https://www.facebook.com/radarboy3000