Will the Robots (Finally) be Arriving?

Exploring the role of consumer robots before, during, and after the current pandemic

Recently, I was walking in a little park in downtown Mountain View, right in the heart of Silicon Valley, when I saw something unexpected. They were little robots, well little “automated carts”, really, with a flag on top and there were a few of them lined up near the library.  Every now and then, one of them would leave the line and navigate the sidewalks and crosswalks of downtown.

I became intrigued.  Being in what is essentially, ground zero of Silicon Valley, I guess I shouldn’t have been that surprised.  When I moved to Mountain View in 2007, I remember being surprised at seeing Google’s fleet of autonomous cars driving around (now called Waymo). This was a precursor to the boom in self-driving car companies that happened over the past decade, and which has yet not quite born fruit.

The little automated “delivery bots” are actually from a company called Starship, and as I would later learn, they could deliver everything from groceries to pizzas. What made them interesting is that unlike automated cars, which still have drivers in them as backup, these little guys were navigating sidewalks, waiting at crosswalks and crossing streets as necessary, completely on their own. Starship, it turns out, is actually an Estonia-based startup that has done campus deliveries, delivering everything from breakfast to pizza on campuses like George Mason university.

Delivery Robots from Starship in downtown Mountain View

Robots helping consumers have been predicted for a long time, particularly in science fiction, but have almost always disappointed and never really materialized. Unlike industrial robots, which have been working away busily at factories around the world for years, real world robots are few and far between.

Part of the reason consumer robots haven’t really taken off is the distinction between AI and robots. Historically, in both science and science fiction, this distinction used to be fuzzier, which is what led to a prediction of “robots everywhere”.

The other reason is that navigating in the real world is tricky and difficult. While the “starship delivery robots” seemed to do fine on a college campus, in downtown mountain view, as I observed them I’d notice that they’d get stuck waiting at crosswalks and never cross the road, leading a human to emerge from somewhere to come in and intervene.

In this article, I want to look at the state of consumer robots, and speculate if the current “social distancing” requirements because of the COVID-19 virus will actually lead to a “robot boom” in both funding and practical applications that have been predicted for a long time?

Robots vs. AI

Given the “AI Boom” over the past decade, with billions of dollars in funding pumped into artificial intelligence, and new AI startup popping up on every corner, you think you’d see more useful robots out and about in the world by now.

Historically, the line between AI and robots is one that might have seemed muddy in the past in both science and science fiction. As we have been making strides in artificial intelligence to help humans in many tasks, the distinction between AI and robots has become clearer and clearer.

In the 1990 movie, Total Recall, when Arnold Schwarzenegger’s character gets into a self-driven taxi, there was a human-like, or anthropomorphic robot, which took directions from the passenger. Today, we know there’s no reason to have a human-like robot in the driver’s seat for autonomous vehicles, and in fact, when they become available as taxis, there will just a be screen and a voice interface.

You can talk to AI without needing a robot. I like to call this Star Trek vs. Star Wars distinction.

In Star Wars, whenever a human needed a machine to do something, they told a droid, like R2D2 or C3PO, perhaps the most famous robots of all time. C3PO, the. More anthropomorphic of the two, was fluent in over 6 million forms of communication and meant to serve as a translator, or protocol droid, whereas R2D2 could interface with systems and machinery in binary and understand what the humans wanted him to do.

Perhaps the most famous robots ever, C3PO and R2D2 exemplified the difference between AI and Robotics

In Star Trek, which was meant to be more “realistic”, there is no need for robots to do the things that machines can do for humans. In Star Trek: The Next Generation, for example, the crew members of the Enterprise simply spoke to the ship, using what we call a “wake word” today: ”Computer” and told the computer to do X or Y or Z. This echoed the earlier representation of HAL in 2001. The only real robot in Star Trek, Data, was a unique android and a bit of a curiosity and not essential to the operating of the starship.

Of course, we now know that the Star Trek model is closer to how our technology is developing – we don’t need a walking tin can to do translation, we just need an app on our smartphone that we can call upon to translate for us. We don’t need a robot to tell our TV what to do, we can just tell it using voice commands, as long as it has the AI to understand us.

Robots in Science Fiction: Killer AI or Anthropomorphic Servants?

This distinction wasn’t always so clear. In fact, if you look at early writings on artificial intelligence, ranging from information theory pioneers Claude Shannon and Alan Turing, you’ll see that when they are talking about AI they are referencing “machines”. Shannon built a famous chess playing “machine” rather than an algorithm, and when Turing proposed his famous “imitation game” test (now known as the Turing test), what was behind the curtain was either a human or a machine, and the goal was to see if you can tell the difference by passing messages – i.e. which curtain held the human and which held the machine.

Robots, even more so than AI, have been part of science fiction for a long time, well before Star Wars hit the screens in 1977, dating to the black-and-white film classic, Metropolis. Images of Robots have ranged from optimistic (helpful servants of humanity) to threatening, as in threatening to replace workers at key jobs, a fear that was exemplified in Metropolis during the Industrial revolution, to downright terrifying, as in the Terminator, where Skynet, a super-intelligent AI, launches a nuclear attack and then sends out robots (technically, cyborgs) to infiltrate and kill key remaining elements of the dwindling human resistance.

The Terminator, released in 1984, wasn’t the first or the last to explore the Frankenstein theme – that we will create the machines that will lead to our destruction or enslavement, ranging from Battlestar Galactica, where the Cylons, a race of robots created by humans almost wiped out the human race, to Dune, set in a world where artificial intelligence had once enslaved humanity, but was saved by a “rebellion”.

This trope has become so common that we now even have a term for it – the “robot apocalypse”, which is seen by many technologists like Elon Musk and others to be more likely than the also much touted “zombie apocalypse”. Still, this isn’t the only vision for robots to come from science fiction – in fact, one of the laws of robotics put forth by sci fi visionary Isaac Asimov was that “a robot shall not harm a human”.

Rosie the Robot is still "aspirational" for today's robot designers

Perhaps among the most helpful robots in popular media was Rosie, the robot maid with the Brooklyn accent in the Jetsons (which aired in 1962). Rosie has, in a way, stood as both a pioneer and promise for how robots can help humans. Just as machines were helping to automate certain tasks that were too dangerous for humans, Rosie could be an aspirational roadmap for human helpers, taking away the daily chores of the housewife for ordinary families (and the housemaid for wealthier families), taking care of the cleaning and washing.

Although she had wheels, she was an example of an anthropomorphic robot, one that was shaped more or less like a human, a model which would be followed by a lot of science fiction but not in the real world – at least not yet.

At the far end of the Anthropomorphic scale are the androids from Blade Runner, based on the Philip K. Dick Novel, Do Androids Dream of Electric Sheep. In Blade Runner, They were designed by the Tyrell corporation (whose motto was famously “more human than human”) and were limited to working in Outer Space and environments that were too dangerous. Each “model” had certain strengths – including Roy Batty (played by Rutger Hauer, the leader of the group) and Pris (Darryl Hannah), who was the “pleasure model”. As social gender roles were opened up, the 2001 movie, AI, directed by Stephen Spielberg, had Jude Law as a male “gigolo robot” who could satisfy any female (or I would assume, male) customer, complete with romantic music that could be turned “on” to well, turn the customer on.

A Quick History Leading to our Robotic future: Partners, Slaves or Masters?

As we look to a robot future, it’s perhaps appropriate to ask a question that John Markoff explores in his recent book, Machines of Loving Grace: “Will robots be partners, slaves, or our masters?” Of course, the reality is that we are nowhere near Robots being either partners or masters, since they are still “machines” that might do our bidding. Let’s take a quick history lesson on modern robots.

Funded by DARPA and developed at SRI, Shakey was the precursor to many of today's robots

• Shakey the Robot – The first autonomous robot was funded by the Pentagon and called “Shakey the robot”, complete with wheels and a monitor. It was built by the engineers at SRI and was featured in Life Magazine in 1970 as the first “electronic person”, which gave the impression that the robot was much further along than it really was. In reality, though Shakey was a long way away from Rosie. Shakey was a bit of a disappointment, particularly in mobility, and it would fail often, having trouble getting around. around. Like the eponymously named ‘Robot” in Lost in Space, Shakey had wheels and was human sized, though not exactly anthropomorphic. Though the project itself didn’t meet the needs of the Pentagon to send in autonomous agents into “hostile territory” (it could barely navigate the halls of SRI), there were a number of breakthroughs that started with Shakey that still impact us today – the shortest route mapping algorithms used today were developed for Shakey.

• Japanese Robots – For some reason, in Japan, robots haven’t seemed as threatening and consumers have been friendly to the idea of robots. In fact, the few consumer robots that have appeared commercially have been more successful in Japan since the 1980s. Rather than being presented as terminator-style frightening robots, Japanese robots tend to be depicted as “friendly” robots. In the computer history museum, right here in Silicon Valley, the robotics section has plenty of Japanese robots. One journalist wrote in 2017, “On a stroll through Tokyo, you can meet humanoid robots like Pepper, touting the latest smartphone deals outside SoftBank shops.” Moreover, with the aging population in Japan, robots are being touted as a solution for elderly care when children or grandchildren are not around to be with elders. Enter the cuddly Robear

Cuddly Robots may be the solution to elder care in Japan

• Siri, Alexa, Jibo and all that - Unlike truly anthropomorphic robots, a few years ago, many venture capitalists and entrepreneurs realized that the voice interface was the most important thing. Siri, which spun out of SRI, the same place where Shakey the Robot had been created, was bought by Apple and incorporated into the iPhone and released to the world in 2011. This showed that much of what we might want robots to do for us, which didn’t involve physical labor, could be accomplished by AI, in a pre-cursor perhaps to Star Treks’ “computer interface”. Quickly, other tech leaders began to think of having devices with voice interfaces. A few years after Siri came out, MIT researcher Cynthia Braziel launched Jibo, and raised over $3 million in crowdfunding (not to mention over $70 million in venture funding), to launch Jibo, a “social robot” in 2017. The original videos put out by Braziel and team were viewed millions of times. Jibo was slightly anthropomorphic, wasn’t mobile, sat on a shelf, and had a little “face” which changed based on what you asked Jibo to do. By the time Jibo got to market, though, Amazon had already released Alexa which seemed to accomplish everything Jibo was meant to do, and Jibo is considered yet another “robot” that didn’t meet its potential. The technology was purchased from Braziel’s startup by an IP firm and is still available in the market.

Jibo, the social robot which came out of the MIT Media Lab

• Boston Dynamics and the Four Legged Robots - One of the few robot companies to be recognizable is Boston Robotics, which was started in 1992 spun out from the MIT AI lab by Mark Raibert. They made robots that could walk in terrains with the ease of animals, looking more like insects or animals than people – and in fact, they are named after dogs. Perhaps the best known of these is the “big dog”, a robot developed for the US Military to carry equipment into terrain that might be too dangerous for people. In fact, this was the original vision for Shakey, so this might be Shakey v3.0, and was funded by the same group at the DOD, DARPA. The “big dog” video became very popular and went viral and unlike Japanese robots, these robots gave people the willies… there was something lifelike about them, almost too lifelike. It was like seeing a giant spider walking around in front of you – more like something from Jason and the Argonauts or Harry Potter than Star Wars. Boston Dynamics was purchased by Google in 2013.

Boston Robotics quadruped robots were more useful than Shakey, and scarier.

• Sophia and Hansen Robotics – Sophia's introduction led to a discussion not unlike that of Data, in Star Trek, who was a synthetic life form whose rights were a matter of debate. Sophia, which (who?) made her first public appearance in 2016, was created by Hanson Robotics, one of the few companies that aspire to create anthropomorphic consumer robots. Sophia was covered widely in the press and was the first robot to be granted citizenship by a country – in this case, Saudi Arabia. Many were amazed with the videos of Sofia and wondered if we had passed the “Turing Test” – but Hanson’s founders admitted that Sofia used a lot of tricks and he didn’t consider her a robot that had passed the Turing Test. Still, Hansen seems to be proceeding on the path with other robots, including Han and Professor Einstein, which are meant to have completely different personalities from Sophia.

Sophia was so human-like that a country gave her citizenship

Robots Arising From the Quarantine?

So, this brings us to today. With the nation and the world under lockdown, the population has been told to avoid unwanted contact with other human beings. While this has negatively impacted many industries, ranging from traditional hotels to Airbnb, even ride-sharing companies like Uber have suffered, because who wants to get into someone else’s car – especially since you don’t know who has been there and if they could have left the virus behind.

This concern isn’t specifically restricted to the coronavirus, but for anyone who is immune-compromised. After I had surgery a few years ago I was told to avoid Uber’s because my immune system might not protect me from “random viruses”.

Just down the road from where I am in Mountain View, there was another company, Zume, that had robots making Pizza in delivery trucks. Bad timing, I suppose, since Zume shut down its consumer Pizza delivery in February of this year, just before the virus concerns caused wide shutdowns.

While the shutdowns will be over soon (supposedly), many wonder if life will ever be the same – and if we will be wary of going into public spaces which put us in close quarters to others – like movie theaters, concerts, etc., for years to come.

Clearly, a real cleaning robot, like Rosie, or a fully autonomous vehicle, could be “cleansed of viruses” much more easily than a human could. While we can’t bring cleaners into our apartment, if we could be assured that a robot was disinfected, there would be no objection to inviting them in.

A Quick Look at the Present and Future of Consumer Robots

Delivery Robots

After seeing the starship delivery robots mentioned at the start of this article, and posters at a number of restaurants/stores downtown that they can deliver to me robotically, I went home to try the app. Unfortunately, I was *just* outside the delivery zone, on the other side of Shoreline Blvd. While I was probably not less than half a mile from downtown and about halfway between the downtown and the Googleplex, it seemed like a large road like Shoreline was too much for the little guys to cover.

Other startups that have been funded for delivery include Nuro and Robby. Robby’s snackbots are a lot like Starship, with slightly different colored “delivery bots”, and were delivering snacks on PepsiCo’s campus.

More interesting looking is Savioke, who looks like a cross between Dr. Who’s Daleks and R2D2, which focuses on the hospital and hospitality industry. Using a robot in a controlled environment like a hotel to deliver room service or other items makes perfect sense, and Savioke raised over $13 million just for this task.

Perhaps even more interesting during this pandemic is that another of Savioke’s industry targets is the hospital industry – and while the hallways in a hotel can be a little more chaotic than in most hotels, delivery of goods to rooms where patients are infected makes perfect sense.

Savokie, the delivery robot for hotels and hospitals

Between the algorithms pioneered in Shakey almost 50 years ago and the advances in autonomous cars and drones, Fully automated delivery to your house can’t be that far away. Today, some delivery services are offering “contactless delivery” – what this means is that the person delivering the food will be wearing gloves and leave the food outside your door. True contactless deliver can’t be too far behind.

Amazon announced that they were explored flying drones to deliver packages in 2016. In fact, it has been in beta testing in the UK and other locations. The biggest obstacles weren’t technical, but regulatory – including restrictions in the US which allow drones to only be flown remotely where the operator can see the drone visually. Airspace is heavily regulated not just in the US but around the world, and for good reason – you don’t want a commercial plan to crash into a drone. The FAA has been talking with companies like Amazon, Uber (who want to have flying Taxis) and other countries, including Iceland and Israel, are way ahead of the US in this regard. Beta programs being tried in other countries may provide the framework to get drone-based delivery working here.

Hospital Robots

Nuro is an autonomous vehicle company whose cars I’ve seen driving around Silicon Valley often, though the frequency seems to have been reduced since the start of the current shelter-in-place orders here. In fact, Nuro is being used in a stadium in California set up for coronavirus patients, where its self-driving vehicles are limited to under 5mph and can be used to deliver medical supplies around the stadium.

Nuro isn’t the only one. One of the easy ways that Robots are helping is during the pandemic itself, in health care providers that may not have the right care equipment to protect nurses from getting infected. Boston Dynamics Spot robot has been modified to let a doctor or nurse go into a room to interact with a patient.

Boston Dynamics Spot modified for telemedicine

According to the company: “Based on the outreach we received as well as the global shortage of critical personal protective equipment (PPE), we have spent the past several weeks trying to better understand hospital requirements to develop a mobile robotics solution with our Spot robot. The result is a legged robot that can be deployed to support frontline staff responding to the pandemic in ad-hoc environments such as triage tents and parking lots.”

Hospital care is a natural area for robots to assist with, and though they won’t be replacing nurses anytime soon, there are enough repeatable tasks and, assuming that the robots can be dis-infected after each visit into a patient room, this is a promising area to watch and may be the first consumer area where robots become critical in future pandemics.

Elderly Care .

An example of Elderly Care Robot, Stevie

The care of the elderly is a task not just for Japanese Robots like Robear, mentioned above. In fact, it’s estimated that by 2040, that there will be a shortage of almost half a million paid caregivers for the elderly here in the US as well, which means that family members will have to be available nearby.

Most of our robots aren’t ready to provide all the caregiving services that are needed by the elderly, but there are many things that can be easily automated, including social activity, reminders for pills, etc. One such robot, called Stevie, a socially interactive robot, was profiled in Time Magazine in 2019, and which was developed at Trinity College’s Robotics and Innovation Labs, which interacts with residents of an elder care facility in the picture above. Stevie can lead the group in song and dance and otherwise interact in a limited way. Although it’s not ready for the physical demands to help seniors in moving around, you can see the technology developing in that direction – anthropomorphic robots that can provide companionship and automate many simple tasks when there are no loved ones around. Jibo, though it’s not mobile, has had some success in serving as a "companion" as well.

Cleaning Robots

So, now we come to what was one of the original inspirations for the robot industry but has turned out to be a bit of a holy grail – when can we expect cleaning robots to come to our house and clean for us?

The closest thing to Rosie we’ve seen has been the Roomba, which was introduced in 2002 by three members of the MIT AI Lab, who started a company called iRobot. The Roomba is a disk shaped autonomous vacuum cleaner that cleans around the house on its own. It comes in various models, but they pretty much follow the same basic design. There are now many competitors to the Roomba for cleaning carpets and hardwood floors including Ecovacs which has a vacuum version and mob version.

For industrial uses, there are companies like Brain Corp, whose autonomous robots look more like lawn mowers, but which can move around stores and factories on their own cleaning. Similarly, there are “robots” that are shaped similar to the Roomba which can be used to clean windows, including one from Ecovacs. There’s even a Grillbot that will clean your outdoor grill, as well as self-cleaning ovens and toilets, and even little disk shaped robots you can place in your toilet.

While these may be seen as impressive achievements, they are rather modest when compared to what we thought Robots might be able to accomplish: they are very specific machines for specific tasks.

Some missing Ingredients

While consumer robots haven’t fulfilled the purpose foreseen by science fiction, this is because certain tasks have proven more difficult
than originally anticipated.  Chief among these is navigating around our houses and apartments. 

Climbing Stairs and inside navigation. While Roombas and similar “cleaning bots” can move along a horizontal surface, simply climbing stairs to go “clean upstairs” is a task that still hasn’t been mastered by most of the consumer robots that are out there.  Interestingly, the Boston Dynamics four-legged approach has shown very agile robots, who can do backflips and run and conceivably, climb stairs. 

Better Navigation outside . While autonomous vehicles have still to live up to their expectations, they are clearly getting better, and in some cases have gotten pretty good at navigating well-marked streets and obeying traffic rules. But for robots that have to navigate both sidewalks and streets, this process is on-going. And since the delivery robots today don’t have any arms or legs, they can’t do a simple thing like open a gate to come up to the doorstep, though flying drones could help with that part of the problem. Still we are a few years away from robots that can navigate our yards, sidewalks, to get to our houses.

Object detection and manipulation. One of the hardest areas for robots to do is to detect objects and to interact with them. Picking up a book, for example or shaking a hand. Industrial robots are usually optimized for a particular manufacturing task, or for packing or moving boxes in warehouses, and this influences everything from the number of joints, the types of “fingers”, and the hardness of the surfaces. A consumer robot would need to be able to interact with humans (biological flesh) as well as pick up and move objects of different hardness and softness – ranging from clothes to books to chairs. While the AI to recognize objects using machine vision is getting better, the surfaces and parameters for picking up and moving objects without negatively impacting the thing that is being touched are still underway.

Speech Recognition and Interaction. Once thought the most difficult part of building robots, this is an area that probably has made the most progress. While we are not yet close to passing the Turing Test, there are now algorithms for parsing both written and verbal speech that have gotten “pretty good” – especially when there is a limited vocabulary. A truly consumer robot would need to be able to not just recognize specific commands, “Robot, pick this up” but also understand general speech and respond. Sophia can listen and respond as can other chatbots like Google’s new chatbot Meena.

Disinfectants. Perhaps one of the simplest tasks that robots will need to be able to do is to ensure they are “disinfected of biological agents” to make sure that any germs, viruses or bacteria, are cleaned off before entering a new environment. This will be important for robots that interact with humans, as well as for vehicles that are autonomous who might have multiple inhabitants, like taxis. Also, any Robot that can go from one environment to the next needs to be able to do this. While there is no technical reason why this can’t be done on all robots, thus far, only a few robots have been modified. In fact, one was used during this outbreak to use ultraviolet light to disinfect hospital rooms after a patient has left, but we'll need robots to disinfect themselves after every interaction or visit.

Regulatory Considerations. Just like airspace consideration, the hardest part of getting robots to be a part of our daily lives might just end up being regulatory. Since Nevada led the way in 2019 to allow autonomous vehicles, 29 other states have followed suit. There are complex legal issues – who is liable if an autonomous vehicle gets into an accident? The insurance company? The software maker? The hardware maker? As we have robots walking around in our stores and our neighborhoods and our homes, this adds layers of complexity which we may be working on for many years.

Artificial General Robotics. While there has been much talk of an AGI, or Artificial General Intelligence, we need a similar terminology / goal for robots. In this case it would be AGI plus a real knowledge of the physical world. This can most likely be accomplished by training the AGI inside 3d simulated worlds, before setting them loose in the physical world around us.

Conclusion

The history of AI And robotics has been intertwined in both science fiction and science. Many of the robotics companies that exist today have their origins in AI labs like the MIT AI Lab or the Stanford SAIL laboratory.

For much of the past decade, while AI has made great strides, the vision of a consumer robot that can be a companion to us, clean our homes, and otherwise do things we might expect a human to be able to do has been slipping farther away. And economically speaking, while robots have become good at certain industrial tasks, consumer robots have not yet come to fruition.

The recent pandemic and lockdown have made us wary of letting other people into our homes or interacting with them, which only bolsters the case for having consumer robots to help us with daily chores, ranging from cleaning to shopping to delivery and pickup, not to mention helping with elderly care and other tasks.

There are a number of areas that have to be improved before we can get to even a basic Rosie the Robot for cleaning, let alone Blade-runner level anthropomorphic robots, but many companies have been quietly making progress, while much of the focus in Silicon Valley and funding has gone into AI and industrial robots. Though science fiction has been predicting ubiquitous robots like those shown in the movie I, Robot, being rolled out to every home across the nation and world, the age of Robots seemed farther away than ever.

Now, with the virus a reality not just for the present, but with other viruses likely to follow, perhaps we will say that 2020 was, finally, the beginning of the real Age of Robots.


Rizwan Virk is the founder of Play Labs @ MIT and author of Startup Myths & Models: What You Won’t Learn in Business School, Zen Entrepreneurship, and The Simulation Hypothesis. He is also a venture partner at several VC firms. For more information, please visit https://www.zenentrepreneur.com, or www.bayviewlabs.com, and follow him on Twitter.