Telepresence Robots: An Ultimate Guide

Today, there are about 30 models on the telepresence robot market (see https://telepresencerobots.com/robots); however, the vast majority of them are either only experimental samples or are no longer produced, or have a price incommensurable with their capabilities. In this story, I have shared the main characteristics of telepresence robots. Video quality The vast majority of the robots use WebRTC technology developed by Google and standardized by the W3C consortium. This technology already has built-in video compression algorithms with VP8 and H.264 codecs, camera access methods, algorithms for adjusting video resolution to the bandwidth of the communication channel, as well as algorithms for obtaining the IP addresses of the robot and the client. The video quality is determined by the bandwidth of the channel from the robot's video camera to the remote user's monitor screen. It is possible to reduce the resolution of the video programmatically, but it cannot be increased. The bandwidth of the entire video transmission channel is determined by the slowest part. There are three such parts: the Internet, the processor in the robot head, and the processor in the user's gadget that controls the robot. As usual, the Internet is the narrowest link in the communication channel, especially if the robot is at an exhibition where many consumers are connected to the same WiFi access point. In conditions when a powerful desktop computer is used on the client-side and the Internet has a large bandwidth, the video quality is determined by the processor performance in the robot. If the client uses an old low-power smartphone, then the video quality depends on the performance of its processor. Since robot manufacturers try to use the most powerful processors available on the market at the time of the robot's creation, the video quality of all robots turns out to be approximately the same if they are made at the same time and use processors comparable in performance. The exception is cases when not universal processors are used in work but specialized ones for video processing. As new processors become available, they are most quickly introduced into smartphones and tablets due to the huge size of this market. The market of specialized processor modules (printed circuit boards) is more inert for natural reasons associated with the small production volume of such products. Therefore, robots that have the ability to replace the tablet installed in the robot's head allow you to get better video at any time by simply replacing an outdated tablet model with a more modern one. Built-in or replaceable tablet Many telepresence robots are available with already built-in touch screens and with integrated processor boards (for example, Double 3 and Ohmni). Others use ready-made tablets inserted into the robot's head (for example, Double 2, BotEyes, Padbot U1). The built-in touch screen with a built-in processor module, unlike a ready-made tablet, has a number of advantages: it allows you to connect any webcam, for example, a rotary camera with magnification (pan-tilt-zoom) or several cameras, non-standard microphones (microphone array), collision sensors. Theoretically, they can also be connected to a standard tablet via a USB-OTG cable, but this technology is not supported by all tablets and not all operating systems. The disadvantage of robots with integrated processor modules and touchscreens is that the semiconductor element base used in them becomes obsolete very quickly. At the same time, when working with a removable tablet, it can always be replaced with a more powerful, latest modification, and even with an updated OS. The advantage of a removable tablet A robot with a removable tablet has another advantage that is not obvious at first glance: it can be positioned both horizontally and vertically. What does it give? The fact is that the matrix of the video sensor in the camera has a certain aspect ratio, for example, 16:9. If the monitor of your computer that controls the robot has the same aspect ratio, and the camera position and the monitor position are the same (both are located horizontally), then the camera image is displayed entirely on the screen. However, if the tablet in the robot (or, what is the same, the camera) is positioned vertically, and the remote monitor is horizontal, then to the right and left of the video you will see large black stripes occupying most of the screen, and it will look something like in this video: This is how the image from the robot looks when the aspect ratio of the camera and monitor is inconsistent. If we try to stretch the image in width, then part of the image will be cropped vertically. The only correct way out is to coordinate the position of the monitor and the position of the camera in operation. Namely, if you are using a desktop computer with a screen positioned horizontally, then the tablet in the robot head must be positioned horizontally. If you use a smartphone that is convenient to hold vertically, then the tablet in the robot head should be positioned vertically. The ability to change the position of the tablet is available only in the BotEyes robot. An additional advantage of using a removable tablet is the ability to install any familiar program like Skype or Zoom on it instead of the default application. Tablet size The size of the tablet determines how much detail you can view the remote scene. Usually, 10-inch screens are used in robots. The required size depends on what purpose you are using the robot for. The advantage of robots with a removable tablet is the ability to use any tablet, as for example, from 8 to 12 inches (see the figure below). Removable tablet in telepresence robot head Motion control options Most robots use four arrows to control movement: "forward", "backward", "left", "right". The speed is either not regulated at all, or to change it, you need to stop the robot, change the speed and then resume movement. This approach is convenient when you need to move the robot a long distance in a spacious room. However, if you are in a room with a lot of people or chairs, such control requires a lot of unnecessary manipulations, the robot loses maneuverability. Some robots use the so-called "control circle", which allows you to change the speed and direction of movement smoothly, without stopping the robot: Robot moving control circle If the mouse pointer (or finger on the touch screen) is located close to the center of the circle, the speed is minimal or zero, the farther from the center, the greater the speed. The arrows indicate the direction of movement of the robot at the moment when the mouse pointer is in the sector with the arrow. The circle is made transparent so that the main image can be seen through it. Such a control body makes the robot highly maneuverable, allows you to move without touching even very closely located objects. This is especially useful when the robot is traveling through a production shop, where there may be a lot of both large and small objects on the floor. Click-To-Drive + Self-Driving This unique way of motion control is proposed by Double Robotics. Its essence is that the user specifies a point on the floor to which the robot should move, and it moves to it automatically, avoiding obstacles along the way with the help of a sensor system. This property looks very interesting but has a number of limitations. Firstly, although the RealSense technology used to display the 3D scene should work at a distance of up to 10 m, in fact, the robot recognizes the scene at a much shorter distance, after which it stops and you need to press the "Forward" arrow again. I.e., instead of pressing the forward arrow once and driving as long as necessary, in the case of Click-To-Drive, it may be necessary to repeatedly indicate a new destination to the robot. In the screenshot, we took during our remote tour of the British Art Gallery (see figure below), the area of light dots on the floor means the area where the click-to-drive technology works. The length of the area is about two to three meters. The size of the area within which the click-to-drive technology worked during our experiment With good lighting and spacious rooms, this method is quite convenient and almost indispensable in the case when the robot is not able to tilt its head to see an obstacle next to the wheels. The second disadvantage is the inability to ride in a cramped room with many obstacles, since ultrasonic distance sensors are not enough to detect, for example, small or thin tall objects, as well as accurately determine the boundaries of large objects. For orientation in such conditions, it is necessary to use a scanning lidar, which, however, is very expensive. It should also be noted that the problem of motion control exists only when the robot is initially tested by people without experience. If a person uses a robot regularly, then the problems of motion control completely disappear, because it is too simple. Stability and height of the robot The ideal telepresence robot should be light and tall so that a person does not have to bend down to view the image on the screen. But it is possible to keep the tablet on a high and movable stick only with the help of automatic balancing. Therefore, there is an obvious huge temptation to make a robot on the principle of a segway. However, this desire generates a large number of technical problems described below. The main problem is that two-wheeled robots fall, and this leads to damage to the tablet, which is often more expensive than the robot itself. This happens when the robot touches one wheel, for example, a sofa or human, and the second continues to go, or when overcoming a doorstep. In these cases, the performance of the balancing system may not be sufficient to compensate for external influences. That is why robots of this type use soft edging around the tablet. The second problem is lateral stability. When a person rides a segway, he provides lateral stability with the movement of his body. In the case of a robot, it is necessary to make a second auto-balancing system - transverse, along with longitudinal. For this reason, the first two-wheeled robots that do not have transverse balancing could not overcome even a wire lying on the floor without falling. The latest versions of such robots have a transverse balancing system and have become much more stable than the first versions, however, they still fall down when moving even low thresholds, if the wheels do not overcome the threshold simultaneously. The third problem is that in such robots it is difficult to realize the tilt of the tablet since the tilt sensor that provides balancing the whole robot is located in the tablet itself. In addition, the balancing algorithm itself becomes more complicated. To compensate for the inability to change the viewing direction by tilting the tablet, a fisheye camera is usually used, which allows you to see what is at the bottom of the robot. Unfortunately, such a camera greatly distorts the space and generates operator discomfort. The fourth problem is that due to the inertia of the entire system and the limited speed reaction of the motors, the robot strongly deviates backward or forwards during the start of the movement and after stopping, which also causes psychological discomfort. All these disadvantages and problems are compensated by the fact that two-wheeled robots are the lightest on the market, which is very convenient when transporting them. Opposite properties and problems are typical for three- and four-wheeled robots. They solve the problem of stability much easier: with the help of an increased base area and increased chassis weight. This approach immediately allows you to solve a number of problems of two-wheeled robots: tilting the tablet at any angle, easy inspection of the space around the wheels, no need for a fisheye lens for the camera, the presence of transverse and longitudinal stability at the same time, the ability to overcome fairly high thresholds. The price for this is more weight, higher transportation costs, large dimensions of the base. Head tilting In order to avoid obstacles, the robot operator must see them. Telepresence robots use two ways to do this: a second camera that shows the space around the wheels, and the tilt of the camera or tablet. The disadvantage of the option with the second (navigation) camera is the reduction of the useful area of the screen. The photo below shows how the operator's screen of such a robot looks like: In bottom of image you can see the image from the navigation camera There are options when the view of the wheels is superimposed on the view from the main camera. In this case, the area of the main image becomes larger, but still part of it is occupied by the view from the navigation camera, which uses very seldom. If we use the tablet front camera to inspect the floor, there is no such effect, and the useful image occupies the entire area of the screen: When using a tiltable camera, the main view takes up the entire area of the screen Using the same camera simultaneously for navigation and the main purpose is justified since obstacles bypassing takes up a small fraction of the total time of using the camera. The second advantage is that the rear camera of the tablet is not attended for any purpose and can be used to inspect the scene behind the robot by quickly switching cameras. In addition, the rear camera of most tablets has a much higher resolution, and this can be used, for example, to read text on a school board. There is also a useful psychological effect of being able to tilt the tablet: the tilt can be made so that the interlocutors have the feeling that they are looking into each other's eyes. It is very important. Batteries type The type of battery affects three characteristics of the robot: safety, battery life, and robot stability. Lithium batteries used in the vast majority of telepresence robots are . Despite the vigorous efforts made in recent years in their design and production, despite the presence of protection against overcharging, short circuits, balancing, there are still cases of their explosion and ignition. The list of incidents can be viewed on the website of the and they continue to this day. Therefore, the use of impenetrable lead-acid batteries in telepresence robots is preferable from the point of view of safety. dangerous for ignition US Department of Transportation It also follows from the safety documents that the higher the capacity of a lithium battery, the more energy is released when it ignites and the more severe the consequences. Therefore, only batteries with a capacity of fewer than 100 watts per hour are allowed to be transported by air. We think that this circumstance limits the duration of operation of many telepresence robots to five hours. In contrast, robots with lead-acid batteries are safe, so there are no restrictions on their capacity during transportation and any capacity batteries can be installed in the robot. The large weight of lead-acid batteries placed in the lower part of the chassis also ensures the good stability of the robot. Sound volume Modern tablets have powerful speakers installed and their power is enough to communicate with an individual in a quiet environment. However, in a crowded, noisy room (for example, at an exhibition), a powerful sound is needed. It can be obtained using an additional external Bluetooth speaker. Some robots have such speakers already installed in them. However, their power may not be enough. Our experiments with five types of purchased external speakers with Bluetooth have shown that they significantly enhance the sound compared to smartphone speakers, but practically do not increase the volume of tablets. Therefore, in cases where powerful acoustics are needed for a speaker at a meeting, the same power sound equipment is needed for the robot. Choosing a video communication tool Modern telepresence robots, almost without exception, use Google WebRTC technology. However, there are also such generally accepted means for video communication as Skype, Zoom, Google Hangout, etc. The peculiarity of their application is that it is impossible to integrate robot motion controls into their user interface. In this case, it is possible to make a second application that controls the movement independently of the video communication application. When using a desktop computer, it is easy to open two applications on the same screen, but in the case of mobile devices, this is not always possible and causes some inconvenience. When using Skype, a remote scene can be viewed on a modern TV screen. In this case, the robot's movement is controlled by any gadget, for example, a smartphone. Autodocking feature All mentioned robots have the function of automatic installation in the dock from a distance of about a meter. They do this by recognizing the image of the docking station, and it requires good lighting. It should be noted that this function has an advertising purpose rather than a practical one since manual installation in the dock does not create inconvenience for the user compared to automatic, which also does not have 100% reliability. Watchdog timer The robot rides when you press the "forward" button and stops when the button is released. Imagine now that the Internet connection was broken before you release the button. The robot will drive without stopping. The same thing will happen when the program freezes on the computer, in the browser, or the application from which you control the robot. Such problems have long been known in the field of automatic control systems and are solved with the help of a watchdog timer. The watchdog timer works as follows: a motion-controlled controller generates short pulses (about 1 second) during which the motors operate. If the pulses disappear, the engines stop. For continuous movement, a series of pulses must come from the user's browser, triggering pulses in the controller timer if everything is working properly. In case of failure of any link in the transmission path of the triggering pulses, the engines stop after 1 sec. A similar system can have several levels of hierarchy and is called a multi-level watchdog timer. Compatibility with various gadgets When choosing a telepresence robot, you should pay attention to which operating systems and brands of gadgets it can work with. For example, some robots work only with iPads and do not work with Android-based tablets, some robots cannot be controlled from desktop computers. Video zooming Magnifying the video allows you to see small details, for example, the text written on the blackboard. The magnification can be digital and optical. The maximum magnification in the digital version is determined by the maximum resolution of the camera. The most effective and reliable optical magnification, which is realized with the help of a system of lenses and micromotors, although it costs quite a lot. All the robots in question have the magnification function, but it works best for the Double 3 robot. Cameras switching Switching from the front camera to the rear and back is not only convenient to improve the view of the surrounding space, but also makes it possible to use the rear camera of the tablet, which has a much higher resolution than the front. Recording and saving videos and images Video and photo recording is one of the rare functions of the telepresence robot, but it can often be replaced by third-party programs that record video from the gadget screen and take pictures of it. It is useful to record, for example, a lesson at school or a report at a conference in order to view it again later. Screen sharing The screen demonstration allows you to open any file (photos, videos, drawings, text) on the user's screen and show it on the robot screen. This is a rare feature and it is not implemented in all the robots. Conclusion Telepresence robots differ significantly in their characteristics and price, so before purchasing a robot, you need to clearly understand why you need it and, depending on this, choose a robot with the necessary set of technical characteristics. Also published . here