Agents of Assistance

A simple framework to understand value and complexity in consumer IOT products

This article was written in collaboration with Brent Engels and Shelly Walbert.

For consumers, meaningful products exceeded expectations and positivity shape behavior and emotion.

As IOT products, systems, and networks slowly become more integrated into our lives and embedded into our environment, our expectations for these products is simple: we want them to do stuff for us — to make our lives easier, safer, and more efficient. And for these products to sustain, they must do it just as good or even better than we can and with minimal input. Today, there might be a few IOT products approaching this state of meaningful, sustainable utility, but unfortunately most simply fade away. This is largely due to the fact that they simply transpose user tasks from a physical one to a digital or verbal one. We should strive to create products and services that go beyond simply assisting with tasks and instead take agency over tasks.

Going forward, the most meaningful systems will be ones that infer our behaviors and desires while requiring minimal user intervention.

Such products will have the authority to act on our behalf and be more that assistants to us — they will be agents for us.

What follows is a simple model for thinking about and creating meaningful IOT consumer products through the lens of assistance and agency. The model breaks down products into simple categories that build upon each other, increasing in value, but also complexity.

Baseline (dominant) Device:

The Baseline Device is the existing, previously “dumb” or mechanical product being displaced. These are the existing light switches, thermostats, speakers, ovens, etc. These physical devices and infrastructure require explicit user input or action to control. The reward of this input is instantaneous and feedback is explicit. Flick the switch and the light turns on. They tend to be simple, but effective.

These Baseline Devices have evolved over decades and there are residual codes and behaviors embedded in these objects that signify trust, safety, and utility. So the bar to replace these simple, but effective devices is tremendously high.

Smart Device

Smart Devices are a networked, controllable version of the baseline device. Their offering to users is the ability to be controlled remotely via the the network. Smart phones, by providing a universal control surface to connect and control these new devices, have opened up the market for these devices. While these devices provide value through their remoteness and novelty, the task management is mostly just a transposition of offsetting a physical task with a digital one. Significant advantages of these devices, however, comes from the ability run programmed schedules and provide two-way feedback. Scheduling allows users to “set and forget” and be notified if the system detects an error. While the complexity of initial set up varies, most of these Smart Devices on the market today continue to operate reliably and deliver ongoing service. There are many examples of these devices on the market today from Philips Hue to Nest and Amazon Dash buttons.

Extensible Devices

Extensible Devices expand the capability of Smart Devices by allowing them to connect to 3rd party systems. Open APIs provide the benefit of allowing control through third parties systems and devices. This information and control sharing can be two-way: Extensible Devices can become sensors on the network that share data and they can receive commands. Working together, these Extensible Devices can work together to execute basic connected scripts such as “recipes” in IFTT, voice control of third party services via Amazon echo, and the networked smoke detectors of Nest. The value of the Extensible Device network is limited by ability to program the system to work together. While fairly simple command and control capability via 3rd party devices is becoming more prevalent, customization still requires significant technical capability on the part of the user.

Smart and Extensible Devices can be thought of as “Assistants” because they assist the user in the act of doing the tasks associated with baseline devices, but they still require the user to explicitly program and control the system.

They assist the user by performing repeatable tasks or tasks that benefit from remote execution. These devices assist us, but they don’t have the ability to act on their own. Just as the Baseline Devices do, they still require some form of user input or command. We can tell Alexa to dim the lights. But she never knows to do it on her own. No matter how many times we tell her, she will never do it without our command.

What differentiates agency from assistance is the autonomy and authority to act on a users behalf, without the need for user input.

By making decisions on behalf of the user, agency truly alleviates of a particular task or job, not just transposition it from physical to digital. This creates a complete cognitive relief for the user. They no longer have to think about or execute the task. Whereas Assistant Tools focus on automating a task, Smart Agents focus on results and beneficial outcomes for the user.

Smart Agents

The key differentiator between Smart Agents and Assistant Devices is the authority to make decisions on behalf of the user. These decisions can be based on personal preference or personal and community safety. A common example can be found in autonomous vehicles with crash avoidance. Crash avoidance systems have the authority to override user’s cruise control settings because to make sure the driver is safe. High frequency trading systems have the ability to analyze markets and place trades to reach a desired outcome without any user interaction. And we can already see this type of intelligence making it’s way into the home with features like automatic load sensing in washing machines that have the authority to regulate the amount of water to maximize efficiency in a load. Rather than simply following a programmed script like Smart Devices, future Smart Agents will empowered to perform actions based on learned preferences, desires, and outcomes.

Agent Systems

With Agent Systems, networks of Smart Agents work, negotiate and act together on a user’s behalf through open systems. Through the power of these systems working together, much of life can be automated toward desired outcomes. Agent Systems could be as complex as autonomous, coordinated navigation systems or simply ovens and dishwashers working with food delivery systems to preheat, cook, and washing for optimal efficiency and performance without any user intervention. Independent home systems could coordinate lighting, temperature, and security automatically when the household goes to bed as opposed to requiring explicit user commands to achieve similar results. Transportation, entertainment, safety, and education all have the potential to adapt and transform the user environment to meet their personal needs and desires.

Important things to consider:

Value and complexity are directly correlated in this model — they go up as we move from Assistant Tools to Agent Tools. For Smart Agents to have the authority to carry out a task requires user trust. Because this trust is paramount, they must perform flawlessly — continuously sensing, learning, and adjusting to a user’s preferences and cycles. They need realtime operations and fail-safe connectivity. All these factors increase the complexity of Agent Tools considerably.

By being a “representative” of a user, any Agents must make the user are aware of decisions, but importantly, not require explicit confirmation to act. By being aware of Agent actions taken on a user’s behalf, a mutually beneficial transparency and trust can grow between the system and user. The user can understand and correct actions by the Agent and the Agent can learn user preferences and exceptions. The simplest way to do this is with some sort of timely, but not intrusive, notification method coupled with a reversible “undo” feature.

It is always important to consider, plan, and build for how these systems will fail. Failing gracefully should be an important requirement for any IOT system and integral to the design. The hierarchical model provides a simple, yet effective blueprint for failsafe measures. The best practice is to degrade gracefully down the value chain: if an agent system fails, revert to a preprogrammed scheduled of assistance. If assistance fails, the user must still be able to control the devices manually, so the baseline functionality is required. Nest is a great example of this principle. If it loses network, it still runs its automated schedule. While the user cannot control the thermostat remotely in this case, the user can still simply override the program and use Nest like the the baseline, standard thermostat.

The eventual trajectory of autonomous agents is a world is configured to our personal preferences — actively working towards personal safety, efficiency, and opportunity without any user interaction.

Hopefully, this model provides a clear way to think about current and future IOT products, roughly outlines technical complexity, and emphasizes meaningful value for consumers.