Building My First VUI by@tjwells47

Building My First VUI

Terrence Wells HackerNoon profile picture

Terrence Wells

Insights from a summer of learning about conversation design

UPDATE: Unfortunately, we didn’t place in the challenge but we are thankful for the opportunity and proud of what we created. Here are the results

This summer, Joseph Cochrane-Brown and I created Sow Study Aid for the Action On Google Challenge. Our goal was to use Quizlet’s API to create a captivating flashcard study experience through conversation. Before we started, we knew nothing about Bots, VUI’s, or API.AI. Here’s what we learned.


We wanted our persona to mimic the experience of studying with a friend. So, Joseph and I took notes while helping each other study flashcards. A few characteristics stood out that helped us hone in on what makes for a friendly study session.


We discovered that the main challenge was to keep the user interested and motivated while being efficient during the study process. After scripting some demo conversations, we were able to prioritize the top 3 key elements of our design.

1. Personality

The difference between a fun study session and a boring one, is the personality of the person you’re studying with. Unfortunately, it’s not as easy as bottling up the personality of a charismatic study partner.

Some of the ways we crafted the persona of Sow Study Aid were by adding contextual gifs or soundbites based on whether you got the card right or wrong, placing random witty remarks in well traveled parts of the conversation, and by using casual or familiar language to help set a relaxed atmosphere.

Unfortunately, creating a likable personality for a bot is extremely subjective. Attempt at your own risk. But, the reward of nailing the persona of a friendly study partner was too great for us to pass up.



2. Conditional Responses

Technically, every point in a conversation for a bot is a conditional response. To us, a conditional response means thinking through the ways a possible change in context could alter the tone and wording of responses.

The simplest example is when you get 2 cards wrong in a row, Sow Study Aid occasionally adds to the response string,

“That’s 2 in a row. Let’s turn this around!”

That’s a very minute change that doesn’t seem to register the same way a ground breaking feature does. But, adding those conditional responses throughout the application will drastically improve the conversational experience.

3. Intuitive Intents

The notes that we took on our demo study sessions, laid out an obvious set of intuitive intents / features which created the foundation of our application. Some examples include reviewing the last 4 missed cards, navigating through the deck, and switching decks at any time.

When studying, there’s a few things that you naturally expect your partner to do. Those are the features that we focused on making available, and adding enough response variations for them to be invoked naturally.


To keep from frying our brains, we went to the library and used wall-sized whiteboards to map out the conversation visually. Not having to keep track of a mental map of complex context and inputs, freed us up to be more creative with how we laid out the conversation.


Despite all that whiteboard real estate, we initially broke up the conversation into 3 simple sections: the introduction, topic selection, and study process. We simplified the structure of the conversation in a way that allowed us to break down the points of emphasis and most traveled paths of the conversation in ways that a script wouldn’t allow.

In each section, we subdivided further into intents and contexts. After intents and contexts were developed, we created variables that kept track of the things that a study partner would keep track of in the back of their head, like the cards that you’ve missed.

Lastly, we stored those variables in Firebase so the next time the app is invoked, all the information from the last study session is available to be picked up where the student left off.






Nothing would have been possible without the videos, articles, and example applications that we found while building Sow Study Aid. Here’s a curated list of anything and everything that was helpful for us.

Conversation Design: Speaking the Same Language_Six principles of human conversation poised to revolutionize Voice User Interface design The next generation of…

Deploy your Assistant app fulfillment webhook using Cloud Functions for Firebase._A quick analysis of why you should use Cloud Functions for Firebase for your Assistant apps and a walkthrough deploying…

Voice Design Google I/O Talks



In Conversation, There Are No Errors (Google I/O ‘17)

Finding the Right Voice Interactions for Your App (Google I/O ‘17)

Getting Your Assistant App Discovered (Google I/O ‘17)

Applying Built-in Hacks of Conversation to Your Voice UI (Google I/O ‘17)

Building Apps for the Google Assistant (Google I/O ‘17)

Building Rich Cross-Platform Conversational UX with API.AI (Google I/O ‘17)

Github Examples

actions-on-google/interactive-fiction-nodejs_interactive-fiction-nodejs - Interactive Fiction sample for Actions on

actions-on-google/apiai-trivia-game-nodejs_Contribute to apiai-trivia-game-nodejs development by creating an account on

actions-on-google/apiai-number-genie-nodejs_apiai-number-genie-nodejs - Number Genie API.AI sample for Actions on

Voice Design Medium Articles

What Every Designer Should Know About Voice_The rise of voice interfaces will change the way we design

SSML for Actions on Google_An important part of designing great conversation actions for the Google Assistant is thinking about how you want them…

Google Assistant Trivia Game_Today we open sourced a trivia game app for the Google Assistant with 3 personas, 750 prompts and 43

Interactive Fiction Actions (Part 1)_Google launched Actions on Google to allow you to write your own conversational actions for the Google Assistant, which…

Please feel free to contact me with any feedback, thank you!

TL;DR: A friend and I built our first bot over the summer. Conversation design is cool and here’s some links to help you when you build things.

react to story with heart
react to story with light
react to story with boat
react to story with money

Related Stories

. . . comments & more!