AI Dungeon: An AI-Generated Adventure Game by Nick Walton

The original AI Dungeon was made just over a year ago, the result of a curious gamer, a hackathon, and the GPT-2 text transformer. Fast forward to the present day, and AI Dungeon has expanded into a unique example of creative AI technology. The game now boasts 1.5 million players, multiple genres for stories, and even multiplayer adventures.

AI Dungeon is an AI-generated text adventure game in which players progress through the game by typing whatever they want. The game responds to player input thanks to a novel adaptation of GPT-2, an AI text transformer. In this way, the game offers each player their own unique game experience.

Development on the game has shown no signs of stopping or slowing down, so I reached out to creator Nick Walton to talk more. In the following interview, we look back at the progress of AI Dungeon and his plans for the future.

Can you tell us about your background in computer science and AI?

I’ve always been interested in robotics. I actually started out as a mechanical engineering major, then switched to computer science when I discovered all the cool stuff that was happening there, especially with deep learning. I ended up working in the deep learning lab at Brigham Young University.

At that point in time, I was studying computer vision and self-driving cars. I interned in Japan working with autonomous vehicles, and later at Aurora Innovation for autonomous technology.

How did AI Dungeon start?

GPT-2 had just come out and I started playing around with it at a hackathon. I realized you could do some pretty fun things with it.

I’d also recently started playing Dungeons & Dragons, and my first thought was, what if you could make an AI dungeon master? The first prototype of that idea used the smallest GPT-2 model, which was 126 million parameters. It wasn’t super great, but it did highlight for me the possibilities in larger models and more training.

Maybe a few thousand people played that game, which was AI Dungeon 1. But next fall the larger GPT-2 model was released, and I’d also found a dataset of text adventures for training. When the game was fine-tuned on that data with the larger model, we saw a massive improvement in quality.

That’s when I released the Colab version of the game, AI Dungeon 2, which exploded and really jump-started the journey to where we are now.

How do you feel looking back at the last year of AI Dungeon? Was there anything particularly challenging or difficult on the way to getting to where you are now?

It’s pretty amazing how far we’ve come from that first prototype, especially given that I couldn’t really work on it over the summer. So it’s nine months of progress.

One of the hardest challenges that we’re just figuring out now is how to take the super open-ended story of AI Dungeon and add more game-like elements such as progression and world persistence. I think that was lacking in the initial AI Dungeon.

It’s actually really hard to figure out how to integrate continuous natural language elements with discrete game-like elements, because you need to be able to say, “Okay, given the text output, what value should we update in the discrete model?” Basically, you need machine learning models that can detect the different things you want. We’ve actually just released our quest detection model, which is our first step in that direction: you have quests and the game can detect whether or not you completed the quest on each action. It uses a machine learning model to classify whether or not you completed a quest, and it works surprisingly well.

That’s our first step; we’re building it so we understand what’s happening in the story and can keep track of those elements to build to more interesting things.

How long did it take to build that particular model?

Building the infrastructure took longer than we hoped. We’re using T5, which is a very general-purpose language model. We have a way for users to label data so we can train and deploy the model, and now that we’ve built the whole pipeline and model architecture, we’re hoping it will get faster.

Two and a half months sounds quick compared to general game update schedules.

That’s one of the powerful things about using AI in games. We see two axes of improvement. One is the dynamic world, which generates content, responds intelligently, and gives the player complete freedom. The other is developer costs. In a game like Skyrim, you’re talking about employing hundreds of people over roughly five years. It’s so expensive.

Ideally, AI lets us cut down on a lot of what developers normally do. We can define the overarching themes of the game, and let the AI fill in the details.

How will you create world persistence?

There’s some fundamental challenges when it comes to world persistence. The first is that the AI model can only take in so much information in its context window each time it generates. With the original GPT-2 it’s roughly a thousand words. So the problem then is figuring out which parts of the world information are relevant.

We just released a system a few days ago where the user can define things about the world. An example of this might be defining things about a kingdom, a person, or an object. When the system sees those things mentioned in the game, it pulls in that information so it knows about it. That’s our first step in that direction.

On the other side, the system has to be able to say, given the output of this model, how should I update the world information? This is a bit trickier, and it’s where our quest model is a first step; it updates quest information based on output.

The next steps then are building out the other pieces. So maybe there’s a location model that tracks your location, or an item model that tracks what items you gain or lose. But that takes more time because we have to build out the datasets and the systems for them.

Have you been able to add extra data to AI Dungeon as you work on it?

We still mostly use the original data. Most of the improvements we’ve made have been in sampling, such as adding things like repetition penalties so the game doesn’t repeat sentences or words. Those are the primary changes, but right now we’re doing some more model experiments that have been showing some good promise. Probably in a few weeks we’ll be able to share more about our future model upgrades that we’re doing.

So is it more about a smoother game experience than having more data?

I think more data is valuable, but it’s hard because there’s only so much text adventure data in the right format. It’s also hard to create that data because it’s not something you can have just anyone make. The average person is probably a decent writer, but they’re not a great writer. Ideally you want people with the right writing skills creating that data.

We are doing some of that with datasets that fill out some of the holes in genres that aren’t very well represented. But I think what we’ll be looking at next is how to use our current set of user data to improve the model.

What plans do you have for developing AI Dungeon over the next year?

Normal games work like clockwork, so you can design the whole game and how everything will work in advance, because you can set up programming systems. We can’t really do that because that’s not how AI works. It’s not something where you can go in and tune every little gear and every little piece. It’s almost more like gardening, where you have to trim here, or prune there, and build things around it.

We’re treading new territory and it’s hard to plan exactly what we’re going to do in advance. Instead, we’re experimenting and building new things at an extremely rapid pace. Even in the last three or four weeks, we’ve made massive changes. We’ve done the world info, we’ve pushed out the quests update, and we released modding, too. Modding basically allows for people to create custom modifications for their worlds.

One of our visions for AI is not as a tool to replace game designers, but a tool for augmenting their work. We want to make it easier to create awesome games. If it only takes one or two people to build an awesome game because AI fills in the details, it opens up doors for a lot of people.

We really want to enable people to build cool things with this tech. Deploying this kind of AI training and these massive models is really hard for the average person, so our hope is that we build out the infrastructure and platform, and then let people build cool things on top of that.

I think in the long run we don’t want to just be the only guys who make a cool AI game. We’d like to be the guys who make it so tons of people can make their own cool AI games.

Would you say there’s a heavy element of R&D to your work?

We’re very much still an R&D center. We’re building out AI Dungeon, but so much of what we do is trying out concepts that no one has ever tried before. For example, we’re thinking about how to have quests that are automatically detected, but after quest detection we want to work on quest generation. So imagine you get a little notification pop-up telling you that you have a new quest, but it’s a quest that nobody else has ever had before because it’s unique to your story. That idea is really powerful because it’s an example of a world that is dynamically changing.

To try out AI Dungeon for yourself, you can find it on multiple platforms through their website. You can also keep up with Nick Walton’s work directly on Twitter.

Also published on: https://lionbridge.ai/articles/can-ai-make-video-games-how-nick-walton-created-ai-dungeon/