I Used AI To Make a Video Story Using My Short Sci-Fi Novel — Here’s What Happened

A few weeks ago, I wrote an article about how I wrote a sci-fi short story using the OpenAI text-davinci-003 language model. After publishing the article, I started to wonder if it would be possible to create a video of the story using AI.

In the first step, I generated the speech using Amazon Polly. I chose Justin as the narrator because his hypnotic voice fits the story well. The result was surprisingly good. It is completely lifelike, not the usual robotic voice.

The result was very inspiring, so I continued with the image generation. Since Midjourney is the best at generating images, that's what I used as well. Of course, it wasn't as simple as I thought. Maybe in the future, it will be enough to imagine a picture that will be read by an fMRI-like device to generate the result, but until then, generation is based only on descriptions, and creating a good description is very difficult. Producing a good image often required 10-20 attempts, but with enough perseverance, I feel that I managed to create good images.

The last step was putting together the final video. As a developer, I don't really know much about video editing, but luckily my brother does. He put together the video, timed the effects, and added the background music. I think the result is quite good.

With the help of AI, many of us have gained access to new publishing channels in the form of videos. A few weeks ago, I only wrote sci-fi short stories in my native language, and now I can create sci-fi "films" for YouTube. This is a huge step that I owe to AI. The future looks very promising. With tools like RunwayML Gen2 or Wonderdnamics, anyone can become a professional filmmaker in the future. I am looking forward to working with these new tools because I still have other stories that I would like to share with you...

You can watch the movie here: