5 Lessons a Software Engineer Learned about Movie Making

I have 2 teammates at Google who are Oscar winners.

Tom Lokovic and Eric Veach both were Google Engineers when they received 2013 Oscar Awards for their work at Pixar on the movie Monster Inc.

Tom Lokovic and Eric Veach won 2013 Sci-Tech Awards for their work in the Pixar movie Monster Inc. on deep shadow rendering (remember Sully’s hair?).

Tom once told me about the Pixar tradition that babies born during the movie production would be included at the end of the movie, and his kids always loved watching Monster Inc. and seeing their own names.

That gave me a dream that one day, as a Software Engineer, I can make an animation movie, and watch it with my kids together.

For years, I have not gotten anywhere close to that dream.

Until last month.

In February, Leap.ai decided to make a video to illustrate Leap’s Instant Match feature. We were short on time, and short on hands, so I got a unique chance to closely work on it. I joked with my team that I served as Assistant to Director during the process. (Note that Assistant to Director is drastically different from Assistant Director.)

This is the process I experienced.

1. Select a video production team from many candidates.
2. Initial kickoff with the video production team and explain what we want to do and the main idea behind it.
3. Script pitch by the video production team to illustrate several versions of different storylines, and we provide feedback and finalize.
4. Actor audition and selection.
5. Video shooting. This took an entire day, in a house rented from Airbnb. I got to observe how the director, camera crew, mic crew, and supporting crew work together, with a close-up observation how actors work.
6. Rough cut.
7. Voice actor audition and selection. Background music selection.
8. Voice recording. This is done over a Zoom call with the voice actress being remote. So I don’t fully know what equipment is needed here.
9. Post-production including animation, editing, and sound mixing.

Here’s the final video.

I learned a lot during this process. Below are my top 5 lessons.

Lesson #1: Set up camera positions is like engineering

While watching the video shoot, the part I enjoyed the most is observing how cameras were set up in different angles, knowing that later they will be pieced together.

Take a look at the video from start to 0:16 mark. How many camera positions can you find?

The answer is 3.

• The first position is at the door (not visible in the video). Chris walked from the door towards the fridge.
• The second position is set close-up into the fridge, only capturing how Chris’ hand reached in and grabbed the takeout box.
• The third position is right next to the stairs, focusing on Chris while he stands next to fridge and talks while eating Chow Mein.

Different camera angles for the same shooting.

This is very much like software engineering, and I can easily relate to it.

• In movie making, the director needs to think about the different scene breakdown before hand, directs the crew to shoot from the different positions, and finally mixes the scenes together to create a cohesive clip.
• In software engineering, we break down a service into different components during design phase, then implement the different components separately, and finally integrate them together for the final product.

Lesson #2: Taking each shot is not like engineering

With each position set, the crew shot the acting many times. The same line, the same motion, with different variants. Again and again. The hope is after many recordings, different angles are covered and one of them is good enough to be included in the final video.

Many things could go wrong during a shot. The following is only a partial list of what I experienced that day:

• Someone started mowing lawn right next door. -> We gave them money to leave then and come back later.
• Wind blew up the cover on the window and caused extra light to shine into the camera. -> We asked two crew members to go outside and manually hold the cover.
• Fridge compressor kicked off. -> We disabled the fridge compressor, but kept the fridge power since the light still needs to turn on when the door is opened by Chris.
• Airbnb house owner banged on the door. -> We smiled and did whatever the owner asked us to do.

The shot at 0:20 mark was taken at least 15 times, and I felt sorry for Chris. Oh remember that Chris was eating Chow Mein? He finished nearly 2 takeout boxes during the shooting. He definitely didn’t enjoy eating cold Chow Mein for that much. :)

I was told by one crew member that once he had experienced a shot being repeated 80 times, when a (famous) director was in a bad mood.

As a software engineer, repetition is probably one of the most hated things. Whenever something needs to be repeated, we automate it into a script (pun intended) and let the machine runs the script. I’m glad I don’t need to do this repetition part day to day.

Lesson #3: What feels normal in real life feels too slow in video

There were several occasions during the video shooting, the director asked the actor to do a motion right after the previous move. Watching the live action, I felt the motion was rushed and not natural. When I asked the director about it, the director told me “trust me, you won’t feel that way in the video”. Indeed, when the video came out, that rushed feeling in live action was the right pace in video.

Example: Chris stopped speaking “I don’t have time to chase interviews” at 0:27 mark, and started to grab the laptop at 0:28, with only one second of delay. In real action, it was very unnaturally fast, but in the video, you don’t feel that way.

The same goes for audio recording. Listen to Athena’s dialogue from 0:32 to 0:59. When I was on the call with the voice actress recording this piece, I felt she was talking quite fast. But after the video is finally made, we all felt this part could be even faster.

Why is this? My hypothesis is: When we are watching video, our brain is more focused, and therefore absorbs information faster than when we are in real life. If any reader knows the answer, please leave me a comment. I’d love to know.

Original source: NewScientist

Lesson #4: Post production takes a lot of work

After video is shot and audio is recorded, it’s very hard (or nearly impossible) to change these materials. The next step is to make animation to fit with the video and audio materials.

In our video, the clip from 1:00 to 1:12 mark has only 12 seconds of animation, but it contains a lot of information. It needs to demonstrate how our product works, and at the same time, does not go into every single detail of the product. It needs to have the right balance between feeling real and feeling abstract. We also need to decide the right visual style. Font / color / opaqueness, all needs to be decided and experimented with.

Font / Color / Opaqueness were experimented with in this screen.

It took a lot of work and many iterations.

Lesson #5: Have fun

Ultimately this video production project was very fun for me. I have so many deep joy during the process, and my kids were with me every step along the way. They were huge factors in the final selection of video and voice actors, and the background music. I got pretty close to the feeling if I get to make a movie with my kids together.

No matter what project to do (and how unqualified a person is), have fun and enjoy!