Moving Beyond Animations to User Interactions at 60 FPS in React Native

presentation is everything

The async nature of the React Native bridge incurs an inherent performance penalty, preventing JavaScript code from running at high framerates. Modern animation libraries, like Animated, address this by minimizing passes over the bridge. User interactions, where UI continuously reacts to the user’s gestures, are a step further. How can we run those at 60 FPS?

Crossing the last mile

React Native has a lot of appeal as the stack of choice for modern mobile apps. The major advantage this framework offers is a dramatic increase in productivity. Simply put, you develop apps much faster — partly due to the fact you can finally share code between platforms.

There’s always a concern though. Will React Native be able to take me across the last mile? Will the app that I’ll produce measure up to top of the line apps that are implemented in pure native?

I have to admit that this concern is valid. At Wix.com, we switched our mobile stack about a year ago from a purely native one, with separate codebases for iOS and Android, to React Native. The first 95% of development was a breeze. We found ourselves moving forward at about 4 times our previous pace. The last 5% though, was a bit more of a challenge. We found that these 5%, what I call the last mile, is still not straightforward to implement with React Native.

It is our goal as community to improve that.

So what makes great apps?

What are the nuances that set the best apps apart from the mediocre ones? In mobile, we’ve grown to expect that objects no longer just pop up on screen. Things are expected to move around in smooth transitions.

Fluid animations at 60 FPS are an important part of the last 5%. Animations used to be a big issue in React Native. This issue was ultimately resolved using Animated, the excellent animation library that’s part of the core.

Let’s look at the next step beyond animations — dynamic user interactions that mimic reality. An interaction takes place when the user performs a gesture on a view, and this view continuously responds to the user’s gesture with physical realism.

Let’s look at some real life examples to better understand what we’re talking about. I went through my personal phone and started cataloging examples of great interactions in some of my favorite apps:

UX Inspirations

• ListView row actions — On the left we have the official iOS Mail app by Apple and the Inbox by Gmail app from Google. As the user swipes rows in the ListView, buttons for row actions gradually appear from the side.
• Swipeable cards — Second from the left we have the Google Now app by Google and the Flic app by Lifehack Labs which has Tinder-like UI. As the user is swiping, these cards modify their appearance and if swiped with enough force, fly off screen.
• Collapsible views — Second from the right we have Airbnb and the Cal app by Any.DO. Both of these have views that the user can collapse between multiple states. Switching between filter and search in Airbnb and switching between month-view to week-view in Cal.
• Sliding panels & drawers — On the right we have the official iOS top notification panel by Apple and the official iOS Maps app by Apple. The user can drag these panels to reveal additional UI elements that are normally hidden. Much like the popular navigation drawer / side menu.

What do these examples have in common? They are all physical in nature. The views have velocities that are changing as they’re being dragged and tossed. Notice the nuances, like how the notification panel bounces from the ground when thrown with enough force.

Implementation with JavaScript

When using React Native, we would naturally try to implement these interactions in JavaScript. Let’s review such an implementation. The first inspiration example — ListView row actions — is actually implemented in React Native core under the name SwipeableRow.

It has a modern implementation with all the latest and greatest. It puts emphasis on performance and makes heavy use of the Animated library. Let’s focus our attention on the part that implements the interaction itself:

The implementation relies on a PanResponder to calculate the changes to views between touch events. What performance should we expect from this approach?

To analyze performance, we’ll have to look into React Native internals. React Native has two realms running side by side: the JavaScript realm — where we implement our business logic, and the native realm — where our native views reside. Communication between the two realms takes place over the bridge. Since serialization is required to send data over the bridge, frequent communication is expensive.

Touch events are a native construct, they originate in the native realm. For every frame of the interaction, these events are sent over the bridge to be handled by _handlePanResponderMove in the JavaScript realm. Once the business logic calculates the response, an Animated Value is set. Since updating views has to take place in the native realm, we have to cross the bridge one more time.

As you can see, every frame requires data to be serialized over the bridge. If your app is busy, you’ll find that this performance overhead will prevent the interaction from running at 60 FPS.

Implementation with native

While working on the Wix app, we originally started implementing all interactions with JavaScript. When performance wasn’t as smooth as expected, we started porting specific use-cases to native.

This meant implementing everything twice — once in Objective-C for iOS and once in Java for Android. It’s usually easier to reach 60 FPS with a native implementation because we can avoid passing data over the bridge and close the entire loop, business logic and views both, in the native realm.

Since we open source almost all of our native code, we’ve ended up with multiple libraries like react-native-swipe-view that implements swipeable cards and react-native-action-view that implements swipeable row actions. Without a general purpose solution, every new use-case results in yet another custom-tailored library.

The main problem with this approach is that it requires native skillset and usually two different developers. At Wix, we maintain about 10% of our frontend workforce as native engineers with expertise in Objective-C/Swift or Java for this purpose.

This is not good enough. We should aim higher and try to find an elegant general purpose solution.

Learning from animations

Animations actually present a very similar challenge. The naive implementation would tween view properties between frames in JavaScript. This would generate a lot of noise over the bridge and result in frame loss. As we know, the library Animated emerged as the solution to deal with animations at 60 FPS in React Native. How does it work?

The concept behind Animated is using a declarative API to describe animations. If we are able to declare the entire animation in advance, then the declaration in JavaScript can be serialized and sent over the bridge once. From that point on, a general purpose driver will execute the animation frame by frame according to the spec in the declaration.

The original driver for Animated was implemented in JavaScript. The recent versions, though, provide a native driver that is able to execute the animation frame by frame in the native realm and update the native views without going over the bridge.

This method reduces traffic over the bridge to the initialization phase only. This brings us to an interesting conclusion:

Declarative API is how we cross the last mile

This is a very powerful concept. These are the sort of libraries we should be thinking about. Whenever we find a performance boundary in React Native, this is a way to push against it. All we have to do is find several example use-cases and design a declarative API that can cover all of them. That’s exactly what we’re going to do next.

Declarative API for user interactions

In order to design a successful API we should define a couple of goals:

1. Our API should be general purpose. A good way to verify that is make sure it covers all 8 examples we saw in our UX inspirations above.
2. Our API should be simple. A good way to verify that is make sure every interaction takes no more than 3–5 lines of code to define.

Before we jump into the specifics of our API, I want to mention some very interesting work going on in Animated aimed at adding some support for user interactions. One interesting addition is Animated.ScrollView which allows to perform view property interpolations based on ScrollView position. Another interesting work in-progress is a library by Krzysztof Magiera named react-native-gesture-handler which allows to perform view property interpolations based on gesture parameters.

The approach we’re going to take together now is a little different. We’re going to start from the 8 UX inspirations shown above and design the simplest high-level API that can define all of them.

Defining the API — phase 1

Analyzing our 8 , we can see that some of the views are free to move horizontally and some are free to move vertically. Therefore, specifying the direction is a good start for our API.

Another observation is that the views are only free to move while dragged. Once the user lets go, they usually snap to one of predefined snap points. Drawers, for example, snap either to an open position or a closed position.

Lastly, to give the snap behavior a realistic feel, we need to use something like a spring animation curve. If we don’t want the spring to oscillate forever, we should also specify in our API the friction (or damping of the spring).

In summary, the first phase of our declarative API can rely on the props:

• horizontal / vertical
• snap points
• friction

Let’s try to use this simple API to declare the first two UX inspirations — ListView row actions (left) and swipeable cards (right):

In order to allow the swipeable cards to be swiped away, we simply define snap points that are completely off-screen (-360 and 360 logical pixels). Note that we currently use pixel values for simplicity. We can add support later for units that are better suited for multiple screen resolutions — such as percentages.

This is a great start, but designing the declarative API is only the first half. The second half is implementing the native driver. Let’s do that next.

Implementing the native driver — attempt 1

After the specs of the interaction have been declared as props in the JavaScript realm, they are serialized by React Native during initialization and sent over the bridge once to the native realm. Our general purpose native driver will receive these specs and drive the interaction entirely from native. There will be no more passes over the bridge required to calculate each frame, resulting in overhead-free execution at 60 FPS.

Let’s start with a simple implementation in Objective-C. We’ll drag the view by using a UIPanGestureRecognizer and when the pan gesture ends, we’ll find the closest snap point and animate our view to it with a spring curve:

This implementation works well enough. The problem with it is that we’re faking the physics with animation. Consider what happens when the view is tossed by the user at some initial velocity. The animation function we’re using can only apply velocity in the direction of the spring. What happens if the user tosses the view in another direction? Our model isn’t powerful enough to drive this case.

Implementing the native driver — attempt 2

Let’s look into more powerful models to drive the interaction. If you dive into the native SDKs and check how Apple recommends implementing complex interactions with physical realism, you’ll run into UIKit Dynamics.

This crazy API was introduced in iOS 7. It runs a full fledged physics engine under the hood and allows us to apply physical properties like mass, velocity and forces to views. The physical parameters of the scene are defined by applying behaviors. We can easily modify the implementation above:

We’re getting closer, but we’re still not there. Using UIKit Dynamics has two major drawbacks. First, there’s no Android support. This API is exclusive to iOS and there’s no parallel in the Android SDK. Second, some behaviors, such as snap, don’t provide enough control — there’s no way to specify the strength of the snapping force for example.

Implementing the native driver — attempt 3

Let’s get a little bit crazier. Why not try to implement UIKit Dynamics by ourselves? At the end of the day, the physical forces are relatively simple mathematical equations. Building the physics engine from scratch shouldn’t be too difficult.

UIKit Dynamics will show us the way. We can even adopt its behavior pattern. Let’s take the snap behavior for example — we can implement it using a spring. How does a spring behave? Time to recall some Physics 101:

Don’t worry about the math too much, this is something the library will do internally. The Wikipedia entries for Newton’s laws of motion and Hooke’s law can provide you with the full background.

We’ll have to calculate the forces and velocities on every frame. To set this up, we’ll need a high precision timer running at 60 FPS. Luckily, there’s native API designed specifically for this task — CADisplayLink. Putting it all together will yield the following:

Now this feels right, and brings us to a very interesting realisation…

We’re writing a declarative physics engine for React Native

And this is pretty damn cool.

We finally have the native driver under control. It’s time to make use of our powerful engine and add some more capabilities to our declarative API.

Enriching the API — more props

The declarative API we have so far provides a solid foundation but still lacks ability to implement some of the more intricate interactions in our 8 UX inspirations. Consider the official iOS top notification panel by Apple. When the user throws the panel with enough force downwards, the panel bounces off the ground.

We can easily add support for this behavior to our declarative API. We’ll limit the view’s movement with boundaries and add bounce from the edges:

Let’s consider another complex use-case, this time involving ListView row actions. Some rows don’t have action buttons on both sides. When this is the case, the common UX behavior is to allow the row to move freely in the direction exposing the buttons, but when moved in the other direction, movement will be more difficult and meet increasing resistance.

We can add resistance to the row movement by tying one of its edges to the screen’s edge using a constant spring. Unlike snap points, this spring will also be active while dragging.

We still need to sort out another issue. The row should move left without resistance (this direction exposes buttons) but move right with resistance (the direction without buttons). We can add this behavior to our API by giving every force, such as our spring, an optional influence area.

When the view is outside the influence area, the force will disappear.

As you can see, as we meet more and more use-cases we can simply enrich our declarative API and add general purpose abilities to describe them.

Enriching the API — integration with Animated

We’re still missing a large piece of the puzzle. Consider the ListView row actions use-case. As you swipe the row, the action buttons gradually appear from beneath. A common pattern is to change their appearance, like scale and opacity, as they are revealed.

You can see this behavior below (the action buttons in blue):

Also note that the views that we want to animate (the blue action buttons) are different from the view the user is interacting with (the gray row cover).

This effect isn’t trivial to implement because the stage of the animation depends on the horizontal position of the row instead of timing. Nevertheless, this is still an animation — where view properties (scale and opacity) are modified in sequence. We already have a powerful tool for animating view properties at our disposal — the Animated library. Let’s find a way to use it for our purposes.

View property animations with Animated are performed declaratively by defining interpolations over an Animated.Value:

Since the animation depends on the horizontal position of the row, what if we transmit the position into an Animated.Value? This will allow us to define interpolations based on the interactable view’s position that affect other views that aren’t a direct part of the interaction (like the buttons).

How would that work in our declarative API? We can specify this behavior by passing the Animated.Value as a prop (animatedValueX):

Our native driver will perform the actual transmission under the hood. This can be accomplished by using Animated.events. Recent versions of Animated even support driving Animated.events using a native driver. This means that the entire animation — from transmitting the position to interpolating and updating the view properties — can be performed in the native realm without sending data over the bridge. This is great news if we’re aiming at 60 FPS.

Enriching the API — finishing touches

If we’re doing our own physics, we might as well add the rest of the forces. We already have springs, let’s add gravity and magnetism too. This will provide developers with the flexibility needed to define all sorts of crazy physical interactions.

We should also add support for events, so our JavaScript code could be notified when an interaction stopped or when the view snapped to a point. And while we’re at it, adding haptic feedback is also a nice touch — so the device will vibrate slightly whenever a view collides with its surroundings. These nuances add the polish required for a great user experience.

Time to wrap things up…

I want to show you the full power of what we’ve created here. Take a look at the following declaration, can you guess what it implements?

Our mystery view snaps either to the left or right edges of the screen. There’s a gravity well in the bottom, which sucks the view inside when it gets too close. Also, notice that we don’t limit movement and allow the view to move in both directions.

What we have here is a full chat heads implementation in 7 lines of code!

Does it really run at 60 FPS?

Seeing the videos isn’t the same as experiencing the interactions by yourself on a real device. Note that even the simulator doesn’t provide the real experience as it drops frames.

So, on a real device, does it really run at 60 FPS? Judge for yourself. I’ve implemented all 8 with the engine we’ve just created using our declarative API in React Native. You can find the resulting demo app on the Apple App Store (iOS) and Google Play (Android).

Download the demo app

The full implementations of the physics engine, our native driver for iOS and Android, and the demo app are available on GitHub:

wix/react-native-interactable_react-native-interactable - Experimental implementation of high performance interactable views in React Native_github.com

Special thanks to Rotem Mizrachi-Meidan and Tzachi Kopylovitz for helping bring this home in time for ReactConf 2017.

Crossing the last mile

I hope you’ve taken from this interesting experiment more than just a cool way to implement great user interactions in React Native. Our goal as a community is to identify the boundaries of React Native and then to push against them.

When you stumble upon an interesting performance problem in React Native, I urge you to find several example use-cases and try to design a simple declarative API that can define them. If the performance problem stems from bridge overhead (as is usually the case), a native driver for your API will probably provide a good solution.

Let’s cross the last mile together.