gfxinfo + UI Automator + Kotlin = Automated Jank Tests

Introduction

Automated tests are king. Reading any modern programming book like Working Effectively with Legacy Code and it is obvious to see why coders just can’t get enough. Usually, the main focus of our automated tests is on the correctness of our business logic. However, there is one form of automated testing that is maybe too oft neglected and that is the measuring of your app’s Jank. Jank is the term that is used when your app drops frames usually due to poor performance. Analogous to how unit test provide quick feedback about correctness, wouldn’t it be great if we had performance test that provides quick feedback about Jank (or hopefully, lack thereof)? Here is a post on how we can combine Kotlin, UI Automator (or any UI automation tool) and gfxinfo to create such a test.

Let’s ensure that our users are happy by actually monitoring our app’s performance!

Measuring Jank

Since Android 23+, measuring Jank has become a whole lot easier with the increase of information that is now reported by gfxinfo. dumpsys gfxinfo is a shell command you can use on the device via the command line. It provides you a dump of the rendering information of your current application instance and included in that dump is your app’s Jank info.

You can run gfxinfo on Android API 23+ via:

adb shell dumpsys gfxinfo example.jank.performance.myapplication

This outputs the Jank information (which is a subset of the total output) like so:

Total frames rendered: 82189Janky frames: 35335 (42.99%)90th percentile: 34ms95th percentile: 42ms99th percentile: 69ms

With this terminal command, we now have an easy way of measuring our app’s Jank. It should be noted that gfxinfo doesn’t tell you why your app ran slow or what to do to fix it. You have to use a tool like Systrace to find the root cause of the issue. However, this is perfect for our use case as we are building our test simply to identify regressions. Actually fixing the regressions is another blog post for another time…

So now we have we can easily measure our app’s Jank. The next part of the test that must now be built is the instrumentation part. In this post, UI Automator will be our UI testing tool of choice.

Device Instrumentation Prep

Now we need to set up our devices to be ready for instrumentation. We won’t actually ran any tests yet (that’s the next step). The goal here is to get the device under test into a state such that we can easily start our UI automation tests via the command line. Android’s UI automation tool is perfect for this.

Setting up and using UI Automator is out of the scope of this post. However, it is pretty straightforward. You can follow these instructions for setting up your project with UI Automator.

Now, let’s get our device under test ready. First, you should create both your regular and test apks via Gradle the tasks:

./gradlew assembleDebug assembleDebugAndroidTest

Now you want to deploy them to the device:

adb install -r debug.apkadb install -r debug-androidTest.apk

That’s it. Now you’re device should be completely prepped for our use of UI Automator. Starting the test which can be done via the command line like so:

adb shell am instrument -w -r -e debug false -e class example.jank.performance.myapplication.BasicTest example.jank.performance.myapplication.test/android.support.test.runner.AndroidJUnitRunner

So now we have our UI Automator test cases and can easily execute them from the command line. From the previous section we also have a way of measuring our app’s Jank from the command line. Now if only there was a cool new language to glue these two concepts together….

Enter Kotlin!

We will use Kotlin to now build a simple script that executes the UI automation tests while, simultaneously, querying the Jank information of the device via gfxinfo. Out script then outputs a subset of the results of the last gfxinfo query in both JSON and XML formats. (Lets be honest, you could really use any language here but I needed find a way to legitimately include “Kotlin” in the blog post name with the hope that you would actually read it. :) )

Due to the verboseness of the script, I have embedded the actual script itself at the bottom of the blog post. Instead, here is an activity diagram which outlines each of the high level steps that the script is performing.

Activity diagram documenting script steps

And That’s It!

We can now use Kotlin to leverage gfxinfo during our automated UI testing to quickly identify any regressions in Jank so that your users don’t have to. There is still a lot left to be done here, however, hopefully this is a great starting point for getting you thinking about integrating this into your CI for the ultimate Android Jank telemetry set up. For more in depth information about the various performance testing tools at your disposal, take a look at this Google IO /17 Session.

Want to see this in action?

I have created a sample project which applies the concepts talked about in this blog post here: (Specifically, you can execute the exampleTest.sh bash script to see of all these steps in action on an Android 23+ device)

akperkins/automated-jank-test-example_automated-jank-test-example - An example repo on how we can combine Kotlin, UiAutomator and gfxinfo to create an…_github.com

But what if you’re team uses a different UI automation tool?

Not everyone uses UI Automator for their UI automation tests. However, this method should work with any UI automation tool that can be started via the command line. It would just require that you replace the UI Automation steps with whatever steps you would normally use to prep your device under test and then hand off the shell command that you would use to start your UI automation tests to the Kotlin glue script.

Do you have already have automated Jank test running? If so, then feel free to reach out to me and share some tips you use to effectively avoid regression Jank!

Example Kotlin Jank Measuring Script:

Happy Coding :)