Milliseconds Make Millions: How and Why to Speed Up Your App, from InDrive's Playbook

Hello everyone, my name is Vladimir Pchelyakov, I’m an iOS engineer at inDrive. In this article, I will talk about how app startup time affects company revenue, how we sped up our app, and I will cover the tools, approaches, as well as real situations and of course the results. This article can serve as a plan for any team that wants to speed up the app launch time and the user's critical path, because the problems we solved most likely exist in most unoptimized projects. For context, in the inDrive app you can order a taxi as a passenger, or you can take orders and do rides as a driver. In this article, I will cover: How startup time affects company revenue Types of app launch App Launch Time, TTI, and Hang Time Average value and percentile What affects startup time App acceleration cycle The most effective ways to speed up Tools, processes, situations Results How startup time affects company revenue Types of app launch App Launch Time, TTI, and Hang Time Average value and percentile What affects startup time App acceleration cycle The most effective ways to speed up Tools, processes, situations Results How Startup Time Affects Company Revenue Most people think that developers are "too obsessed with optimizations" that bring no real value. But data shows that speeding things up by hundreds of milliseconds can affect business almost as much as launching a new feature or entering a new market. Simply put, if your app takes too long to open, some users will go to a competitor or just give up on their need. inDrive Research Inside the company, we ran a large study which showed that speeding up the passenger's critical path by 750 ms increases the number of orders by 1.5%. If your app has millions of orders, just because the app started showing the interface faster, you get tens or even hundreds of thousands of extra orders per day! Deloitte Research Deloitte published some very impressive numbers: speeding up a mobile website by 100 ms gives +8.5% conversion in e-commerce, and +10% in travel. The numbers look fantastic, but this is Deloitte, and there is no reason to doubt their professionalism. Moreover, some companies deliberately slowed down the app for certain groups of users to prove the hypothesis: the faster the app launches, the more orders the user makes. And the hypothesis was confirmed. If you have always wanted to work on speeding up the app but did not have strong arguments, now you have them. Startup time directly affects company revenue. Types of App Launch The type of app launch directly affects the metric, so it is important to tell them apart. Apple does not provide a direct API for determining the launch type, so each team does it in their own way. Cold Launch Full loading of the app from scratch, the slowest type of launch. The app process was not present, there is no cache, everything is initialized from scratch. For example, this is the first launch after downloading from the store, or after several hours or more after the app was unloaded from memory. Cold Launch is the only honest metric that can be compared from version to version to draw conclusions. Throughout this article, all measurements will be done exclusively on Cold launches. Warm Launch The app was recently terminated, so no process exists. However, parts of the app remain partially in memory — system caches and the dynamic linker cache are still warm. The system must spawn a new process and reinitialize the runtime, but skips much of the I/O work required in a cold launch, making it noticeably faster. This happens when you unload the app from memory and almost immediately open it again, it opens noticeably faster. This metric depends too much on many factors, so it does not work for us. Hot Launch The app was simply minimized and opened again. There is nothing to measure here, the app opens instantly and does not go through any path, it is already in the right place. Prewarm iOS itself launches the app in advance "in the background", predicting that the user will open this app soon. It sounds useful, but it causes a lot of problems: we had "launches" of 15 minutes and 10 hours, because prewarm created the process in advance, and we calculated the time from a variable that was created in the main file, and the user did not launch the app right away but after tens of hours. At first we could not understand where the error was and how the launch time could be equal to hours, and it also badly corrupted the statistics, because other data is measured in ms, and metrics with hours really messed up the picture. Be careful, Prewarm can completely ruin the statistics. App Launch Time, TTI and Hang Time When we started working on speeding things up, it turned out that even basic terms were understood differently by everyone. So we had to develop common definitions. App Launch Time The time from tapping the icon to the appearance of the first frame of our launch controller. Important: this is not about the system Launch Screen, which appears instantly, but about our root view controller — the moment when library linking is done, AppDelegate methods have finished, the view controller is created, and the viewDidAppear method has fired. TTI (Time To Interactive) The time from tapping the app icon to the moment when the user can interact with the app. You can also call this the critical path, because it affects the start of creating or processing an order, and it is what affects business metrics. This path is different for every app, since everyone has a different final point, and it is chosen based on its impact on the business. For example in our case: For the passenger — when the map appears and the user can tap "Where to go". The faster this happens, the more likely the passenger will create an order. For the driver — when the screen with passenger orders is displayed. The faster this happens, the more likely the driver will see and take an order in time. Hang Time This is the time when the interface "freezes" — a button is tapped, but the action does not happen instantly, for example it takes a second. Usually this is the result of heavy computations on the main thread. And yes, Hang Time directly affects TTI, because freezes appear not only when the user taps a button, but also when something on the critical path occupies the main thread and the UI freezes during rendering. For example, a screen slides out not in 300 ms, but in over a second. Average Value and Percentile After collecting data from millions of users, you need to somehow interpret these metrics, and there are at least 2 options: average value and percentiles. Why the Average Does Not Give the Full Picture The average seems convenient and easy to calculate, but sometimes it is hard to draw conclusions from it. Here is an example: We have 5 users and the launch times for 5 users = 1, 2, 3, 9, 12 seconds. Average = 5.4 sec. But this does not reflect reality: 3 out of 5 users launch the app in 3 seconds or faster. 2 users are very slow and skew the entire statistics. And if one launch is recorded as "1 hour" (or infinity comes due to a metric error), the average becomes completely meaningless. This is where percentiles come to the rescue. How Percentiles Work and Why They Are More Accurate A percentile shows how much time the app needs so that X percent of users fit within that time. On the same 5 values: 1, 2, 3, 9, 12 50th percentile = 3 seconds → half of the users launch the app in 3 sec or faster. 80th percentile = 9 seconds → 80% of users see the app in ≤9 seconds. 99th percentile = 12 seconds → the "heaviest" users: weak devices, poor internet. We look at the 50th, 75th, 90th and 95th percentiles. In addition to the main mass of users, we always try to speed up the 95th percentile, to improve life for people whose app loads the longest, most likely due to the device or poor internet in their country. By the way, I will note that we operate in 47 countries around the world, and we have many countries where many users have weak devices and poor unstable internet. What Affects Startup Time Let us look at the main areas that can significantly slow down the app. Network requests The number of requests in the chain, the order of requests, the ability to cache, internet speed. App Initialization At startup, a huge number of operations happen: library linking, launching Objective-C runtime, Swift Reflection, SDK initialization (analytics, ads, Firebase), loading dylib, dyld, rebase, symbol binding, DI container initialization. And all of this happens before the first frame. Main Thread Any heavy operations on the main thread increase TTI: JSON decoding of large and complex models, synchronous calls, unnecessary computations "at startup" because it was convenient, even though they can be moved somewhere else. Device Characteristics We cannot speed up old devices, but we can adapt the logic for them or reduce the amount of load. An interesting fact: for countries with weak devices and slow internet, Meta released the Facebook Lite app. The Facebook Lite app is small, allowing you to save space on your phone and use Facebook in 2G conditions. App Acceleration Cycle To speed up the app, you need to systematically measure and improve, not "fix things by eye". Our cycle looks like this: Mark up the app for data collection App Launch Splash TTI of all verticals (critical paths) Steps of the critical path Collect and visualize data Raw telemetry data Long-term storage, Big Query Monitor Xcode Organizer Find places of slowdown Instruments: LaunchTime, Profile, Network, Hangs Timeline analysis Fix problems Restructure requests Cache Offload the main thread Remove everything unnecessary from startup Run experiments Compare results, verify hypotheses Control startup time before a new release UI-performance tests on CI, alerts on degradations Mark up the app for data collection App Launch Splash TTI of all verticals (critical paths) Steps of the critical path App Launch Splash TTI of all verticals (critical paths) Steps of the critical path App Launch Splash TTI of all verticals (critical paths) Steps of the critical path Collect and visualize data Raw telemetry data Long-term storage, Big Query Raw telemetry data Long-term storage, Big Query Raw telemetry data Long-term storage, Big Query Monitor Xcode Organizer Find places of slowdown Instruments: LaunchTime, Profile, Network, Hangs Timeline analysis Instruments: LaunchTime, Profile, Network, Hangs Timeline analysis Instruments: LaunchTime, Profile, Network, Hangs Timeline analysis Fix problems Restructure requests Cache Offload the main thread Remove everything unnecessary from startup Restructure requests Cache Offload the main thread Remove everything unnecessary from startup Restructure requests Cache Offload the main thread Remove everything unnecessary from startup Run experiments Compare results, verify hypotheses Control startup time before a new release UI-performance tests on CI, alerts on degradations UI-performance tests on CI, alerts on degradations UI-performance tests on CI, alerts on degradations This way, from version to version the app becomes faster, more stable and more predictable. Of course, markup and visualization are not done in every cycle. The Most Effective Ways to Speed Up The biggest effect comes from a few obvious but systematically done steps: changing the order of network requests caching network requests offloading the main thread removing heavy computations at startup changing the order of network requests caching network requests offloading the main thread removing heavy computations at startup These are exactly the changes that gave us the maximum TTI improvement with minimal effort and risk of breaking something. I would recommend everyone to start with this. Tools, Processes, Situations In this chapter I will tell you about our tools and walk through several real examples, in which I will show how one tool complements another and what each of them looks like. Optimization is impossible without measurements. First you need to mark up the project. Inside our app, the key launch stages are marked: AppLaunchTracker — app launch until the first controller appears. SplashTracker — time of different splash screen stages (getting feature toggles, profile, determining location). CriticalPathTracker — the user's critical path with step breakdown and time between them VerticalsTTITracker — TTI for each product vertical. AppLaunchTracker — app launch until the first controller appears. AppLaunchTracker SplashTracker — time of different splash screen stages (getting feature toggles, profile, determining location). SplashTracker CriticalPathTracker — the user's critical path with step breakdown and time between them CriticalPathTracker VerticalsTTITracker — TTI for each product vertical. VerticalsTTITracker All data is visualized in charts and dashboards (Kibana, Redash). Additionally, we have established processes: version comparison by performance UI-performance tests on CI alerts on degradations regular weekly meetings and a roadmap for speeding up version comparison by performance UI-performance tests on CI alerts on degradations regular weekly meetings and a roadmap for speeding up We use native tools: Xcode Instruments — LaunchTime, Profile, Network, Hangs Xcode Organizer — Hangs, App Launch Xcode Instruments — LaunchTime, Profile, Network, Hangs Xcode Organizer — Hangs, App Launch I also recommend creating a roadmap to understand the order of task execution. Ours looks like this: Now let us look at everything in action. Working with Network Requests Order One of the most effective ways to speed up TTI is to rearrange the order of network requests and run independent requests in parallel. For analysis we used Xcode Instruments → Network, where you can clearly see what requests are going and their dependencies. This is faster than digging through the codebase. What we did: ran the specific flow captured a trace of network requests overlaid the moments when screens appeared on it ran the specific flow captured a trace of network requests overlaid the moments when screens appeared on it This way it became clear which requests actually block the UI from showing. After that we went to the product team, showed the findings, discussed how to optimize the order of network requests, and then created tasks. Simple Case A request that the first screen depended on was sent too late. It could be moved to the beginning of the screen lifecycle, because it actually had no dependency. As a result, the user would see the ready interface sooner, which means faster TTI. On Android we found a classic "staircase": toggles → driver config → reasons → offers All requests went strictly sequentially. Only after that did the loading of orders begin. Solution: Combine several requests into one or run requests in parallel if there are no dependencies. The simplest and most effective way is to remove the artificial sequence where it is not needed. There is a chance that developers wrote this sequence long ago for convenience or other reasons, and it stayed that way because it did not bother anyone. A More Complex Case: Driver feed Originally, before loading orders, a chain of several requests was executed: general taxi settings taxi driver settings courier settings general taxi settings taxi driver settings courier settings All of them went sequentially, and each request slowed down TTI, especially on slow internet this was noticeable, since each of the three requests increased the total path time. We got together with the team and significantly simplified the scheme: independent requests were launched in parallel сombined two settings requests into one request the request for orders started right after receiving the minimum necessary data. And only in rare cases would we need to wait for a parallel request, but even then, it was launched in parallel with the first one, not from scratch after the first one. independent requests were launched in parallel сombined two settings requests into one request the request for orders started right after receiving the minimum necessary data. And only in rare cases would we need to wait for a parallel request, but even then, it was launched in parallel with the first one, not from scratch after the first one. As a result: instead of three sequential requests, there was one in the best case in the worst case — one fewer request than before instead of three sequential requests, there was one in the best case in the worst case — one fewer request than before After that we ran an experiment. Experiment result: TTI acceleration of 0.5–0.8 seconds on average at the 95th percentile — up to 2 seconds TTI acceleration of 0.5–0.8 seconds on average at the 95th percentile — up to 2 seconds Timelines Charts by version show that the app became faster or slower, but do not answer the question why. Without them we had to guess why we were getting such a result. To solve this problem we use timelines — a breakdown of TTI by stages. That is, we collect all the main stages of the critical path: app Launch delegate methods getting toggles getting location getting profile navigation to the module module initialization screen rendering app Launch delegate methods getting toggles getting location getting profile navigation to the module module initialization screen rendering This allows us to see precisely: which stage sped up where a regression appeared what exactly affected the final TTI which stage sped up where a regression appeared what exactly affected the final TTI It is timelines that allow us to safely make large changes and understand their consequences. This also became the basis for decision-making and finding regressions or places for acceleration — the longest pieces interest us the most. I highly recommend you implement timelines. This is a very good level of detail, but plan them to be detailed and high-quality right away, because changing them later will be inconvenient, and comparing charts with a previous breakdown will be impossible. App Launch Report and Removing Typhoon Xcode App Launch Report shows what code slows down the launch. In our case: The DI container Typhoon took ~30% of launch time several of its initializations consistently showed up in the top slowest places The DI container Typhoon took ~30% of launch time several of its initializations consistently showed up in the top slowest places After removing Typhoon: The first stage of launch was reduced by almost 3 times. From ~1.3 seconds to ~445 ms in version 5.122. On the timeline you can see how the first stage sped up 3 times, but the overall TTI became larger. If we did not have the breakdown, we would not have understood what was happening and how removing DI could slow everything down like that. But it turned out that along with the speedup, a bug made it into this version. I will talk about it in the next section. Hang Rate Now we can see how one tool complements another — timelines gave a picture that the launch sped up, but further something broke and affected several initial stages. The next tool that helped us — Xcode Organizer Hangs. It showed that the new version of the app started slowing down significantly. Something was blocking the main thread. The tool shows that 92% of hang time was coming from a third-party SDK. Moreover, it points to the specific line. The reason — a network request from a third-party library on the main thread with a server response wait of up to 10 seconds. The problem only reproduced on slow internet, so it went unnoticed by our testers and developers, since everyone has fast internet on laptops and devices. We filed an issue with the library, and after the fix, Hang Rate returned to normal. Research From time to time we profile the main user path by running it on different devices, and it looks roughly like this. Here we found micro-freezes, and we immediately see which stages of the critical path they affect. It turned out that certain code for working with user location was heavily loading the main thread. We fixed the problem, random micro-freezes disappeared, and the Hang rate became even lower. Also, the profile fetching stage was reduced from 480 ms → 21 ms. UI Performance Tests on CI Finding problems and solving them is good, but even better is to learn about a slowdown before the app goes to the App Store, and even before the version regression starts. We made a native UI test that goes through a real scenario on the production server and sends the same metrics as for the production version. It works like this: goes through the real user path to the key screen runs at night runs several times without network mocks sends metrics to telemetry goes through the real user path to the key screen runs at night runs several times without network mocks sends metrics to telemetry If the time exceeds the threshold — an alert comes to Slack. This gives the ability to catch a problem before release, which is currently the most important element of the entire system. We also see all the same timelines, and the test is as close as possible to real usage conditions. Also, to be sure that our caching works well, we added internet throttling on CI — first the test runs at normal speed, everything is cached, then the same thing but on 3G slow internet. In my opinion, this is one of the most important components of working on startup speed — not letting such a version go to release. Next we want to run this on every Pull Request asynchronously, so as not to block the merge, since performance tests run on the same Mac Mini to keep measurements honest, and with a large team there would be a queue. But we can do this after the merge and understand which PR broke the startup time, so we can quickly revert changes. Version Comparison Before adding a new dependency to the project, we have a version comparison process — an instruction for the developer on how to compare the current version of the app and the one with the new dependency added. Once during such a run we saw a 250 ms increase at app startup and decided to roll out this feature only temporarily and only to a portion of users, so as not to slow down the app. This process also gives an understanding of the reason for version slowdown in advance. Toggle Caching Let us look at the last case — looking at the timelines, one of the longest stages on our path is getting feature toggles (700 ms), the black bars on the chart. We implemented progressive caching: if the toggles are fresher than a certain time — we use the cache. Then we update the data asynchronously. Naturally, we run an A/B experiment which shows improvement. Result: the toggle stage was reduced in versions 126–127 TTI became noticeably faster users who open the app frequently and those with slow internet benefited the most. the toggle stage was reduced in versions 126–127 TTI became noticeably faster users who open the app frequently and those with slow internet benefited the most. The chart shows driver toggles. It became so small because they open the app very frequently. Traffic Lights and Quick State Control When there are too many metrics, you need aggregates. We use "traffic lights". They use complex aggregation formulas per day, automatic evaluation of improvements and degradations. I will not describe all of this now since it would take a lot of time, but globally it is all for a quick answer to the question: "what happened with the version". For example, on one of the versions there was a visible exceptional boost in TTI — the result of offloading the main thread. Final Results Let us look at our results over half a year of work. From the beginning of the year to September (versions 115 → 140): the time from app launch to transition to the product vertical (50th percentile) was reduced by almost 3 times: 2.8 sec → 1 sec The share of users with launch faster than 5 seconds: 58% → 84% For 75 percent of users the app launches in 4 sec, and it used to be 6.2 sec. At the same time, improvements happened gradually from version to version, except for one bug spike, which we quickly caught thanks to the tools.The percentage of users who run the app on cold starts faster than 5 seconds. Conclusion Speeding up the app is not a one-time optimization, but a continuous process: clear metrics understandable timelines automatic control before release regular analysis and experiments clear metrics understandable timelines automatic control before release regular analysis and experiments And the most important thing — speeding up by hundreds of milliseconds really affects the business. Not theoretically, but in real percentages of orders.