Thanks! We'll be in touch in the next 12 hours
Oops! Something went wrong while submitting the form.

How to Test the Performance of Flutter Apps - A Step-by-step Guide

The rendering performance of a mobile app is crucial in ensuring a smooth and delightful user experience. 

We will explore various ways of measuring the rendering performance of a mobile application, automate this process, and understand the intricacies of rendering performance metrics.

Internals of Flutter and the Default Performance

Before we begin exploring performance pitfalls and optimizations, we first need to understand the default performance of a basic hello-world Flutter app. Flutter apps are already highly optimized for speed and are known to perform better than existing cross-platform application development platforms, such as React Native or Apache Cordova.

By default, Flutter apps aim to render at 60 frames per second on most devices and up to 120 frames per second on devices that support a 120 Hz refresh rate. This is made possible because of Flutter’s unique rendering mechanism.

It doesn’t render UI components like a traditional mobile application framework, which composes native widgets on the screen. Instead, it uses a high-performance graphics engine, Skia, and renders all components on the screen as if they were part of a single two-dimensional scene.

Skia is a highly optimized, two-dimensional graphics engine used by a variety of apps, such as Google Chrome, Fuchsia, Chrome OS, Flutter, etc. This game-like rendering behavior gives Flutter an advantage over existing applications SDK when it comes to default performance.

Common Performance Pitfalls:

Now, let’s understand some common performance issues or pitfalls seen in mobile applications. Some of them are listed below: 

  • Latency introduced because of network and disk IO 
  • When heavy computations are done on the main UI thread
  • Frequent and unnecessary state updates
  • Jittery UX due to lack of progressive or lazy loading of images and assets
  • Unoptimized or very large assets can take a lot of time to render

To identify and fix these performance bottlenecks, mobile apps can be instrumented for time complexity and/or space complexity.

Most of these issues can be identified using a profile. Profiling an app means dynamically analyzing the application’s code, in a runtime environment, for CPU and memory usage and sometimes the usage of other resources, such as network and battery. Performance profiling entails analyzing CPU usage for time complexity to identify parts of the application where CPU usage is high and beyond a certain threshold. Let’s see how profiling works in the Flutter ecosystem.

How to Profile a Flutter App

Below are a set of steps that you may follow along to set up profiling on a Flutter app.

  1. Launch the application in profile mode. To do so, we can run the app using the command

CODE: https://gist.github.com/velotiotech/a9ba66218ec418e6cc2c509529b7074c.js

on the terminal or set up a launch configuration for the IDE or code editor. Testing the performance of Flutter apps in profile mode and not in debug (dev) mode ensures that the true release performance of the application is assessed. Dev mode has additional pieces of code running that aren’t part of release builds.

  1. Some developers may need to activate Flutter ‘devtools’ by executing this command: 

CODE: https://gist.github.com/velotiotech/534c99566e81bdef322ab2f2212fd8f3.js

  1. To set up ‘profile mode’, launch the configuration for a Flutter app in VSCode; edit or create the file at project_directory/.vscode/launch.json and create a launch configuration “Profile” as follows:

CODE: https://gist.github.com/velotiotech/08f9b47b626d74e83e1996d0b20cc043.js 

  1. Once the application is running on a real device, go to the timeline view of the DevTools and enable performance overlays. This allows developers to see two graphs on top of each other and overlaid on top of the application. The top graph represents the raster thread timeline, and the second graph below it represents the UI thread timeline. 

⚠️ Caution: It is recommended that performance profiling of a Flutter application should only be done on a real device and not on any simulator or emulator. Simulators are not an exact representation of a real device when it comes to hardware and software capabilities, disk IO latency, display refresh rate, etc. Furthermore, the profiling is best done on the slowest, oldest device that the application targets. This ensures that the application is well-tested for performance pitfalls on target platforms and will offer a smooth user experience to end-users.

Understanding the Performance Overlays

Once the timeline view is enabled in profile mode, the application’s running instance gets an overlay on the top area. This overlay has two charts on top of each other.

Both charts display timeline metrics 300 frames at a time. Any frame going over the horizontal black lines on the chart means that the frame is taking more than 16 milliseconds to render, which leads to a frame drop and eventually a jittery user experience.

Reference screenshot from Dart Profiler for optimal rendering
Fig:- Dart profiler for optimal rendering

Look at the timeline above. No frames are going over the black lines,  i.e., no frame takes more than 16 milliseconds to render. This represents an optimal rendering with no frame drops, i.e., no jank for end users.

Reference screenshot from Dart Profiler for suboptimal rendering
Fig:- Dart profiler for suboptimal rendering

Here, some frames in the timeline above are going over the horizontal black lines, i.e., some frames are taking more than 16 milliseconds to render. That is because the application was trying to load an image from the network while the user was also scrolling through the page. This means there is some performance bottleneck in this part of the application, which can be further optimized to ensure smoother rendering, i.e., a jank-free end-user experience.

The two graphs mentioned above can be described as:

  1. UI thread: This is the first chart, and it portrays the timeline view of all the dart code executions. Instructions written by developers are executed on this thread, and a layer tree (for rendering) is created, which is then sent to the raster thread for rendering. 
  1. Raster thread: The raster thread runs the Skia engine and talks to the GPU and is responsible for drawing the screen’s layer tree. Developers can not directly instruct the GPU thread. Most performance optimizations are applicable to the UI thread because the raster thread is already optimized by the Flutter dev team.

Automatically Testing for Jank:

Profiling the app gives some idea of which screens and user interaction may be optimized for performance, but it doesn’t actually give a concrete reproducible assessment. So, let’s write some code to automate the process of profiling and detecting sources of lag in our Flutter app.

First, include the Flutter driver extension in the application’s main entrypoint file and enable the Flutter drive extension. In most cases, this file is called main.dart and invokes the runApp() method.

CODE: https://gist.github.com/velotiotech/8a9373a79c67db9927cd10765e336d4c.js

Next, let's write a Flutter driver script to drive parts of the application that need to be profiled. Any and all user behavior such as navigation, taps, scroll, multipoint touches, and gestures can be simulated by a driver script.

To measure the app’s rendering performance, we will make sure that we are driving and testing parts of the application exactly like a user would do, i.e., we need to test interactions like click or scroll and transitions like page changes and back navigation. Flutter driver makes this simpler by introducing a huge set of methods such as find(), tap(), scroll(), etc.

The driver script will also have to account for and mock any sources of latency, such as time taken during API calls or while reading a file from the local file system.

We also need to run these automated tests multiple times to draw conclusions from average render times.

The following test driver script checks for a simple user interaction:

  • Launches the app
  • Waits for a list of items
  • Finds and clicks on the first list item, which takes users to a different page
  • Views some information on the page
  • Presses the back button to go back to the list

The script also does the following:

  • Tracks time taken during each user interaction by wrapping interactions inside the driver.traceAction() method
  • Records and writes the UI thread and the raster thread timelines to a file ui_timeline.json

CODE: https://gist.github.com/velotiotech/aa4ff2e8ca9333d40cd529eb790d9557.js

To run the script, the following command can be executed on the terminal:

CODE: https://gist.github.com/velotiotech/9ea36b9f3fb67534513f6f9d8db827cd.js

The test driver creates a release-like app bundle that is installed on the target device and driven by the driver script. This test is recommended to be run on a real device, preferably the slowest device targeted by the app.

Once the script finishes execution, two json files are written to the build directory.

CODE: https://gist.github.com/velotiotech/a75085b34e5d1034e66fb69f1c92d390.js

Viewing the Results:

Launch the Google Chrome web browser and go to URL: chrome://tracing. Click on the load button on the top left and load the file ui_timeline.timeline.json.

The timeline summary when loaded into the tracing tool can be used to walk through the hierarchical timeline of the application and exposes various metrics, such as CPU duration, start time, etc., to better understand sources of performance issues in the app. The tracing tool is versatile and displays methods invoked under the hood in a hierarchical view that can be navigated through by mouse or by pressing A, S, D, F keys. 

Chrome tracing in action
Fig:- Chrome tracing in action

The other file, i.e., the timeline_summary file, can be opened in a code editor and eye-balled for performance data. It provides a set of metrics related to the performance of the application. For example, the flutter_driver script above outputs the following timeline on a single run:

CODE: https://gist.github.com/velotiotech/1ddf5eb3ce7dcb0606d0ba27564a0799.js

Each of these metrics can be inspected, analyzed, and optimized. For example, the value of average_frame_build_time_millis should always be below 16 milliseconds to ensure that the app runs at 60 frames per second. 

More details about each of these fields can be found here.

Conclusion

In this blog post, we explored how to profile and measure the performance of a Flutter application. We also explored ways to identify and fix performance pitfalls, if any.

We then created a Flutter driver script to automate performance testing of Flutter apps and produce a summary of rendering timelines as well as various performance metrics such as average_frame_build_time_millis.

The automated performance tests ensure that the app is tested for performance in a reproducible way against different devices and can be run multiple times to calculate a running average and draw various insights. These metrics can be objectively looked at to measure the performance of an application and fix any bottlenecks in the application.

A performant app means faster rendering and optimal resource utilization, which is essential to ensuring a jank-free and smooth user experience. It also contributes greatly to an app’s popularity. Do try profiling and analyzing the performance of some of your Flutter apps!

Related Articles

1. A Primer To Flutter

2. Building High-performance Apps: A Checklist To Get It Right