Test Coverage for Android multi-module project

Hello there!

We are writing unit tests quite extensively for our Android project, covering domain models, use cases, domain to database mappings, data factories and ViewModels. When we create a merge request (or pull request if you come from BitBucket), we are running all the unit tests in our CI pipeline, and only if all the tests are passing we can merge this MR.

We were wondering how to improve the code and app quality and we had a couple of options like adding snapshot tests or instrumentation test, both quite time consuming to implement if we are hoping to cover all (or most of) the app. So, talking about covering the app we found what we were looking for, test coverage. Creating test coverage reports to know what we are missing to test seems like the quickest way to improve our testing while we start planning to add another testing strategy. If you have a single module project that will be straightforward, but we have a multi-module project, and it was a challenge to setup everything and here I will share how to setup an Android multi-module project with Kotlin and Android libraries to create test coverage reports for every module and merge all of them in a single report that will show the overall test coverage of our project.

Create test coverage reports for every module

We have different feature modules that might be divided in domain, data and view modules, so for featureFirst we could have featureFirstDomain, featureFirstData and featureFirstView. Domain modules will be Kotlin modules, with no Android related logic at all. Data and View modules are Android libraries. To keep our sample project simple, we will have domain and view modules.

To create the test coverage reports we will use JaCoCo Gradle plugin. We can start creating a new gradle file, in our sample it will be in the folder gradleCommon where we will have all our gradle configurations. We can name this file jacoco-config.gradle.

The first thing we will do in this file will be the creation of the JaCoCo configuration. We can start with the configuration for the Kotlin modules. In those modules the task jacocoTestReport will be available automatically when applying the JaCoCo plugin therefore the configuration will be easier as we just need to set the type of report we would like to generate for the module and the classes we will be including/excluding in the report.

We are going to generate HTML reports, just in case we would like to check the code coverage for single modules and we are excluding Dagger generated classes. In the sample project we are not adding Dagger or Hilt dependencies but it is the perfect sample of classes we would like to exclude from our reports.

Next step is the creation of the configuration for the Android modules. The jacocoTestReport task won’t be available out of the box just applying the JaCoCo plugin therefore we have to create it.

As we did for the Kotlin configuration, we are going to generate HTML reports for the Android modules as well. The filters will be more extensive, excluding classes from Android, DataBinding, Dagger, Kotlin and some UI classes that we will not be covering with unit test but that we can include in case we have instrumentation tests. For the next lines we have the real difference between Kotlin and Android modules. In the Kotlin modules, JaCoCo knows where to find the classes and the source directories but it doesn’t for Android modules and we have to define those manually. Also we have to define the location of the execution data, the .exec file that JaCoCo creates automatically with the information about the code that has been covered with tests.

Now that we have the configuration to generate the coverage reports for each module type we need to find a way to filter the Kotlin modules from the Android modules. We can do so checking if the module applies the android.library or android.application plugins.

The final step will be the configuration of each module.

We have our project ready to generate test coverage reports. We can run ./gradlew testDebugUnitTest to create all the unit test reports and then ./gradlew jacocoTestReport to generate all the JaCoCo test coverage reports for every single module.

One report to rule them all

If we would like to get accurate metrics of the overall test coverage we need to create a coverage report that merges all the reports created at each module. This is the step where I struggled the most and I could not find a proper updated solution. There was a 3rd party plugin that did all the job almost without configuration but this was deprecated some time ago because of Gradle changes and none of the other solutions I could find on the Internet was working for me therefore I decided to explore the JaCoCo and Gradle documentations and try to achieve a solution by myself.

To create a single report we will start creating a new task in the project level gradle file.

In this task we are going to define the variable project to store all the modules that contain the task jacocoTestReport. Our task will depend on jacocoTestReport task because without running this task we have no reports to merge.

Now that we know all the modules that potentially will have a report, we need to define the source and class directories. We are going to relay on our jacocoTestReport task to get this from us. Same applies to the executionData.

You might be wondering why we need to define all those directories. This is because the way JaCoCo stores the coverage. We talked about .exec files in the previous section, the file that stores the coverage information. This information knows nothing about line numbers. We need the class files to generate XML reports as the .exec file will be applied on the class files and combined will produce the number of lines of code that are test covered. For HTML reports we need the sources as well.

Going back to the script, there is a chance that a module has the jacocoTestReport task but it doesn’t produce a coverage report. To prevent a crash in our generation task we have to filter the executionData list and rebuild it with the files that actually exist. The last bit is the configuration of the reports we would like to generate, in this case we are going to create CSV, XML and HTML. Running ./gradlew jacocoFullReport will create the combined report in our root build folder or in the path we have specified in the report configuration.

Enforce test coverage

We can do a last thing to enforce that we are keeping the test coverage to a minimum. JaCoCo has a task where we can set some rules.

In this task we are setting the minimum coverage on 85% and that the build will fail if test coverage is less than this number. Running ./gradlew jacocoCoverageVerification will do the check.

Wrapping up

You can find a demo project in this GitHub repository. I hope you found this post useful and if you have some feedback or question feel free to drop a message.

Thanks for reading!