Enforcing Team Rules with Lint: Tests 🧐

A few months ago, my team came upon an agreement that when leaving a TODO anywhere in our code, we need to always provide several things:

• the person who is expected to address the TODO
• date when the TODO was left
• a comment or explanation on what needs to be done

I created a live template to support adherence to this rule, but why not go one step further and integrate the rule into our daily workflow?

We have previously seen:

• the foundations we have started
• what detectors are and how to write them

Having tests for our custom Lint rule is really important. We do not want Lint to flag errors, uhm, erroneously. It results in frustration and users might just turn off our rule.

Serving up files 💁

Lint checks run on files, so for each of our test cases we need to provide mock files.

Lint provides a TestFile API that allows us to create these mock files inline.

TestFiles.java(
    """
        package test.pkg;
        public class TestClass1 {
            // In a comment, mentioning "lint" has no effect
         }
    """
)

TestFiles.kotlin(
     """
        package test.pkg
        class TestClass {
            // In a comment, mentioning "lint" has no effect
        }
    """

I am using raw strings so that I do not have to worry about escaping special characters.

It also gives us very nice syntax highlighting!

Furthermore, if you choose “Edit Kotlin Fragment” from the hint, Android Studio will open up a file editor. Any changes you make in this editor will immediately reflect in your TestFile. Pretty cool!

Let’s get testing 🔬

The gateway to our test cases is TestLintTask. We need to provide it the following:

• files we want to run our checks on
• the issue we are testing against
• expected result

@Test
fun testKotlinFileNormalComment() {
    TestLintTask.lint()
        .files(
            TestFiles.kotlin(
                """
                    package test.pkg
                        
                    class TestClass {
                        // In a comment, mentioning "lint" has no effect
                    }
                """
            )
        )
        .issues(TodoDetector.ISSUE)
        .run()
        .expect("No warnings.")
}

Note that we have the option here of using expect("No warnings.") or expectClean().

For the test cases where we expect an error to occur, we need to put in the text that Lint spits out (i.e. similar to what you see in the console when running .gradlew :app:lintDebug). The trickiest thing about this is that the string has to match exactly, including where the squiggly lines are.

The easiest way to do this is to pass an empty string to expect() and let the test fail. You can then copy-paste the error message into your test.

Retrieving the message for an error scenario

I wrote a few tests for the detector covering Java and Kotlin files, incorrect date formats, and “TODO” casing. You can find them all here.

Bringing it all together 🤝

Now that we have written our tests, it’s finally time to integrate our Lint rule into our app!

First we need to create our IssueRegistry to let Lint know about our custom rule. We also need to provide an API value; for custom rules we can use the constant defined by the Lint API.

@Suppress("UnstableApiUsage")
class IssueRegistry : IssueRegistry() {
    override val issues: List<Issue> = listOf(
        TodoDetector.ISSUE
    )

    override val api = CURRENT_API
}

We then register our IssueRegistry by creating a file with the fully qualified name of our file under a META-INF/services/ folder:

src/main/resources/META-INF/services/dev.zarah.lint.checks.IssueRegistry

Most posts mention that adding this registry is enough, and I am probably doing something wrong but I found out that for my project, I still have to include my IssueRegistry in the Manifest by adding this to my build.gradle.kts file:

tasks {
  jar {
    manifest {
      attributes(
          "Lint-Registry-v2" to "dev.zarah.lint.checks.IssueRegistry"
      )
    }
  }
}

We have set up everything and now it’s time to consume our custom lint rule! In the modules where you want the rules applied, add a lintChecks entry in the dependencies closure, build your project, and everything should be good to go!

dependencies {
    lintChecks project(':checks')
}

Seeing it in action 📽️

Finally! We have come to the end of our journey! Here’s our detector in action:

Custom lint rule with quickfix

The source code for this Lint series is available on Github (diff).

I have learned so many things whilst I was doing this task.

As I mentioned on Twitter, there’s barely any public documentation at all. The talks I’ve seen are way too advanced for me (for example, I needed to understand what PSI and UAST were before it clicked and most talks barely even define them).

There was a lot of trial and error, and so. much. guesswork. It was incredibly frustrating. Major props to Mike Evans who patiently guided me through the hours and hours of pair programming that we did. If he didn’t help, I wouldn’t even know where to start. Sure I can copy-paste from samples but I want to understand why things are done the way they are – that’s how I learn.

I probably have made wrong assumptions whilst writing these posts but again, no documentation so this is the best I could do.

Anyway, what I want to say is, I usually only see the final output of other people’s work and sometimes I cannot help but feel jealous – how come they find it super easy whilst I’m out here crying because I don’t understand anything? I needed to remind myself that it’s okay to be frustrated, it’s okay to take your time to learn, and it’s okay to ask for help.

Further reading (and/or watching) 📖

If you’re keen to learn more, here are some resources about Lint and custom Lint rules:

• Coding in Style: Static Analysis with Custom Lint Rules, Android Dev Summit 2019 (Alan Viverette, Rahul Ravikumar)
• Kotlin Static Analysis with Android Lint (Tor Norbye)
• Lint API source code
• AndroidX documentation
• Android Lint Checks demo (Alex Lockwood)
• lint-dev Google group

Enforcing Team Rules with Lint: Detectors 🕵️

A few months ago, my team came upon an agreement that when leaving a TODO anywhere in our code, we need to always provide several things:

• the person who is expected to address the TODO
• date when the TODO was left
• a comment or explanation on what needs to be done

I created a live template to support adherence to this rule, but why not go one step further and integrate the rule into our daily workflow?

In this post, we build upon the foundations we have started.

Now that we have our module set up, we can start writing our detector.

As mentioned previously, detectors do the heavy lifting for our custom rule. To do so, it has to fulfil several roles:

• Look for the relevant locations
• Find issues, if any, in those locations
• Report back any found issues to the user
• Suggest fixes for issues, if possible

We will look at each of these roles in turn.

Coming to terms with Lint

Before we dive into detectors, it is useful to understand a bit of Lint lingo. This took me a really long time to understand and it was really frustrating. To say that there is no documentation at all is not a lie, there is no one place I can link to. A lot of the information here I sort of cobbled together from all the talks, posts, and hours and hours of research on Lint.

When dealing with detectors, most things refer to something called UAST or PSI. For illustration purposes, let’s take this sample file from Florina Muntenescu’s gist:

package com.zdominguez.sdksandbox

/*
 * Copyright (C) 2018 The Android Open Source Project
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

import android.graphics.Typeface
import android.text.TextPaint
import android.text.style.MetricAffectingSpan

/**
 * Span that changes the typeface of the text used to the one provided. The style set before will
 * be kept.
 */
open class CustomTypefaceSpan(private val font: Typeface?) : MetricAffectingSpan() {

    override fun updateMeasureState(textPaint: TextPaint) = update(textPaint)

    override fun updateDrawState(textPaint: TextPaint) = update(textPaint)

    private fun update(textPaint: TextPaint) {
        textPaint.apply {
            val old = typeface
            val oldStyle = old?.style ?: 0

            // keep the style set before
            val font = Typeface.create(font, oldStyle)
            typeface = font
        }
    }
}

PSI

PSI, or Program Structure Interface, is traditionally used by IntelliJ to model source files via elements. The PsiViewer plugin is really useful in helping you visualise what this means.

I like to think of PSI as the blueprint of the file. It shows each and every single element, including whitespaces and braces (it can even tell you if it’s a left brace or a right brace!)

PSI for the `CustomTypefaceSpan` Kotlin file

PSI for an XML file

UAST

UAST, or Universal Abstract Syntax Tree, was created by Jetbrains to describe Java and Kotlin syntax trees. A syntax tree shows the hierarchical structure of our code, illustrating all the rules and constructs that the code will follow via nodes.

Partial UAST for the `CustomTypefaceSpan` Kotlin file

When dealing with UAST, we do not really care if we are looking at a Java file or a Kotlin file. Instead we see the logical branches in our code.

PSI vs UAST

It took a while for me to wrap my head around these concepts, and I finally sort of understood it with with an IKEA analogy. UAST is like the photos you see on the catalogue. It describes what the furniture is – a chest of drawers with white knobs, a shelf, it has a specific kind of door. PSI, on the other hand, is like the assembly instructions for that furniture. Get this plank, put a screw in here, a bolt there.

Just get on with it, Zarah

Let’s go ahead and make our detector:

import com.android.tools.lint.detector.api.Detector

@Suppress("UnstableApiUsage")
class TodoDetector : Detector() {
}

Detector is the base class of all detectors. It is marked as @Beta, so I added the @Suppress annotation there to confirm that yes, I know this might break.

Detector is pretty generic (except for ResourceXmlDetector), and we can get much more out of Lint if we can indicate what specific detector we need. There are a few specialised interfaces we can use depending on what type of thing we need to run our rule on:

For the purposes of our TODO detector, we will need to implement the SourceCodeScanner interface since we want to look at Java and Kotlin files:

@Suppress("UnstableApiUsage")
class TodoDetector : Detector(), SourceCodeScanner {

}

Being specific

There is another term that most talks on Lint mention a lot, and that is “visit”. I haven’t seen an exact accurate definition of the term, but from what I gather this is what we call the act of Lint getting to a particular UAST node or PSI element.

This means if we want to look at usages of a method for example, we need to “visit” methods and figure out if the issue exists there. If we want to write a rule that checks constraints in a layout file, then we probably need to “visit” XML attributes and values.

There is, indeed, a proper way to be specific in Lint about what locations we want to be visited. The Detector povides several methods for this. Note that MOST of them start with get but not all of them do.

Available getApplicable* methods

All of these methods are available to you regardless of what kind of *Scanner interface our detector implements as they are all in the base Detector() class. We need to make sure to implement at least one of these methods to signal to Lint that we care about those locations.

Since we want to look at comments, the best fit for our usecase is getApplicableUastTypes().

Next we need to specify what specific kind of UAST node we are looking for. There is a UComment UAST type which sounds like exactly the type we need.

There is one caveat though – if we look at the generated UAST for our sample file, UComment does not make an appearance at all! However, we do see via PsiViewer that a PsiComment is present.

If we traverse the UAST backwards (upwards?), we eventually get to UFile which encompasses all the possible places that a comment can reside in. Perfect, let’s go ahead and use that as the UAST type we care about.

@Suppress("UnstableApiUsage")
class TodoDetector : Detector(), SourceCodeScanner {
    override fun getApplicableUastTypes(): List<Class<out UElement>> {
        return listOf(UFile::class.java)
    }
}

Receiving callbacks

Now that we have told Lint what kind of location we care about, we need to tell it to let us know if it encounters that location.

Again, there are numerous options available to us. Note that MOST of them of them start with visit, but not all of them do.

Available visit*** methods

In fact, judging by the names, none of these match what we need. Since we have selected that we care about UAST types, what we need is actually called createUastHandler(). Our next task is to create this handler:

override fun createUastHandler(context: JavaContext): UElementHandler {
    return TodoScanner(context)
}

A very important side note

It is important to remember that each of the getApplicable* methods map to a corresponding visit* method.

Let’s take this example from the ADS 2019 talk on Lint. We want to create a rule that prevents users from calling Log.wtf() anywhere in an app. In it, they are overriding getApplicableMethodNames() so the corresponding callback method to override is visitMethodCall().

Similarly, if we want to look at XML attributes we would override getApplicableAttributes() and the corresponding visitAttribute() callback. Alex Lockwood demonstrates this in this sample project.

It is extremely important to double check that you are using the correct combination because Lint will not tell you if are doing it incorrectly (i.e. everything will still compile). I haven’t found any documentation on the mapping of getApplicable* against visit* but this what I gather from my experiments:

Implementing the scanner

I felt it worth the effor of understanding the relationship between get* and visit* methods because that same principle applies when implementing our handler.

When we defined our detector, we told Lint that we want to watch for UAST type of UFile and thus we must implement the appropriate callback in our scanner:

class TodoScanner(private val context: JavaContext) : UElementHandler() {

    override fun visitFile(node: UFile) {
        
    }
}

For each UAST type we provide in getApplicableUastTypes(), we should also have the corresponding the visit* implementations. I will not enumerate them here because there are a LOT but this time Lint actually helps you out!

For example, if we say:

override fun getApplicableUastTypes(): List<Class<out UElement>> {
    return listOf(UMethod::class.java, UClass::class.java, UFile::class.java)
}

We would need to implement visitMethod, visitClass, and visitFile in our scanner. Lint will tell you at runtime if it encounters a particular UAST type but cannot find the appropriate callback.

If like me you have no idea what the possible U-values are or how a file’s UAST would look like, it is quite challenging to figure out what to do at this point. Unfortunately there is no UastViewer that I know of, and the only way to explore this realm is through good old-fashioned println.

override fun visitFile(node: UFile) {
    val nodesString = node.asRecursiveLogString()
    println(nodesString)
}

This gives you a string with the full UAST structure of the file being analysed.

Finding fault

Now that we have a better understanding of what is going on, let’s go ahead and implement the soul of our scanner:

override fun visitFile(node: UFile) {

    val allComments = node.allCommentsInFile
    allComments.forEach { comment ->
        val commentText = comment.text

        // Ignore regular comments that are not TODOs
        // If we find a TODO that does not follow the convention, show an error
        if (commentText.contains("TODO", ignoreCase = true) && !isValidComment(
                commentText)) {
            reportUsage(context, comment)
        }
    }
}

The isValid method checks if the comment follows our team rules:

private fun isValidComment(commentText: String): Boolean {
    val regex = Regex("//\\s+TODO-\\w*\\s+\\(\\d{8}\\):.*")
    return commentText.matches(regex)
}

And if it does not, report this as an issue.

Responsible reporting

When we find a comment that violates the contract, we want to give our users a clear definition of what went wrong and how to fix the issue.

Remember the anatomy of an issue we talked about in the previous post? Let’s use that knowledge to define our issue:

val ISSUE: Issue = Issue.create(
    id = "UnassignedTodo",
    briefDescription = "TODO with no assignee",
    explanation =
    """
        This check makes sure that each TODO is assigned to somebody.
    """.trimIndent(),
    category = Category.CORRECTNESS,
    priority = 3,
    severity = Severity.ERROR,
    implementation = IMPLEMENTATION
)

The last parameter required by Issue.create() is an Implementation that maps the issue being reported with the detector responsible for finding that issue.

An Implementation requires a Scope – which tells Lint what kind of files our implementation is interested in.

private val IMPLEMENTATION = Implementation(
    TodoDetector::class.java,
    Scope.JAVA_FILE_SCOPE
)

The naming is indeed misleading but JAVA_FILE_SCOPE means both Java and Kotlin files will be considered for our rule when running Lint.

This means at this point in our implementation it is absolutely critical that these information need to be compatible:
the interface we are implementing for our detector
the overridden getApplicable* method
the overridden visit* method
(since we using UAST type) the overridden visit* methods in our UElementHandler
the Scope of the issue’s implementation

A little help from my friends

We can help our users get more value out of our custom Lint rule by helping them fix the issue. Lint allows us to provide a quickfix option and users can press SHIFT+ALT+ENTER (or ALT+ENTER then ENTER) and apply the changes we have proposed.

For our detector, we want to format the comment properly.

// TODO This is an improperly formatted comment

A comment like the one above would be flagged by our detector as improperly formatted (remember we look for the TODO string). To fix it, we would need to add the user’s name and today’s date. Luckily LintFix allows us to do it pretty easily:

// Look for the instance of the "TODO" literal
val oldPattern = Regex("TODO|todo")

// Our proposed fix concatenates the user's name
// and today's date in the correct format
val replacementText = "TODO-${System.getProperty("user.name")} " +
    "(${
        LocalDate.now()
            .format(DateTimeFormatter.ofPattern("yyyyMMdd"))
    }):"

val quickfixData = LintFix.create()
    .name("Assign this TODO")
    .replace()
    .pattern(oldPattern.pattern)
    .with(replacementText)
    .robot(true) // Can be applied automatically.
    .independent(true) // Does not conflict with other auto-fixes.
    .build()

Now that we have the issue, implementation, and quickfix created, the only thing remaining is to tell Lint where to indicate that an issue was encountered. It is important to provide an accurate location because this tells the IDE where to show the error:

And where to put the red squiggly lines in the Lint report:

For this rule, it’s good enought for me to highlight the full comment. We can now finish up our reportUsage method:

private fun reportUsage(
    context: JavaContext,
    comment: UComment
) {
    context.report(
        issue = Companion.ISSUE,
        location = context.getLocation(comment),
        message = "Please make sure to assign the TODO, include today's date in YYYYMMDD format, and the comment is properly formatted.",
        quickfixData = quickfixData
    )
}

We have finally finished our detector! This has been an honest-to-goodness brain dump. Writing my first Lint rule has truly been a challenge and there are stretches of time where I was literally staring at the screen periodically yelling WHAT at no one in particular. Good thing we’ve all been working from home.

In the next post in this series, we get to write some tests and if everything goes well, we get to actually use our detector! Stay tuned!

Enforcing Team Rules with Lint 👩‍🔧

A few months ago, my team came upon an agreement that when leaving a TODO anywhere in our code, we need to always provide several things:

• the person who is expected to address the TODO
• date when the TODO was left
• a comment or explanation on what needs to be done

But memories fade and sometimes people forget. So I made a live template to make it easier for everyone to adhere to the rule. A simple ALT+ENTER and tada, there’s the TODO template for you to fill in:

Custom template in action

If you’re curious to know more about how this was done, I wrote about implementing a live template here.

Everything seemed fine and dandy, but after a few months I noticed that even though we have the template, people occasionally forget to fill it in.

We can keep on reminding people to use the template, but I don’t want to be that annoying person saying things over and over. Wouldn’t it be better if we incorporate this team rule into the developer workflow? So it got me thinking, why not write a custom Lint rule to remove the human intervention and let the tool do its job?

A great idea

Lint is a static analyser and the built-in rules are great! They help identify common problems (forgetting to call super()), possible bugs (forgetting to constrain a view), or potential optimisations (detecting overdraw) in our code and gives us the chance to correct them immediately.

These rules are flexible. We can pick and choose which ones we want to exclude from running, we can choose to fail the build if something goes wrong, we can even identify a specific issue to be ignored.

There are a lot of great talks out there that give an overview of Lint’s history, philosophy, and features especially this one by Tor Norbye from KotlinConf 2017.

Lint has been around for a while, and support for writing our own Lint checks has also been around for a while, so surely there are references out there right? There’s this good overview by John Rodriguez from Droidcon NYC 2017 and this very informational talk by Alan Viverette and Rahul Ravikumar from Android Dev Summit 2019.

They make it look easy enough. Surely I can do this.

The worst idea

Turns out that it is easy enough… if you know what you’re doing. And ooooh mama I DO NOT know what I was doing.

The Android tools site looks abandoned . The sample rules are a bit basic but aside from helping me figure out how to set things up, it doesn’t really tell me anything.

But Zarah, you say, why not just copy off of the ones in the platform? I did, and there are HUNDREDS of them. It is overwhelming to figure out which one to look at first.

I guarantee you that if someone asks me to do this again in about three months, I would have totally forgotten what I did or how I made it work (to be honest I still do not understand how it actually works ). Fair warning: I will constantly be calling out things that I do not understand or was just guessing at.

Let’s get to it.

Getting cosy with it

There are some key concepts we need to understand before diving into the code. Lint is fueled by issues and detectors.

Issues identify implementations that we want to highlight as incorrect or potential sources of bugs. Take this excerpt from a report and let us look at the anatomy of an issue from this perspective:

An issue has:

Detectors do the heavy lifting and figure out where the issues are. They can look in virtually any sort of file – Manifest? , resource file? Gradle files?

A single detector can identify any number of related issues, and can report each of those issues individually.

Note that Lint calls all existing detectors in a pre-defined order. This means that if a Lint rule needs to check something in both Kotlin and XML files (like resource usages, for example) the order in which checks are executed matters.

Setting up

Our custom rules need a home; and their home is a new module. Right click on your project and select New > Module then choose Java or Kotlin Library

In the freshly-created build.gradle file, add the dependencies for Lint. (Note: I use the Kotlin DSL)

// The Lint version is strongly tied to the Android Gradle Plugin (AGP) version
// Add 23 to the major version number of AGP (x in x.y.z)
// The project is currently dependent on AGP v4.1.1
val lintVersion = "27.1.1"

// Lint
compileOnly("com.android.tools.lint:lint-api:${lintVersion}")
compileOnly("com.android.tools.lint:lint-checks:${lintVersion}")

// Lint testing
testImplementation("com.android.tools.lint:lint:${lintVersion}")
testImplementation("com.android.tools.lint:lint-tests:${lintVersion}")
testImplementation("junit:junit:4.13.1")

The next part of our Lint journey is going to be long and treacherous. So let’s take a quick break here, make sure to watch the talks linked above and stay tuned for the next post in this series where we finally get to write our very own detector!