How to Test Your ESLint Config

We all know testing is something we should do to make our future selves happy. The continuous integration system checks all the things we instructed and we rest assured that nothing is going haywire.

Breadboard with a forest of electric wires.
Photo by Nicolas Thomas on Unsplash

By testing, we usually mean unit tests, integration tests and end-to-end tests. Looking at the testing trophy from Kent C. Dodds, we shall not forget about static testing.

This can take the form of type-checking our code, using Flow or TypeScript.

It can also take the form of using a JavaScript linter to “find and fix problems in your code”, with ESLint for example.

However, how can we be sure that ESLint will reliably inform us that some rule has been trampled?

What happens when we update some ESLint dependency providing the shareable config we rely on?

In other words, how can we go about testing our ESLint config?

The devil is in the details

In this day and age, when more and more JavaScript gets shipped to the browser despite our bundler’s tree shaking capabilities, it is important to avoid shipping unnecessary code because the bundler rightly bundled code that we did not intend to.

Let’s have a look at this example:

import dateFnsFormat from 'date-fns/format'
import * as locale from 'date-fns/locale'

export default function formatWithDefaultFrench(date, format, options) {
  return dateFnsFormat(date, format, {
    locale: locale['fr'],
    ...options,
  })
}

This code looks kind of reasonable and would likely go unnoticed in code review.

However, it has a serious problem.

The import statement just added roughly 88 kilobytes of compressed JavaScript to your final bundle. 😱

It would then seem fitting to configure our tooling to let us know when such an expensive statement was added to the codebase, so that we may rewrite it in a better way.

If you’re a VSCode user, you could use the Import Cost extension to display visual cues like in the screenshot above. But not everyone uses VSCode, and we cannot really use that on our CI/CD pipelines.

With ESLint, we could add the following rule to our ESLint config file:

// eslintrc.js

module.exports = {
  // ...
  rules: {
    'no-restricted-imports': [
      'error',
      {
        paths: [
          {
            name: 'date-fns/locale',
            message:
              "Please, import a specific locale instead, e.g. `import enUS from 'date-fns/locale/en-US';`",
          },
        ],
        patterns: ['!date-fns/locale/*'],
      },
    ],
  },
}

This configuration effectively instructs ESLint to show an error for all the imports of date-fns/locale; it sadly prevents the valid and harmless statement:

import { enUS } form 'date-fns/locale'

but more importantly, it prevents the harmful

import * as locales from 'date-fns/locale'

In addition, it provides a helpful message to the developer who just wanted to import and destructure enUS from date-fns/locale: it is possible, and preferred, to import it in such a way:

import enUS from 'date-fns/locale/en-US'

There are of course different (and maybe better!) ways to go about this specific problem. We might instead want to disable all import * as statements for example.

So we just saved the world from bloated JavaScript bundles! Hooray! We can now go on our merry way and call it a day. 🥳 🎉

Except we have no assurance that someone will not modify our ESLint config in the future, and mistakenly remove the detection of this bundle-bloating line. This is usually called a regression.

It would call for a test to attract attention to this change, don’t you think?

So let’s write a test for it!

Prevent regressions in our config

The approach is not complex, but requires to think in double negatives. It relies on the report-unused-disable-directives flag in the ESLint CLI command. It makes ESLint report an error for each unused eslint-disable directive.

So given the following file.js,

/* eslint-disable no-unused-vars */

// eslint-disable-next-line no-restricted-imports
import * as locale from "date-fns/locale";

import enUS from "date-fns/locale/en-US";

running eslint file.js --report-unused-disable-directives will pass as long as our ESLint configuration stays the same as above.

If someone removes the rule about restricting the import for date-fns/locale, this command will fail because the eslint-disable statement will not be necessary anymore, which --report-unused-disable-directives will happily tell us about.

We can now go on and create an __eslinttests__ folder, containing our offending code with eslint-disable directives.

Finally, adding eslint __eslinttests__ --report-unused-disable-directives as an NPM script called test:eslintConfig means you can create another job on your continuous integration pipeline to ensure these ESLint configuration regressions will not go unnoticed.

Not a silver bullet

Of course, anyone could add a local .eslintrc.js file to any nested folder and disable that rule, or, even simpler, just use a // eslint-disable comment for any offending line.

There are ways to mitigate these behaviours, but as a last resort, they always rely on the code being properly reviewed (just like anyone could remove a non-passing test 🙈🙉🙊).

For example:

• it is possible to harden your ESLint config to prevent rogue // eslint-disable statements not followed with a specific list of rules:

  // ❌ Not this
  // eslint-disable-next-line
  
  // ✅ but this
  // eslint-disable-next-line rule1,rule2

• it would also be possible to count the number of .eslintrc, .eslintrc.js, .eslintrc.json files there are in your projects and make a bash script fail if there are more than the expected number;
• you could set up code owners for these ESLint configuration files,
• you could finally write a script that detects whether an ESLint configuration was passed through the package.json files in your repository…

I repeat: this is not a silver bullet and proper code review is still necessary.

Feel free to try yourself! Either clone the accompanying GitHub repo or remix this project on Glitch.

That’s it for this post, hope you enjoyed it!

The comments are open below 👇 or tag me (@RobinCsl) on Twitter.