Testing

Learn how to test your extension.

On this page, you'll find several strategies you can use to test your extension.

Unit tests

Some tests require you to mock production files. Mocked files are generated right in the test with mock(..) or cachedMock(...) automatically. You can also generate the mocked file by passing the filename from the /sdk/api/ directory:

npx webpack --config ./test/unit/mock/webpack.config.js --env filename="subscriptions.js"

Run the unit tests:

npm run unittest

Functional tests

Serving the test pages

Regardless of whether you're manually loading the test extension or using the test runner, functional test suites require locally served test pages:

npm run test-server

Using the test extension

The test extension will be built on both /dist/test-mv2 and /dist/test-mv3 folders, which can be loaded as unpacked extensions under chrome://extensions in Chromium-based browsers, and under about:debugging in Firefox. Once the extension is loaded, it opens the test suite in a new tab.

Keep the following in mind when you use the test extension:

• test-mv2 contains a manifest version 2 extension, and test-mv3 contains a manifest version 3 extension.

• test-mv2-custom contains a manifest version 2 extension using custom (test) subscriptions.

• For the popup tests to work, you have to disable the browser's built-in popup blocking (on localhost).

You can also inspect the extension's background page to manually test the API through the global EWE object.

Test options

• The timeout option overrides the per-test timeout in milliseconds.

• The grep option filters the tests to run with a regular expression.

• The incognito checkbox is used to inform the tests whether the browser started in incognito/private mode or not. This does not cause tests to run in incognito mode.

• The testKinds option is used to run only a certain subset of the functional tests. For example, you can target only the service worker fuzzing tests by running --testKinds fuzz, or a combination of the functional, reload and update tests by running tests with --testKinds functional reload update mv2-mv3-migrate.

Using the test runner

npm test -- {v2|v3|v2-custom} {chromium|firefox|edge} [version|channel] [options]

Runner options need to be preceded by two dashes (--), for example --timeout 10000.

In the test runner context, the --incognito flag forces the browser to start in incognito/private mode. This is only supported by Firefox.

Using the test runner on docker

The CI/CD pipeline runs the functional tests using this docker image. Using docker locally to run the same tests may be useful to recreate the infrastructure the CI uses:

docker build -t functional -f test/dockerfiles/functional.Dockerfile .
docker run --shm-size=512m -e TEST_PARAMS="{v2|v3|v2-custom} {chromium|firefox|edge}" -it functional

The TEST_PARAMS argument can also take the additional options described previously.

Tip: Matching CI system resources

The following resources setup on a local macOS Docker installation has proven to be close enough to the GitLab Linux shared runners behaviour:

• CPUs: 1

• Memory: 1.75 GB

• Swap: 1.5 GB

This can also by done by adding the flags --cpus=1 --memory=3.75g to docker run.

docker run --cpus=1 --memory=3.75g {other flags as above}

Tip: Maintaining browser cache

This works only on Linux machines. If you have a slow internet connection or are running the test many times, you can add the following flags to docker run to use the browser download cache from the host machine.

-v $(pwd)/browser-snapshots:/webext-sdk/browser-snapshots

Tip: Testing flaky tests

Flaky tests in our CI pipelines can prove frustrating because they don't fail or succeed every time. To help in measuring if a fix improves these, you can run the measureTestFlakiness test to see the percentage of test runs that fail.

npm run measureTestFlakiness

This script has various options to tweak how many runs to do, which tests to run, or the docker options to use. You can run the script with the --help option for a list of options.

npm run measureTestFlakiness -- --help

Integration tests

Building ABPUI extension and running testpages integration tests

If there are no breaking changes in the code, you will be able to build the ABPUI extension with the current codebase.

To build and extract the custom extension on Docker, run the following commands:

docker build -t extension -f test/dockerfiles/extension.Dockerfile .
docker run extension
docker cp $(docker ps -aqf ancestor=extension):/extension .

The extension code uses the most recently released ABPUI Tag by default.

extension folder will contain two versions: mv2 and mv3.

To run testpages integration tests on Docker with the custom extension previously created, you may clone the Testpages project and follow the instructions on how to run tests with packed extensions. Example:

cd testpages.adblockplus.org
cp <location/to/extension/> .
docker build -t testpages --build-arg EXTENSION_FILE="extension/extensionmv<2|3>.zip" .
docker run --shm-size=512m -e SKIP_EXTENSION_DOWNLOAD="true" -e GREP="^.*$BROWSER((?!Snippets).)*\$" testpages

Custom extensions don't support snippets. As a result, eyeo recommended using the GREP parameter to exclude those tests. On a zsh shell (default for MacOS), you'll need to escape the ! symbol from the previous example (\!).

Non-functional tests

Bundle test

This checks that the bundled code can be imported and re-bundled:

npm run test-bundle

Scripts test

npm run test-scripts

Manifest V3

See Manifest V3 Subscriptions for documentation specific to Manifest V3.