Writing Automated Tests in Drupal 8, Part 1: Test types and set up
Automated testing is a cornerstone of modern programming. Tests ensure that the promised functionality of an application works as expected. Drupal has used automated testing for years, and even provides a vast, public infrastructure to run those tests on Drupal.org.
Drupal 7 relied on its own testing framework, Simpletest. Even tests against Javascript functionality were written in PHP and executed by the Simpletest framework. While this had a high degree of integration with Drupal, it was also a drupalism, something so specific to Drupal itself that it became yet another barrier to new developers.
In Drupal 8, it was decided we should "get off the island" and adopt a more standard testing framework. PHPUnit is used by many different PHP projects, making it easier for new Drupal developers to write tests. Furthermore, a standard framework allowed Drupal core maintainers to maintain less code directly.
In this series, we'll take a closer look at how Drupal 8 provides testing functionality, the tools available, and how to write tests.
When I was studying computer science in college, we had a unit on writing automated tests. At the time, I thought that writing software inside your software to test your own software was a little circuitous. After all, I thought at the time, why would software ever need to be tested more than once? Why would it ever change?
Oh, how much I have learned since them.
I can't fault my professors for not explaining the point to me. The real-world grind of writing and maintaining a software project isn't something that often makes it into a college classroom. Implementing a new feature or changing some seemingly banal detail can lead to all kinds of wonkiness and broken functionality. Testing an application completely ensures that existing, unchanged functionality remains just that: existing and unchanged.
But why automated testing? Surely, people would be better at this? Some are, of course. At my first job in the computer industry, I developed a great deal of respect for QA testers who could sniff out bugs I never would have expected. Nevertheless, people can be inconsistent too. Fatigue, stress, a lapse of attention is all that's needed for some critical detail or step to be lost. In most automated testing frameworks, it's better not to think of it as the software testing itself, as an external framework acting as in the role of the tester. All the software under test really provides is the instructions on what steps to perform. Since a computer is performing the execution, the tests are executed in the same way each time. Automated tests do not replace experienced QA testers, but they can catch obvious bugs and broken functionality earlier in the development process.
In Drupal 7, all tests were done with the Simpletest framework. It didn't matter if you were testing a tiny utility function or the Javascript on a complete application. This did make it easy to learn, as once you knew Simpletest, you could write any test for Drupal. The downside was performance. Running tests could take minutes to hours. Why? In order for each test to run consistently, we would need to a clean install of Drupal each time. Both testing a tiny utility function (even something a simple as return 5;
) or a JS integration would both require the same installation overhead.
To reduce the time needed to run tests, it was decided to break them up into several different types depending on requirements:
Functional tests
Functional tests provide a complete clean install of Drupal. These tests are the closest to what you were expecting if you used Simpletest. The downside is that they do not support Javascript.
- Framework: PHPUnit
- Written in: PHP
- Use when: You need to test the UI, acting as an end-user, but do not need Javascript.
Nightwatch.js tests
The newest testing type in Drupal 8 is Javascript tests powered by Nightwatch.js. This is a new, popular testing framework that is written entirely in Javascript.
- Framework: Nightwatch.js
- Written in: Javascript
- Use when: You need to test anything in the UI that depends on Javascript.
Functional JS tests
Functional JS tests perform the same function as Nightwatch tests, but use a different framework and are written in PHP. This type of test is considered deprecated as a critical infrastructure dependency, PhantomJS, is now only minimally maintained. If you need to test JS, use Nightwatch instead.
- Framework: PHPUnit
- Written in: PHP
- Use when: Never. Write JS tests in Nightwatch instead.
Unit tests
On the opposite side of the scale, Unit tests are intended to test the smallest parts of an application. The best way to think about Unit tests in a Drupal context is, "They test the parts of your code that can run without Drupal." This means functions, methods, and some classes.
- Framework: PHPUnit
- Written in: PHP
- Use when: Testing individual functions or classes that would work without Drupal.
Kernel tests
At first blush, Kernel tests don't seem to belong anywhere. They aren't full integration tests like Functional tests, nor do they have the granular scope of Unit tests. Furthermore, Kernel tests do not guarantee a working UI. So what are they? Often, you'll need to test parts of your module that interact with Drupal, but do not need a UI. This includes using hooks, services, plugins, and anything API-side within Drupal.
- Framework: PHPUnit
- Written in: PHP
- Use when: Testing parts of your module that interact with Drupal, but do not require a UI.
Before we can run any tests, we need to set up a local test framework. Why? Doesn't Drupal.org do this for us? It does, but it isn't best to rely exclusively on Drupal.org's testing function when writing tests -- or whenever you're working on your module, really. Running a module's tests on Drupal.org often requires a wait of 20 minutes or more while the test runners process other requests. This can be particularly frustrating if you make a simple typo or syntax error. Instead, it's better for us to run the tests locally, using the testing feature on Drupal.org as a final check.
For this series, I will assume you already have a local development environment test up with whatever web server stack you prefer. This can include virtualized options such as DrupalVM or containerized options such as Flight Deck. If developing a Drupal 8 module, Dropwhale is a containerized local development environment that comes with testing configured out of the box.
Install Drupal using Composer
If you've Downloaded Drupal from a *.zip or *.tar.gz file, you may have a pre-built version of Drupal running on your system. This is fine, nut often the archive files exclude development related dependencies such as...PHPUnit. We...kinda need that. For that reason, we need to run composer install
locally in order to get all the required dependencies.
If you are working with a virtualized or containerized environment, be sure to enter into the environment first before running these commands.
cd /path/to/my_project
composer install
Configure phpunit.xml
PHPUnit relies on a configuration file in order to know how to connect to the database, and from what URL the site is accessible. This is provided by the phpunit.xml file. This file isn't provided by Drupal core, but an example file, phpunit.xml.dist, is available in the core/ directory.
- Using the command line or a file browser navigate to your the core/ directory of your site.
- Copy phpunit.xml.dist to phpunit.xml
- Open the file for editing.
- Locate the env tag with the name attribute of SIMPLETEST_BASE_URL. Set the value to the URL you use to access your site locally.
<env name="SIMPLETEST_BASE_URL" value="http://docker.test"/>
- Locate the env tag with the name of SIMPLETEST_DB, enter your database URL:
<env name="SIMPLETEST_DB" value="mysql://db_user:db_password@database_server_name/my_db_name"/>
- Save the file.
Create the browser test output directory
Next we need to create a directory for our functional tests. This directory is often used by the test runner to save screenshots, or HTML files generated by failing tests. It's a really, really useful feature that'll save us a lot of head scratching later.
- Using the command line or a file browser, create the sites/default/files/simpletest/ directory.
mkdir -p sites/default/files/simpletest
- On Linux or macOS, be sure to open the directory permissions. Since we're only working locally, let's open the permissions so that everyone can read, write, and execute:
chmod 777 sites/default/files/simpletest
- Open the phpunit.xml file editing.
- Locate the env tag with the name of BROWSERTEST_OUTPUT_DIRECTORY. Set the value to the path of the directory you just created:
<env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/sites/default/files/simpletest"/>
- Save the file.
Validating configuration
Now that we have the configuration file created, we can start running PHPUnit tests. Instead of writing our own -- we'll get to it! -- let's run one provided by Drupal core instead:
$ vendor/bin/phpunit --configuration core core/modules/datetime/tests/src/Unit/Plugin/migrate/field/DateFieldTest.php
PHPUnit testing framework version 6 or greater is required when running on PHP 7.0 or greater. Run the command 'composer run-script drupal-phpunit-upgrade' in order to fix this.
Other deprecation notices (1)
1x: The each() function is deprecated. This message will be suppressed on further calls
$
Hm. That didn't go well. What happened?
At the time of this writing, Drupal 8 ships with a slightly older version of PHPUnit in order to remain compatible with PHP 5.6. If you get the above error and are on PHP 7, run the composer run-script drupal-phpunit-upgrade
as directed, then re-run the test.
$ composer run-script drupal-phpunit-upgrade ...composer running the script... $ vendor/bin/phpunit --configuration core core/modules/datetime/tests/src/Unit/Plugin/migrate/field/DateFieldTest.php PHPUnit 6.5.13 by Sebastian Bergmann and contributors. Testing Drupal\Tests\datetime\Unit\Plugin\migrate\field\DateFieldTest . 1 / 1 (100%) Time: 134 ms, Memory: 6.00MB OK (1 test, 2 assertions) $
Much better!
In this part, we looked at the current state of Drupal 8's testing options. Unlike Drupal 7, There are multiple kinds of test types available with different capabilities. By choosing a test type carefully, we only use the capabilities we need, improving our test performance. While we didn't get to writing any tests in this part, we did set up our local development environment such that we can run a basic, core-provided PHPUnit test.
In the next part of this series, we'll start writing our first test.
This post was created with the support of my wonderful supporters on Patreon.
If you like this post, consider becoming a supporter at:
Thank you!!!