Writing Automated Tests in Drupal 8, Part 2: Functional tests

 

In the last post, we looked at automated testing in Drupal 8. Compared to Drupal 7, creating automated tests become slightly more complex due to the introduction of multiple test types. Drupal 8 has Unit tests, Kernel tests, Functional tests, and Browser-based testing. Each of these testing frameworks have different uses, and different levels of complexity in order to write a successful test. 

If I had to choose a favorite test type, however, I would choose functional tests. Functional tests are sometimes called full-stack or "integration" tests. These tests put all the pieces of software together and execute them. The attraction to me is conceptual simplicity. In a functional test, you are working (more or less) with a complete and working version of the software you are writing. When you write a functional test, you can take the perspective of an end user attempting to perform a scripted set of tasks:

  1. Login as a user with a given set of permissions.
  2. Navigate to a given page.
  3. Perform a specific action on the page.
  4. Check if the page shows expected results.

In Drupal 8, functional tests are guaranteed to have a working user interface, with the primary caveat that JavaScript is not supported. While this sounds like a key drawback, it isn't has deleterious as you might expect. Many Drupal interface elements have non-JS fallbacks that work consistently. As a result, you can write a great deal of tests once you understand functional tests.

Let's start by creating the necessary directories for a functional test. For this series, I'm going to assume we are working as a module developer and writing tests for a Drupal 8 module. This also implies a few other important details:

  • We do not have an existing database. Instead, we assume the site is fresh from installation with no additional configuration.
  • We have already created your module directory structure, *.info.yml, and *.module files.
  • We have a local development environment capable of running your Drupal site and running tests. I will be using Docker, but it could be any sort of environment you prefer.

If you've written tests using Drupal's older testing framework, Simpletest, you may expect to create your tests under your module's src/ directory. This is incorrect. Instead, you want to create a new, top-level directory in your module called tests/:

modules/custom/my_module
├── my_module.info.yml
├── my_module.module
├── src/
└── tests/

Inside the tests/ directory, you want to create a src/ directory, and inside that a Functional/ directory:

modules/custom/my_module
├── my_module.info.yml
├── my_module.module
├── src/
└── tests/
    └── src/
        └── Functional/

That's a lot of directories! Why so many? Drupal 8 relies on the PSR-4 naming standard to locate and autoload PHP classes. To keep things clean, unit, kernel, and functional tests are in their own Testing namespace separate from your module. This seems strange at first, but it allows some overlap in class names without conflict. It also has the added advantage of not burying your test code with the rest of your module code. 

But why tests/src/? Why not just tests/? Remember that in tests we start with a Drupal site with no database each time we run the test. While this gives us additional consistency, it also places the burden of configuring the site on us, the test writers. Toward this end, many tests rely on test-only submodules. To keep end users from enabling them, these tests are kept in a special directory:

modules/custom/my_module
├── my_module.info.yml
├── my_module.module
├── src/
└── tests/
    ├── modules/
    └── src/
        └── Functional/

Having a test/src/ allows us to put test-supporting submodules in tests/modules/, keeping everything separate from your regular module code.

With the necessary directories created, we can now create the skeleton of a functional test. Like nearly everything in Drupal 8, each functional test is encapsulated in a PHP class. First, we create the class file. Let's say we need to test the module's settings form, so we create a new settings form class file in the tests/src/Functional/ directory:

modules/custom/my_module
├── my_module.info.yml
├── my_module.module
├── src/
└── tests/
    ├── modules/
    └── src/
        └── Functional/
            └── SettingsFormTest.php

In general, it's considered good practice to use a class name ending with "Test".

Inside the file, we first stub out the class itself:

<?php

namespace Drupal\Tests\my_module\Functional;

/**
 * Test the module settings page
 *
 * @group my_module
 */
class SettingsPageTest {

}

The @group annotation tells the testing framework that this is a test that belongs to the "my_module" group. Later in this series, we'll use this to run multiple test classes with a single command. For now, we use the same group name as our module name.

This alone doesn't get us anything. Even in the tests/src/Functional/ directory, it's just a class. To make it a real functional test, we need to derive from the BrowserTestBase class provided by Drupal core:

<?php

namespace Drupal\Tests\my_module\Functional;

use Drupal\Tests\BrowserTestBase;

/**
 * Test the module settings page
 *
 * @group my_module
 */
class SettingsPageTest extends BrowserTestBase {

}

The BrowserTestBase class provides us all the class methods and utilities necessary to run our functional test. It also has a few demands on our part:

<?php

namespace Drupal\Tests\my_module\Functional;

use Drupal\Tests\BrowserTestBase;

/**
 * Test the module settings page
 *
 * @group my_module
 */
class SettingsPageTest extends BrowserTestBase {

  /**
   * The modules to load to run the test.
   *
   * @var array
   */
  public static $modules = [
    'user',
    'my_module',
  ];

  /**
   * {@inheritdoc}
   */
  protected function setUp() {
    parent::setUp();
  }
}

We added two more things to our test class. The first is we added a new static class variable, $modules. We also implemented the ::setUp() method, although we're not using it yet. 

The $modules variable is used by the testing framework to know what modules to enable when we run our test. Hold on, didn't I say earlier we have a fully functioning Drupal environment for functional tests? It's in the name! Functional!. Well, yes. We do have a functional Drupal environment, but that doesn't mean we need the complete set of all core modules enabled. For many tests, we want to restrict the number of core modules enabled so that we are only leveraging the functionality we need.

In the above, we enabled two separate modules, "user" and our custom "my_module". We enable the user module because we know we are going to interact with a settings form. This involves logging in and having appropriate permission to access the form -- user stuff, in other words. So we enable the user module.

Since we're testing our own module, we also enable it here. This can be monotonous, but it does give us some flexibility for testing that we previously didn't have. One could write a functional test that enables the module as a user action, allowing us to check if it deploys initial configuration correctly.

Now that we have the test class skeleton created, we can try to run it and see if everything is set up correctly. Please be sure that you have configured phpunit.xml as described in part 1 of this series first.

Let's try to run the test! We'll change to the directory containing our Drupal site, then run the phpunit command from the vendor/bin/ directory:

$ cd path/to/my_drupal_site

$ vendor/bin/phpunit modules/custom/my_module/tests/src/Functional/SettingsPageTest.php
 
PHP Fatal error:  Class 'Drupal\Tests\BrowserTestBase' not found in /var/www/html/modules/custom/my_module/tests/src/Functional/SettingsPageTest.php on line 12

Yikes! What happened here? When we validated the testing framework was set up in Part 1, we didn't do anything different. We ran the phpunit command right from the site root. This time, however, the testing framework completely failed to find our module. What happened?

It turns out, for PHPUnit to be aware we're testing a Drupal module, we have to execute it from inside the core/ directory. This helps PHPUnit to resolve the class namespaces correctly so that it can find our test class.

Let's try that again, but we'll change to the core/ directory first:

$ cd core/

$ ../vendor/bin/phpunit ../modules/custom/my_module/tests/src/Functional/SettingsPageTest.php 

PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

Testing Drupal\Tests\my_module\Functional\SettingsPageTest
W                                                                   1 / 1 (100%)

Time: 128 ms, Memory: 6.00MB

There was 1 warning:

1) Warning
No tests found in class "Drupal\Tests\my_module\Functional\SettingsPageTest".

WARNINGS!
Tests: 1, Assertions: 0, Warnings: 1.

A warning!? Wait no, a warning! That's actually a good thing! This time, PHPUnit found our class correctly, and ran the test class. The reason we got a warning is that our test class....doesn't test anything. True, we configured the $modules variable and provided the ::setUp() method, but we didn't have it test anything.

We'll do that in the next step.

Each test class can be thought of as a suite of test cases. Each test case within the suite (class) shares the same $modules and the same ::setUp(). For this reason, each case within a test suite class tends to be highly related.

So how do we provide test cases to PHPUnit? Fortunately for us, it's really, really easy. PHPUnit will assume any public method starting with "test" in the name is a test case. So, all we need to do is create a new method:

<?php

namespace Drupal\Tests\my_module\Functional;

use Drupal\Tests\BrowserTestBase;

/**
 * Test the module settings page
 *
 * @group my_module
 */
class SettingsPageTest extends BrowserTestBase {

  /**
   * The modules to load to run the test.
   *
   * @var array
   */
  public static $modules = [
    'user',
    'my_module',
  ];

  /**
   * {@inheritdoc}
   */
  protected function setUp() {
    parent::setUp();
  }

  /**
   * Tests the setting form.
   */
  public function testForm() {
  }
}

Now that a test case method has been added, we should be able to re-run the test:

$ ../vendor/bin/phpunit ../modules/custom/my_module/tests/src/Functional/SettingsPageTest.php 

PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

Testing Drupal\Tests\my_module\Functional\SettingsPageTest
E                                                                   1 / 1 (100%)

Time: 7.35 seconds, Memory: 6.00MB

There was 1 error:

1) Drupal\Tests\my_module\Functional\SettingsPageTest::testForm
Exception: User warning: mkdir(): Permission Denied
Drupal\Component\PhpStorage\FileStorage->createDirectory()() (Line: 174)


[...stack track goes here...]

ERRORS!
Tests: 1, Assertions: 1, Errors: 1.

Now what!? Why is the test failing when trying to create a directory? Why is it trying to create a directory at all!?

This problem, as the error indicates, is due to permissions. The Running PHPUnit tests guide on Drupal.org indicates:

Functional tests have to be invoked with a user in the same group as the web server user. You can either configure Apache (or nginx) to run as your own system user or run tests as a privileged user instead.

In the above error, I'm running the tests from inside a Docker container. I'm even running the tests from the same container as the web server runs:

$ whoami

root

$ ps -ef | grep apache

root         1     0  0 16:03 ?        00:00:00 /bin/sh /usr/sbin/apachectl -D FOREGROUND
root        11     1  0 16:03 ?        00:00:00 /usr/sbin/apache2 -D FOREGROUND

So what's the problem? If I'm running the web server as root, and the tests as root, I should have superuser permission to the entire system. I should be able to create or write to any directory I want.

Yet, even if I use chmod -R 777 path/to/my_drupal_site to grant read, write, and execute permission to everyone, for every directory, for every file in my web site, the test will still fail. 

So what the heck is going on? From what I gather, PHPUnit is smart enough to know -- rightly -- that running tests as the operating system superuser is a VERY. BAD. IDEA. And, rightly, it tries to save our collective butts by running the tests as the only user it knows for sure exists. Every UNIXalike has both a root user, as well as a special unpermissioned user named nobody. The nobody user has no permissions to anything, even if the directory is writable by...everybody. 

The solution is to reconfigure the Docker container so that the web server runs as a non-root user, and you run the tests as the same user. Many custom containers will run as root by default, so you will need to either build a custom container, or use a container that does not run as root by default, such as Dropwhale or Flight Deck.

Once the container is reconfigured (or switched out completely) so the web server is running as non-root, and the tests are running as non-root, the tests should complete successfully:

$ ../vendor/bin/phpunit ../modules/custom/my_module/tests/src/Functional/SettingsPageTest.php 

PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

Testing Drupal\Tests\my_module\Functional\SettingsPageTest
.                                                                   1 / 1 (100%)

Time: 4.75 seconds, Memory: 4.00MB

OK (1 test, 2 assertions)

Now that our -- admittedly empty -- test case is running successfully, we can now focus on writing a test case. For my_module, our settings form has a single text field with the machine name of my_text_field. There are no other settings. In order to access the settings page, the module also provides the configure my module permission. At this point, we have tested the settings for manually and it stores the value of my_text_field as expected.

Now we can write the automated test. First we need to create a new user that has the configure my module permission and then log in. In the body of our ::testForm() method, we can write the following:

    // Create the user with the appropriate permission.
    $admin_user = $this->drupalCreateUser([
      'configure my module',
    ]);
    
    // Start the session.
    $session = $this->assertSession();

    // Login as our account.
    $this->drupalLogin($admin_user);

First, we create the admin user. The BrowserTestBase class provides an easy method for us just for this purpose, ::drupalCreateuser(). It takes an array of permission machine names. Next we start the session. This instructs the underlying test framework to begin a browsing session so that we might interact with the site. We don't use the $session it returns just yet, but we'll hang on to it for now. Finally, we login using the user we created earlier. Again, BrowserTestBase provides a method explicitly for this purpose.

Now, we want to load the settings page, just to make sure we have access to it as expected:

    // Get the settings form path from the route
    $settings_form_path = Url::fromRoute('my_module.my_module_settings_form');

    // Navigate to the settings form
    $this->drupalGet($settings_form_path);
    
    // Assure we loaded settings with proper permissions.
    $session->statusCodeEquals(200);  

So far so good. While the BrowserTestBase::drupalGet() method only takes a raw path, our test can be more flexible if we load the path from the route name instead. So, we use the Url class to load the route path, then navigate to it, then check for a 200 OK HTTP response code.

Now, we get down to business. Let's update the form value and submit the form:

    // Update our text field with a new value.
    $edit = [
      'my_text_field' => 'why hello there',
    ];
    $this->drupalPostForm($settings_form_path, $edit, 'Save configuration');

Wait, what? This is probably the most confusing part about writing functional tests. Instead of populating each form field and then submitting the form, we do everything in a single step. The BrowserTestBase::drupalPostForm() method takes an array of form values, and the page at which the form can be found. Since multiple forms can be on the same page, we identify which one to submit by providing the text of the submit button. In this case, Save configuration.

Finally, let's check that our need field value was stored and the form will not show it as the default:

    // Reload the page.
    $this->drupalGet($settings_form_path);

    /** @var NodeElement $omit */
    $my_text_field = $session->fieldExists('my_text_field')->getValue();

    // Check that we will not run every cron run.
    $this->assertTrue($my_text_field == 'why hello there');

We reload the page, then we get the value of my_text_field. Here is where the $session object we created earlier comes into play. We use this to check if the ::fieldExists(). If so, we can ::getValue() on it. This alone doesn't check what is the value, it only returns it.

To check it, we use one of BrowserTestBase's many ::assert*() methods. There are many of these available, but the one we use here, ::assertTrue(), expects the first parameter to be a boolean value of TRUE. We compare the value we got from the field to our expected value. 

And that's it! We've written our first function test!

Let's run our test to see how we did. We know that the form works because we tested it manually.

$ ../vendor/bin/phpunit ../modules/custom/my_module/tests/src/Functional/SettingsPageTest.php

PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

Testing Drupal\Tests\my_module\Functional\SettingsPageTest
E                                                                   1 / 1 (100%)

Time: 3.53 seconds, Memory: 6.00MB

There was 1 error:

1) Drupal\Tests\my_module\Functional\SettingsPageTest::testForm
Drupal\Core\Config\Schema\SchemaIncompleteException: Schema errors for my_module.settings with the following errors: my_module.settings missing schema

[...stack trace goes here...]

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

What the complete hell. Instead of a successful test, we instead get a new error, SchemaIncompleteException. What in the heck is this thing?

In Drupal 8, modules can save settings using the configuration API. This API stores data as a hierarchical set of key-value pairs. When writing the module, we can define whatever settings we desire without much concern as to what goes in them. When we start writing tests, however, the test framework wants to know what kind of data to expect to store as configuration. This description is called a schema, and it must be included with the module in the config/schema/ folder.

modules/custom/my_module
├── config/
│   ├── install/
│   └── schema/
│       └── my_module.schema.yml
├── my_module.info.yml
├── my_module.module
├── src/
└── tests/
    ├── modules/
    └── src/
        └── Functional/
            └── SettingsFormTest.php

The schema document has the name of your_module_name.schema.yml. As a YAML document, it mimics the hierarchical set of data stored in the configuration. So how do you know what your schema is if you've never done this before? The Configuration Inspector module can provide you much needed data on what schema items are missing, what they should be named, and where they should go. While it's not necessary for an experienced developer, it can really help you out when you're learning to write schemas.

After installing the module, we determined we need the following schema:

my_module.settings:
  type: config_object
  label: 'My Module Settings'
  mapping:
    my_text_field:
      type: string
      label: 'My text field'

We save the file, and rerun the test...

$ ../vendor/bin/phpunit ../modules/custom/my_module/tests/src/Functional/SettingsPageTest.php 

PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

Testing Drupal\Tests\my_module\Functional\SettingsPageTest
.                                                                   1 / 1 (100%)

Time: 5.29 seconds, Memory: 6.00MB

OK (1 test, 10 assertions)

Success! We now have a new functional test!

In this part, we've created our first functional test. We learned that functional tests are written as test suite classes in the tests/src/Functional/ directory of our module. Each test suite class has a $modules array of modules to enable to run the test. We are also provided a ::setUp() method to do any set up we need for all tests in the class. Individual test cases are written as class methods that start with the name "test". For functional tests, the BrowserTestBase base class provides many utilities for interacting with the test framework. 

Functional tests still are my favorite kind of tests to write. They have the conceptual simplicity of acting as an end user in a full Drupal environment (minus the JavaScript of course). The downside, however, is that they can be rather....slow. Installing Drupal all over again for each test suite is time and resource intensive. 

Next time we'll see how to test the smallest parts of our application, by writing Unit Tests.

This post was created with the support of my wonderful supporters on Patreon.

If you like this post, consider becoming a supporter at:

Thank you!!!