In a previous post, we went over how to set up your local development environment with Docksal for Drupal 8 or Drupal 9 projects. Now, let’s see how we can leverage that Docksal setup to validate each push to GitHub. This will make it much easier for the project lead to make sure that new branches or updates don’t break anything and that all the tests pass before reviewing the branch.
Testing
Let’s add a command to run all of the tests in the project. Based on the project in the prior blog post, let’s add a test command that will run all of our Behat and PHPUnit tests. Create .docksal/commands/test:
#!/usr/bin/env bash
## Run site tests.
##
## Usage: fin test
run_tests ()
{
# We stop and restart the project to start the test run with full memory.
fin project stop
fin project start
# These are mostly split up because of high memory usage.
echo "Executing Behat JS tests..."
fin behat --tags=javascript
js_exit_code=$?
echo "Executing Behat non-JS tests..."
fin behat --tags=~javascript
non_js_exit_code=$?
# We stop and restart the project to start the test run with full memory.
fin project stop
fin project start
echo "Executing PHPUnit tests..."
fin phpunit web/modules/custom
php_unit_exit_code=$?
echo "Exit $((js_exit_code + non_js_exit_code + php_unit_exit_code))"
exit $((js_exit_code + non_js_exit_code + php_unit_exit_code))
}
run_testsThe VM that GitHub gives us has 7GB of RAM, so we’ll run tests in a few distinct sets. ChomeDriver does eat memory so we’ll run the JavaScript Behat tests separately from the non-JS Behat tests, then we’ll restart the project and run the PHPUnit tests on our custom modules, which can include Unit, Kernel, Functional, FunctionalJavascript, and ExistingSite DTT tests (more on Drupal Test Traits to come!).
To install PHPUnit and what it needs to run on Drupal 9, we need to run:
fin composer require --dev phpspec/prophecy-phpunit:^2 symfony/phpunit-bridge. Create the file .docksal/settings/phpunit.xml with:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="tests/bootstrap.php" colors="true"
beStrictAboutTestsThatDoNotTestAnything="true"
beStrictAboutOutputDuringTests="true"
beStrictAboutChangesToGlobalState="true"
failOnWarning="true"
printerClass="\Drupal\Tests\Listeners\HtmlOutputPrinter"
cacheResult="false">
<php>
<!-- Set error reporting to E_ALL. -->
<ini name="error_reporting" value="32767"/>
<!-- Do not limit the amount of memory tests take to run. -->
<ini name="memory_limit" value="-1"/>
<env name="SIMPLETEST_BASE_URL" value="http://web"/>
<env name="SIMPLETEST_DB" value="mysql://user:user@db/default"/>
<env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/web/sites/simpletest/browser_output"/>
<env name="BROWSERTEST_OUTPUT_BASE_URL" value="http://web"/>
<!-- To disable deprecation testing completely uncomment the next line. -->
<env name="SYMFONY_DEPRECATIONS_HELPER" value="disabled"/>
<!-- Example for changing the driver class for mink tests MINK_DRIVER_CLASS value: 'Drupal\FunctionalJavascriptTests\DrupalSelenium2Driver' -->
<env name="MINK_DRIVER_CLASS" value=''/>
<!-- Example for changing the driver args to mink tests MINK_DRIVER_ARGS value: '["http://127.0.0.1:8510"]' -->
<env name="MINK_DRIVER_ARGS" value=''/>
<!-- Example for changing the driver args to phantomjs tests MINK_DRIVER_ARGS_PHANTOMJS value: '["http://127.0.0.1:8510"]' -->
<env name="MINK_DRIVER_ARGS_PHANTOMJS" value=''/>
<!-- Example for changing the driver args to webdriver tests MINK_DRIVER_ARGS_WEBDRIVER value: '["chrome", { "chromeOptions": { "w3c": false } }, "http://localhost:4444/wd/hub"]' For using the Firefox browser, replace "chrome" with "firefox" -->
<env name="MINK_DRIVER_ARGS_WEBDRIVER" value=''/>
<env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/tmp"/>
<!-- To disable deprecation testing completely uncomment the next line. -->
<!--<env name="SYMFONY_DEPRECATIONS_HELPER" value="disabled"/>-->
</php>
<testsuites>
<testsuite name="unit">
<directory>./web/modules/custom/*/tests/src/Unit</directory>
<!--<directory>./web/profiles/custom/*/tests/src/Unit</directory>-->
</testsuite>
<testsuite name="kernel">
<directory>./web/modules/custom/*/tests/src/Kernel</directory>
<!--<directory>./web/profiles/custom/*/tests/src/Kernel</directory>-->
</testsuite>
</testsuites>
</phpunit>We can test this by running the tests for one of the modules we have installed. Note here that we have not configured our Simpletest output directory and the admin_toolbar module has a couple of skipped tests.
Run fin test from the command line now and you’ll run all of your Behat and PHPUnit tests.
We capture and add up the return codes because we want GitHub Actions to run all of our tests whether any of the previous tests have failed or not. We want a complete picture of the state of the branch.
Codesniffing
We want our GitHub action to run codesniffer on our project so we can see if any branch or pull request breaks some Drupal coding standards.
Add a new command to .docksal/commands called cs:
#!/usr/bin/env bash
## Run project's Codesniffer inspections.
##
## Usage: fin cs
# Environment variables passed from fin:
#
# $PROJECT_ROOT - (string) absolute path to NEAREST .docksal folder
# $VIRTUAL_HOST - (string) ex. projectname.docksal
# $DOCROOT - name of the docroot folder
# $DOCKER_RUNNING - (string) "true" or "false"
fin exec "./vendor/bin/phpcs --standard=Drupal,DrupalPractice --extensions=php,module,inc,install,test,profile,theme,css,info,txt,md --ignore=node_modules,bower_components,vendor ./web/modules/custom"
fin exec "./vendor/bin/phpcs --standard=Drupal,DrupalPractice --extensions=php,module,inc,install,test,profile,theme,css,info,txt,md --ignore=node_modules,bower_components,vendor,.min.css ./web/themes/custom"
fin exec "./vendor/bin/phpcs --standard=Drupal --extensions=php ./tests/features/bootstrap"This runs the codesniffer against all of our custom modules, custom themes, and Behat context classes.
We’ll need a few Composer dependencies to make this work:
fin composer require drupal/coder dealerdirect/phpcodesniffer-composer-installer --devThen we can run fin cs and get our linting/codesniffing output:
The FeatureContext class that Behat generated for us has a bunch of coding standards violations. We’ll leave these for now, but every push to GitHub will remind us that we need to come back and fix this class.
GitHub Action Job
Let’s set up our job for GitHub Actions. This tells GitHub how we want to structure the job, what events we want this job to respond to, and what actions the job will run.
Create a .github/workflows directory, then we’ll add ci.yml:
# Run tests after push.
name: CI
# Controls when the action will run.
on: [push]
env:
# Disable CI mode so Docksal is setup as if it were a local workstation.
CI: ''
TEST_SITE_NAME: testtest
DOCKSAL_DNS_DISABLED: 1
defaults:
run:
shell: bash
jobs:
docksal:
name: Setup Docksal
runs-on: ubuntu-latest
steps:
# Hack to fix docker-gen in Github Actions
# Drops docker daemon config with custom cgroup (which docker-gen does not support)
# See https://github.com/jwilder/docker-gen/issues/315
- name: Fix docker-gen in Github Actions
run: |
sudo rm /etc/docker/daemon.json
sudo service docker restart
- name: Checkout code
uses: actions/checkout@v2
- name: Install Docksal
run: |
bash <(curl -fsSL https://get.docksal.io)
- name: Get versions
run: |
fin version
fin sysinfo
docker version
- name: Initialize project
run: |
fin init
- name: Set environment variables to secrets
run: |
echo ${{ secrets.API_ENDPOINT }} > keys/api_endpoint.key
echo ${{ secrets.API_PASSWORD }} > keys/api_password.key
echo ${{ secrets.API_USER }} > keys/api_user.key
- name: Reset cache
run: |
fin drush cr
- name: Test
run: |
fin test
- name: Codesniff
run: |
fin csGitHub Actions has some pretty good documentation, so we won’t repeat all of that here. Instead of going over all of the various options for each setting, we’ll discuss what the settings we have here do.
There are a number of GitHub events the job can respond to, but we are only concerned about the push event. This means that whenever someone pushes a new branch to the repository, GitHub will start this job.
The env settings make sure we’re running a fairly bare-bones environment. We’ll be using Docksal to run our project instead of running them natively on the VM they give us. Similarly, the defaults.run.shell setting lets GitHub know we want to use the regular bash shell.
Now things get interesting! Let’s look at how our job is set up. We have a docksal element and we specify that it should run on the latest Ubuntu release. We don’t have to be picky about setting up this VM because all of the magic happens using our Docksal project configuration.
In steps, the first thing we need is a little bit of a hack in order to make sure that docker-gen does what we need it to do, which is mostly staying out of the way!
Next, we checkout our code, then we install Docksal on the VM. We install on this VM the same way we install Docksal on our development environment. And since this is running on Linux, we can use regular Docker without worrying about slowness or VirtualBox like we would need on an OSX environment.
Next, we’ll output all of the versions. This is a cheap but useful step. It doesn’t take any time to execute so it won’t cost any GitHub Actions minutes, but it makes it much easier to troubleshoot if something goes wrong because you can see in the log what version of everything it’s running.
Then we have our Initialize Project step. This runs our fin init command, which installs all of the Composer dependencies, runs the site install, imports configuration, and whatever else we have set up, such as importing migrations to add default data to the site.
The Set environment variables to secrets step is optional and you should only include it if you need it. In this example, we are using the Key module to override configuration settings with key files for settings necessary to integrate with a third-party API. We also use these secrets to do things like save Commerce payment gateway settings so we can have our GitHub Action run an end-to-end test against the actual payment sandbox. The values will be **’d out in the output so that your secrets never get revealed to anyone looking at the log.
The next step resets the cache. This does add a bit of slowness but since the entire process is running unattended after doing things like config import and module installation, it’s a safe step to clear those caches and plugin definitions before we start running tests.
The Test step runs our fin test command, which executes all of our Behat and PHPUnit tests. Again, we want all of the tests to run without stopping if there is a failure, so we capture the exit codes from each test command, then return those at the end so that the job will fail if any of the tests fail.
The last step is our codesniffer step, which runs the cs command we defined earlier and the job will fail if there are coding standard violations.
When we push this code, our job will fail:
It failed on the Test step, which is expected. We are trying to run PHPUnit tests against web/modules/custom, but that path doesn’t exist. When we have custom modules with tests, they’ll run here. We’ll cover more on custom module testing soon.
With this GitHub Action in place, every time a developer on your project pushes to a branch, these tests will run. Prior to having this in place, the lead developer/merge-meister would have to pull down pull requests locally, run the tests, codesniff, and validate each branch before merging it. Additionally, much of our development team is on OSX, so running these tests on Ubuntu against Docksal will run quite a bit faster than on the Mac.
Additionally, GitHub sends an email when the job fails, and each pull request or branch is flagged with the status of the jobs. When you get a green checkmark, all of the tests have passed and there are no coding standard violations! It’s good to go!
You can see the code for this post at https://github.com/jonathanfranks/docksal_drupal9 in the github-actions branch.
Comments