Testing is Hard (But Totally Worth It)

Testing is hard. I think it was Plato who said, “The best time to start adding automated tests to your site was right at the beginning. The next best time is today.” I can’t argue with wisdom like that, so let’s say you’re working on a complicated, custom Drupal site with no tests at all. Should you be writing tests now? 

Great question. 

Let’s talk about why it matters, how to convince the people paying the bills, and most importantly, where to actually start.  Oh, and, yes. The answer is emphatically yes, you should be writing tests now. 

Why Writing Tests is Important 

Why is it important to write tests? I’ll answer this question with another question. 

Have you ever gotten an email (maybe urgent, maybe just confused) that asks, “Hey, we added this cool new feature and that’s working great, but now the thing we built last month doesn’t work anymore. What’s going on?” 

Would you like to be asked that question less? 

I almost take for granted that I don’t have to answer a lot of questions like that anymore. My projects have testing from the beginning, so in my case, the basic “happy path” functionality is already covered by code that proves that every acceptance criteria we implement works, and that it continues to work after we build new things on top of it. Before a branch with a new feature is merged, it needs to pass all of the tests. The new code won’t be deployed to QA until all of the tests pass, not only for the new feature, but for the existing features, too. This way, we can be satisfied that none of our new changes break anything that was working before. 

But that’s my project. Let’s talk about yours, where you need to convince people that this is a good idea. One of the most important things in writing tests to cover your functionality is that you aren’t only saving dev. You’re saving QA. 

Dev and QA can sometimes clash because dev doesn’t understand why anyone would ever put the knife in the peanut butter upside down, and QA can’t understand why dev didn’t just put the peanut butter in a squeeze bottle anyway.  Then dev reminds QA that some peanut butter is extra chunky and that to implement creamy and extra chunky with the same UI, it needs to be a jar and not a squeeze bottle, and they finally settle on adding a link to instructions that show a user how to properly operate the knife. 

We can’t suddenly test our site’s features at all at once, but we can add them one issue at a time. I’m going to assume that issues and features both come to you in the form of some sort of ticketing system. If not, I am so sorry, and you have bigger issues than testing. Follow along anyway, though, there is a lot to learn here! 

Let’s start with a defect. You get a ticket saying that as a user with a specific role, a particular view never shows any results. You have good QA, so they let you know that they tested it with each of the four roles who can get to the view. Administrator works (of course it did, that’s how you tested it!), but the other three all show no results. 

Hmm. Didn’t you test it with those other roles? You can’t really remember. If only there was some way you could document (both to yourself, and to others if that is the kind of place you work) that you tried it and that it passed. Enter the automated test. 

How Do I Convince my Client to Pay For it? 

Okay, now, hear me out, here. You don’t. 

Let me be clear: I am not advocating that you lie to your client. Deceit is not the answer. What I am saying is, as of right now, it is simply part of your process. If you had written a route with a custom table you needed to sort, and then you find out Core already has an incredible TableSort class, do you need to get buy-in from the client to use the better tool for a new requirement? No, you just use the better solution. From now on, you write code and configure sites to solve problems, and to prove that you did, you test that code and configuration. 

When you add a new link to a page, do you not click that link to make sure it works? 

Prove it. Prove you clicked it before you deliver it to a client and cost them however much it costs them to have their customers upset that something they need for their business doesn’t work anymore. 

If you have only been proving that with manual testing, imagine how much cheaper it will be if you don’t need to have your testers go through all of the same heavy duty repetitive tests every single time you deploy a minor feature. It’s all automatic. You wrote it once and it runs every time and the client doesn’t have to pay a tester to validate that your view headers still click sort every time you add a new permission to your site. 

Take that defect from above. Let’s go through how we would do it manually. First, we would set up a development environment. Maybe this means we have a local instance with a download of a database from QA or production or wherever the report of this defect came from. We can also start from a fresh development environment and make sure in a fresh environment, the behavior QA reported is actually the issue. The issue boils down to something like this: Users who don’t have the administrator role can’t see results in the view. 

To test this manually, we would log in as an administrator and make sure that there are results in the view that the admin can see. Next, we log in as one of the other roles, and make sure that those results are not visible to this user. Then you would dig into the view and figure out what the problem is. You solve it (not crack it, which would have a pejorative connotation…) and go through your very proper workflow to deploy it. 

Immediately, QA sends you the ticket back. 

What? You fixed it! You tested it! It worked! Can you prove that it worked? 

Maybe you took a screenshot. That doesn’t really help anyone, though, because, sure enough, you reproduce your steps, and now when you are logged in as an administrator, the rows are out of order. Your fix didn’t quite work. You fix that. You don’t want to get caught out again, so you test the other role again, then you deploy. 

And you get the ticket back again. 

The second non-admin role also sees no results. Imagine how many more times this could go back and forth. Cut to: FINALLY, you have the fourth role working, and you deploy… And, of course, something is wrong with the second role again that you didn’t catch. Your manager didn’t want to pay for automated testing, but for some reason, paying for dev and QA to go round and round so many times is not a hard sell.

Let’s imagine what this would look like with automated testing. 

Again, the issue: Users who don’t have the administrator role can’t see results in the view. 

You know there are four user roles that are allowed to hit this view. Let’s ignore negative testing for now (where we make sure that users who do NOT have those roles are NOT allowed to see the view) and only focus on validating this specific defect. Also, let’s trust our QA at this point only use the four roles they told us about. We’ll make one test for each of the roles mentioned in the ticket.

Let’s assume this is a functional or behavior test that will walk through the site and do stuff, rather than, say, a Kernel test which looks at module functionality. The test makes sure that there is a test user with that specific role, log in with the user, navigate to the view, and check for the expected results. 

The test for the administrator passes. The other three tests all fail. That is good! We have replicated what was reported to us! We all know that sometimes it works perfectly on our machines and something’s plain weird on QA, but in this case, we match. 

Now we go into the site, tweak the view, tweak the code, whatever we need to do to get it going, and for our test user, now it works. We run the tests and immediately we can see that we didn’t fix it for everything. Only some of the roles’ tests are passing! We don't need to bother QA yet or spend the time doing a deployment. We know right now that we aren’t done. A couple more fixes and we can prove now that all four roles pass. 

So far, these are the only tests that exist for our site. It’s possible our fix today may have broken the password reset function, for example, and we won’t know until someone goes and makes sure it works. And for now, don’t worry about writing a test for reset password until you get a new feature near it or a defect for it. 

For this feature, for this view with these roles, you and QA both can prove this page works. And you did it with a single deployment and didn’t involve any additional people to manually test anything until you proved out that it was working. Additionally, now that you have these tests that you’ll run every time you want to merge something, you can make sure that no future changes ever unexpectedly undo your hard work. 

Let’s say you get a new requirement that a role change scope, and maybe one of those three roles can’t see that view anymore. Now you know right away that the change in permissions for that role do or do not affect your previously defined acceptance criteria. You have a record of the results that you expect for a particular behavior (like a user with a specific role hitting a particular page) and you can prove it works before you hand it to a tester. 

When you write tests for all of your new work, you will see the impact on your tickets right away. Defects won’t be re-opened, because dev knows that some previous bad behavior was already fixed and validating it won’t fall through the cracks because there was a test for it that ran before any new work was merged. You know that new changes do not break tested functionality. 

How Do I Get Started? 

Start small. Yes, learning how to write tests takes time. Thinking about how to solve problems in a test-friendly way takes time. But start small and don’t overwhelm yourself. Here we picked one defect that needs a few small things: test data (users with roles and results for the view), navigating to a page, and checking results. 

All of your favorite behavior test tools have a tutorial to show you how to navigate a site and fill in fields and press buttons. That’s all we’re doing here, and with only that knowledge, our work is considerably more solid. 

Stay tuned because I will get deeper into the technical aspects of testing. Next time, we’ll get Drupal Test Traits going against a Drupal site and write some tests. The concepts will apply to other solutions, but this series will focus on DTT. For the impatient, here is a starting point:

Install DTT. Follow the examples. Starting writing a test! https://www.drupal.org/project/dtt