Test-driven development of WordPress plugins

Saturday, 25 November 2006

Evermore, one of my WordPress plugins, has been around for a year or so. It’s worked very well, but occasionally I did receive reports of mysterious failures. In an effort to fix it once and for all, I decided to take a new approach. So for the last couple of updates I have used a test-driven development (TDD) approach to iron out all remaining wrinkles. The results were interesting.

The heart of Evermore is the algorithm it uses to detect where a paragraph ends and another begins. It’s simplest to do this before WordPress reformats the paragraphs, because then Evermore can take advantage of WordPress’s tag-balancing and other functions. (For example, Evermore can break a post in the middle of a <div>, and WordPress will insert a </div> in the right place.) However, this makes it tricky to find paragraph breaks. For example, a double linebreak delimits a paragraph, but Evermore also allows a double <br/>. Also, a paragraph ends whenever another begins (for example, with a <p>).

In case all this sounds a bit simple, here is the regular expression Evermore would use to find the end of the second paragraph of a post.

!^(\s*.+?(?:(?:(?:\r\n *\r\n|\r *\r|\n *\n|<br\s*/?>\s*<br\s
){1})((?:(?:\r\n *\r\n|\r *\r|\n *\n|<br\s*/?>\s*<br\s*/?>)|

This kind of thing is hard to test manually — there are so many possible situations. So I wrote an automated test suite. This contains a number of different snippets of post content (paragraphs separated by <p> tags, paragraphs separated by newlines, tables embedded in paragraphs, DOS and Unix line breaks and so on). The test harness takes every possible combination of these, and runs them through Evermore with each possible combination of options. In total, it tests 536 test cases.

When I started this test approach, Evermore 1.0.1 was the current version. It worked pretty well except in a few obscure cases. But there are a lot of different obscure cases — Evermore 1.0.1 scores a miserable 6.7% pass rate in the test suite: 36 out of 536. I released Evermore 2.0 shortly after I started working on the test suite, and it has a much better pass rate of 28.4% (152/536).

After 2.0, I focused on passing all the tests. Evermore 2.1 and 2.2 (the latest) both have a nice 100% pass rate. I’m pretty confident that Evermore can handle not just every case I can think of, but every combination of every case I can think of. Of course, I haven’t thought of everything; and when a new bug comes to light, I will first expand my test suite to cover it, and then modify Evermore until it passes 100% of the new suite. Then I will know that not only I have fixed the new bug, but I haven’t re-introduced any old ones.

I’m using this approach in other plugin development work too. In fact, I’ve had to use it to help update the Code Markup plugin that i use on Semicolon, because Code Markup wasn’t able to correctly display the regular expression in this post! So you can expect a new release of Code Markup soon. TDD is a wonderful thing. (And so are regular expressions.)

Tags: ,


You can leave a comment, or trackback from your own site.

  1. Hi Bennett and great post. Awesome to see how TDD is helping you – It’s always great to see success stories with test driven development. Thanks for sharing.

  2. Hi, I’m trying to load Evermore to a 2.5.1 WP blog – free theme – I keep getting a “CANNOT LOAD EVERMORE.PHP error.

    Eeeek – how do I get this to work, I would love to use this plugin.


  3. Remove the evermore.php from the evermore directory. Just add evermore.php to your plug-ins directory. Re-activate and you should be good to go.

  4. This is the only post I could find using Google on TDD for WordPress plugins. I’m just starting to explore this, and I’m wondering exactly how you did it. My experience with TDD so far is doing it through the command line, but WordPress plugins are typically filled with dependencies on a running instance of WordPress. If you have a minute, I’d really like to hear how you went about setting up the test environment. Thanks!

  5. I’m in the same boat as Mike. It sounds like you were able to test a specific component of your plugin against an independent fixture of test cases. Do your test cases set up a working WordPress environment that has access to all of the APIs that a plugin would out there in the wild?

    If so I’d be thrilled to hear about how you did it. Thanks for the post!

Leave a comment