• How to run PHPUnit tests for WordPress plugins in situ

    For every new release of WordPress, plugin and theme developers are expected to test that their products are compatible with the new version, and to update their “Compatible up to” status. But if you have lots and lots of plugins, how do you test them all? WordPress uses a tool called PHPUnit. Plugins and themes can do the same. This post discusses the solution that I’ve been developing to PHPUnit test my WordPress plugins in situ.

    Contents

    What do I mean by in situ?

    WordPress’s core tests using PHPUnit basically start with and end with an empty database, with no plugins activated and the default theme. For more complex tests the data has to be programmatically created.

    But what if my problem happens in a particular situation in a particular environment? To help me track down a problem I’d like to be able to run the same set of tests in a number of different environments.

    And when I’ve found the problem, I want to be able to perform a test that fails when the system is exhibiting the problem and passes when the problem’s been fixed.

    My requirements for in situ testing are:

    • Running tests in my normal development environment.
    • Don’t destroy and recreate content.
    • Easily run the same tests in multiple environments
    • Testing in different WordPress Multisite domains.
    • Being able to develop tests to use existing data.
    • Being able to use my trace functions.

    This is different from the method employed for most other plugins, which work in the test environment created for WordPress core, and which often have to artificially load themselves in PHPUnit bootstrap processing.

    Assumptions and pre-requisites

    • You want to write and run your own PHPUnit tests.
    • You can use the command line interface.
    • You can create batch scripts for your operating system.
    • You have an up to date WordPress development installation.
    • You can clone Git repositories into your WordPress development directories.
    • You can write and test PHP code.
    • You can read manuals and follow instructions.
    • You’ll let me know if there’s something wrong with these notes.

    Installation

    phpunit command – Windows batch file

    setlocal
    set PRE_PHPUNIT_CD=%CD%
    php C:\path-to-file\phpunit-5.5.2.phar "--verbose" "--log-json=phpunit.json" "--disallow-test-output" %*
    endlocal
    

    Notes:

    • setlocal and endlocal control the visibility of environment variables in a batch file.
    • PRE_PHPUNIT_CD is required by oik-wp to help locate the correct wp-config.php file to use. It will contain the current working directory before calling PHPUnit.
    • Here we’re running phpunit-5.5.2.phar passing the following parameters
      • --verbose – to output more verbose information.
      • --log-json=phpunit.json – log test execution in JSON format.
      • --disallow-test-output – be strict about output during tests.
      • %* – pass passed parameters to PHPUnit

    Test installation

    For the purpose of these notes let’s assume your development environment is installed in C:\apache\htdocs\wordpress, and that you’re using a standard installation not multisite.

    Test that PHPUnit works

    cd \apache\htdocs\wordpress\wp-content\plugins\oik-batch
    phpunit --version
    

    Expected output:

    PHPUnit 5.5.2 by Sebastian Bergmann and contributors.
    

    Test that oik-batch’s PHPUnit tests work

    cd \apache\htdocs\wordpress\wp-content\plugins\oik-batch
    phpunit
    

    Expected output:

    Searching for wp-config.php in directories leading to: C:\apache\htdocs\wordpress\wp-content\plugins\oik-batch
    Found wp-config.php in: C:\apache\htdocs/
    Found wp-config.php in: C:\apache\htdocs\wordpress/
    Domain: localhost
    cli
    oik-wp running WordPress 4.6
    C:\apache\htdocs\wordpress\wp-content\plugins\oik-batch
    cli
    Using wordpress-develop-tests plugin
    There you go: C:\apache\htdocs\wordpress\wp-content\plugins\oik-batch\tests\test-phpunit.php
    PHPUnit 5.5.2 by Sebastian Bergmann and contributors.
    
    Runtime:       PHP 7.0.7
    Configuration: C:\apache\htdocs\wordpress\wp-content\plugins\oik-batch\phpunit.xml
    
    .....                                                               5 / 5 (100%)
    
    Time: 8.76 seconds, Memory: 24.00MB
    
    OK (5 tests, 5 assertions)
    

    The important part of the output is the OK!

    Test sample plugin – allow-reinstalls

    There are two scenarios for the allow-reinstalls plugin; activated and deactivated.

    When the plugin is deactivated the tests should fail.

    Expected output:

    PHPUnit 5.5.2 by Sebastian Bergmann and contributors.
    
    Runtime:       PHP 7.0.7
    Configuration: C:\apache\htdocs\wordpress\wp-content\plugins\allow-reinstalls\phpunit.xml.dist
    
    FF                                                                  2 / 2 (100%)
    
    Time: 5.81 seconds, Memory: 20.00MB
    
    There were 2 failures:
    
    1) Tests_allow_reinstalls::test_allow_reinstalls_hooked_in
    Failed asserting that false is true.
    
    C:\apache\htdocs\wordpress\wp-content\plugins\allow-reinstalls\tests\test-allow-reinstalls.php:16
    
    2) Tests_allow_reinstalls::test_upgrader_package_options
    Failed asserting that true is false.
    
    C:\apache\htdocs\wordpress\wp-content\plugins\allow-reinstalls\tests\test-allow-reinstalls.php:23
    
    FAILURES!
    Tests: 2, Assertions: 2, Failures: 2.
    

    When the plugin is activated the test should work. Expected output:

    PHPUnit 5.5.2 by Sebastian Bergmann and contributors.
    
    Runtime:       PHP 7.0.7
    Configuration: C:\apache\htdocs\wordpress\wp-content\plugins\allow-reinstalls\phpunit.xml.dist
    
    ..                                                                  2 / 2 (100%)
    
    Time: 7.1 seconds, Memory: 20.00MB
    
    OK (2 tests, 2 assertions)
    

    Create your own PHPUnit tests

    Here we’ll create tests for the dinlo plugin.

    • Clone the bobbingwide/dinlo plugin from Github.
    • Activate the plugin.
    • Change directory to the plugin.
    • Copy the phpunit.xml.dist file from oik-batch.
    • Run phpunit.
    cd ..\dinlo
    mkdir tests
    copy ..\oik-batch\phpunit.xml.dist
    phpunit
    

    Expected output:

    No tests executed!
    

    Now write some tests in the test folder. e.g. tests\test-dinlo.php


    <?php
    class test_dinlo extends WP_UnitTestCase {
      function test_something() {
        assertTrue( true );
      }
    }
    &#91;/bw_geshi&#93;
    Then try running phpunit again.
    Expected output:
    &#91;bw_geshi none&#93;
    PHPUnit 5.5.2 by Sebastian Bergmann and contributors.

    Runtime:       PHP 7.0.7
    Configuration: C:\apache\htdocs\wordpress\wp-content\plugins\dinlo\phpunit.xml.dist

    .                                                                   1 / 1 (100%)

    Time: 5.87 seconds, Memory: 20.00MB

    OK (1 test, 1 assertion)
    &#91;/bw_geshi&#93;

    Now write a proper test which demonstrates a known problem. Show that the test fails with original code, but works when the problem with the original code is fixed.

    ....Currently left as <em>an exercise for the reader</em>.

    <a name="moreinsitu"></a>
    <h3>More about in situ</h3>

    Some of the challenges of <em>in situ</em> testing are that the configuration and data will be different between environments. Tests may pass or fail simply because a plugin is activated or not. The tests I write will have to take this into account, verifying the pre-conditions for performing the tests. My tests will also need to be fairly flexible when checking the output.

    I expect that I will have to develop some helper functions, by extending the WP_UnitTestCase class.
    <strong>It's early days yet, so watch this space.</strong>

    <a name="readmore"></a>
    <!-- &#91; bw_readmore&#93; -->
    <h3>Further information</h3>
    <h4>phpunit.xml.dist</h4>
    The configuration file used by oik-batch, allow-reinstalls and other plugins is basically the same.
    [bw_geshi html5]
    <phpunit
            bootstrap="../oik-batch/oik-wp.php"
            backupGlobals="false"
            colors="false"
            convertErrorsToExceptions="true"
            convertNoticesToExceptions="true"
            convertWarningsToExceptions="true"
            >
            <!-- for PHPUnit 4.8.0 or 5.5.0 -->
            <testsuites>
                    <testsuite>
                            <directory prefix="test-" suffix=".php">tests/</directory>
                    </testsuite>
            </testsuites>
    </phpunit>

    • The routine that’s run to bootstrap the test cases is oik-batch’s oik-wp.php.
    • We’re following WordPress’s conventions for test file names.

    Note: If phpunit just displays its help it probably means it can’t find a phpunit.xml or phpunit.xml.dist file.

    References

    Plugins to clone from GitHub bobbingwide

    Plugin Link
    oik-batch plugin bobbingwide/oik-batch
    wordpress-develop-tests plugin bobbingwide/wordpress-develop-tests
    allow-reinstalls plugin bobbingwide/allow-reinstalls
    dinlo plugin bobbingwide/dinlo

    Reference documentation

    Content Link
    WordPress and PHPUnit https://make.wordpress.org/core/handbook/testing/automated-testing/phpunit
    Getting Started with PHPUnit https://phpunit.de/getting-started.html
    PRE_PHPUNIT_CD bobbingwide/oik-batch#9
    WordPress Develop tests a2z – dynamic API reference develop.wp-a2z.org

    Other articles on PHPUnit testing plugins

    These are quite good for background reading, but note that the solutions are designed to run in the vanilla WordPress environment; a fresh install, not in situ.

    Each one of these posts is a good read in itself and also has a number of links worth following.


    ,

    Published:

    Last updated:

    March 21, 2019

Today’s word is this:

Friday

Food-le.com

aspic

Foodlewordle.io

ranch

Categories

Tide times from tidetimes.org.uk

Tide Times & Heights for Langstone Harbour on
24th May 2024
00:39 High Tide ( 4.72m )
05:52 Low Tide ( 1.15m )
13:04 High Tide ( 4.63m )
18:10 Low Tide ( 1.25m )

Tide times from tidetimes.org.uk

Tide Times & Heights for Northney on
24th May 2024
00:44 High Tide ( 4.4m )
05:50 Low Tide ( 0.77m )
13:11 High Tide ( 4.31m )
18:05 Low Tide ( 0.97m )