PHPCoverage - an open-source code coverage measurement tool for PHP Applications

About

PHPCoverage is an open-source tool for measuring and reporting code coverage provided by the test suite of a PHP application. PHPCoverage can instrument and record the line coverage information for any PHP script at runtime.

PHPCoverage also provides an extensible reporting mechanism with a standard HTML report implemented out of the box. The default report displays the summary information about the code coverage for an application and also shows the detailed information about a file including which lines were actually executed and with what frequency. It is possible to specify the directories and files that should be included and/or excluded from a coverage measurement. Screenshots of the summary and detailed line coverage reports are available here.

PHPCoverage works on PHP 5.0 and newer and uses Xdebug Extension for gathering the coverage data.

Requirements

• PHP 5.0 or newer
• Xdebug Extension configured with PHP
• XML_Parser PEAR module version 1.2.5 or newer.
• Web browser to view the coverage report

Frequently Asked Questions

1. How do I install PHPCoverage?

   Just expand the tarball

   tar -zxvf spikephpcoverage-0.6.tar.gz

   This will create a directory called spikephpcoverage-0.6 and expand all files in it. You should export an enviroment variable PHPCOVERAGE_HOME or define a constant in your test script that refers to the location of src directory and use it when including the PHPCoverage files:

   $ export PHPCOVERAGE_HOME=/path/to/phpcoverage/src

   OR

   define("PHPCOVERAGE_HOME", "/path/to/phpcoverage/src"); 
   require_once PHPCOVERAGE_HOME . "/CoverageRecorder.php";

2. How do I use PHPCoverage for recording code coverage locally?

   This topic explains how PHPCoverage can be used to record local code coverage information. Any PHP script that runs in the same context as that of the invoker is said to run locally. In practice, if you are running tests that do not exercise PHP code through a web server, you can use the normal CoverageRecorder class. On the other hand, if your test suite contains web tests that access PHP code running in a web server (on same or different) machine, you will need to follow a different procedure for code coverage measurement.

   • Install the Xdebug Extension on the machine.
   • You will need to include at least two files in your main PHP test script or executable file.

     require_once PHPCOVERAGE_HOME . "/CoverageRecorder.php";
     require_once PHPCOVERAGE_HOME . "/reporter/HtmlCoverageReporter.php";
     

     The first file is the coverage recorder class that can start and stop coverage recordings and configure parameters. The second file is the HTML Report Generator class.

   • Instantiate the HTML Reporter class.

     $reporter = new HtmlCoverageReporter("Code Coverage Report", "", "report");

     The first argument to the constructor is the title of the coverage report. The second argument is an optional style sheet name. If not specified, default style sheet is used. The third argument is the name of the directory where the report files should be generated. The directory will be created if it does not exist.

   • Instantiate the Coverage Recorder class. The constructor for this class can take an array of file names or directories to be included in the coverage recording, an array of file names or directories to be excluded from the coverage recordings, and a Reporter object.

     $includePaths = array(".");
     $excludePaths = array("test", "foo.php");
     $cov = new CoverageRecorder($includePaths, $excludePaths, $reporter);
     

     Please note that the PHPCoverage directory will always be omitted from the recordings irrespective of whether it is explicitly mentioned as an "excludePath" or not.

   • Start the code coverage measurement just before starting the tests and stop it just after.

     $cov->startInstrumentation();
     ... // run tests
     $cov->stopInstrumentation();

     Make sure that you exclude the directory containing the actual tests.

   • Generate the report and optionally print the summary to console.

     $cov->generateReport();
     $reporter->printTextSummary();
     

     The report files can be accessed in the output directory specified.

3. How do I use PHPCoverage for recording remote code coverage?

   If your tests are trying to exercise PHP code that is running within a web server on same or different machine, you should use RemoteCoverageRecorder to measure code coverage. You should also instrument the application code before running your tests. The actual process of instrumenting a file is straightforward. PHPCoverage files need to be included at the top and bottom of each of the application files. PHPCoverage comes with a command line utility to do this for you.

   Deploy your web application into your web server. This typically involves copying the application files into a web directory managed by the webserver or creating a symlink from such directory to your application.

   Execute the command line utility to instrument your application.

   $ cd /path/to/application/base/dir
   $ export PHPCOVERAGE_HOME=/path/to/phpcoverage/location
   $ php /cli/instrument.php -b . file1.php file2.php ... 

   You need to specify your application base directory with -b option, followed by a list of files or directories or both. The application base directory is where your index.php is stored. PHPCoverage files will be copied to this directory and need to be accessed over HTTP to initialize code coverage measurements. instrument.php also supports following options:

   • -b application-base-path
   • -p phpcoverage-home-path
   • -r recursive
   • -u undo instrumentation
   • -h help

   The undo option can only uninstrument files that were instrumented with this script. After this step, make sure that two files phpcoverage.remote.top.inc.php and phpcoverage.remote.bottom.inc.php have been copied to your application base path. Also, the instrumented files should include these two files at the top and the bottom.

   • Make sure that your application URL is private and nobody else is accessing the application except for your test script. This is important to ensure correct code coverage recording.
   • Change your test driver script to include PHPCoverage calls. A sample driver script is given here. The script uses Simple Test web test suite along with PHPCoverage.

   addTestFile('TestMainLinks.php');
   
       // PHPCoverage includes
       require_once dirname(__FILE__) . "/phpcoverage.inc.php";
       require_once PHPCOVERAGE_HOME . "/remote/RemoteCoverageRecorder.php";
       require_once PHPCOVERAGE_HOME . "/reporter/HtmlCoverageReporter.php";
   
       $cov_weburl = $weburl . "phpcoverage.remote.top.inc.php";
       // Initialize RemoteCoverageRecorder
       file_get_contents($cov_weburl . "?phpcoverage-action=init&cov-file-name=" 
          . urlencode("phpcoverage.data.xml") . "&tmp-dir=". urlencode("/tmp"));
   
       // Run the simple test web test suite
       $testsuite->run(new TextReporter());
   
       // The coverage data is available in xml form at following url
       // You may download this file and save it locally for debugging
       $xmlUrl = $cov_weburl . "?phpcoverage-action=get-coverage-xml";
       
       // Configure reporter, and generate report
       $covReporter = new HtmlCoverageReporter(
         "MyApp Web Test Code Coverage", "", "php-coverage-remote-report");
       $excludePaths = array("../"); 
       // Set the include path for the web-app
       // PHPCOVERAGE_APPBASE_PATH is passed on the commandline
       $includePaths = array(realpath($PHPCOVERAGE_APPBASE_PATH));
   
       // Notice the coverage recorder is of type RemoteCoverageRecorder
       $cov = new RemoteCoverageRecorder($includePaths, $excludePaths, $covReporter);
       // Pass the XML data url to generateReport function
       // Alternatively, you can pass the complete file path of a local file here.
       // The second parameter signifies that this is a url stream and an XML string
       $cov->generateReport($xmlUrl, true);
       $covReporter->printTextSummary("php-coverage-remote-report/report.txt");
   
       // Clean up
       file_get_contents($cov_weburl . "?phpcoverage-action=cleanup");
   ?>
   

   Note that calls are made to the phpcoverage.remote.top.inc.php file over HTTP to intialize coverage recorder object. Following points distinguish remote code coverage measurement from local one:

   • $weburl points to your application's URL. Something like http://server-name/myapp/. The phpcoverage.remote.top.inc.php should be reachable by the subsequent HTTP calls from this URL.
   • phpcoverage-action should be set to init in order to initialize the coverage recorder. This must be done exactly once before any of your tests begin. The two parameters passed for initialization are cov-file-name which is set to a filename where the coverage recording XML file will be written on the server. The second parameter tmp-dir is a directory where the scripts executing in web server can write information. The usual place for this is the /tmp directory on UNIX systems. The coverage XML data file will be created in this directory. Make sure that the data file name is not an existing file, as it will be deleted as part of the initialization.
   • Once the test execution is complete, coverage data must be retrieved from remote site, and the report will be generated on the client machine. This is done by setting phpcoverage-action to get-coverage-xml. Starting from PHPCoverage 0.6.5, this url itself can be passed to the generate report function. This is the recommended method since it avoids having to save the entire XML in a string which can eat up huge amount of memory for large applications.
   • The coverage recorder object of the type of RemoteCoverageRecorder is created. The generateReport method of this object accepts a XML string which was retrived earlier.
   • The code coverage recording is cleaned up by setting phpcoverage-action to cleanup.
   • The remote application base path can be passed on the commandline as PHPCOVERAGE_APPBASE_PATH=/path/to/webapp/basedir. This will be parsed by the phpcoverage.inc.php and made available to the test driver script.
   • Ensure that you are running only once code coverage measurement at any given time between a client-server pair. Otherwise, different coverage recordings might corrupt each other's data.
   • PHPCOVERAGE_HOME location must also be available on the remote machine where the web application is running. The easiest way to do this is export it as a shell variable before starting the webserver.
   • If your application is running on a different machine, you need to do some additional setup. Create the exact same directory structure where your application is deployed on the remote machine on your local machine. You do not need a web server running on the local machine. Deploy the application (copy all files) as before in this directory and run the instrumentation script as advised above. This will make sure that at the time of generating the report from the code coverage data gathered on the remote machine, PHPCoverage finds the application files in the same system path as the remote machine. Note that PHPCoverage needs to be installed on both machines. Also, the instrumented application needs to be present on both machines - even though it may not be running on the local machine.

   • How do I configure and install the Xdebug extension for PHP?

     Recent versions of PHPCoverage already come with a pre-compiled Xdebug shared library (for 32-bit Linux) or DLL (for Windows). For most Linux and Windows versions they should work right out of the box. In that case, proceed to configuration section below.

     If they don't, you may have to compile one for yourself. Doing so is very simple and is outlined below.

     The steps outlined here are for PHP 5 installed along with SpikeSource Core Stack. They should be fairly similar for other PHP installations.

     Compiling Xdebug

     Download the Xdebug 2.x tar ball (xdebug-2.0.0beta2, as of this writing) from Xdebug download page.

     $ tar -xzf xdebug-2.0.0beta2.tgz
     $ cd xdebug-2.0.0beta2
     $ sudo -s
     % source /opt/oss/env.sh
     % phpize
     % ./configure --enable-xdebug
     % make
     % cp modules/xdebug.so /opt/oss/lib/php
     

     This should install the xdebug.so module inside /opt/oss/lib/php.

     Configuring Xdebug

     Hand edit your php.ini file and add the following line with the correct location of xdebug.so module.

     ## For Windows, please use the parameter: zend_extension_ts
     zend_extension="/opt/oss/lib/php/xdebug.so"
     

     For more details on installing Xdebug, please visit their website.

   • How do I install XML_Parser PEAR module?

     Assuming you have PEAR installed along with PHP 5 and your machine is able to connect to the Internet, just type:

     $ pear install XML_Parser
     

   • What php.ini settings do I need to change (if any)?

     Apart from adding the zend_extenstion line in your php.ini file for Xdebug, you might have to bump up the memory limit for code coverage measurements. Depending on the size of the application, PHPCoverage may require a large amount of memory than generally configured in php.ini, by default. The default 8M limit is almost always not enough. Since coverage measurement is not supposed to happen when the system is being accessed by other users, it is recommended to increase the limit at least to 128MB. It might also be advisable to remove the memory limit by setting the memory_limit parameter to -1 in php.ini file.

   • What information is gathered by PHPCoverage?

     PHPCoverage is designed to record how many times a line of PHP code gets executed when instrumentation is enabled. This information is important in validating a test suite that tests the functionality of your application. A good test suite will exercise most lines of code at least once thus giving a fairly high code coverage reading. Code coverage information helps highlight untested code and unused functionality. Code coverage thus provides a metric to measure the completeness of a test suite for an application.

     There are different types of code coverage measurements possible such as line coverage, conditional coverage, method (function) coverage, etc. PHPCoverage currently reports only line coverage information. As part of its summary report, it measures and displays the following parameters:

     • Total Files: This is the total number of source code files included in the code coverage measurement.
     • Total Lines: These include lines of code, comments, whitespace, and embedded HTML code (if any) in a source code file.
     • Total Lines of Code: These are the lines of code that can be executed. Thus, this value excludes comments, blank lines, variable and function declarations that do not contain initializations, include and require statements, etc.
     • Total Covered Lines of Code: These are the lines of code that were executed at least once during the code coverage recording. Needless to say, these can only be part of executable lines of code. These lines will be shown in green color with the frequency of their execution.
     • Total Missed Lines of Code: Lines of executable code that were not executed even once during the recording. These lines are highlighted in red color to flag that they were never tested or exercised.
     • Code Coverage: The percentage of lines of executable code that were actually executed at least once.

     No information is reported on the internet. All reports are generated locally.

   • How do I test if PHPCoverage is working?

     After expanding the distribution, go to the 'samples' directory. There are two very basic samples included in this directory (as of release 0.6.2). One is a local code coverage measurement sample and the other is a remote coverage measurement sample. Please see the README files in the respective sample directories for instructions on how to run those samples. These samples include very simple one file applications. You should be able to run and get expected code coverage from both these samples if you have performed all the set up steps correctly. The expected code coverage percentages are given in the respective README files.

   • I cannot see anything when I access my application after instrumenting. What is wrong? OR I get 0% code coverage in the report, but I am pretty sure that at least some portion of the code is being tested. What is wrong?

     First and foremost thing to do when you sense a problem is check the PHP error log. By default the error logging might be turned off in the php.ini file. To enable error logging, change the "log_errors" parameter to "On", and assign a log file path to "error_log" parameter in php.ini file. Make sure that the log file is writable by the user who is running the web server. Restart the web server after this change if you are measuring code coverage remotely. PHPCoverage writes a lot of debug information in the error log and chances are that you will find a hint as to what went wrong. In addition, make sure that you have all the dependencies installed on your system. For remote code coverage measurement, PHPCoverage and its dependencies must be installed on both the client and server machines.

   • Which license is PHPCoverage released under?

     Starting from release 0.6.6, PHPCoverage is dual-licensed under the Open Software License (v2.0) and the GNU Lesser General Public License (v2.1). Users are free to choose the license that best suites their need between these two.

   • Which platforms can I use PHPCoverage on?

     Starting from release 0.6.2, PHPCoverage works on Linux and Windows. The same version should work on both these platforms. There is no separate distribution for Windows. Also, you are encouraged to test it on any other platforms. The goal of PHPCoverage project is to make it a cross-platform tool.

   • Where can I report problems with PHPCoverage?

     Please visit PHPCoverage tracker page for reporting bugs, feature requests or asking for help.