PHPUnit code coverage : wrong files coverage rate

Laurent Polese

Created March 07, 2013 18:47

Hello,

I'm not sure this is about PhpStorm, but maybe someone here could help.
We're just beginning to use unit testing with PHPUnit / PhpStorm, and we're facing an issue.
Our test classes are next to tested classes in our file system.
In my example, I have lib/Chrono.php and lib/ChronoTest.php
Phpstorm file coverage probleme.png
But we didn't manage to exclude test files from coverage metrics for data displayed in the Project view.
So the test files are counted in total files in each folder, which displays 50% coverage while 100% are covered.

This is a real issue for us, because we aim 100% file coverage and wanted to rely on PhpStorm to get this information.

Here is my PHPUnit xml file

<phpunit bootstrap="CONFIG.php" colors="true">
    <testsuite name="Test Laurent conf Fred">
        <directory suffix="Test.php">lib</directory>
    </testsuite>
    <filter>
        <whitelist>
            <directory suffix=".php">lib</directory>
            <exclude>
                <directory suffix="Test.php">lib</directory>
            </exclude>
        </whitelist>
    </filter>
    <logging>
        <log type="coverage-html" target="./codeCoverage" charset="UTF-8"
             yui="true" highlight="false"
             lowUpperBound="35" highLowerBound="70"/>
        <log type="coverage-clover" target="./codeCoverage/codeCoverage.xml"/>
        <log type="metrics-xml" target="./metrics/metrics.xml"/>
        <log type="test-xml" target="./testResults/logfile.xml" logIncompleteSkipped="false"/>
    </logging>
</phpunit>

I tried many combinations with excludes, blacklists but nothing worked.

Any idea ? Thanks a lot ;)
Laurent

3 comments

Andriy Bazanov

Created March 07, 2013 20:38

Hi Laurent,

#1

So the test files are counted in total files in each folder, which displays 50% coverage while 100% are covered.

This is a real issue for us, because we aim 100% file coverage and wanted to rely on PhpStorm to get this information.

You only reliable option is to PROPERLY separate tested classes from actual tests (this will work fine in both any IDE and PHPUnit itself).

Shortly speaking: tested classes in one folder and tests in another (which should be marked as Test Source Root in "Settings | Directories")

Example:

[MyProject]
|-- [lib]
|  |-- [MyFramework]
|    |-- Chrono.php
|-- [tests]
|  |-- [MyFramework]
|  |  |-- ChronoTest.php
|  |-- [assets] (resources/files used during testing -- to check if tests doing well, e.g. expected output)

|  |-- [reports] (for generated reports; e.g. HTML code coverage)

|  |-- [var] (for temp files created during tests execution)
|  |-- bootstrap.php (bootstrap file executed before running phphunit tests -- used to initialise require classes/autoloading etc)

|  |-- phpunit.xml (phpunit configuration file)
|-- build.xml (phing build file -- to cleanup temp/reports folders etc)

Of course, you are free to submit Feature / Bug ticket to the Issue Tracker (if it is not there already) and it may get fixed .. but separating tests from actual classes is more correct approach (and will work everywhere).

#2 -- If you are referring to wrong percentage for specific file (fro your screenshot: "98% lines") -- it's either

PHPUnit's own bug (I guess) -- when it generates coverage clover.xml file, it write a bit different/wrong values;
or PhpStorm's way of calculating it (PhpStorm uses ceratin fields/values from coverage clover.xml file as is while PHPUnit's own HTML report is either uses different fields or applies some extra logic)

In any case: http://youtrack.jetbrains.com/issue/WI-9910

Laurent Polese

Created March 08, 2013 08:45

Hi Andriy

thanks for your quick and very clear answer.

#1 Files coverage metric
We are currently discussing about where to place the test classes, and we are not sure yet.
That's how we found this issue, so I guess we'll separate test classes and actual ones.

#2 Lines coverage metric
We have this issue too, I didn't mention it here because I found the issue you pointed on Youtrack.
I tried to add return statement to each method with no effect.
But when I use IDE settings to point tests instead of xml configuration, I get 100% coverage as expected.

Thanks again
Laurent

Patrice Bourgougnon

Created May 16, 2017 08:51

Hi Laurent,

More and more people think keeping test files close to their tested file is a best practice. I do so.

If you wish to keep your folder structure, you could just include your test files in your coverage whitelist (remove it from your whitelist exclusions). It would solve your count issue, and also show you very interesting things in your test code.

Please sign in to leave a comment.