Hello! Today I would like to share experience in test coverage reporting performance optimisation for PHP project.

Once we decided to measure and watch test coverage for the project. There is a Symfony components project, which partially covered by unit and integration tests.

We tried to run tests with coverage reporting. It was about 65% test coverage. Project contains about 45000 lines of code. We also realized that coverage reporting affects performance significantly.

There are some useful references:

• https://medium.com/@nicocabot/speed-up-phpunit-code-coverage-analysis-4e35345b3dad
• https://kizu514.com/blog/phpdbg-is-much-faster-than-xdebug-for-code-coverage/

Without coverage.

Without coverage reporting tests ape running 5 min. 47 sec.

Configuration example:

name: Tests
jobs:
  php-tests:
      runs-on: ubuntu-latest
      steps:
        - name: Git checkout Core
          uses: actions/checkout@v2
        - name: Setup PHP
          uses: shivammathur/setup-php@v2
          with:
              php-version: '7.4'
              coverage: none
        - name: Run tests
          run: vendor/bin/phpunit

Xdebug

There is the most common debugger for PHP. It also can calculate tests coverage. Affects performance significantly, which might be sensitive for the large project.

Tests ape running 30 min. 32 sec. So, it's problematic to run them for each commit.

Configuration example:

name: Tests
jobs:
  php-tests:
      runs-on: ubuntu-latest
      steps:
        - name: Git checkout Core
          uses: actions/checkout@v2
        - name: Setup PHP
          uses: shivammathur/setup-php@v2
          with:
              php-version: '7.4'
              extensions: xdebug
              coverage: xdebug
        - name: Run tests
          run: vendor/bin/phpunit --coverage-html coverage --coverage-clover coverage.xml

Phpdbg

There is an alternative lightweight debugger.

Tests ape running 14 min. 31 sec. Seems to be better! But it's still 3 times slower, then without coverage. Is it possible to run faster?

Configuration example:

name: Tests
jobs:
  php-tests:
      runs-on: ubuntu-latest
      steps:
        - name: Git checkout Core
          uses: actions/checkout@v2
        - name: Setup PHP
          uses: shivammathur/setup-php@v2
          with:
              php-version: '7.4'
              extensions: phpdbg
              coverage: none
        - name: Run tests
          run: phpdbg -qrr -d memory_limit=-1 vendor/bin/phpunit --coverage-html coverage --coverage-clover coverage.xml

Pcov

There is a special tool for tests coverage calculation.

Tests ape running 9 min. 35 sec.

Configuration example:

name: Tests
jobs:
  php-tests:
      runs-on: ubuntu-latest
      steps:
        - name: Git checkout Core
          uses: actions/checkout@v2
        - name: Setup PHP
          uses: shivammathur/setup-php@v2
          with:
              php-version: '7.4'
              coverage: pcov
        - name: Run tests
          run: php -d memory_limit=1024M -d pcov.enabled=1 vendor/bin/phpunit --coverage-html coverage --coverage-clover coverage.xml

Finally, decided to use Pcov.

That's all for today. Thank you for your attention!