Hello! Today I would like to share Code Coverage Analysis configuration for PHP(Symfony) project. Then I'm going to demonstrate integration with Github Actions.

Code Coverage Analysis is important for understanding quality of tests, and how much they are testing code. More information about Code Coverage Analysis.

Initially I configured Code Coverage Analysis for this blog. Then I also made it for one of my open source projects - placeholder-service. Idea behind is to make it more clear and share all configs.

Xdebug

Firstly we need to install Xdebug. It can be installed different way depends on environment. Docker environment:

FROM php:8.0-fpm

# ... #

RUN pecl install xdebug
RUN docker-php-ext-enable xdebug

Local Linux Ubuntu 20.04 environment:

sudo apt install php8.0-xdebug

Code Coverage Analysis

Then we need to add additional params for phpunit to generate code coverage report.

./bin/phpunit --coverage-html coverage --coverage-clover coverage.xml

We will add --coverage-html coverage parameter for saving html report to coverage directory.

And --coverage-clover coverage.xml parameter for saving xml report. We will use it for generating code coverage badge. It also might be useful for loading coverage to separate service.

We can add shortcut to composer.json. Then we will only need to run composer test.

{
    ...
    "scripts": {
        ...
        "test": [
            "./bin/phpunit --coverage-html coverage --coverage-clover coverage.xml"
        ],
        ...
    }
    ...
}

Finally, we will have such report. There is a general coverage percent and statistics for each specific file. Uncovered lines are marked as red, so we can analyze this report and add missing tests.

Generate code coverage badge

Code coverage badge can be generated with PHPCoverageBadge library. Installation:

composer require --dev jaschilz/php-coverage-badger

Generate badge based on coverage statistics:

vendor/bin/php-coverage-badger coverage.xml .github/badges/coverage.svg

We also can add shortcut to composer.json. Then we will only need to run composer update-badges.

{
    ...
    "scripts": {
        ...
        "update-badges": [
            "vendor/bin/php-coverage-badger coverage.xml .github/badges/coverage.svg"
        ],
        ...
    }
    ...
}

Then let's add badge to README.md. It won't work for a private repository, but we can save badge on external web-server in this case.

...
![Code Coverage](https://raw.githubusercontent.com/antonshell/placeholder-service/master/.github/badges/coverage.svg)

Integration with Github Actions

Now we will setup Code Coverage Analysis for Github Actions. Then we could see code coverage report for each Pull Request, and set required coverage percentage. Firstly need to add Xdebug to PHP installation:

# https://github.com/shivammathur/setup-php (community)
- name: Setup PHP
  uses: shivammathur/setup-php@v2
  with:
    php-version: ${{ matrix.php-versions }}
    coverage: xdebug
    tools: composer:v2
    extensions: mbstring, xml, ctype, iconv, intl, pdo_sqlite, dom, filter, gd, iconv, json, mbstring, pdo, xdebug
  env:
    update: true

Then we will add check to .github/workflows/tests.yml workflow. It will check that code coverage greater or equal then 40%. We will use coverage.xml file, that was generated on tests run.

- name: Check test coverage
  id: test-coverage
  uses: johanvanhelden/gha-clover-test-coverage-check@v1
  with:
    percentage: "40"
    filename: "coverage.xml"

Generate html report for each Pull Request

We also going to have detailed code coverage report for each Pull Request. Probably we could somehow use artifacts storage for that. Or 3rd party service, like codecov.io (which is paid). Or we can use our own external web server.

For a private repository it's very important to protect web server with basic auth or other methods!

## —— Upload tests coverage report to remote server
- name: Extract branch name
  shell: bash
  run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
  id: extract_branch
- name: Print branch name
  shell: bash
  run: echo ${{ steps.extract_branch.outputs.branch }}
- name: Create directories for reports
  uses: appleboy/ssh-action@master
  env:
    BRANCH: ${{ steps.extract_branch.outputs.branch }}
    FILES_PATH: ${{ secrets.GA_FILES_PATH }}
  with:
    envs: BRANCH,FILES_PATH
    host: ${{ secrets.REMOTE_HOST }}
    username: ${{ secrets.REMOTE_USER }}
    key: ${{ secrets.SERVER_SSH_KEY }}
    passphrase: ''
    script: mkdir -p $FILES_PATH/$BRANCH/coverage
- name: Uploads reports to server
  uses: appleboy/scp-action@master
  with:
    host: ${{ secrets.REMOTE_HOST }}
    username: ${{ secrets.REMOTE_USER }}
    key: ${{ secrets.SERVER_SSH_KEY }}
    passphrase: ''
    rm: true
    source: "coverage"
    target: ${{ secrets.GA_FILES_PATH }}/${{ steps.extract_branch.outputs.branch }}/coverage

We will extract branch name, then connect to Web server over SSH and create target directory. Then we will save html report to the target directory. THere is a double /coverage/coverage because otherwise branch directory will be deleted.

We also need to add GA_FILES_PATH variable in Github Secrets storage. We will save a path to target directory on Web server, like /var/www/files/placeholder-service.

And we also need REMOTE_HOST, REMOTE_USER, SERVER_SSH_KEY variables for SSH connection. There are more details in my previous article or in this tutorial.

After workflow will be completed, report will be available by link: http://files.antonshell.me/github-actions/placeholder-service/configure_code_coverage/coverage/coverage.

Add Pull Request comment

Now we need to show generated report on Pull Request page. We can't do it directly, however we can add a link to generated report. We will create .github/workflows/pull_request_comments.yml workflow, that will post comment to Pull Request.

name: Pull request comments
on:
  pull_request:

jobs:
  pull_request_comments:
    runs-on: ubuntu-20.04
    steps:
      - name: Extract branch name
        shell: bash
        run: echo "##[set-output name=branch;]$(echo ${GITHUB_HEAD_REF})"
        id: extract_branch
      - name: Print branch name
        shell: bash
        run: echo ${{ steps.extract_branch.outputs.branch }}
      - name: Add test coverage report comment
        uses: mshick/add-pr-comment@v1
        with:
          message: |
            Test coverage report: [http://private.antonshell.me/github-actions/antonshell/${{ steps.extract_branch.outputs.branch }}/coverage/coverage](http://private.antonshell.me/github-actions/antonshell/${{ steps.extract_branch.outputs.branch }}/coverage/coverage)
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          repo-token-user-login: 'github-actions[bot]' # The user.login for temporary GitHub tokens
          allow-repeats: false # This is the default

Demo project is available on github. There is also Pull Request with code coverage implementation. That's all for today. Thank you for your attention!