Automated Testing in CI
Run unit tests, integration tests, upload coverage, and fail the pipeline on regressions
Skills taught: cicd.testing, cicd.coverage, cicd.test-artifacts, cicd.test-reporters
Running tests in CI — the complete setup
name: Tests
on:
push:
branches: [main]
pull_request:
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '20', cache: 'npm' }
- run: npm ci
# Run tests with coverage
- name: Run unit tests
run: npm test -- --coverage --coverageReporters=json-summary --coverageReporters=lcov
# Upload coverage to Codecov (free for open source)
- name: Upload to Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage/lcov.info
fail_ci_if_error: true
# Upload coverage report as artifact
- name: Archive coverage
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage/
retention-days: 7
integration-tests:
runs-on: ubuntu-latest
# Start a PostgreSQL service container alongside the test runner
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_PASSWORD: testpass
POSTGRES_DB: testdb
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis:7-alpine
ports:
- 6379:6379
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '20', cache: 'npm' }
- run: npm ci
- name: Run integration tests
env:
DATABASE_URL: postgres://postgres:testpass@localhost:5432/testdb
REDIS_URL: redis://localhost:6379
run: npm run test:integration
Enforcing coverage thresholds
# In package.json / vitest.config.ts:
# coverage: { thresholds: { lines: 80, functions: 80, branches: 70 } }
# Test command fails with non-zero exit if thresholds aren't met
# In workflow:
- name: Test with coverage check
run: npm test -- --coverage
# Exits 1 if below threshold → step fails → job fails → PR is blocked
# For a more nuanced approach: check the JSON summary in the workflow
- name: Check coverage threshold
run: |
COVERAGE=$(node -e "const c = require('./coverage/coverage-summary.json'); \
console.log(c.total.lines.pct)")
echo "Line coverage: $COVERAGE%"
node -e "if ($COVERAGE < 80) { console.error('Coverage below 80%!'); process.exit(1); }"
Test result annotations
- name: Run tests
run: npm test -- --reporter=json --outputFile=test-results.json
# Publish test results as PR check with inline annotations
- name: Publish test results
uses: dorny/test-reporter@v1
if: always() # run even if tests fail
with:
name: Test Results
path: test-results.json
reporter: jest-json
# Now failing tests show as inline PR annotations
# and a 'Test Results' check appears in the PR
Tip: Use if: always() for artifact uploads and reports. By default, later steps are skipped if an earlier step fails. But you want to upload test results even when tests fail — that's exactly when you need the report. Adding if: always() to the upload step ensures it runs regardless of test outcome.
Frequently Asked Questions
Should I run tests on every commit or only on PRs?
Both. Run a fast subset (unit tests only, ~30s) on every push to any branch — instant feedback. Run the full suite including integration and E2E tests on PRs targeting main — thorough validation before merge. This gives speed on developer branches and safety before merging.