igergov.com
← Writing
DevOpsCI · 5 min · BG EN ES DE

A broken test is good news

A red pipeline means the bug was caught early, at a point where fixing it is cheap.

When you push code and the pipeline turns red, the first reaction is usually irritation. In the DevOps course at TU-Sofia we spend time on exactly this moment, because it is the whole point of the pipeline.

Try it yourself

A pipeline is a series of steps that run automatically on every commit: build, tests, a vulnerability scan, deployment, monitoring. If a step fails, the ones after it do not start. Run the two commits below and compare.

$ git push origin main e07b3d1 · quick fix

  1. ✓ passed Build
  2. ✕ stopped Test
  3. · skipped Scan
  4. · skipped Deploy
  5. · skipped Monitor

The test caught the bug. The pipeline stopped before it reached any user. That is what it is for.

The later, the more expensive

The same bug costs a different amount depending on where we catch it. Each further step is further away from the moment you wrote the code, and it affects more people. Click “One step later” and watch the cost grow.

Where we catch it
Where it shows up

You run pytest before you push. One test fails.

What it costs · On your machine

Minutes. You are still in the same file and remember why you wrote every line. You fix it and carry on.

$ pytest
FAILED tests/test_notes.py::test_title_is_required

That is why a pipeline that stops saves time. It is the last cheap place for a bug. After it, every step costs more.

What it looks like in the labs

All the exercises in the course are done on the lab template, a small note-taking app in Python. From day one it has tests with pytest and style checks with ruff. The pipeline the students write runs exactly those:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements-dev.txt
      - run: ruff check .
      - run: pytest   # if this fails, the next steps do not start

What to do when it is red

  1. Open the step that failed and read the first error. The ones after it are often a consequence of it.
  2. Run the same check locally (pytest or ruff check .) so you can see it on your own machine.
  3. Fix the code. The test only changes if the requirement really has changed.

Red means that this time the bug was caught in time.