Skip to main content
CI CD quality gatesPlaywrighttest automation

CI/CD Quality Gates That Don't Block the Team

12 September 2026 · OpenCrevo

CI/CD quality gates fail when they block every flake or when they block nothing. The useful gate is narrow: new failures on @critical paths, plus a flake budget you already track. Do not gate on coverage percent. Pair this with which tests to run on every PR (/blog/which-tests-to-run-on-every-pull-request/) and the CI trust playbook (/blog/ci-pipeline-full-of-failed-tests-nobody-trusts/). Score the current mess with the QA maturity assessment (/qa-maturity/).

The failure: one gate, 400 tests, no owners

A monorepo runs the full Playwright suite on every PR. Two known flakes sit in checkout. Developers rebase until green. A real refund bug rides through because nobody looks at the 18th red check. The gate trained people to ignore it.

A flake-aware CI/CD quality gate

import { test, expect } from "@playwright/test";

test("refund confirmation @critical", async ({ page }) => {
  await page.goto("/orders/1/refund");
  await page.getByRole("button", { name: "Confirm refund" }).click();
  await expect(page.getByText("Refund queued")).toBeVisible();
});
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npx playwright install --with-deps chromium
      - name: Critical paths (merge gate)
        run: npx playwright test --grep @critical --reporter=github
      - name: Full suite (informational)
        if: always()
        continue-on-error: true
        run: npx playwright test --grep-invert @critical
  • Merge is blocked only by @critical. Everything else can be red without stopping the train — but it still publishes a report.
  • Known flakes get a ticket and a quarantine tag, not a silent retry loop. See AI quality engineering services if you need the factory to own that queue.
  • Never add a coverage-percentage gate. Coverage can rise while refund is untested.

Pass, fail, cleanup

  • Pass: Breaking refund confirmation fails the PR; a flake in settings does not.
  • Fail: The gate still runs 400 tests serially and people rebase instead of reading traces.
  • Cleanup: Remove coverage-percent required checks from the branch protection rules.
START YOUR QUALITY JOURNEY

Your next chapter starts with a conversation.

Book a free quality audit. We'll review your AI system, identify the highest-risk failure modes, and map a quality roadmap tailored to your stack.