Intelligent Flaky Tests Capabilities
Three-Signal Detection
Identify flaky jobs and tests automatically using pass-on-retry attempts, same-commit mixed outcomes, and default-branch flip rates across 9+ test frameworks.
Wasted CI Time & Cost
See exact wasted runner minutes and estimated dollar cost per flaky job or test over a 14-day window so engineering teams can fix the most expensive flakes first.
30 & 90-Day Trend Analytics
Track active, newly detected, and auto-resolved flakes over time with breakdowns by repository and test framework (Jest, Vitest, Playwright, pytest, Go, JUnit, RSpec, and .NET).
One-Click Retry, Alerts & PR Context
Create pre-filled auto-retry rules in one click, dismiss accepted flakes with a 30-day cooldown, receive Slack or email digests, and surface known-flaky notes on PR failure comments.
See It In Action
Inspect every flaky job and test with commit-level evidence, captured error excerpts, and wasted runner spend.
Gitera groups flaky tests under their parent job, highlights pass-on-retry evidence, and resolves items automatically after 7 flake-free days.
How Flaky Tests Detection Works
Zero workflow changes or custom XML uploads required — Gitera analyzes the runs and logs it already receives.
Ingest Runs & Parse Failure Logs
As GitHub Actions runs complete, Gitera tracks job attempts and parses failed job logs across Jest, Vitest, Mocha, Playwright, pytest, Go test, JUnit, RSpec, and .NET.
Evaluate Flakiness Signals & Thresholds
Every 6 hours, Gitera evaluates 14 days of history for pass-on-retry, same-commit mixed outcomes across at least 2 commits, and default-branch flip rates while separating broken jobs (≥80% failure rate).
Rank By Wasted CI Cost & Alert Teams
Flaky items are ranked by wasted runner minutes and SKU cost. Digest alerts notify Slack or email channels when new flakes appear, and PR comments flag known flaky jobs automatically.
Fix, Retry, or Auto-Resolve
Inspect captured failure stack traces, create a one-click auto-retry rule while working on a fix, or dismiss low-priority items. Fixed tests resolve automatically after 7 days without a flake.
Key Benefits
Restore trust in your test suite and stop burning CI budget on retries
Eliminate Wasted Minutes
See exactly how many runner minutes and dollars each flaky test burns so you can fix the highest-impact culprits
Zero Workflow Changes
Detect flaky jobs and tests directly from existing GitHub Actions runs and standard console output
Seamless Remediation
Jump from a flaky job to a pre-filled auto-retry rule or inspect failure excerpts without opening raw logs
Automatic flaky job and test detection for GitHub Actions
A flaky test fails and passes on the same commit, training engineers to click Re-run instead of investigating and burning billable runner minutes. Gitera detects flaky jobs and individual tests automatically from the runs and logs it already receives — no workflow changes or test report uploads required.
Every finding is backed by three signals — pass on retry, same-commit mixed outcomes, and default-branch flip rate — and ranked by wasted CI time and estimated runner cost using the same SKU pricing as cost analysis. Tests from Jest, Vitest, Mocha, Playwright, pytest, Go test, JUnit, RSpec, and .NET are grouped under their parent job with captured error excerpts from log search.
From any flaky job row you can open a pre-filled rule in auto-retry, dismiss accepted flakes with a 30-day cooldown, or configure Slack and email digests through the alerting system. Consistent breakages (80% or higher failure rate) are separated so teams can triage them in failure analysis.
Flaky Tests FAQ
How does Gitera detect flaky tests in GitHub Actions without test report uploads?
Gitera evaluates completed workflow runs every six hours over a 14-day window using three signals: pass on retry within the same run, mixed pass-and-fail outcomes on the same commit, and high pass/fail flip rates on the default branch. For failed jobs, Gitera parses standard console output from frameworks like Jest, Vitest, Playwright, pytest, Go test, JUnit, RSpec, and .NET to identify the exact test that flaked alongside failure analysis.
How is wasted CI time and cost calculated for flaky tests?
Wasted time is the duration of failed job attempts on commits where the item flaked over the last 14 days. Gitera multiplies each failed job’s billable minutes (rounded up to the whole minute) by the runner SKU rate used in cost analysis — including standard Linux, Windows, macOS, ARM, and larger runners — while counting shared job runs once in summary totals.
How does Gitera distinguish a flaky test from a broken test?
Pass-on-retry and same-commit signals must appear on at least two distinct commits before a job or test is flagged as flaky, preventing a single unlucky run from creating noise. Any job or test that fails in 80% or more of its runs is classified as broken rather than flaky so engineers can fix it as a deterministic incident in failure analysis instead of masking it with auto-retry.
When are flaky tests marked as resolved?
Active flaky jobs and tests resolve automatically once they go seven consecutive days without flaking, which prevents noisy status flipping between runs. Engineers can also manually dismiss a known flake to silence alert digests for a 30-day cooldown period, or create a one-click auto-retry rule while a permanent code fix is in progress.
Related Gitera capabilities
Explore all features on the Gitera homepage or read insights on the blog.