Skip to main content

Test Case Success Rate Report (Test Case Success Rate Analysis)

The Test Case Success Rate report calculates success rates for individual test cases across multiple executions to identify unreliable tests, track test quality, and prioritize test maintenance efforts. This report is essential for maintaining a reliable and trustworthy test suite.

What This Report Shows​

Test Case Success Rate report

The Test Case Success Rate report displays:

  1. Overall Success Rate: Aggregate success percentage across all test cases
  2. Per-Test Success Rates: Individual success percentages for each test case
  3. Execution Counts: Number of times each test has been executed
  4. Reliability Metrics: Statistical significance of success rates
  5. Folder Organization: Tests grouped by folder structure

When to Use This Report​

Use the Test Case Success Rate report when you need to:

  • Identify flaky tests that pass/fail inconsistently
  • Track test reliability across multiple executions
  • Prioritize test maintenance based on failure frequency
  • Evaluate test quality before expanding test suite
  • Build confidence in test results through consistency metrics
  • Quarantine unreliable tests from critical pipelines

Understanding the Report​

Overall Success Rate​

At the top of the report, you'll see the aggregate success rate displayed in a circular progress indicator showing the overall percentage of successful test executions.

Example:

Success Rate: 68.1%

Interpretation: Across all test executions, tests passed 68.1% of the time. This indicates room for improvement in test reliability.

Success Rate Table​

The detailed table shows per-test metrics organized by folder structure:

ColumnDescription
NameTest case name and description
Execution CountNumber of times test has been executed
Success RatePercentage of executions that passed

Success Rate Indicators:

  • 90-100%: Reliable test ✅
  • 70-89%: Acceptable but needs monitoring ⚠️
  • 50-69%: Flaky test requiring attention ❌
  • Below 50%: Broken test needing immediate fix 🚫

Generating the Report​

Generating the Success Report

  1. Open your QA Sphere project
  2. Click Reports in the top navigation
  3. Select Test Case Success Rate
  4. Click Next
  5. Choose the way to select test cases for the report: by Milestone, Test Run or time Period
  6. Click Build report

Export Options​

Export XLSX:

  • Raw data export for further analysis
  • Open in Excel, Google Sheets, or BI tools
  • Click ... and then Export XLSX button

Export PDF:

  • Full report with charts and tables
  • Professional formatting for stakeholder distribution
  • Click ... and then Export PDF button

Print:

  • Direct print for physical documentation
  • Formatted for standard paper sizes
  • Click ... and then Print button

Getting Help​

For assistance with this report:

  1. Review the report parameters to ensure correct configuration
  2. Check the Reports Overview for general guidance
  3. Contact QA Sphere support: [email protected]

Quick Summary: Use the Test Case Success Rate report to identify unreliable tests. Target 95%+ success for critical tests, 90%+ for others. Investigate tests below 80% immediately. Track improvement over time and quarantine flaky tests from critical pipelines.