GitHub Repository

The QA Sphere CLI is open source: github.com/hypersequent/qas-cli

Track development, report issues, and contribute on GitHub.

QA Sphere CLI Tool Description

What is the QA Sphere CLI?

The QAS CLI (Command Line Interface) is a command-line tool for submitting your test automation results to QA Sphere. It provides the most efficient way to collect and report test results from your test automation workflow, CI/CD pipeline, and build servers.

The QAS CLI is language- and framework-agnostic. It works with any testing tool that produces JUnit XML reports — which covers virtually every language and framework including Playwright, Cypress, pytest, Go, Java (JUnit/TestNG), Rust, .NET (NUnit/xUnit), Ruby (RSpec), and many more. It also supports Playwright JSON reports natively.

Supported Report Formats

The CLI supports two report formats:

1. JUnit XML (junit-upload command) — The universal standard supported by nearly all test frameworks across all programming languages. If your framework can produce a JUnit-style XML report, the QAS CLI can upload it.
2. Playwright JSON (playwright-json-upload command) — Native support for Playwright's JSON report format, with support for test annotations to link test cases.

Key Features

1. Automated Test Run Creation: The CLI creates new test runs automatically when no specific run URL is provided, or uploads to an existing run when a URL is specified.

2. Multi-Format Test Result Upload: Upload test results from JUnit XML or Playwright JSON reports to QA Sphere test runs — covering any language, framework, or CI/CD environment.

3. Intelligent Test Case Matching: The tool matches test results to QA Sphere test cases using test case markers. It supports multiple marker formats to accommodate different naming conventions across languages:

   • Hyphenated markers (all languages): PRJ-002: Login test
   • Underscore markers (pytest, Go, Rust, etc.): test_prj002_login_test
   • CamelCase markers (Go, Java): TestPrj002LoginTest

4. Automatic Test Case Creation: The --create-tcases option automatically creates test cases in QA Sphere for results that don't have valid markers, generating a mapping file for future reference.

5. Custom Run Naming: The --run-name option supports templates with date/time placeholders ({YYYY}, {MM}, {DD}, etc.) and environment variables ({env:BUILD_NUMBER}), making it easy to organize runs in CI/CD.

6. Attachment Support: With the --attachments option, the CLI can detect and upload relevant attachments (screenshots, videos, traces) along with your test results.

7. Flexible Configuration: The tool supports configuration through environment variables, .env files, or a .qaspherecli configuration file.

8. Output Control: The --skip-report-stdout and --skip-report-stderr options let you control when to exclude stdout/stderr blocks from test reports (e.g., skip on success to reduce payload size).

9. Graceful Error Handling: The --force option ignores API errors, invalid test cases, or attachment issues. The --ignore-unmatched option suppresses individual unmatched test messages and shows only a summary.

Works With Any Language and Framework

The QAS CLI is not limited to JavaScript or Node.js projects. While the CLI itself is distributed as an NPM package, it can upload test results from any testing tool that produces JUnit XML reports. Here are just a few examples:

Language	Frameworks / Tools
JavaScript / TypeScript	Playwright, Cypress, Jest, Mocha, WebdriverIO, Vitest
Python	pytest, unittest, Robot Framework
Java / Kotlin	JUnit 4/5, TestNG, Selenium, Appium
Go	go test (with -v and go-junit-report)
C# / .NET	NUnit, xUnit, MSTest
Rust	cargo test (with cargo2junit)
Ruby	RSpec, Minitest
PHP	PHPUnit

As long as your test tool can output JUnit-style XML, the QAS CLI can upload it to QA Sphere.

Why Use QAS CLI?

1. Streamlined Workflow: Eliminate manual result entry, reducing human error while integrating testing directly into your QA processes.

2. CI/CD Integration: Incorporate automated test reporting into your pipelines with automatic test run creation at every development stage.

3. Comprehensive Reporting: Automate data collection to understand your testing process efficiency and prevent bottlenecks.

4. Time Savings: Simplify administrative tasks and free your team to focus on analysis and coverage improvements.

5. Consistency: Unify your testing processes and data to ensure a consistent approach to test execution management across different languages and frameworks.

By leveraging the QAS CLI tool, you can significantly enhance your test management process, bringing automation, consistency, and efficiency to your quality assurance workflows — regardless of which programming language or test framework your team uses.