Example Project

See a complete working example: bistro-e2e-python — Playwright for Python (pytest) E2E tests with QA Sphere integration.

QA Sphere CLI Tool: Python (pytest) Integration

This guide shows you how to integrate pytest test results with QA Sphere using the QAS CLI tool. While this guide uses Playwright for Python as the browser automation library, the same approach works with any Python test framework that produces JUnit XML output — including pytest with Selenium, Robot Framework, or plain unit tests.

Installation

Prerequisites

• Python 3.8+ (3.13+ recommended)
• pytest test project
• Node.js 18.0.0 or higher (for the QAS CLI tool only)

Install QAS CLI

The QAS CLI is distributed as an NPM package. You don't need Node.js for your Python project itself — only for running the CLI tool.

1. Via NPX (Recommended) Simply run npx qas-cli. On first use, you'll need to agree to download the package.

   Verify installation:

   npx qas-cli --version

2. Via NPM

   npm install -g qas-cli

   Verify installation:

   qasphere --version

Configuration

The QAS CLI requires two environment variables to function properly:

• QAS_TOKEN - QA Sphere API token
• QAS_URL - Base URL of your QA Sphere instance (e.g., https://qas.eu2.qasphere.com)

You can create your API key under the QA Sphere settings.

To do that:

1. Go to Settings by clicking the gear icon in the top right corner.
2. Click API Keys
3. Click Create API Key and generate one

These variables can be defined in several ways:

1. Environment Variables: Export them in your shell:

   export QAS_TOKEN=your_token_here
   export QAS_URL=your_qas_url

2. .env File: Place a .env file in the current working directory:

   # .env
   QAS_TOKEN=your_token
   QAS_URL=your_qas_url

3. Configuration File: Create a .qaspherecli file in your project directory or any parent directory:

   # .qaspherecli
   QAS_TOKEN=your_token
   QAS_URL=your_qas_url
   
   # Example with real values:
   # QAS_TOKEN=tst0000001.1CKCEtest_JYyckc3zYtest.dhhjYY3BYEoQH41e62itest
   # QAS_URL=https://qas.eu1.qasphere.com

pytest Configuration

pytest has built-in support for JUnit XML output via the --junitxml flag. Configure it in your pyproject.toml so it runs automatically:

# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = [
    "--junitxml=junit-results/results.xml",
]

Alternatively, pass the flag directly:

pytest --junitxml=junit-results/results.xml

With Playwright for Python

If you're using Playwright for Python, you can also configure screenshots and tracing:

# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = [
    "--screenshot=on",
    "--tracing=retain-on-failure",
    "--junitxml=junit-results/results.xml",
]

Key Configuration Points

• JUnit XML Output: The --junitxml flag generates an XML file that QA Sphere can process
• Output File: Specify a consistent output path (e.g., junit-results/results.xml)
• Screenshots (Playwright): Use --screenshot=on to capture screenshots
• Tracing (Playwright): Use --tracing=retain-on-failure to keep trace files for failed tests

Using the junit-upload Command

The junit-upload command uploads JUnit XML test results to QA Sphere. It will automatically create a new test run if no run URL is provided, or upload to an existing run if a run URL is specified.

Basic Syntax

# Create new test run automatically
npx qas-cli junit-upload <path-to-junit-xml>

# Upload to existing test run
npx qas-cli junit-upload -r <run-url> <path-to-junit-xml>

Options

Option	Description
-r, --run-url	Upload results to an existing test run URL
--project-code	Project code for creating a new test run (can also be auto-detected from markers)
--run-name	Name template for the new test run. Supports {YYYY}, {MM}, {DD}, {HH}, {mm}, {ss}, {env:VAR} placeholders
--create-tcases	Automatically create test cases in QA Sphere for results without valid markers
--attachments	Detect and upload attachments with test results (screenshots, traces)
--force	Ignore API errors, invalid test cases, or attachment issues
--ignore-unmatched	Suppress individual unmatched test messages, show summary only
--skip-report-stdout	Control when to skip stdout blocks (on-success or never, default: never)
--skip-report-stderr	Control when to skip stderr blocks (on-success or never, default: never)

pytest-Specific Examples

1. Run tests and upload results:

   # Run pytest (generates JUnit XML automatically via pyproject.toml)
   pytest --browser chromium -v
   
   # Upload results to QA Sphere
   npx qas-cli junit-upload --project-code BD --attachments junit-results/results.xml

2. Upload to an existing test run:

   npx qas-cli junit-upload -r https://qas.eu1.qasphere.com/project/BD/run/23 junit-results/results.xml

3. Create a new run with a custom name template:

   npx qas-cli junit-upload --project-code BD --run-name "pytest {YYYY}-{MM}-{DD} {HH}:{mm}" junit-results/results.xml

4. Auto-create test cases for unmatched results:

   npx qas-cli junit-upload --project-code BD --create-tcases junit-results/results.xml

Test Case Marker Requirements

To map test results to QA Sphere test cases, your pytest tests should include a QA Sphere test case marker in the function name.

Underscore-Separated Markers (Recommended for Python)

Since Python function names cannot contain hyphens, the QAS CLI supports underscore-separated hyphenless markers. The test name must start with test (standard pytest convention), and the marker can appear at the start or end of the function name.

Format: test_PROJECTSEQUENCE_description — where PROJECT is the project code and SEQUENCE is the test case number (minimum 3 digits, zero-padded if needed).

pytest Test Examples with Markers

# tests/test_ui_cart.py
from playwright.sync_api import Page


def test_bd023_cart_operations(page: Page, demo_base_url: str) -> None:
    """BD-023: User should see product list on the Checkout page."""
    page.goto(f"{demo_base_url}/")
    # Add items to cart
    page.locator(".menu-item").first.locator(".add-to-cart").click()
    # Navigate to checkout and verify
    page.locator(".cart-btn").click()
    assert page.locator(".checkout-items").is_visible()


def test_bd022_order_with_cash_payment(page: Page, demo_base_url: str) -> None:
    """BD-022: User should place the order successfully with Cash payment."""
    page.goto(f"{demo_base_url}/checkout")
    page.locator("#name").fill("John Doe")
    page.locator("#email").fill("[email protected]")
    page.locator("#payment").select_option(label="Cash on Delivery")
    page.locator(".place-order").click()
    assert page.locator(".success-message").is_visible()

Valid function names with markers:

• test_bd023_cart_operations — marker bd023 at the start (after test_)
• test_cart_operations_bd023 — marker bd023 at the end

note

The underscore-separated marker format is matched case-insensitively. Both test_bd023_... and test_BD023_... will match test case BD-023.

Hyphenated Markers in Docstrings

While the function name uses the underscore format for matching, it's good practice to include the hyphenated marker in the docstring for readability:

def test_bd023_cart_operations(page: Page) -> None:
    """BD-023: User should see product list on the Checkout page."""
    ...

Troubleshooting

Common Issues

1. JUnit XML not found

   Error: File not found: junit-results/results.xml

   Solution: Ensure --junitxml=junit-results/results.xml is configured in your pyproject.toml or passed directly to pytest.

2. Test cases not matching

   • Ensure your function names include the marker (e.g., test_bd023_description)
   • The function name must start with test (standard pytest convention)
   • Verify the marker matches an existing test case in QA Sphere
   • Use --create-tcases to auto-create test cases for unmatched results

3. Node.js not installed

   • The QAS CLI requires Node.js 18+. Install it from nodejs.org or use your package manager
   • On macOS: brew install node
   • On Ubuntu: sudo apt install nodejs npm

Getting Help

For help with any command, use the -h flag:

npx qas-cli junit-upload -h

Complete Example Workflow

Here's a complete example of running pytest with Playwright and uploading to QA Sphere:

# 1. Set up environment (or use .env/.qaspherecli file)
export QAS_TOKEN=your_token_here
export QAS_URL=https://qas.eu1.qasphere.com

# 2. Create and activate virtual environment
python3 -m venv .venv
source .venv/bin/activate

# 3. Install dependencies
pip install -e ".[dev]"
playwright install --with-deps

# 4. Run tests
pytest --browser chromium -v

# 5. Upload results to QA Sphere
npx qas-cli junit-upload --project-code BD --attachments junit-results/results.xml

By following these guidelines, you can effectively integrate your Python test results with QA Sphere, streamlining your test management process.