CI/CD Integration¶

AI TestPilot X is a drop-in quality gate for any CI pipeline. The testpilot run command exits with:

0 — GO (pipeline passes)
1 — GO WITH RISK (pipeline fails with warning)
2 — NO GO (pipeline hard fails)

GitHub Actions¶

Quality gate on every push¶

# .github/workflows/ai-quality-gate.yml
name: AI Quality Gate

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

jobs:
  quality-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: pip

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Install AI TestPilot X
        run: pip install -e .

      - name: Run AI quality gate
        run: |
          python -m cli.main run \
            --story "User should be able to login and checkout" \
            --mode MOCK
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
          EXECUTION_MODE: MOCK

      - name: Upload report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: ai-testpilot-report
          path: .testpilot/reports/
          if-no-files-found: ignore

Full test suite + quality gate¶

# .github/workflows/testpilot-ci.yml
name: TestPilot CI

on:
  push:
    branches: [main, master, develop]
  pull_request:
    branches: [main, master]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      EXECUTION_MODE: MOCK
      DB_URL: sqlite:///./testpilot_ci.db
      CHROMA_PATH: ./chroma_ci
      GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: pip
      - run: pip install -r requirements.txt
      - run: ruff check . --ignore E501
      - run: black --check .
      - run: pytest tests/ -v --html=report.html --self-contained-html
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-report
          path: report.html

GitLab CI¶

# .gitlab-ci.yml
stages:
  - test
  - quality

unit-tests:
  stage: test
  image: python:3.11
  script:
    - pip install -r requirements.txt
    - pytest tests/ -v
  artifacts:
    reports:
      junit: test-results.xml

ai-quality-gate:
  stage: quality
  image: python:3.11
  script:
    - pip install -r requirements.txt
    - pip install -e .
    - python -m cli.main run --story "User can login and checkout" --mode MOCK
  variables:
    GEMINI_API_KEY: $GEMINI_API_KEY
    EXECUTION_MODE: MOCK
  artifacts:
    when: always
    paths:
      - .testpilot/reports/

Jenkins¶

// Jenkinsfile
pipeline {
    agent any

    environment {
        GEMINI_API_KEY = credentials('gemini-api-key')
        EXECUTION_MODE = 'MOCK'
    }

    stages {
        stage('Install') {
            steps {
                sh 'pip install -r requirements.txt && pip install -e .'
            }
        }

        stage('Unit Tests') {
            steps {
                sh 'pytest tests/ -v'
            }
        }

        stage('AI Quality Gate') {
            steps {
                sh '''
                    python -m cli.main run \
                      --story "User can login and checkout" \
                      --mode MOCK \
                      --output report.json
                '''
            }
            post {
                always {
                    archiveArtifacts artifacts: '.testpilot/reports/**', allowEmptyArchive: true
                }
            }
        }
    }
}

Setup secrets¶

GitHubGitLabJenkins

Go to Settings → Secrets and variables → Actions
Click New repository secret
Name: GEMINI_API_KEY, Value: your key

Go to Settings → CI/CD → Variables
Add GEMINI_API_KEY as a masked variable

Go to Manage Jenkins → Credentials
Add a new Secret text credential with ID gemini-api-key

Execution mode recommendations¶

Environment	Recommended mode	Reason
GitHub Actions / GitLab CI	`MOCK`	No Chrome available on runners
Self-hosted runner with Chrome	`LOCAL`	Real browser validation
Selenium Grid	`GRID`	Parallel cross-browser testing
Local dev	`LOCAL` or `MOCK`	Your choice

MOCK mode in CI

MOCK mode returns realistic simulated test results generated by the AI — it's not skipping the QA. The AI generates test cases, verifies coverage, analyzes bugs, and produces a real GO/NO GO decision. Only the actual browser clicks are simulated.