CI/CD Integration

GitHub Actions

Here’s a complete workflow for running vanya tests in GitHub Actions:

.github/workflows/e2e.yml

name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install vanya
        run: npm install -g @vanya-lang/vanya

      - name: Check syntax
        run: vanya check tests/*.vanya

      - name: Build tests
        run: vanya build tests/*.vanya -o out/ --format pytest-function

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: |
          pip install pytest playwright pytest-playwright
          playwright install chromium --with-deps

      - name: Run tests
        run: pytest out/ -v

GitLab CI

.gitlab-ci.yml

e2e:
  image: mcr.microsoft.com/playwright:v1.44.0-jammy
  script:
    - npm install -g @vanya-lang/vanya
    - vanya check tests/*.vanya
    - vanya build tests/*.vanya -o out/ --format pytest-function
    - pip install pytest pytest-playwright
    - pytest out/ -v

Key Points

1. Install vanya - Use npm for fastest installation (no network access during test run)
2. Check first - Run vanya check to catch syntax errors before building
3. Use —format pytest-function - Generates proper test functions for pytest
4. Install Playwright browsers - Use playwright install chromium --with-deps for system dependencies
5. Cache dependencies - Consider caching npm/pip/playwright for faster builds

Binary Caching

Both npm and pip installations of vanya download a native binary on first run (~5MB from releases.vanya-lang.org). For hermetic CI runners without external network access, you can pre-fetch and cache the binary.

Cache the Binary Directory

npm

- name: Cache vanya binary
  uses: actions/cache@v4
  with:
    path: ~/.vanya
    key: vanya-binary-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}

- name: Pre-fetch vanya binary
  run: vanya --version  # Triggers download if not cached

pip

- name: Cache vanya binary
  uses: actions/cache@v4
  with:
    path: ~/.vanya
    key: vanya-binary-${{ runner.os }}-${{ hashFiles('**/requirements.txt') }}

- name: Pre-fetch vanya binary
  run: vanya --version  # Triggers download if not cached

The binary is stored in ~/.vanya/bin/vanya-{arch}. Caching this directory eliminates the network fetch on subsequent runs.

Testing Against a Real App

For browser tests, you’ll need to start your app before running tests:

- name: Start app
  run: npm start &
  working-directory: ./app

- name: Wait for app
  run: npx wait-on http://localhost:3000

- name: Run tests
  run: pytest out/ -v --base-url=http://localhost:3000