CI/CD Integrations

A pipeline can refresh test data before tests run, on a schedule, or for each pull request. It either asks Governor to run a workflow you built in the web UI, or runs the TDK CLI with a configuration from your repository.

Choose an approach

	Governor’s REST API	TDK CLI
What runs	A workflow you built in Governor, on Governor’s workers	A configuration file from your repository, in the pipeline’s runner or container
The pipeline needs	An access key and the workflow’s ID	The database URLs and credentials, and your license key
Network	The runner reaches Governor over HTTPS	The runner reaches the databases directly
Records	Runs, logs and the audit log in Governor	The output of the CLI, in the pipeline log
Use it when	Workflows are managed in Governor, and databases are reachable only from your workers	The configuration lives in version control, or there’s no Governor

Governor’s REST API

TDK CLI

What runs

A workflow you built in Governor, on Governor’s workers

A configuration file from your repository, in the pipeline’s runner or container

The pipeline needs

An access key and the workflow’s ID

The database URLs and credentials, and your license key

Network

The runner reaches Governor over HTTPS

The runner reaches the databases directly

Records

Runs, logs and the audit log in Governor

The output of the CLI, in the pipeline log

Use it when

Workflows are managed in Governor, and databases are reachable only from your workers

The configuration lives in version control, or there’s no Governor

Call Governor’s REST API

The pipeline starts a run, polls its status until it’s final, and prints the log if the run fails:

Authenticate with an access key in the X-Access-Key header.
Start a run: POST /external/api/v1/workflow/{id}/run.
Poll GET /external/api/v1/workflow-run/{run_id} until the status is COMPLETED, FAILED or STOPPED.
If it didn’t complete, fetch GET /external/api/v1/workflow-run/{run_id}/logs.

See Run a workflow from a pipeline for a script, and the Azure DevOps page for a complete pipeline with a reusable step.

Generate the pipeline’s access key as a dedicated user that has access only to the workflows the pipeline runs. A key can do everything its user can, and can’t be revoked yet, so choose a short expiry.

Run the TDK CLI

Run the CLI, or its Docker image, as a pipeline step. Keep the passwords and the license key in the CI system’s secret store.

tdk --config-file workflow.yaml \
  --input-url "$INPUT_URL" --input-username "$INPUT_USER" --input-password "$INPUT_PASSWORD" \
  --output-url "$OUTPUT_URL" --output-username "$OUTPUT_USER" --output-password "$OUTPUT_PASSWORD"

The CLI reads the license key from SYNTHESIZED_KEY. See Command-Line Interface and Run the CLI in Docker.

Check the configuration in pull requests

Run validate-config on changes to the configuration file, so a broken configuration fails the pull request instead of the nightly run:

tdk --config-file workflow.yaml validate-config

Common patterns

Nightly refresh

Schedule the pipeline, for example at 02:00, so testers start each day with fresh data. In GitHub Actions:

on:
  schedule:
    - cron: '0 2 * * *'

Before the test suite

Make the test data step the first stage, and run the tests only if it succeeds.

Per environment

Use a different workflow, or a different output database, for each test environment. Two runs at the same time can’t write to the same output database.