This documentation is available to agents as Markdown at https://docs.synthesized.io/tdk/latest/faq/synthesized-platform.md. For the complete documentation index, see llms.txt.

Synthesized Platform FAQ

Short answers to the questions people ask most about the Synthesized Platform. Each one links to the page that covers it in full.

For security and compliance questions, such as encryption, access control, the audit log and SOC 2, see Security & Compliance.

The platform

What does the Synthesized Platform do?

It makes test data from your production databases and files, in three ways you can combine in one workflow:

  • Masking replaces sensitive values, and keeps the relationships between tables.

  • Subsetting makes a smaller copy that still holds together.

  • Generation creates new rows that follow the patterns of the input, at any size.

The platform runs inside your network, and the output goes to another database or to files. See Core Concepts.

What are Governor, workers and the TDK CLI?

  • Governor is the web UI and the REST API. You connect data sources, build workflows, run them and see their history there.

  • Workers run the workflows that Governor gives them. Put them close to the databases they read and write.

  • The TDK CLI runs the same engine from the command line, with a YAML configuration and without Governor, for example in a pipeline.

See Architecture.

How can I try it?

Run the demo on your own machine. It starts Governor with a sample PostgreSQL database and a ready-made masking workflow. You need the demo package, a license key, and Docker with 8 GB of memory. See the Quick Start.

Which databases and files does it work with?

Relational databases, including PostgreSQL, Oracle, SQL Server, MySQL, Db2 and Snowflake; SAP S/4HANA and ECC, through SAP HANA or at the application layer; and CSV and XML files. See Database Integrations for the full list and the connection formats, and SAP TDM for SAP.

Do I need a license key?

For most uses, yes. Without one, Governor and the CLI run in free mode:

  • Oracle, SQL Server, Db2, Snowflake, SAP and CSV data sources are blocked.

  • PostgreSQL, MySQL and MariaDB data sources are limited to 20 tables and 100,000 rows per table.

See Free mode.

Does any data leave our environment?

No. Nothing is sent to Synthesized, there’s no telemetry, and the license key is applied offline. The only outbound connections are ones you turn on yourself, such as the Assistant, which sends its requests to the LLM endpoint you configure, never to Synthesized. See Security Overview.

Configuration

How do I create a configuration for my database?

In Governor, create a workflow on the data sources, and edit its configuration there. See Create a Workflow.

In YAML, start small. This masks every table:

default_config:
  mode: MASKING

The engine chooses a transformer for each column from its type and role. If a column matches no rule, the default safety mode, STRICT, stops the run and names the column, so you can set a transformer for it. The sensitive data scan can suggest transformers for the columns that hold personal data.

To see every table and column with the transformer it will get, print the effective configuration with dry-run, then edit it:

tdk --config-file workflow.yaml \
  --input-url "jdbc:postgresql://db.example.com:5432/production" \
  --input-username tdk --input-password "$INPUT_PASSWORD" \
  dry-run -ec effective.yaml

See Workflows and the CLI commands.

Should I use safety_mode: RELAXED to make the run finish?

Only after you’ve checked which columns it affects. With RELAXED, a column that no rule matches is copied unchanged in MASKING mode, with its real values. Keep the default, STRICT, and set transformers for the columns it stops at. See How each column gets a transformer.

What’s the difference between MASKING, GENERATION and KEEP?

Each table has one mode, so one workflow can mix them:

Mode What happens to the table Output size

KEEP

Rows are copied as they are.

The same, or smaller

MASKING

Every column gets a masking transformer, unless you set another one.

The same, or smaller

GENERATION

New rows are generated from what the engine learns from the input.

Any size

Keys stay consistent between tables in different modes. See Modes, How Masking Works and How Generation Works.

Can I change only some columns and keep the others as they are?

Yes. Put the table in KEEP mode, and set transformers for the columns you want to change. In KEEP mode, every other column is copied unchanged:

tables:
  - table_name_with_schema: "public.customer"
    mode: KEEP
    transformations:
      - columns: ["first_name", "last_name", "email"]
        params:
          type: person_generator
          column_templates: ["${first_name}", "${last_name}", "${email}"]

In a MASKING table, give the columns you want to keep the passthrough transformer instead. Either way, the columns you keep hold real values. See Data Protection.

How do I leave a table or a schema out?

  • A table: set its target_ratio to 0, and it’s empty in the output. Rows in other tables that reference it are left out too, because the platform keeps only rows whose parent rows it kept.

    tables:
  - table_name_with_schema: "public.audit_log"
    target_ratio: 0

  • A schema: list the schemas to process in schemas. See schemas.

How do I keep only part of the data, such as the last six months?

Give the table a filter, a condition on its rows, or a target_ratio below 1 to keep a share of them. Tables that depend on it are reduced to match:

default_config:
  mode: MASKING
tables:
  - table_name_with_schema: "public.rental"
    filter: "rental_date >= '2026-05-01'"
  - table_name_with_schema: "public.customer"
    target_ratio: 0.1

See Data Filtering, Subset a Database and How Subsetting Works.

How do I add relationships that aren’t in the database?

Declare them as virtual foreign keys, in the configuration’s metadata. The platform then treats them like foreign keys in the database. For example, if orders.user_id refers to users.id but the database has no foreign key for it:

metadata:
  tables:
    - table_name_with_schema: "public.orders"
      foreign_keys:
        fk_orders_user:
          referred_schema: "public"
          referred_table: "users"
          columns:
            - column: "user_id"
              referred_column: "id"

See Virtual Foreign Keys.

Does Governor keep the scan’s results and my relationships for the next run?

Yes. The sensitive data scan writes its suggestions into the workflow’s configuration, and virtual foreign keys are part of the configuration too. Governor saves the configuration with the workflow, and keeps a history of every change. See Version History.

Does the platform profile the data?

Yes, as part of each run. In GENERATION mode, the engine learns each column’s distribution or categories from the input. The sensitive data scan checks the values in each column for personal data. See What the engine learns and PII Detection.

How do I check a configuration before I run it?

tdk validate-config checks the file against the configuration’s JSON schema, and tdk dry-run prints the effective configuration without running anything. See the CLI commands.

The output

How does the platform keep referential integrity?

It follows the foreign keys in the database, and any virtual foreign keys you add:

  • Tables are processed parents first.

  • A subset keeps only rows whose parent rows were kept, so no row points to a missing parent.

  • In MASKING mode, keys are hashed the same way in every table, so references still match.

  • In GENERATION mode, every foreign key gets the key of a parent row that exists in the output.

See What a run does.

What happens to data that’s already in the output?

It depends on table_truncation_mode:

  • DO_NOT_TRUNCATE, the default, needs an empty output. Rows from an earlier run cause duplicate key errors.

  • TRUNCATE empties the output tables first.

  • IGNORE leaves the rows there and adds to them. In GENERATION mode, new rows can refer to existing ones; in MASKING mode, rows are added without checking what’s already there.

See Data Append mode and Schema Creation.

Can two runs write to the same output at once?

No. A run locks its output database, so a second run on the same output fails with "Failed to claim sink". Give runs that happen at the same time different outputs. See Database Locking Issues.

Are the results repeatable?

  • Masking: hashing transformers give the same output for the same input, run after run, under the same masking key.

  • Generation: set global_seed. With the same input data, configuration, seed and platform version, the result is the same.

See Repeatable runs.

Can I generate more rows than the input has?

Yes, in GENERATION mode. Set target_ratio above 1, for example 2 to double a table, or an exact target_row_number. KEEP and MASKING tables can’t be larger than the input. See The output size.

Can I generate data from the schema alone, without any data?

Yes, in GENERATION mode. Set the number of rows with target_row_number, because a ratio of an empty table is still empty. The engine generates values from the column types and keeps the primary and foreign keys. With no values to learn from, give generators their parameters, such as categories or ranges. See Generation Transformers.

Production databases

Will a run affect the input database?

A run only reads the input, and doesn’t block other reads and writes to it. Reading a large database adds load, though. To limit it:

  • Run against a replica or a copy, rather than the production instance.

  • Lower the platform’s connection pool, TDK_DB_MAXIMUM-POOL-SIZE (default 20). See Connection pool.

  • Cap the platform user’s connections in the database. See Limit concurrent connections.

  • Filter on indexed columns, so a filter doesn’t read the whole table.

What if the input changes during a run?

The platform is designed to keep the output consistent when rows are inserted, updated or deleted in the input during a run. For a predictable result, run against a replica or a snapshot that doesn’t change while the run reads it.

Which database permissions does the platform need?

Read access on the input, and write access on the output, including creating tables if the platform creates the schema. See Database Permissions.

Performance and scale

What hardware do I need?

Start from this baseline for the machine that runs workflows, then add capacity as your data grows:

Resource Minimum

CPU

8 cores

Memory

32 GB

Storage

SSD, with at least 100 GB free

Network

10 Gbps

The demo needs less: 8 GB of memory for Docker. See Size the servers.

What makes a run slow, and how do I speed it up?

These make the most difference:

  • Network latency between the worker and the databases. Run workers in the same network or cloud region as the databases.

  • The amount of data. Filter or subset large tables, and leave out tables you don’t test with, such as audit logs.

  • The relationships between tables. Tables that don’t depend on each other are processed in parallel, so many foreign keys mean less parallelism.

  • Subsetting. It reads less, but costs more per row than a full copy.

Two settings help too: the working directory, which keeps working data on disk, and the connection pool size:

TDK_WORKINGDIRECTORY_ENABLED=true
TDK_WORKINGDIRECTORY_PATH=/home/tdk/working-directory
TDK_DB_MAXIMUM-POOL-SIZE=20

On a worker, set WORKER_WORKING_DIRECTORY_PATH for the working directory. See Improving Performance and Performance Troubleshooting.

Is there a limit on the number of tables, columns or rows?

With a license key, there’s no fixed limit on the number of tables or columns, and the platform is used with databases of several terabytes. Free mode limits PostgreSQL, MySQL and MariaDB data sources to 20 tables and 100,000 rows per table.

For large databases, subset or filter the big tables, and size the servers for the data. See What makes a run slow, and how do I speed it up?.

Automation

Can I run workflows from a CI/CD pipeline?

Yes. Start a workflow in Governor through the REST API, or run the TDK CLI in the pipeline. See CI/CD.

Can I run workflows on a schedule?

Run them from a scheduler: a scheduled CI/CD pipeline, or the TDK CLI as a Kubernetes CronJob. See CI/CD and CLI on Kubernetes.

Help

Where do I start when a run fails?

Read the run’s log: in Governor, open the run in the workflow’s run history; with the CLI, read its output. The error usually names the cause. Then see Common Issues, or find your problem in Troubleshooting.

How do I contact support?

Write to support@synthesized.io, with the platform’s version, the run log and the configuration, without passwords, keys or data. See Troubleshooting for what to collect.