# Synthesized Docs

> Documentation for the Synthesized Test Data Kit (TDK) and the Synthesized platform:
> generating synthetic test data, masking production data, and running both from CI.
> Every page below is also available as HTML by removing the .md suffix.

## Get Started

- [Welcome to Synthesized](https://docs.synthesized.io/tdk/latest/user_guide/010_get_started/welcome.md): The Synthesized platform is your complete solution for database masking, synthetic data generation, and subsetting.
- [5-Minute Quick Start](https://docs.synthesized.io/tdk/latest/user_guide/010_get_started/quickstart.md): Get up and running with the Synthesized platform in just 5 minutes.
- [Connect Data Sources](https://docs.synthesized.io/tdk/latest/user_guide/010_get_started/datasources.md): In order to create and execute a workflow, users needs to define the input and output (destination) data sources.
- [How to Create a Workflow](https://docs.synthesized.io/tdk/latest/user_guide/010_get_started/workflow.md): A workflow is a manageable entity which is used to audit any configured data transformation.
- [Projects](https://docs.synthesized.io/tdk/latest/user_guide/010_get_started/projects.md): Projects help you organize your workflows and manage who can access them.
- [Database Permissions](https://docs.synthesized.io/tdk/latest/user_guide/010_get_started/database_permissions.md): In order to run the Synthesized platform successfully, the following minimum users permissions are required.

## Guides

- [Masking Data](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/masking_data.md): Learn how to mask sensitive data while preserving data utility and referential integrity.
- [Masking Specific Column Types](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/masking_data/masking.md): If we run the platform with an empty configuration file, that would be the equivalent of using:
- [Masking Transformers](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/masking_data/transformers.md)
- [Generating Data](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data.md): Generate synthetic data for testing, development, and demo environments.
- [Viewing the XML Schema](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data/xml_schema_view.md): View the structure of an XML data source so you can target the right paths when customising XML generation.
- [Generation Overview](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data/generation.md): To run the following examples on your machine, you should first:
- [Generation Transformers](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data/transformers.md)
- [Generating XML Files](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data/xml_files_generation.md): Generate realistic, schema-valid XML documents from an XSD definition, with optional customisation of individual field values.
- [Generating CSV Files](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data/csv_generation.md): Synthesized supports transforming CSV files using either the CLI or web interface.
- [Linking CSV Files in a Workflow](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data/linked_csv_files.md): Generate multiple CSV files in a single workflow and link columns across them with virtual foreign keys.
- [Subsetting Data](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/subsetting_data.md): Extract a smaller, referentially intact subset of your database for testing.
- [Subsetting Overview](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/subsetting_data/subsetting.md): To run the following examples on your machine, you should first:
- [Data Filtering Techniques](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/subsetting_data/data_filtering.md): To run the following examples on your machine, you should first:
- [Version History](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/managing_data_properties/version_history.md): Review who changed a workflow’s configuration and when, and restore an earlier version.
- [Adding Tables from the Database View](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/managing_data_properties/manage_tables_database_view.md): Add tables that the metadata scan missed, directly from the Database view.
- [Schema Creation](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/managing_data_properties/schema_creation.md): Synthesized can only insert data into the target database if the target schemas are compatible with the input schemas.
- [Virtual Foreign Keys](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/managing_data_properties/virtual_foreign_keys.md): The Synthesized platform is able to automatically understand and maintain the relationships defined using foreign keys.
- [Changing Data Types](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/managing_data_properties/value_mapping.md): Allows transforming columns as they are being read from the input database and written to the output database.
- [Append Mode](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/configuration/append.md): Starting from version 1.58, Synthesized TDK supports not only provisioning data into an empty database, but also appending data to an existing database.
- [Advanced Workflows](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/advanced_workflows.md): Advanced platform features for complex use cases and enterprise deployments.
- [Custom Transformations](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/advanced_workflows/scripting_transformer.md): The Scripting Transformer is Synthesized’s most flexible transformer.
- [Synthesized Workers](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/advanced_workflows/workers.md): The Synthesized platform serves as the core of database processing within Governor.
- [Creating blank SAP Workflows](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/sap_workflows/create_sap_workflow_from_scratch.md): The SAP workflow wizard lets you build a workflow without a preselected business-process or module template, so you control exactly which SAP tables and masking rules go into it.

## Core Concepts

- [What is the Synthesized Platform?](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/what_is_tdk.md): A comprehensive test data transformation platform that provides secure, privacy-preserving versions of production like data for development, testing, and data engineering.
- [Architecture Overview](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/architecture.md): Understand the high-level architecture and components that power the platform’s data masking, generation, and subsetting capabilities.
- [How Masking Works](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/how_masking_works.md): Understand the masking engine’s internals, how transformers replace sensitive data, and how the platform preserves referential integrity during masking.
- [How Generation Works](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/how_generation_works.md): Understand how the platform’s generation engine creates synthetic data from scratch while maintaining referential integrity and realistic distributions.
- [How Subsetting Works](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/how_subsetting_works.md): Learn how to create smaller, representative subsets of production databases while maintaining referential integrity.
- [PII Detection & Protection](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/pii.md): Automatically detect and protect personally identifiable information (PII) using the platform’s built-in PII scanning capabilities.

## Reference

- [Transformers](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/transformers.md): Complete reference for all platform transformers and their configuration options.
- [All Transformers](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/transformers/transformations.md): optional Object.
- [Masking Transformers](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/masking_data/transformers.md)
- [Generation Transformers](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/generating_data/transformers.md)
- [Configuration](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/configuration.md): Reference documentation for the platform configuration files and settings.
- [Configuration Reference](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/configuration/configuration_reference.md): Object.
- [Advanced Configuration](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/configuration/advanced_configuration.md): This page documents advanced configuration properties and environment variables that are not commonly used but provide fine-grained control over TDK behavior.
- [API Reference](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/api_reference.md): Complete reference for the platform APIs - CLI, REST API, and programmatic access.
- [CLI Reference](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/api_reference/command_line_interface.md): In order to run the application, the host needs to have the following installed:
- [REST API Reference](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/advanced_workflows/external_api_reference.md): This section provides detailed information about the available endpoints in the Governor REST API.
- [API Access Keys](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/advanced_workflows/public_api_access_keys.md): To generate a new access key, please follow the steps below:
- [Data Types](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/data_types.md): Comprehensive reference for data types, mappings, and supported formats.
- [Data Type Inventory](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/data_types/inventory.md): Object.

## Deployment & Operations

- [Getting TDK into Production](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/production_setup.md): Production deployment guides for the platform across different platforms.
- [Docker Compose](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/production_setup/dc.md): Modify the following lines in the docker-compose file:
- [Kubernetes (Helm)](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/production_setup/helm.md): DB_URL is the JDBC URL for Governor database in the following format: jdbc:postgresql://[host]:5432/[database-name].
- [Multi-Worker Deployment](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/production_setup/multi_worker.md): Running multiple Governor workers in parallel allows concurrent workflow execution.
- [Worker–Governor mTLS](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/production_setup/worker_mtls.md): Enable mutual TLS on the worker–Governor gRPC connection using your own certificate files, so only workers holding a certificate signed by a trusted CA can register.
- [Offline Installation](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/production_setup/offline.md): This document provides detailed instructions on downloading and loading Docker images when the target machine does not have internet access.
- [Installation Methods](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods.md): Different ways to install and run the platform based on your use case.
- [CLI Overview](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/cli.md): Please contact us for more information about obtaining the full version.
- [CLI Installation](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/cli_installation.md): In order to run the platform you need to install Java 17+ first.
- [CLI with Docker](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/run_cli_with_docker.md): In order to run the application, the host needs to have the following installed:
- [CLI with Kubernetes](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/run_cli_with_kubernetes.md): You can find the list of all parameters for the TDK docker image here.
- [AWS Marketplace](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/aws_marketplace.md): The platform can be purchased at AWS Marketplace as a container product.
- [AWS EKS](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/run_cli_with_aws_eks.md): Find video instruction on how to start an AWS free trial using the EKS deployment option:
- [AWS ECS Fargate](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/run_cli_with_aws_ecs_fargate.md): Find the video instructions on how to start an AWS free trial using ECS deployment option:
- [GCP Marketplace](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/installation_methods/gcp_marketplace.md): Find a video tutorial on how to start your first GCP Kubernetes deployment:
- [Updating TDK](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/updating_tdk.md): Keep the platform up-to-date with the latest features, fixes, and security patches.
- [Update Docker Compose](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/updating_tdk/update_dc.md): Before performing the upgrade, ensure the following steps are completed: * The Governor Docker Compose setup archive (e.g., governor-compose-prod_{new-version}.zip) is downloaded…
- [Update Helm](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/updating_tdk/update_helm.md): Before performing the upgrade, ensure the following steps are completed: * Helm is installed and configured to interact with your Kubernetes cluster.
- [Update Offline](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/updating_tdk/update_offline.md): Before performing the upgrade, ensure the following steps are completed: * PostgreSQL server (version 13 or later) is installed and running.
- [Administration](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/administration.md): Administrative tasks for managing platform deployments.
- [User Management](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/administration/administration.md): In the Administration page you can view and manage the table of users.
- [Activation & Licensing](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/administration/activation_product_license.md): Governor can be used to manage and activate product licenses for Synthesized.
- [Roles and Permissions](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/administration/roles_and_permissions.md): User permissions are a predefined list of application resources and actions that can be accessed and executed if that permission is granted to the user.

## Security & Compliance

- [Role-Based Access Control](https://docs.synthesized.io/tdk/latest/user_guide/060_security_compliance/rbac.md): Control access to platform resources using roles, permissions, and projects.
- [Authentication](https://docs.synthesized.io/tdk/latest/user_guide/060_security_compliance/authentication.md): Configure authentication methods for the platform including SSO, LDAP, Active Directory, and Kerberos.
- [Single Sign-On (SSO)](https://docs.synthesized.io/tdk/latest/user_guide/060_security_compliance/authentication/sso.md): Governor supports Single Sign-On using OAuth2 or SAML.
- [LDAP/Active Directory](https://docs.synthesized.io/tdk/latest/user_guide/060_security_compliance/authentication/ldap_activedirectory.md): LDAP (Lightweight Directory Access Protocol) is often used by organizations as a central repository for user information and as an authentication service.
- [Kerberos](https://docs.synthesized.io/tdk/latest/user_guide/060_security_compliance/authentication/kerberos.md): In enterprises, it is common to use Kerberos for authentication.

## Integrations

- [SAP TDM](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/sap.md): Synthesized is the only TDM platform with end-to-end test data capabilities (synthetic data generation, copying, timeslicing, PII detection, masking, subsetting, provisioning) for…
- [SAP HANA Connection](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/sap/sap_hana.md): Configure connections to SAP HANA databases for source and target data operations.
- [SAP App Level Integration](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/sap/Odata.md): Synthesized supports both JDBC and OData API connections for SAP.
- [Database Integrations](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/databases.md): This page is the single reference for supported databases, JDBC connection strings, required and permissions.
- [CI/CD Integrations](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cicd.md): Automate platform workflows in your CI/CD pipelines with GitHub Actions, GitLab CI, Jenkins, and more.
- [GitHub Actions](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cicd/github_actions.md): To enable easy integration of TDK into CI and CD pipelines, there is a GitHub action available that runs the free version of TDK as part of GitHub Actions workflows.
- [Azure DevOps](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cicd/azure_devops.md): Run Synthesized workflows from an Azure DevOps pipeline: mask production data, generate synthetic edge cases, then run your test suite — all as pipeline stages.
- [Cloud Integrations](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cloud.md): Deploy and integrate the Synthesized platform with AWS, Google Cloud Platform, and Microsoft Azure.
- [Secret Management](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/secret_management.md): Securely store and access database credentials using AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, or HashiCorp Vault.
- [AWS Secrets Manager, Azure Key Vault, HashiCorp Vault](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/secret_management/secret_managers.md): Several ways to provide database credential are supported.
- [Cloud Storage](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cloud_storage.md): Store platform configurations, scripts, and data files in AWS S3, Google Cloud Storage, or Azure Blob Storage.
- [AWS S3](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cloud_storage/aws_s3.md): To be able to read the configuration file from AWS S3 you need to enable the TDK_AWS_ENABLED property.
- [CSV Files](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cloud_storage/csv.md): CSV (Comma Separated Values) is a textual file format used to store tabular data.
- [External Tools](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/external_tools.md): Integrate the platform with development and testing tools including GitHub Actions, Testcontainers, and AWS License Manager.
- [GitHub Actions](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/cicd/github_actions.md): To enable easy integration of TDK into CI and CD pipelines, there is a GitHub action available that runs the free version of TDK as part of GitHub Actions workflows.
- [Testcontainers](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/external_tools/testcontainers.md): Testcontainers for Java is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that c…
- [AWS License Manager](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/external_tools/aws_license_manager.md): If the TDK was purchased from AWS Marketplace, it is required to enable integration with AWS License Manager in order to use the license.
- [Test Automation](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/test_automation.md): Feed realistic, compliant synthetic test data into your test automation platforms so functional, regression, and end-to-end suites run against representative data without exposing…
- [UiPath Test Suite](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/test_automation/uipath_test_suite.md): The Synthesized platform integrates with UiPath Test Suite to supply your automated tests with realistic, compliant synthetic data, so your suites run against representative data…
- [Tricentis Tosca](https://docs.synthesized.io/tdk/latest/user_guide/070_integrations/test_automation/tricentis_tosca.md): The Synthesized platform integrates with Tricentis Tosca to drive your model-based test cases with realistic, compliant synthetic data, so your suites run against representative d…

## Troubleshooting

- [Common Issues](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/common_issues.md): Quick solutions to frequently encountered problems.
- [Diagnostic Tools](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/common_issues/diagnostic.md): The diagnostic tool is capable of collecting the most important information about a database without exposing sensitive data.
- [Performance](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/performance.md): Diagnose and resolve performance issues in platform workflows.
- [Improving Performance](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/performance/improving_performance.md): The machine that Synthesized is hosted on can impact how long it takes to process data.
- [Database Locking Issues](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/performance/locking.md): This error message indicates that Synthesized Platform has insufficient permissions to create an object in the output database.
- [Database-Specific](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/database_specific.md): Solutions to issues specific to particular database platforms.
- [PostgreSQL Partitioned Tables](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/database_specific/postgres_partitioned_table.md): TDK supports PostgreSQL partitioned tables.
- [Oracle Sequences](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/database_specific/sequence_in_oracle.md): Certain columns can be configured to draw their values from a sequence, for example:
- [MySQL TINYINT Issues](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/database_specific/tinyint1_in_mysql.md): When copying a MySQL schema, TINYINT(1) fields in the source database can sometimes be transformed into BIT(1) in the target database.
- [Zero Date Value Issues](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/database_specific/zero_date_value_prohibited.md): As we know, MySQL supports a "zero" value for date types, as stated in the documentation:
- [SSL Connection Issues](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/database_specific/ssl_issues.md): This guide addresses common SSL certificate issues encountered when connecting TDK to databases.
- [Type Handling](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/type_handling.md): Troubleshoot issues related to data type handling, conversion, and compatibility.
- [Auto-generated Types](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/type_handling/autogenerated_types.md): Handle issues with auto-generated database types and sequences.
- [Custom Types](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/type_handling/custom_types.md): Handle custom database types including ENUMs, composite types, and user-defined types.
- [Geometric Types](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/type_handling/geometric_types.md): Handle PostgreSQL geometric and spatial data types.
- [Governor](https://docs.synthesized.io/tdk/latest/changelog/governor.md): Thai is now available as a locale for person and address generation, and format-preserving hashing can keep Thai text in the Thai script.
- [TDK](https://docs.synthesized.io/tdk/latest/changelog/tdk.md): Metadata scan and schema copy for SAP HANA and S4 sources now cover only the tables the workflow uses (plus their foreign-key neighbours) instead of the whole schema.