Application properties

How to set

There are several ways how to set system properties:

  • Create application.properties file in the same directory as the application JAR file and set values there. The content of application.properties:

    PROPERTY_NAME = PROPERTY_VALUE

  • Set properties when running a JAR file:

    export PROPERTY_NAME=PROPERTY_VALUE; java -jar app-[version].jar

Properties supported for both API and CLI

  • SECRETS_GCP_SECRET_MANAGER_ENABLED: indicates if Google Cloud Platform Secret Manager is enabled (default: false).

    If this feature is enabled, you can pass JDBC user and password credentials in the following format: {"type": "gcp", "project": "project", "secret": "secret", "version": "version"}, where:

    • project: GCP project ID,

    • secret: Secret Manager secret ID,

    • version (optional, default - latest): Secret Manager secret version ID.

    Please note that access to GCP should be configured via built-in GCP credentials or by GOOGLE_APPLICATION_CREDENTIALS environment variable.

    If this property is enabled you can see the message Google Secret Manager enabled in the application logs.

    The value of the secret should be a UTF-8 encoded string.

  • SECRETS_AWS_SECRET_MANAGER_ENABLED: indicates if AWS Platform Secrets Manager is enabled (default: the value of the property TDK_AWS_ENABLED: false).

    If this feature is enabled, you can pass JDBC user and password credentials in the following format: {"type": "aws", "secret": "secret", "version": "version"}, where:

    • secret: The ARN or name of the secret to retrieve,

    • version (optional): The unique identifier of the version of the secret to retrieve. If you don’t specify the version, then the AWSCURRENT version is used.

    Please note that if this property is enabled, you must ensure that your environment is set up correctly, see how to configure credentials

    If this property is enabled you can see the message AWS Secrets Manager enabled in the application logs.

    The value of the secret should be a UTF-8 encoded string.

  • SECRETS_VAULT_SECRET_MANAGER_ENABLED: indicates if Vault Secret Manager is enabled (default: false).

    If this feature is enabled, you can pass JDBC user and password credentials in the following format: {"type": "vault", "storage": "storage", "path": "path", "secret": "secret", "versioned": "versioned", "version":"version"}, where:

    • storage: The name of the secret storage (e.g. "secret", "database", "cubbyhole", etc).

    • path: The mount path of the secret.

    • secret: The name of the secret to retrieve.

    • versioned: Indicates if the current secret storage is versioned ("false" by default).

    • version: The unique identifier of the version of the secret to retrieve. Using 0 as the default value for the current version.

    If this property is enabled you can see the message Vault Secrets Manager enabled in the application logs.

    The value of the secret should be a UTF-8 encoded string.

    You also need to set additional properties, such as vault.uri, vault.ssl.*, vault.token, and other authentication parameters. For more details, refer to the documentation.

  • TDK_WORKINGDIRECTORY_ENABLED

    If set to true (default: false), enables the usage of a directory on a local file system to optimize some TDK internal operations. Enabling this can noticeably increase the TDK throughput at the cost of occupied disk space. In the worst case, the disk space needed will be linear to the size of the input tables. However, there is a way to opt out of using the working directory for some tables (see use_working_directory setting in configuration section).

    If this property is enabled you can see the message Working directory enabled, path: $path in the application logs.

    Currently, the working directory usage will only affect the subsetting mode performance (when TDK is operating in MASKING mode with target_ratio less than 1). In the future, more disk-based optimizations will be added.

    Example:

    TDK_WORKINGDIRECTORY_ENABLED=true

  • TDK_WORKINGDIRECTORY_PATH

    If TDK_WORKINGDIRECTORY_ENABLED is set to true, this setting is mandatory and must contain an existing path to an empty directory which will be used by the TDK to optimize its operations. Although both relative and absolute paths are supported, it is recommended to specify an absolute path to make the operation reproducible regardless of what directory the TDK is run from.

    Example:

    TDK_WORKINGDIRECTORY_PATH=/home/tdk/working-directory

  • TDK_DB_MAXIMUM-POOL-SIZE

    Maximum database connection pool size. By default, it’s 20.

Note: the level of parallelism for data transferring depends on this parameter.

  • TDK_AWS_ENABLED

    When the parameter is set to true, enables integration with the AWS Cloud Platform.

    By default, the parameter is set to false.

    If this property is enabled, you can provide configuration files located in S3 storage and use credentials from AWS Secrets Manager.

    If this property is enabled, you must ensure that your environment is set up correctly, see how to configure credentials

    Example:

    TDK_AWS_ENABLED=true

  • TDK_AWS_LICENSE_ENABLED

    When the parameter is set to true, enables loading a license from AWS License Manager.

    By default, the parameter is set to false.

    If the TDK was purchased from https://aws.amazon.com/marketplace/pp/prodview-qniisetx55ofg [AWS Marketplace], this property is required. The property is embedded to the AWS marketplace Docker container.

    Example:

    TDK_AWS_LICENSE_ENABLED=true

  • TDK_GCP_ENABLED

    When this parameter is set to true, TDK can use certain Google Cloud Platform features, such as reading configuration and inventory files from Google Cloud Storage.

    By default, the parameter is set to false.

    If this property is enabled, you can use TDK as usual without a special GCP environment. Simply set the path to the file with credentials in the GOOGLE_APPLICATION_CREDENTIALS variable.

    Example:

    TDK_GCP_ENABLED=true

  • TDK_GCP_LICENSE_ENABLED

    When the parameter is set to true, TDK becomes available for use in the Google Cloud Platform. This includes reporting usage statistics, integrating with the GCP License Manager and Cloud Billing.

    By default, the parameter is set to false.

    If this property is enabled, you must ensure that your environment is set up correctly, see how to configure

    Example:

    TDK_GCP_LICENSE_ENABLED=true

  • TDK_AZURE_LICENSE_ENABLED

    When the parameter is set to true, TDK becomes available for use in the Azure Cloud environment.

    By default, the parameter is set to false.

    Example:

    TDK_AZURE_LICENSE_ENABLED=true

Properties supported only for API

  • DB_URL: internal database JDBC URL (default: jdbc:h2:mem:db).

  • DB_USER: internal database user.

  • DB_PASSWORD: internal database password.

  • SQL_DIALECT: internal database SQL Dialect (default: H2).

  • SERVER_PORT: application port (default: 8081).