Workflows

Workflow terms

A Workflow (WF) is a manageable entity which is used to audit any configured data transformation.

There are the following parts of WF:

Input and output DB configurations

The Input database marks the source of data and the output database defines where the transformed data will be written after a synthesis.

YAML editor

Here you can see the default configuration of data transformation, that can be redefined. You can modify it according to the TDK documentation.

You can check a YAML file validity by clicking "Validate", (TODO: add more info about validation), upload or download a configuration.

DB view

DB is a visual representation of metadata of the input DB with applied configuration of data transformations. There is a list of tables on the left side of the table, you can navigate through and see columns with applied transformers.

Clicking on the row allows opening the documentation of the appropriate transformer.

Activity log

It’s a log of all manipulations conducted with a WF allowing to perform an audit.

Here you can see the information on when, how, and by whom a WF is changed.

Workflow Creation

To create a WF click "New Workflow" in the top right corner of the Workflows page.

Set input and output DBs. If there are no created data sources yet, you are to press the button "+ Create a new data source", otherwise, select the existing one in the list.

Pay attention: only data sources with read and write permissions are available as output DB (see more in Data sources).

Then change YAML configuration if it’s necessary. The changes are saved on demand.

Workflow Run

Workflow running connects to a database, extract the database model, learn all of the necessary information from the database, and generate a Synthetic copy that will be written to a destination database.

To run workflow click "Run" button.

While the data is being synthesized, you can see its stages and the time spent on each of them.

Then the process is completed you can download a report. (TODO: // reports ??)

Workflow validation

To send a WF on validation to a compliance officer, click Send to validation.

If a user has the role "COMPLIANCE" they will see the status "Validation requested", check the applied transformers in the DB view tab or right in the YAML configuration and can reject or approve the WF.