Single Table Synthesis

This example will use a modified public dataset from Kaggle detailing the attendance of a range of fitness classes available at a gym:

This dataset has a number of interesting features:

This tutorial will demonstrate how the SDK can be used create synthetic data with the same statistical fidelity as the original dataset, and new values of booking_id in order to supplement the original data. This is a common task within an organisation, especially where the volume of the original dataset is small.

Before any synthetic data is generated, it is best practice to perform some exploratory data analysis (EDA) to gain a deeper understanding of the dataset. For example, EDA could be done to determine whether there are missing values, understand the distribution of outliers or calculate/plot any univariate and multivariate statistics that may aid in the development of any hypothesis testing. This is by no means a comprehensive list of techniques and tools encompassed within EDA and the actual EDA process is highly individual and dependent on how the synthetic data will be leveraged downstream.

The first step is to extract the metadata from the original dataset. The metadata stores the information regarding the inferred data types that will be used during model training. The MetaExtractor object can be used to extract the meta data, which is then available for inspection

More information on the metadata, including how to override the default behaviour, can be found in the documentation.

Following the metadata extraction, a HighDimSynthesizer object is created. The HighDimSynthesizer is our generative model for tabular data, using a mixture of deep generative statistical models. It is trained on the original dataset and used to generate the synthetic data

The intelligent inference built-in to the SDK will determine how each column should be modelled using the metadata extracted in the previous step. The HighDimSynthesizer object can be inspected in order to examine how each column will be modelled

There are three types of models available as part of the SDK

It is worth noticing that even though the day_of_week column is stored as an Integer meta (and int dtype in the original data), the intelligent inference in the HighDimSynthesizer has correctly determined that it should be modelled categorically. More information on the models used by the HighDimSynthesizer is available in the documentation

The distributions and correlations in the original data can now be learnt by the deep generative engine utilised by the HighDimSynthesizer

before the trained HighDimSynthesizer instance is used to generate data

Before performing any evaluation, by simply visually inspecting the synthetic dataset it is clear that the values of booking_id have been sampled from the original dataset, rather than any new ones actually being created. The next stage is to evaluate the statistical quality of the synthetic data in order to determine which aspects of the training process need reconfigured, along with configuring the ability to synthesize new values in "booking_id".

When evaluating the quality of synthetic data, there are generally three perspectives to consider:

It is recommended best practice to evaluate the statistical quality of the synthetic data (i.e. its fidelity compared to the original data) before considering other metrics, to ensure that the synthetic data quality meets the requirements of the user.

Here, the focus will be on evaluating only the statistical quality of the synthetic data. However, the documentation details the comprehensive set of tools which can be combined to form a framework that can be used to evaluate the quality of the data across the three key dimensions listed above.

A simple first step is to analyse whether similar numbers of missing values are present in the original and synthetic dataframes. Where there are no missing values in the original, it should be confirmed there are none in the synthetic

Next, the Assessor class can be utilized in order to compare the univariate and bivariate statistics across the synthetic and original datasets. As an initial sanity check, the distributions of the continuous and categorical features in the original and synthetic datasets are visually compared (excluding the "booking_id" feature since this is an enumerated unique ID):

A quick visual inspection confirms that the synthetic data is qualitatively similar to the original, however by calculating appropriate distances between distributions in the two datasets the user can better quantify the statistical quality of the data. For continuous distributions, the Kolmogorov-Smirnov (KS) distance is calculated

while for categorical distributions the earth mover’s distance (EMD) is shown

Analysis can also be extended to two column correlation and interaction metrics. For example, when examining the correlations across all ordinal variables it is appropriate to use the Spearman-rho correlation

A full description of the metrics available in the Assessor module can be found in the documentation

Finally, certain constraints may be present in an original dataset, such as business rules, that may be required to be adhered in the synthetic dataset. Recall that in the original data, "Anytime" membership allowed gym users to attend a class on any day of the week while "Weekend" members could only book classes on a Saturday or Sunday. By plotting a cross-table of the number of times a value in one column appears beside a value in another, it is possible to determine whether this business logic has been adhered to

Although the statistical quality of the synthetic data is very good, there are a few instances where the business logic regarding membership is not respected. When this occurs, it is usually due to limited training data. In the next section, a custom synthesis job will be run which will enforce this business logic.

While the statistical quality of the synthetic data generated through the default configuration is very good, it did not meet all the requirements laid out at the beginning of the tutorial. In this section, a custom configuration will be applied such that:

To accomplish the first of these goals, the Associations class will be used in order to generate custom scenarios within the synthetic data.

The Association class will automatically detect how two variables are related, model their joint-probability and ensure that "impossible" events (e.g. a gym user with a "Weekend" membership attending a class on a weekday) do not appear in the synthetic data. More information on the Association class, as well as other rules and constraints, can be found in the documentation.

Secondly, to ensure that new values are created in the booking_id column, a type_override is defined when instantiating an instance of the HighDimSynthesizer to ensure that the column is modelled properly. As described in Overrides, a starting value and step size can be explicitly defined when using the Enumeration model

As before, the model is trained

and used to generate synthetic data

In contrast to the default set-up, the new configuration allows the generation of new values for booking_id. A full evaluation of the statistical quality of the new synthetic dataset should once again be completed as in the previous section.

To demonstrate that the use of the Associations class has constrained the synthetic data to follow the correct business logic, with regards to the membership and booking_id column, the cross tables are plotted once again

With the custom configuration, the HighDimSynthesizer has adhered to the correct business logic by only producing data points where "Weekend" members can only attend weekend classes.