synthcity

A library for generating and evaluating synthetic tabular data.

Features:

• :key: Easy to extend pluginable architecture.
• :cyclone: Several evaluation metrics for correctness and privacy.
• :fire: Several reference models, by type:
  • General purpose: GAN-based (AdsGAN, CTGAN, PATEGAN, DP-GAN),VAE-based(TVAE, RTVAE), Normalizing flows, Bayesian Networks(PrivBayes, BN), Random Forrest (arfpy), LLM-based (GReaT).
  • Time Series & Time-Series Survival generators: TimeGAN, FourierFlows, TimeVAE.
  • Static Survival Analysis: SurvivalGAN, SurVAE.
  • Privacy-focused: DECAF, DP-GAN, AdsGAN, PATEGAN, PrivBayes.
  • Domain adaptation: RadialGAN.
  • Images: Image ConditionalGAN, Image AdsGAN.
• :book: Read the docs !
• :airplane: Checkout the tutorials!

Please note: synthcity does not handle missing data and so these values must be imputed first HyperImpute can be used to do this.

:rocket: Installation

The library can be installed from PyPI using

$ pip install synthcity

or from source, using

$ pip install .

Other library extensions:

• Install the library with unit-testing support

 pip install synthcity[testing]

• Install the library with GOGGLE support

 pip install synthcity[goggle]

• Install the library with ALL the extensions

 pip install synthcity[all]

:boom: Sample Usage

Generic data

• List the available general-purpose generators

from synthcity.plugins import Plugins

Plugins(categories=["generic", "privacy"]).list()

• Load and train a tabular generator

from sklearn.datasets import load_diabetes
from synthcity.plugins import Plugins

X, y = load_diabetes(return_X_y=True, as_frame=True)
X["target"] = y

syn_model = Plugins().get("adsgan")

syn_model.fit(X)

• Generate new synthetic tabular data

syn_model.generate(count = 10)

• Benchmark the quality of the plugins

# third party
from sklearn.datasets import load_diabetes

# synthcity absolute
from synthcity.benchmark import Benchmarks
from synthcity.plugins.core.constraints import Constraints
from synthcity.plugins.core.dataloader import GenericDataLoader

X, y = load_diabetes(return_X_y=True, as_frame=True)
X["target"] = y

loader = GenericDataLoader(X, target_column="target", sensitive_columns=["sex"])

score = Benchmarks.evaluate(
    [
        (f"example_{model}", model, {})  # testname, plugin name, plugin args
        for model in ["adsgan", "ctgan", "tvae"]
    ],
    loader,
    synthetic_size=1000,
    metrics={"performance": ["linear_model"]},
    repeats=3,
)
Benchmarks.print(score)

Static Survival analysis

• List the available generators dedicated to survival analysis

from synthcity.plugins import Plugins

Plugins(categories=["generic", "privacy", "survival_analysis"]).list()

• Generate new data

from lifelines.datasets import load_rossi
from synthcity.plugins.core.dataloader import SurvivalAnalysisDataLoader
from synthcity.plugins import Plugins

X = load_rossi()
data = SurvivalAnalysisDataLoader(
    X,
    target_column="arrest",
    time_to_event_column="week",
)

syn_model = Plugins().get("survival_gan")

syn_model.fit(data)

syn_model.generate(count=10)

Time series

• List the available generators

from synthcity.plugins import Plugins

Plugins(categories=["generic", "privacy", "time_series"]).list()

• Generate new data

# synthcity absolute
from synthcity.plugins import Plugins
from synthcity.plugins.core.dataloader import TimeSeriesDataLoader
from synthcity.utils.datasets.time_series.google_stocks import GoogleStocksDataloader

static_data, temporal_data, horizons, outcome = GoogleStocksDataloader().load()
data = TimeSeriesDataLoader(
    temporal_data=temporal_data,
    observation_times=horizons,
    static_data=static_data,
    outcome=outcome,
)

syn_model = Plugins().get("timegan")

syn_model.fit(data)

syn_model.generate(count=10)

Images

Note : The architectures used for generators are not state-of-the-art. For other architectures, consider extending the suggest_image_generator_discriminator_arch method from the convnet.py module.

• List the available generators

from synthcity.plugins import Plugins

Plugins(categories=["images"]).list()

• Generate new data

from synthcity.plugins import Plugins
from synthcity.plugins.core.dataloader import ImageDataLoader
from torchvision import datasets


dataset = datasets.MNIST(".", download=True)
loader = ImageDataLoader(dataset).sample(100)

syn_model = Plugins().get("image_cgan")

syn_model.fit(loader)

syn_img, syn_labels = syn_model.generate(count=10).unpack().numpy()

print(syn_img.shape)

Serialization

• Using save/load methods

from synthcity.utils.serialization import save, load
from synthcity.plugins import Plugins

syn_model = Plugins().get("adsgan")

buff = save(syn_model)
reloaded = load(buff)

assert syn_model.name() == reloaded.name()

• Saving and loading models from disk

from sklearn.datasets import load_diabetes
from synthcity.utils.serialization import save_to_file, load_from_file
from synthcity.plugins import Plugins

X, y = load_diabetes(return_X_y=True, as_frame=True)
X["target"] = y

syn_model = Plugins().get("adsgan", n_iter=10)

syn_model.fit(X)

save_to_file('./adsgan_10_epochs.pkl', syn_model)
reloaded = load_from_file('./adsgan_10_epochs.pkl')

assert syn_model.name() == reloaded.name()

• Using the Serializable interface

from synthcity.plugins import Plugins

syn_model = Plugins().get("adsgan")

buff = syn_model.save()
reloaded = Plugins().load(buff)

assert syn_model.name() == reloaded.name()

📓 Tutorials

• Tutorial 0: Getting started with tabular data
• Tutorial 1: Writing a new plugin
• Tutorial 2: Benchmarking models
• Tutorial 3: Generating Survival Analysis data
• Tutorial 4: Generating Time Series
• Tutorial 5: Generating Data with Differential Privacy Guarantees
• Tutorial 6: Practice using Custom Time series data
• Tutorial 7: Image generation
• Tutorial 8: hyperparameter optimization
• Tutorial 9: dealing with missing data

🔑 Methods

Bayesian methods

Generative adversarial networks(GANs)

Variational autoencoders(VAE)

Normalizing Flows

Graph neural networks

Diffusion models

Random Forest models

LLM-based models

Static Survival analysis methods

Time-Series and Time-Series Survival Analysis methods

Privacy & Fairness

Domain adaptation

Images

Debug methods

:zap: Evaluation metrics

The following table contains the available evaluation metrics:

• Sanity checks

• Statistical tests

• Synthetic Data quality

• Privacy metrics

Quasi-identifiers : pieces of information that are not of themselves unique identifiers, but are sufficiently well correlated with an entity that they can be combined with other quasi-identifiers to create a unique identifier.

:mag: Use-cases of synthetic data beyond fidelity & privacy

The following table contains research projects and papers that have been completed using Synthcity. Feel free to explore them and be inspired to use Synthcity in your own research project!

:hammer: Tests

Install the testing dependencies using

pip install .[testing]

The tests can be executed using

pytest -vsx

Contributing to Synthcity

We want to make contributing to Synthcity is as easy and transparent as possible. We hope to collaborate with as many people as we can.

Development installation

First create a new environment. It is recommended that you use conda. This can be done as follows:

conda create -n your-synthcity-env python=3.9
conda activate your-synthcity-env

Python versions 3.7, 3.8, 3.9, and 3.10 are all compatible, but it is best to use the most up to date version you can, as some models may not support older python versions.

To get the development installation with all the necessary dependencies for linting, testing, auto-formatting, and pre-commit etc. run the following:

git clone https://github.com/vanderschaarlab/synthcity.git
cd synthcity
pip install -e .[testing]

Please check that the pre-commit is properly installed for the repository, by running:

pre-commit run --all

This checks that you are set up properly to contribute, such that you will match the code style in the rest of the project. This is covered in more detail below.

Our Development Process

Code Style

We believe that having a consistent code style is incredibly important. Therefore Synthcity imposes certain rules on the code that is contributed and the automated tests will not pass, if the style is not adhered to. These tests passing is a requirement for a contribution being merged. However, we make adhering to this code style as simple as possible. First, all the libraries required to produce code that is compatible with Synthcity's Code Style are installed in the step above when you set up the development environment. Secondly, these libraries are all triggered by pre-commit, so once you are set-up, you don't need to do anything. When you run git commit, any simple changes to enforce the style will run automatically and other required changes are explained in the stdout for you to go through and fix.

Synthcity uses the black and flake8 code formatter to enforce a common code style across the code base. No additional configuration should be needed (see the black documentation for advanced usage).

Also, Synthcity uses isort to sort imports alphabetically and separate into sections.

Type Hints

Synthcity is fully typed using python 3.7+ type hints. This is enforced for contributions by mypy, which is a static type-checker.

Tests

To run the tests, you can either use pytest (again, installed with the testing extra above). The following testing command is good for checking your code,as it skips the tests that take a long time to run.

pytest -vvvsx -m "not slow" --durations=50

But the full test suite can be run with the following command.

pytest -vvvs  --durations=50

Some plugins may be included in the library as extras, the associated tests for these need to be run separately, e.g. the goggle plugin can be tested with the below command:

pytest -vvvs -k goggle --durations=50

Pull Requests

We actively welcome your pull requests.

1. Fork the repo and create your branch from main.
2. If you have added code that should be tested, add tests in the same style as those already present in the repo.
3. If you have changed APIs, document the API change in the PR.
4. Ensure the test suite passes.
5. Make sure your code passes the pre-commit, this will be required in order to commit and push, if you have properly installed pre-commit, which is included in the testing extra.

Issues

We use GitHub issues to track public bugs. Please ensure your description is clear and has sufficient instructions to be able to reproduce the issue.

License

By contributing to Synthcity, you agree that your contributions will be licensed under the LICENSE file in the root directory of this source tree. You should therefore, make sure that if you have introduced any dependencies that they also are covered by a license that allows the code to be used by the project and is compatible with the license in the root directory of this project.

Citing

If you use this code, please cite the associated paper:

@misc{https://doi.org/10.48550/arxiv.2301.07573,
  doi = {10.48550/ARXIV.2301.07573},
  url = {https://arxiv.org/abs/2301.07573},
  author = {Qian, Zhaozhi and Cebere, Bogdan-Constantin and van der Schaar, Mihaela},
  keywords = {Machine Learning (cs.LG), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Synthcity: facilitating innovative use cases of synthetic data in different data modalities},
  year = {2023},
  copyright = {Creative Commons Attribution 4.0 International}
}