To benchmark its synthetic data model, Sarus joined forces with NIST to develop SDNist: an open source library enabling easy benchmarking of synthetic data on the NIST PSCR Differential Privacy Temporal Map Challenge.
The library is really easy to use, first install the sdnist python package:
You can then use the library to evaluate a synthetic dataset:
You can also submit the generative model itself:
And get the score for various levels of privacy loss (ε).
The results can be displayed on a map to figure out where the synthetic data model performed better.
Some examples using sdnist to evaluate some of the top performing generative models from the Differential Privacy Temporal Map Challenge have been implemented and shared on Github.
This work was presented at a AAAI-22 workshop.
If you like sdnist, feel free to star it on Github.