Generating Diverse Counterfactual Explanations without accessing training data¶
If only the trained model is available but not the training data, DiCE can still be used to generate counterfactual explanations. Below we show an example where DiCE uses only basic metadata about each feature used in the ML model.
[1]:
# import DiCE
import dice_ml
from dice_ml.utils import helpers # helper functions
# supress deprecation warnings from TF
import tensorflow as tf
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
Defining meta data¶
We simulate “adult” income dataset from UCI Machine Learning Repository (https://archive.ics.uci.edu/ml/datasets/adult) by providing only meta information about the data: range is provided for continuous features and categories are provided for categorical features. Please note for Python<=3.6, “features” parameter should be provided as an OrderedDict in the same order that was used to train the ML model.
[2]:
d = dice_ml.Data(features={'age':[17, 90],
'workclass': ['Government', 'Other/Unknown', 'Private', 'Self-Employed'],
'education': ['Assoc', 'Bachelors', 'Doctorate', 'HS-grad', 'Masters', 'Prof-school', 'School', 'Some-college'],
'marital_status': ['Divorced', 'Married', 'Separated', 'Single', 'Widowed'],
'occupation':['Blue-Collar', 'Other/Unknown', 'Professional', 'Sales', 'Service', 'White-Collar'],
'race': ['Other', 'White'],
'gender':['Female', 'Male'],
'hours_per_week': [1, 99]},
outcome_name='income')
Loading trained ML model¶
Below, we use a trained ML model which produces high accuracy on test datasets, comparable to other popular baselines. This sample trained model comes in-built with our package.
The variable backend below indicates the implementation type of DiCE we want to use. We use TensorFlow 1.x in the notebooks with backend=‘TF1’. You can set backend to ‘TF2’ or ‘PYT’ to use DiCE with TensorFlow 2.x or with PyTorch respectively. We want to note that the time required to find counterfactuals with Tensorflow 2.x’s eager style of execution is significantly greater than that with TensorFlow 1.x’s graph execution.
[3]:
backend = 'TF'+tf.__version__[0] # TF1
ML_modelpath = helpers.get_adult_income_modelpath(backend=backend)
m = dice_ml.Model(model_path= ML_modelpath, backend=backend)
Generate diverse counterfactuals¶
[4]:
# initiate DiCE
exp = dice_ml.Dice(d, m)
[5]:
# query instance in the form of a dictionary; keys: feature name, values: feature value
query_instance = {'age':22,
'workclass':'Private',
'education':'HS-grad',
'marital_status':'Single',
'occupation':'Service',
'race': 'White',
'gender':'Female',
'hours_per_week': 45}
[6]:
# generate counterfactuals
dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=4, desired_class="opposite")
INFO:root: MAD is not given for feature age, so using 1.0 as MAD instead.
INFO:root: MAD is not given for feature hours_per_week, so using 1.0 as MAD instead.
Diverse Counterfactuals found! total time taken: 00 min 04 sec
[7]:
# visualize the results
dice_exp.visualize_as_dataframe(show_only_changes=True)
Query instance (original outcome : 0)
age | workclass | education | marital_status | occupation | race | gender | hours_per_week | income | |
---|---|---|---|---|---|---|---|---|---|
0 | 22.0 | Private | HS-grad | Single | Service | White | Female | 45.0 | 0.01904 |
Diverse Counterfactual set without sparsity correction since only metadata about each feature is available (new outcome : 1)
age | workclass | education | marital_status | occupation | race | gender | hours_per_week | income | |
---|---|---|---|---|---|---|---|---|---|
0 | 60.0 | Self-Employed | Prof-school | Married | Professional | - | - | 43.0 | 0.911 |
1 | 38.0 | Other/Unknown | Assoc | Married | - | - | - | 55.0 | 0.74 |
2 | 90.0 | - | Doctorate | - | - | - | - | 99.0 | 0.755 |
3 | 70.0 | - | - | - | White-Collar | Other | Male | 73.0 | 0.525 |
When the training data is available, by default, the distance between two values of a continuous feature is scaled by the inverse median absolute deviation (MAD) of the feature using the training data. This is done to capture the relative prevalence of observing a continuous feature at a particular value, as discussed in our paper. However, when there is no access to the training data, as in the above case, no scaling is done and hence all features are weighted equally in the normalized form. As a result, the counterfactuals generated above are different from those in DiCE_getting_started notebook where the training data was available. Nonetheless, you can manually provide the scaling constants through a parameter feature_weights to the generate_counterfactuals() method as shown in this advanced notebook, or you can provide the MADs directly to the data interface dice_ml.Data if you know them.
[ ]: