Accuracy: Classification

Experiment initialization and data preparation

from piml import Experiment
from piml.models import XGB2Classifier

exp = Experiment()
exp.data_loader(data="TaiwanCredit", silent=True)
exp.data_summary(feature_exclude=["LIMIT_BAL", "SEX", "EDUCATION", "MARRIAGE", "AGE"], silent=True)
exp.data_prepare(target="FlagDefault", task_type="classification", silent=True)

Train Model

exp.model_train(XGB2Classifier(), name="XGB2")

Accuracy table

exp.model_diagnose(model="XGB2", show="accuracy_table")

          ACC      AUC      F1 LogLoss   Brier

Train  0.8219   0.7978  0.4759  0.4196  0.1316
Test   0.8290   0.7728  0.4797  0.4252  0.1319
Gap    0.0071  -0.0251  0.0038  0.0057  0.0004

Plot confusion matrix, ROC and Recall-Precision

exp.model_diagnose(model="XGB2", show="accuracy_plot", figsize=(10, 4))

Plot residual with respect to the feature PAY_1

exp.model_diagnose(model="XGB2", show="accuracy_residual", show_feature="PAY_1",
                   use_test=False, original_scale=True, figsize=(5, 4))

Plot residual with respect to the target feature

exp.model_diagnose(model="XGB2", show="accuracy_residual", show_feature="FlagDefault",
                   use_test=False, figsize=(5, 4))

Plot residual with respect to the predicted response

exp.model_diagnose(model="XGB2", show="accuracy_residual", show_feature="FlagDefault_predict",
                   use_test=False, figsize=(5, 4))

Estimated memory usage: 54 MB