ReLU DNN Classification (Taiwan Credit)

Experiment initialization and data preparation

from piml import Experiment
from piml.models import ReluDNNClassifier

exp = Experiment()
exp.data_loader(data="TaiwanCredit", silent=True)
exp.data_summary(feature_exclude=["LIMIT_BAL", "SEX", "EDUCATION", "MARRIAGE", "AGE"], silent=True)
exp.data_prepare(target="FlagDefault", task_type="classification", silent=True)

Train Model

exp.model_train(model=ReluDNNClassifier(hidden_layer_sizes=(40, 40), l1_reg=0.0002, learning_rate=0.001),
                name="ReLUDNN")

Evaluate predictive performance

exp.model_diagnose(model='ReLUDNN', show="accuracy_table")

          ACC      AUC      F1  LogLoss    Brier

Train  0.8200   0.7723  0.4722   0.4334   0.1357
Test   0.8300   0.7708  0.4817   0.4250   0.1317
Gap    0.0100  -0.0015  0.0095  -0.0083  -0.0039

Local Linear Model (LLM) summary plot

exp.model_interpret(model="ReLUDNN", show="llm_summary", figsize=(5, 4))

      Count  Response Mean  Response Std  Local AUC  Global AUC
0    6535.0         0.1053        0.3069     0.6037      0.7349
1    4166.0         0.1066        0.3086     0.6477      0.6261
2    1307.0         0.2295        0.4207     0.5999      0.5003
3    1158.0         0.3100        0.4627     0.7819      0.7348
4     903.0         0.1595        0.3663     0.6118      0.5528
..      ...            ...           ...        ...         ...
300     1.0         1.0000        0.0000        NaN      0.7334
301     1.0         0.0000        0.0000        NaN      0.4979
302     1.0         1.0000        0.0000        NaN      0.7322
303     1.0         1.0000        0.0000        NaN      0.7364
304     1.0         0.0000        0.0000        NaN      0.6900

[305 rows x 5 columns]

Local Linear Model (LLM) parallel coordinate plot

exp.model_interpret(model="ReLUDNN", show="llm_pc", figsize=(5, 4))

Local Linear Model (LLM) violin plot

exp.model_interpret(model="ReLUDNN", show="llm_violin", figsize=(5, 4))

Global feature importance

exp.model_interpret(model="ReLUDNN", show="global_fi", figsize=(5, 4))

Global effect plot: with one feature

exp.model_interpret(model="ReLUDNN", show="global_effect_plot", uni_feature="PAY_1",
                    original_scale=True, figsize=(5, 4))

Global effect plot: with two features

exp.model_interpret(model="ReLUDNN", show="global_effect_plot", bi_features=["PAY_1", "PAY_3"],
                    original_scale=True, figsize=(5, 4))

Local feature importance without centering

exp.model_interpret(model="ReLUDNN", show="local_fi", sample_id=0, centered=False,
                    original_scale=True, figsize=(5, 4))

Local feature importance with centering

exp.model_interpret(model="ReLUDNN", show="local_fi", sample_id=0, centered=True,
                    original_scale=True, figsize=(5, 4))

Estimated memory usage: 120 MB