.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples\4_testing\plot_4_robustness_cls.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_4_testing_plot_4_robustness_cls.py>`
        to download the full example code or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_4_testing_plot_4_robustness_cls.py:


Robustness: Classification
=====================================

.. GENERATED FROM PYTHON SOURCE LINES 8-9

Experiment initialization and data preparation

.. GENERATED FROM PYTHON SOURCE LINES 9-17

.. code-block:: default

    from piml import Experiment
    from piml.models import ReluDNNClassifier

    exp = Experiment()
    exp.data_loader(data="TaiwanCredit", silent=True)
    exp.data_summary(feature_exclude=["LIMIT_BAL", "SEX", "EDUCATION", "MARRIAGE", "AGE"], silent=True)
    exp.data_prepare(target="FlagDefault", task_type="classification", silent=True)








.. GENERATED FROM PYTHON SOURCE LINES 18-19

Train Model

.. GENERATED FROM PYTHON SOURCE LINES 19-21

.. code-block:: default

    exp.model_train(ReluDNNClassifier(), name="ReluDNN")








.. GENERATED FROM PYTHON SOURCE LINES 22-23

Robustness test with default settings

.. GENERATED FROM PYTHON SOURCE LINES 23-24

.. code-block:: default

    exp.model_diagnose(model="ReluDNN", show="robustness_perf", figsize=(6, 4))



.. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_001.png
   :alt: Model Performance: Perturb on All Features
   :srcset: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 25-26

Robustness test with custom perturbation features

.. GENERATED FROM PYTHON SOURCE LINES 26-29

.. code-block:: default

    exp.model_diagnose(model="ReluDNN", show="robustness_perf",
                       perturb_features=["BILL_AMT1", "BILL_AMT2", "BILL_AMT3"], figsize=(6, 4))




.. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_002.png
   :alt: Model Performance: Perturb on 3 Features
   :srcset: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 30-31

Robustness test with custom perturbation size

.. GENERATED FROM PYTHON SOURCE LINES 31-33

.. code-block:: default

    exp.model_diagnose(model="ReluDNN", show="robustness_perf", perturb_size=0.2, figsize=(6, 4))




.. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_003.png
   :alt: Model Performance: Perturb on All Features
   :srcset: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 34-35

Robustness test with custom perturbation method

.. GENERATED FROM PYTHON SOURCE LINES 35-37

.. code-block:: default

    exp.model_diagnose(model="ReluDNN", show="robustness_perf", perturb_method="quantile", figsize=(6, 4))




.. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_004.png
   :alt: Model Performance: Perturb on All Features
   :srcset: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 38-39

Robustness test with custom metrics

.. GENERATED FROM PYTHON SOURCE LINES 39-41

.. code-block:: default

    exp.model_diagnose(model="ReluDNN", show="robustness_perf", metric="AUC", figsize=(6, 4))




.. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_005.png
   :alt: Model Performance: Perturb on All Features
   :srcset: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_005.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 42-43

Robustness test on worst alpha-percent samples

.. GENERATED FROM PYTHON SOURCE LINES 43-44

.. code-block:: default

    exp.model_diagnose(model="ReluDNN", show="robustness_perf_worst", alpha=0.3, figsize=(6, 4))



.. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_006.png
   :alt: 30%-Worst Sample Performance
   :srcset: /auto_examples/4_testing/images/sphx_glr_plot_4_robustness_cls_006.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 1 minutes  9.751 seconds)

**Estimated memory usage:**  69 MB


.. _sphx_glr_download_auto_examples_4_testing_plot_4_robustness_cls.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/selfexplainml/piml-toolbox/main?urlpath=lab/tree/./docs/_build/html/notebooks/auto_examples/4_testing/plot_4_robustness_cls.ipynb
        :alt: Launch binder
        :width: 150 px



    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_4_robustness_cls.py <plot_4_robustness_cls.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_4_robustness_cls.ipynb <plot_4_robustness_cls.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_