.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples\4_testing\plot_5_resilience_reg.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code or to run this example in your browser via Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_4_testing_plot_5_resilience_reg.py: Resilience - Regression ===================================== .. GENERATED FROM PYTHON SOURCE LINES 8-9 Experiment initialization and data preparation .. GENERATED FROM PYTHON SOURCE LINES 9-17 .. code-block:: default from piml import Experiment from piml.models import XGB2Regressor exp = Experiment() exp.data_loader(data="BikeSharing", silent=True) exp.data_summary(feature_exclude=["yr", "mnth", "temp"], silent=True) exp.data_prepare(target="cnt", task_type="regression", silent=True) .. GENERATED FROM PYTHON SOURCE LINES 18-19 Train model .. GENERATED FROM PYTHON SOURCE LINES 19-21 .. code-block:: default exp.model_train(model=XGB2Regressor(n_estimators=100), name="XGB2") .. GENERATED FROM PYTHON SOURCE LINES 22-23 Resilience performance against worst sample scenario .. GENERATED FROM PYTHON SOURCE LINES 23-24 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_perf", resilience_method="worst-sample", figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_001.png :alt: Resilience Test :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 25-26 Resilience performance against outer sample scenario .. GENERATED FROM PYTHON SOURCE LINES 26-27 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_perf", resilience_method="outer-sample", figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_002.png :alt: Resilience Test :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 28-29 Resilience performance against worst cluster scenario .. GENERATED FROM PYTHON SOURCE LINES 29-30 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_perf", resilience_method="worst-cluster", figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_003.png :alt: Resilience Test :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 31-32 Resilience performance against hard sample scenario .. GENERATED FROM PYTHON SOURCE LINES 32-33 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_perf", resilience_method="hard-sample", figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_004.png :alt: Resilience Test :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 34-35 Marginal distributional distance between full sample and worst sample with worst-sample scenario .. GENERATED FROM PYTHON SOURCE LINES 35-37 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_distance", resilience_method="worst-sample", distance_metric="PSI", alpha=0.3, figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_005.png :alt: Distribution Shift: 30%-Worst vs Remaining :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 38-39 Marginal distributional distance between full sample and worst sample with not None immutable features .. GENERATED FROM PYTHON SOURCE LINES 39-41 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_distance", resilience_method="worst-sample", distance_metric="PSI", immu_feature="hr", alpha=0.3, figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_006.png :alt: Distribution Shift: 30%-Worst vs Remaining :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 42-43 Marginal distributional distance between full sample and worst sample with worst-cluster scenario .. GENERATED FROM PYTHON SOURCE LINES 43-45 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_distance", resilience_method="worst-cluster", distance_metric="WD1", n_clusters=10, figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_007.png :alt: Distribution Shift: 10-Cluster-Worst vs Remaining :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 46-47 Marginal histogram plot for full sample and worst sample .. GENERATED FROM PYTHON SOURCE LINES 47-49 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_shift_histogram", resilience_method="worst-sample", show_feature="hr", original_scale=True, figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_008.png :alt: Distribution Shift: 10%-Worst vs Remaining :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_008.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 50-51 Marginal density plot for full sample and worst sample .. GENERATED FROM PYTHON SOURCE LINES 51-53 .. code-block:: default exp.model_diagnose(model="XGB2", show="resilience_shift_density", resilience_method="worst-sample", show_feature="hr", original_scale=True, figsize=(5, 4)) .. image-sg:: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_009.png :alt: Distribution Shift: 10%-Worst vs Remaining :srcset: /auto_examples/4_testing/images/sphx_glr_plot_5_resilience_reg_009.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 1 minutes 25.540 seconds) **Estimated memory usage:** 23 MB .. _sphx_glr_download_auto_examples_4_testing_plot_5_resilience_reg.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/selfexplainml/piml-toolbox/main?urlpath=lab/tree/./docs/_build/html/notebooks/auto_examples/4_testing/plot_5_resilience_reg.ipynb :alt: Launch binder :width: 150 px .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_5_resilience_reg.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_5_resilience_reg.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_