.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples\1_train\plot_1_hpo_random.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_1_train_plot_1_hpo_random.py>`
        to download the full example code or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_1_train_plot_1_hpo_random.py:


HPO - Random Search
=========================================

.. GENERATED FROM PYTHON SOURCE LINES 7-8

Experiment initialization and data preparation

.. GENERATED FROM PYTHON SOURCE LINES 8-17

.. code-block:: default

    import scipy
    from piml import Experiment
    from piml.models import GLMClassifier

    exp = Experiment()
    exp.data_loader("SimuCredit", silent=True)
    exp.data_summary(feature_exclude=["Race", "Gender"], silent=True)
    exp.data_prepare(target="Approved", task_type="classification", silent=True)








.. GENERATED FROM PYTHON SOURCE LINES 18-19

Train Model

.. GENERATED FROM PYTHON SOURCE LINES 19-21

.. code-block:: default

    exp.model_train(model=GLMClassifier(), name="GLM")








.. GENERATED FROM PYTHON SOURCE LINES 22-23

Define hyperparameter search space for grid search

.. GENERATED FROM PYTHON SOURCE LINES 23-26

.. code-block:: default

    parameters = {'l1_regularization': scipy.stats.uniform(0, 0.1),
                  'l2_regularization': scipy.stats.uniform(0, 0.1)}








.. GENERATED FROM PYTHON SOURCE LINES 27-28

Tune hyperparameters of registered models

.. GENERATED FROM PYTHON SOURCE LINES 28-32

.. code-block:: default

    result = exp.model_tune("GLM", method="randomized", parameters=parameters, n_runs=100,
                            metric="AUC", test_ratio=0.2)
    result.data






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>Rank(by AUC)</th>
          <th>AUC</th>
          <th>time</th>
        </tr>
        <tr>
          <th>params</th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>{'l1_regularization': 0.020239411195815772, 'l2_regularization': 0.002493911719174813}</th>
          <td>1</td>
          <td>0.729472</td>
          <td>0.058955</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.0801649627572255, 'l2_regularization': 0.0009505206902733599}</th>
          <td>2</td>
          <td>0.729471</td>
          <td>0.049783</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.07054458515086691, 'l2_regularization': 0.002443422808146689}</th>
          <td>3</td>
          <td>0.729470</td>
          <td>0.070307</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.0200826243066503, 'l2_regularization': 0.0038925653368613645}</th>
          <td>4</td>
          <td>0.729469</td>
          <td>0.054850</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.08763141285690618, 'l2_regularization': 0.0011317924903120004}</th>
          <td>5</td>
          <td>0.729469</td>
          <td>0.038847</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.014018374389893852, 'l2_regularization': 0.08885523820070576}</th>
          <td>96</td>
          <td>0.729274</td>
          <td>0.046059</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.06720040139301987, 'l2_regularization': 0.08585578494124796}</th>
          <td>97</td>
          <td>0.729273</td>
          <td>0.041242</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.05105133900697683, 'l2_regularization': 0.0845254996471495}</th>
          <td>97</td>
          <td>0.729273</td>
          <td>0.047853</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.05346229516211951, 'l2_regularization': 0.09880860499665617}</th>
          <td>99</td>
          <td>0.729267</td>
          <td>0.046578</td>
        </tr>
        <tr>
          <th>{'l1_regularization': 0.09599608264816151, 'l2_regularization': 0.09690348478707304}</th>
          <td>100</td>
          <td>0.729255</td>
          <td>0.044768</td>
        </tr>
      </tbody>
    </table>
    <p>100 rows × 3 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 33-34

Refit model using a selected hyperparameter

.. GENERATED FROM PYTHON SOURCE LINES 34-37

.. code-block:: default

    params = result.get_params_ranks(rank=1)
    exp.model_train(GLMClassifier(**params), name="GLM-HPO-RandSearch")








.. GENERATED FROM PYTHON SOURCE LINES 38-39

Compare the default model and HPO refitted model

.. GENERATED FROM PYTHON SOURCE LINES 39-41

.. code-block:: default

    exp.model_diagnose("GLM", show="accuracy_table")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

               ACC     AUC      F1 LogLoss   Brier
                                              
    Train   0.6722  0.7309  0.6965  0.6047  0.2088
    Test    0.6690  0.7318  0.6976  0.6073  0.2095
    Gap    -0.0032  0.0009  0.0011  0.0026  0.0008




.. GENERATED FROM PYTHON SOURCE LINES 42-43

Compare the default model and HPO refitted model

.. GENERATED FROM PYTHON SOURCE LINES 43-44

.. code-block:: default

    exp.model_diagnose("GLM-HPO-RandSearch", show="accuracy_table")




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

               ACC     AUC      F1 LogLoss   Brier
                                              
    Train   0.6721  0.7309  0.6964  0.6047  0.2088
    Test    0.6690  0.7318  0.6976  0.6073  0.2095
    Gap    -0.0031  0.0009  0.0012  0.0026  0.0008





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 1 minutes  24.760 seconds)

**Estimated memory usage:**  15 MB


.. _sphx_glr_download_auto_examples_1_train_plot_1_hpo_random.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/selfexplainml/piml-toolbox/main?urlpath=lab/tree/./docs/_build/html/notebooks/auto_examples/1_train/plot_1_hpo_random.ipynb
        :alt: Launch binder
        :width: 150 px



    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_1_hpo_random.py <plot_1_hpo_random.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_1_hpo_random.ipynb <plot_1_hpo_random.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_