piml.scored_test.test_reliability_table

piml.scored_test.test_reliability_table(x, y, prediction, prediction_proba=None, feature_names=None, feature_types=None, target_name=None, train_idx=None, test_idx=None, task_type=None, random_state=None, alpha=None)

Get empirical coverage and average bandwidth for regression or Brier Loss for classification.

Parameters:

xndarray of shape (n_samples, n_features), default=None

The covariate data.

yndarray of shape (n_samples, ), default=None

The actual response.

predicitonndarray of shape (n_samples, ), default=None

The model prediction for regression tasks and predicted class for binary classification tasks.

prediciton_probandarray of shape (n_samples, ), default=None

The predicted probability of the positive class in binary classificaiton tasks. Not need for regression tasks.

task_type{‘regression’, ‘classification’}, default=None

The task type.

feature_nameslist, default=None

Feature names.

feature_types: list, default=None

Feature types, can be ‘numerical’ or ‘categorical’.

target_namestr, default=None

Target name.

train_idxarray-like of shape (n_samples_train,), default=None

If train_idx and test_idx are not None, it will be ignored.

test_idxarray-like of shape (n_samples_test,), default=None

If train_idx and test_idx are not None, it will be ignored.

random_stateint, default=None

Random seed for train / test split. If None, it will be 0.

alphafloat, default=None

The error rate in Split Conformal Prediction (only for regression tasks). If None, it will be 0.1.