cleverly.validation.NuisanceModelReport#

class cleverly.validation.NuisanceModelReport(name: str, kind: str, metrics: dict[str, float], calibration: dict[str, list[float]], learner_weights: dict[str, float], learner_risks: dict[str, float], calibration_omission: str | None = None)[source]#

Fit quality for one nuisance model.

Parameters:

namestr: Which nuisance this report describes.
kindstr: Whether it is a regression or a classification.
metricsdict of str to float: Fit metrics at the report’s evaluation rows.
calibrationdict of str to list of float: Binned mean prediction and observed rate.
learner_weightsdict of str to float: Ensemble weight per candidate.
learner_risksdict of str to float: Cross-validated risk per candidate.
calibration_omissionstr or None: Why a probability report has no calibration_slope, for example CALIBRATION_CONSTANT_WITHIN_FOLDS. None when the slope is available or the report is not a probability model.

Methods#

NuisanceModelReport.row() → list[str][source]#

Return this model’s metrics as one row of the diagnostics table.

Returns:

list of str: The formatted cells, in the column order NuisanceDiagnostics.summary() prints. A cell with no finite value reads "-".