cleverly.validation.ScoreCheckRow#
- class cleverly.validation.ScoreCheckRow(name: str, kind: str, score: float, threshold: float, std_error: float, passed: bool, converged: bool, n_iter: int, method: str, score_initial: float = nan, failure: str = '', hessian_condition: float = nan, folds_converged: tuple[int, int] | None = None, stopping: str = 'relative', residual_se: float = nan, coverage_bound: float = nan, rule_bound_se: float = nan)[source]#
The score diagnostic for one estimand or one targeted family.
score_initialandhessian_conditionarenanon the rows that do not carry them, which is a sentinel and not an unknown – seesentinel_equality()for why comparing two of these rows needs saying so rather than inheriting the generated__eq__.- Parameters:
- name
str Estimand alias or targeted family.
- kind
str Which score equation this row checks.
- score
float The solved score, on the scale
thresholdis given in.- threshold
float Magnitude the score may reach and still pass.
- std_error
float Standard error the threshold is expressed in, read through
plugin_std_error. It is a scale for the tolerance. On a fit that supplies no inference it is the plug-in diagnostic and not a standard error, andScoreCheck.to_frame()does not publish it.- passedbool
Whether the score stayed inside it.
- convergedbool
Whether targeting reached its own tolerance.
- n_iter
int Targeting iterations taken.
- method
str Targeting algorithm used.
- score_initial
float The score before targeting, for comparison.
- failure
str What went wrong, when something did.
- hessian_condition
float Condition number of the targeting Hessian. A large value says the solve was ill-conditioned even where the score came out small.
- folds_converged
tupleofbool Whether each fold’s own targeting converged, under fold-specific targeting.
- stopping
str The stopping rule that judged the row.
"relative"is the shipped bar, and any other value meansthresholdis the threshold the solver stopped at.- residual_se
float On a
"diagnostic"row of a fit under a declared rule,|P_n IC| / seof the estimand.nanelsewhere.- coverage_bound
float The coverage a Wald interval keeps at that residual (
wald_coverage_bound()).- rule_bound_se
float Under
scaled_var, the residual the rule implies before the fit is read:sum_j |g_j| threshold_j / se, over the score columns the estimand is a combination of. It equalscfor a unit-weight arm mean, and it has no a priori value for a contrast. On a stratified fit it sums the stratum blocks.nanfor a doubly-robust fit, a fit with several draws, an incremental tilt, a working-model coefficient, and a shift, policy, regime or ratio row (_rule_bound_segives each reason).
- name
Methods#
Attributes#
- ScoreCheckRow.folds_converged: tuple[int, int] | None = None#
For a fold-targeted fit, how many of the per-fold solves converged.
- ScoreCheckRow.hessian_condition: float = nan#
Conditioning of the last Newton Hessian. Large means
epsilonis barely identified even where the score looks solved.
- ScoreCheckRow.ratio#
Score relative to its threshold; below 1 passes.
- ScoreCheckRow.reduction#
How far targeting moved the score, as a factor. Above 1 means it shrank.
- ScoreCheckRow.residual_se: float = nan#
The three columns of the estimand rows of a fit under a declared rule.
nanis a sentinel for “this row does not carry it”.
- ScoreCheckRow.score_initial: float = nan#
The same score before the fluctuation moved anything. A step that starts near zero had nothing to do, which is a different situation from one that started large and was driven down – and only the latter is evidence targeting worked.