cleverly.validation.ScoreCheck

class cleverly.validation.ScoreCheck(rows: tuple[ScoreCheckRow, ...], tolerance: float, n: int, corrected: bool = False, corrections: CorrectionCheck | None = None, backend: str | None = None, inference: Literal['influence_curve', 'working_mechanism_plugin', 'generated_design_plugin', 'estimated_weight_plugin', 'adaptive_baseline_plugin', 'no_cluster_variance_plugin', 'few_cluster_plugin', 'constant_node_plugin', 'stack_failure_plugin'] = 'influence_curve', stopping: str = 'relative', alpha_sig: float = 0.05)

Whether the targeting step solved the score equations it was supposed to.

“The efficient score equation” for a plain fit, and deliberately not that phrase for a doubly-robust one – see corrected.

Parameters:

rowstuple of ScoreCheckRow

One row per estimand or targeted family.

tolerancefloat

Score magnitude, in standard errors, a row may reach and pass.

nint

Number of observations.

correctedbool

Whether the fit carried DR-TMLE correction equations.

correctionsCorrectionCheck or None

Those equations’ own check, when it did.

backendstr or None

Dataframe backend to_frame() returns when data is omitted.

inferencestr

Inference status of the checked fit. One of InferenceStatus, which the inference reference lists. A failing verdict names the spread the fit reports by it, so a fit that supplies no inference reads “plug-in standard errors”.

stoppingstr

The stopping rule the fit declared, "relative" for the shipped one.

alpha_sigfloat

The significance level of the fit’s intervals.

Methods

ScoreCheck.one_line() → str

The verdict as a block a report can append, without the table.

Phrased here rather than at the call site so that every place a verdict is spoken – this module’s own report, and cleverly.TMLEResult.summary() – says the same thing. It names the failing rows, because “the check failed” without them sends the reader back to the method call this line exists to spare them.

Returns:

str

The verdict alone, for a report to append without the table.

ScoreCheck.raise_if_failed() → None

Raise when the check failed – for use in pipelines that must not ship a bad fit.

ScoreCheck.rule_failure_line() → str

The pinned line when the targeting did not meet the rule, else "".

Returns:

str

The line RULE_FAILURE_LINE of cleverly.fluctuation._score. Its detail names each equation that missed its rule and how many times its threshold its largest score is. An empty string means the targeting met the rule, or the fit declared none.

ScoreCheck.rule_line() → str

The pinned line of a fit under a declared rule, or "" under the shipped one.

Returns:

str

The largest score residual in standard errors, the estimand it is on, and the coverage a Wald interval keeps at that residual (RESIDUAL_LINE).

ScoreCheck.summary() → str

Return a printable summary.

Returns:

str

A printable table, one line per checked score equation.

ScoreCheck.to_frame(data: Any = None) → Any

Return tabular output in the input dataframe backend.

Parameters:

dataAny

A dataframe or fitted container whose backend to match. None uses the backend recorded on this object.

Returns:

dataframe

One row per checked score equation.

Attributes

ScoreCheck.alpha_sig: float = 0.05

The significance level of the fit’s intervals, which the coverage bound reads.

ScoreCheck.backend: str | None = None

Name of the dataframe backend the fit’s data arrived in, so that to_frame() honours “results come back in the backend you passed in” without a caller having to thread the container back in by hand.

ScoreCheck.corrected: bool = False

Whether this fit reports the corrected curve \(D = D^* - D^*_Q - D^*_g\) – or the one term of it a single guard solves for, which _curve() derives from the rows – rather than \(D^*\). True for a DRTMLE fit with a non-empty guard, and what the verdict branches on: the equations such a fit solves are its own three, and the curve they leave is the estimator’s influence function at the nuisance limits, which is generally not the efficient one. Signing that off as “the estimated efficient score equation” was the one place the package asserted the thing reduced_corrections() exists to deny.

ScoreCheck.corrections: CorrectionCheck | None = None

The per-arm corrections behind this check’s identity and correction rows, for a reader who wants the recomputation itself rather than the verdict on it. None on a check built by hand; empty rows on any fit that reports no corrections, which is every fit but a guarded DRTMLE one.

ScoreCheck.declared

Whether the fit declared a stopping rule other than the shipped one.

ScoreCheck.failures

Return rows that failed their check.

ScoreCheck.identity_failures

Failures that are software defects rather than unsolved equations.

A row here says the score the targeting step recorded and the term the reported curve carries are not the same functional of the state the fit returned – the DR-TMLE correction identity. Iterating longer cannot fix one, which is why it is worded apart from every other failure this class reports.

ScoreCheck.inference: Literal['influence_curve', 'working_mechanism_plugin', 'generated_design_plugin', 'estimated_weight_plugin', 'adaptive_baseline_plugin', 'no_cluster_variance_plugin', 'few_cluster_plugin', 'constant_node_plugin', 'stack_failure_plugin'] = 'influence_curve'

A default, so a check built by hand reads as the ordinary fit it describes.

ScoreCheck.passed

True when every row is within tolerance.

ScoreCheck.rule_failures

The failing rows that a declared rule judged, which are the rule not being met.

ScoreCheck.stopping: str = 'relative'

The stopping rule the fit declared, "relative" for the shipped one. Under a declared rule the fluctuation rows are held to the rule’s thresholds and the estimand rows are diagnostics.

ScoreCheck.rows: tuple[ScoreCheckRow, ...]

ScoreCheck.tolerance: float

ScoreCheck.n: int