cleverly.validation.RefutationTest

class cleverly.validation.RefutationTest(name: str, estimand: str, original: float, values: tuple[float, ...], expectation: str, passed: bool, detail: str, standard_errors: tuple[float, ...] = (), declaration: Any = None, rule: EmpiricalInclusionRule | None = None, records: tuple[EmpiricalRefitRecord, ...] = (), failures: tuple[ReplicationFailure, ...] = (), family: str | None = None, empirical_pvalue: float | None = None, declared_effect: float | None = None, requested_draws: int | None = None, resampling: str | None = None, inference: Literal['influence_curve', 'working_mechanism_plugin', 'generated_design_plugin', 'estimated_weight_plugin', 'adaptive_baseline_plugin', 'no_cluster_variance_plugin', 'few_cluster_plugin', 'constant_node_plugin', 'stack_failure_plugin'] = 'influence_curve', mean_kind: Literal['arithmetic', 'geometric'] = 'arithmetic', spread_kind: Literal['reported', 'log'] = 'reported')

The outcome of one refutation test.

Parameters:

namestr

Which refutation was run.

estimandstr

Alias the test was run for.

originalfloat

The estimate before the refutation.

valuestuple of float

Estimates the refutation produced.

expectationstr

What those values should look like if the fit is sound.

passedbool

Whether they did.

detailstr

What was seen, in the test’s own terms.

standard_errorstuple of float

Standard errors from successful empirical refits.

declarationobject or None

Immutable declaration, for an empirical refit test. It holds the outcome process, or the measurement-error request.

ruleEmpiricalInclusionRule or None

Recorded inclusion rule, for an empirical refit test.

recordstuple of EmpiricalRefitRecord

Successful empirical refit draws.

failurestuple of ReplicationFailure

Failed empirical draws retained by the shared failure contract. Every operation retains a refit failure. bootstrap_measurement_error also retains a draw it cannot sample or perturb.

familystr or None

Family supplied by successful refitted results.

empirical_pvaluefloat or None

Two-sided empirical rank p-value for the declared effect. None when no draw succeeded, and on an older refuter.

declared_effectfloat or None

Effect the declared process implies for this test’s own contrast direction, and None for an older refuter. It carries the sign of the parameter key: a fit that reports code zero against code one declares the negated process effect.

requested_drawsint or None

Requested empirical draw count.

resamplingstr or None

Resolved bootstrap mode for a bootstrap-based refuter.

inferencestr

Inference status of the refitted estimates. One of InferenceStatus, which the inference reference lists. On a fit that supplies no inference, standard_errors and each record’s std_error hold the plug-in diagnostic plugin_std_error, and to_frame() publishes that column as plugin_std_error.

mean_kindstr

"geometric" for a ratio placebo or noise refutation tested on the log scale. Other tests use the arithmetic mean. The underlying values remain on the reported scale.

spread_kindstr

"log" for ratio subset scatter tested on the log scale. Other reports use reported-scale scatter. The underlying values remain on the reported scale.

Methods

RefutationTest.to_frame(data: Any = None, *, backend: str | None = None) → Any

Return one row per empirical refit draw.

Parameters:

dataAny

Dataframe or fitted container whose backend to match.

backendstr or None

Backend name used when data is omitted.

Returns:

dataframe

Successful estimates and standard errors alongside retained failures. The standard-error column is plugin_std_error when inference is a diagnostic status.

Attributes

RefutationTest.child_seeds

Return every successful and failed draw seed in draw-index order.

RefutationTest.declaration: Any = None

RefutationTest.declared_effect: float | None = None

RefutationTest.empirical_pvalue: float | None = None

RefutationTest.failures: tuple[ReplicationFailure, ...] = ()

RefutationTest.family: str | None = None

RefutationTest.inference: Literal['influence_curve', 'working_mechanism_plugin', 'generated_design_plugin', 'estimated_weight_plugin', 'adaptive_baseline_plugin', 'no_cluster_variance_plugin', 'few_cluster_plugin', 'constant_node_plugin', 'stack_failure_plugin'] = 'influence_curve'

RefutationTest.mean

Return the displayed mean of refitted estimates.

Returns:

float

Arithmetic mean by default, or geometric mean for a ratio placebo or noise refutation. nan when no replicate is finite. Failed replicates are dropped.

RefutationTest.mean_kind: Literal['arithmetic', 'geometric'] = 'arithmetic'

RefutationTest.n_failed

Return the number of retained failed empirical draws.

RefutationTest.n_replicates

Return the number of requested draws represented by this test.

RefutationTest.process

Return the empirical refit declaration, or None for older refuters.

RefutationTest.records: tuple[EmpiricalRefitRecord, ...] = ()

RefutationTest.requested_draws: int | None = None

RefutationTest.resampling: str | None = None

RefutationTest.rule: EmpiricalInclusionRule | None = None

RefutationTest.spread

Return how far this test’s replicates spread on the verdict scale.

Returns:

float

Sample standard deviation over the finite replicates, on the log scale for a ratio subset test. Returns nan with fewer than two values.

RefutationTest.spread_kind: Literal['reported', 'log'] = 'reported'

RefutationTest.standard_errors: tuple[float, ...] = ()

RefutationTest.name: str

RefutationTest.estimand: str

RefutationTest.original: float

RefutationTest.values: tuple[float, ...]

RefutationTest.expectation: str

RefutationTest.passed: bool

RefutationTest.detail: str