cleverly.validation.refute

cleverly.validation.refute(result: TMLEResult, *, estimand: str = 'ate', tests: Sequence[str] = ('placebo', 'random_common_cause', 'subset'), n_replicates: int | None = None, subset_fraction: float = 0.7, negative_control_outcome: Any = None, dummy_outcome: GaussianIndependentOutcome | None = None, simulated_outcome: GaussianAdjustmentOutcome | None = None, outcome_rule: EmpiricalInclusionRule = EmpiricalInclusionRule(alpha=0.05, minimum_draws=40, failure_policy='fail'), bootstrap_measurement_error: BootstrapMeasurementError | None = None, measurement_error_rule: EmpiricalInclusionRule = EmpiricalInclusionRule(alpha=0.05, minimum_draws=40, failure_policy='fail'), random_state: int | None = None, tolerance: float = 3.0) → RefutationResult

Run refutation tests against a fitted model.

Parameters:

resultTMLEResult

A fitted result to refute.

estimandstr

Alias the tests are run for.

testssequence of str

Which refutations to run. Parameters without a fixed reported-scale no-effect value refuse placebo and negative_control_outcome before any refit. Differences use zero; risk and odds ratios use one. subset and random_common_cause keep their stability interpretations. A fit given split_plan= refuses subset and bootstrap_measurement_error before any refit. refute_refusal holds these refusals, and every refusal of a generated-outcome or measurement-error request. The refute capability row reads it for the same request, and defers on tests when the default tests meet one and a smaller set runs. random_common_cause adds a column of independent noise. A CTMLE with an explicit ordering= places that column after the declared ordering, and a DRTMLE refits without an evaluation= companion that lacks it.

n_replicatesint or None

Replicates per randomized test. None uses five for each established perturbation, and 100 for each generated-outcome test and for bootstrap_measurement_error. Each of those three tests refuses a budget below its rule’s minimum_draws before any refit.

subset_fractionfloat

Share of rows the subset test refits on.

negative_control_outcomestr or None

An outcome the treatment cannot affect. Required to run that test.

dummy_outcomeGaussianIndependentOutcome or None

Independent Gaussian process declaration. None uses the default declaration.

simulated_outcomeGaussianAdjustmentOutcome or None

Adjustment-dependent additive process. None uses the default declaration.

outcome_ruleEmpiricalInclusionRule

Recorded rule applied to generated-outcome refits.

bootstrap_measurement_errorBootstrapMeasurementError or None

Measurement-error declaration. Required when the named test is requested.

measurement_error_ruleEmpiricalInclusionRule

Recorded rule that compares the original estimate with measurement-error refits.

random_stateint or None

Seed for the randomised tests. None uses the seed the fit was run with, so a seeded fit gives the same refutation every time. An unseeded fit draws a seed, and the refits run under it too. Either way the report records it under random_state, and passing that value back repeats the report.

tolerancefloat

How many standard errors a null test may deviate before failing. The default of 3 keeps the false-alarm rate low across several tests.

Returns:

RefutationResult

One record per test run, with what it expected and what it saw.

Raises:

ValueError

If a test name is unknown, n_replicates is not a positive integer, or the negative-control test has no outcome. These checks precede every refusal and refit.

CapabilityError

If refute_refusal refuses the request. Each such refusal precedes every refit.