cleverly.validation.EmpiricalInclusionRule

class cleverly.validation.EmpiricalInclusionRule(alpha: float = 0.05, minimum_draws: int = 40, failure_policy: Literal['fail'] = 'fail')

Declare a two-sided empirical inclusion rule for generated outcomes.

Parameters:

alphafloat

Two-sided tail probability.

minimum_drawsint

Minimum successful draw count needed to apply the empirical rule. The rule refuses a count below 2 / alpha, where the declared level is unreachable.

failure_policy{“fail”}

Policy for a failed refit. The first catalog always fails if any refit fails.

Methods

EmpiricalInclusionRule.evaluate(values: Sequence[float], truth: float, failures: Sequence[ReplicationFailure]) → tuple[bool, float | None, str]

Apply the declared draw-count, failure, and empirical-tail gates.

Parameters:

valuessequence of float

Successful refitted estimates.

truthfloat

Declared process effect.

failuressequence of ReplicationFailure

Failed refits retained by the operation.

Returns:

tuple of bool, float or None, str

Pass status, empirical probability, and readable detail. The probability is None when no draw succeeded, because an empty sample carries no rank information and a reported zero would read as a strong rejection.

EmpiricalInclusionRule.pvalue(values: Sequence[float], truth: float) → float

Return a two-sided empirical rank p-value with inclusive half-ties.

Parameters:

valuessequence of float

Successful refitted estimates.

truthfloat

Declared process effect.

Returns:

float

Two-sided empirical rank probability. An empty sample returns 0.0, which is a rank over nothing rather than a rejection. evaluate() reports None for that case, and it is the value a report carries.

Attributes

EmpiricalInclusionRule.alpha: float = 0.05

EmpiricalInclusionRule.failure_policy: Literal['fail'] = 'fail'

EmpiricalInclusionRule.minimum_draw_count

Return the minimum successful draw count under this rule.

EmpiricalInclusionRule.minimum_draws: int = 40