Scope and refusals#
cleverly refuses a composition it has not derived, rather than returning a convenient
approximation to a different estimand. This page says how to read such a refusal. It also tabulates
the one axis where the answer is least obvious: which surfaces take more than two treatment arms.
How to read a refusal#
A refusal is always by name. The keyword is accepted and rejected with a stated reason. It does
not arrive as an unexpected keyword argument that names none.
Every CleverlyError carries the record of its refusal in error.refusal. The record is a frozen
Refusal from cleverly.exceptions. str(error) prints error.refusal.message().
field |
holds |
|---|---|
|
the |
|
the sentences that state the failed condition |
|
the sentence that names a working call, or |
|
the id of the roadmap item that tracks a missing capability, or |
The reason of the record tells you where the problem is. That decides what to do about it.
A RefusalReason is a string enumeration, so error.refusal.reason == "inference_gap" is true
for an inference gap. The error class does not fix the code. Read error.refusal.reason and not
the class.
code |
where the problem is |
what the message holds |
what to do about it |
|---|---|---|---|
|
in the declared sample or design |
the failed condition |
Correct the declaration, the data, or the design. The request is not well posed as declared |
|
in the question |
both questions: the one you asked and the one the option answers |
Decide which question you mean. No flag produces the other, and one that quietly did would answer something nobody asked |
|
in this package |
the alternatives and the roadmap item that tracks the gap |
The parameter is well defined and nobody has written it here. Use the alternative, ask for it, compute it elsewhere, or contribute it. Proposed work is on the roadmap |
|
in the uncertainty |
the supported inference |
The estimate exists, and the requested interval, p-value, or band is not derived. Report the supported inference |
|
in the call |
a working call |
The feature exists through another argument or object. Make the call that the message names. This code does not mark a missing capability |
A capability row carries the same record before any call. A row of
result.diagnostics.capabilities, an AssessmentItem, and a MethodAvailability row of
CausalStudy.available_methods() each hold it in refusal. For these rows refusal is None when no refusal
applies. The call, when you make it, raises an error with the same code and text. The row and the raise
therefore state one reason in one text.
A built-in ValueError or TypeError means a malformed argument, such as a wrong type or a value
out of range. It is an invalid declaration, and it carries no record. A misspelled keyword raises
TypeError for the same reason.
Some refused requests would run and return a plausible number whose target or interval is wrong. The Wrong by construction section lists them. Read them as warnings about the analysis, and not about this package’s coverage.
A fit whose data cannot support what you declared is an invalid_declaration that the package
refuses where the problem arises. Examples are a regimen nobody followed and two absorbing causes
firing at one node. A known policy that can draw a level no fitted row of its node received is
another, and known stochastic policies states
why. Those are statements about the sample.
A horizon at which no follower of a regimen had the event is not refused. Its regression is zero, the maximum-likelihood hazard, and a node with no event gives the rule.
Cross-fitting narrows what the sample supports. Each outer fold fits its regressions and its mechanism on its training rows alone, so every fold must carry the followers and treatment levels that each node needs. The split reads no treatment and no outcome, so it cannot protect a rare level. The package checks the realized draw instead, before the first learner, and the refusal names no redraw. The fold and outcome-scale rules give every such message.
The sections below list refusals by topic. Each table has a code column. The column holds the
code that the raise site assigns, so a section can hold rows of several codes.
section |
what it lists |
|---|---|
refusals of the request, mostly |
|
the refusals of |
|
requests that run and carry the experimental label, and the ones that stay refused |
|
the omission codes of a result that cannot replay, all |
|
requests that ask for another estimand, mostly |
|
mistakes that would run and return a wrong number |
Not written yet#
Nothing is wrong with wanting any of these. Most rows are gaps in coverage, and the message says
so rather than implying the request was ill-posed. A row marked implementation_gap names an
alternative and the roadmap item that tracks the gap. tests/unit/test_refusal_codes.py checks
that each such refusal names an item. A row marked inference_gap names an alternative, and the
roadmap item when one tracks the gap. A row marked invalid_declaration or
api_redirection names the declaration to correct or the call to make.
refused |
code |
where |
|---|---|---|
missing-outcome |
|
missing-outcome natural-course contracts lists every refusal. Observed-data extensions defines both estimators, and PAR and PAF with missing outcomes defines the stack |
cross-fitted arm-indexed means and contrasts with missing outcomes outside the stacked CV-TMLE contract |
|
missing-outcome arm-indexed contract lists every refusal. Stacked CV-TMLE for arm-indexed targets defines the estimator |
a cross-fitted shift, incremental, regime, MSM, or controlled-direct-effect fit with missing outcomes ( |
|
the refusals a caller can meet. The hard-stop review reads Díaz et al. (2023), Theorem 3, for the shift, regime, MSM and controlled-direct-effect targets. The incremental target waits on a source read. The package implements neither. The fit raises |
a continuous-dose MSM with missing outcomes ( |
|
MSM projections. The clever covariate that the package builds divides by the treatment density at each grid dose. In that construction, this composition also needs the response or intermediate mechanism at each grid dose, and the package does not write it. |
|
|
method presets, and the DR-TMLE refusals |
a declared missing treatment ( |
|
the composite indicator. The recording may depend on the treatment, so |
an undeclared missing treatment value |
|
the package does not infer a missing treatment from a missing value. |
a declared known treatment mechanism beside a continuous treatment, a vector treatment with a dose, or |
|
known treatment mechanism. A known density is not supported, and the classifier learns the ratio that the declaration fixes. A declared zero on the arm a regimen needs leaves no unit able to follow it. Each refuses before any learner. |
baseline strata ( |
|
weights, strata, and clusters. Fold targeting fits the fluctuation on the validation rows. Montoya et al. (2023), Section 3.2, Step 2(c), publish that update, and the package does not implement it with strata (X53 and X11 part (h) hold its other compositions). X61 part (b) holds fold targeting with strata. Fold evaluation needs stratum shares and a stratum-indexed fold average in each validation fold (X28). The other three are data conditions where the root may not exist or no regression is fittable. Each refuses before any learner. |
the MNAR tilt, |
|
missingness tilt and tipping gamma. The tilt re-mixes the arm-indexed means and their linear contrasts. No derivation here covers the tilt of these parameters. Both entry points raise the sentence of |
a learned-rule value from an in-sample or one-fold fit, the stacked report, fold-specific targeting, a categorical treatment, or a contrast with a known regime or an arm, |
|
learned rules. Each request has a published source and no implementation here. |
a learned-rule value with a continuous treatment, |
|
learned rules. No reviewed source gives the result. The fit refuses before any learner. |
|
|
|
a hazard-based survival fit outside a held baseline categorical treatment with no covariate after baseline, and the isotonic projection on an end-of-study fit or a working model |
|
hazard and isotonic survival refusals lists every refusal. On a design that H1, H2, H7 or H11 refuses, |
an odds tilt on |
|
incremental interventions over time. |
an odds tilt on |
|
incremental interventions over time. |
an |
|
incremental interventions over time. |
|
|
incremental interventions over time. The estimator fits no fluctuation. |
|
|
incremental interventions over time. An average of influence-function values can leave the outcome range. |
|
|
repeated outcomes. |
|
|
what the reader refuses. The package fits no intercept-only nuisance design. The three layouts share one container, so |
|
|
sequential g-computation and IPW. Neither baseline has a root-n result with a data-adaptive learner, in sample or cross-fitted. The fit raises |
|
|
sequential g-computation and IPW. The fit raises |
a fluctuation setting on |
|
sequential g-computation and IPW. A fluctuation setting raises |
|
|
repeated outcomes. The fit raises the shared repeated-outcome refusal before any learner. X71 owns the reduction |
a static level other than the reference, a contrast, or |
|
structural dynamic effects. The fit reports the blips and |
a rule, a modified treatment policy, a known stochastic plan, an incremental policy, censoring, a survival outcome, a repeated outcome, a held decision, a declared mechanism, or |
|
structural dynamic effects. The fit raises |
a static dose outside the observed range, a dose node whose range excludes the reference dose 0, a singular diagonal Jacobian block, a blip feature that reads a later covariate or an unknown column, and |
|
structural dynamic effects. The message names the range, the block or the node |
|
|
parametric g-formula. The fit raises |
|
|
parametric g-formula. The fit raises |
an undeclared time-varying column that is not coded 0/1, a learner that is not a |
|
parametric g-formula. The fit raises |
sample sensitivity-bound estimation for |
|
longitudinal diagnostics. See F16 for the contracts Tan (2025) leaves open |
|
|
the sequentially doubly robust estimator. |
|
|
the sequentially doubly robust estimator. The estimator is the sequential regression of a longitudinal fit. |
|
|
the sequentially doubly robust estimator. An average of transforms can leave the outcome range (Díaz et al. 2023, page 854), and the scale is undefined there. |
an |
|
MSM projections. A transform mean outside the link’s range has no finite solution. The fit raises |
blocked-temporal and rolling-origin splits |
|
|
a confidence interval, a p-value, or a standard error on a |
|
collaborative TMLE. No result shows the reported curve is the estimator’s influence curve when the selected working mechanism is not consistent for the treatment law. |
a confidence interval, a p-value, or a standard error on every |
|
collaborative TMLE. The fit takes the |
a confidence interval, a p-value, or a standard error on a |
|
DR-TMLE refusals. The fit takes the |
a confidence interval, a p-value, or a standard error on a |
|
clusters. The fit takes the |
|
|
clusters. No source read here gives a \(t\)-calibrated joint band. |
a contrast, a simultaneous band, any E-value branch, |
|
inference status. Each one is a confidence statement, or reads an interval that these fits do not supply. |
|
|
fold and outcome-scale rules. No shipped result covers a partition read off the data the fit then conditions on |
a cross-fitted continuous outcome with |
|
fold and outcome-scale rules. The sample outcome range would take the scale from held-out rows |
|
|
fold and outcome-scale rules. No clustered result covers the outcome-adaptive mechanism. Selector-based fits also lack a result for grouped selection and nested folds or candidate-selection variance. A |
|
|
the algorithm as implemented. Each fold’s variance takes the variance of the cluster totals inside the fold, and 1 total has none. |
the |
|
reusable outer split plans. Each one refits on rows the fit never ran, and no rule here says which fold labels those rows inherit. |
replicate weights (BRR, jackknife) |
|
observation weights. These are a set of designs rather than one weight vector, so the shape they want is a refit per replicate outside the estimator |
omitted-variable sensitivity after |
|
validation and sensitivity methods. The median bound needs an influence function; a coordinatewise median of per-draw influence terms is not one. The five omitted-variable capability rows declare this refusal as |
omitted-variable sensitivity on a |
|
validation and sensitivity methods. Neither estimator assumes a consistent treatment mechanism, and no derivation gives \(\nu^2\) or the bound’s standard error without that assumption. A |
omitted-variable sensitivity on a fit with a response mechanism |
|
validation and sensitivity methods. The bound is well posed, and the implementation is missing. The implemented representer omits the response indicator, and \(\sigma^2\) averages the respondents only. Use the missingness tilt for response on the arm means. The tilt does not cover |
omitted-variable sensitivity on a fit with a declared missing treatment |
|
validation and sensitivity methods. The Riesz representer of the bound is derived for a recorded treatment. This fit divides by the composite mechanism, whose representer and confounding strengths are not derived |
omitted-variable sensitivity on a fit with an intermediate variable |
|
validation and sensitivity methods. The bound is well posed, and the implementation is missing. The representer carries the intermediate weight, so the treatment strength \(c_D\) would also measure the intermediate mechanism |
omitted-variable sensitivity on a |
|
validation and sensitivity methods. Each parameter has a Riesz representer, so the bound is well posed. Only the implementation is missing |
omitted-variable sensitivity on an |
|
validation and sensitivity methods. An incremental intervention tilts the mechanism, so the mechanism is part of the estimand rather than a nuisance |
omitted-variable sensitivity on an arm-indexed fit that reports no counterfactual mean and no linear contrast. A ratio-only fit, a PAR or PAF fit, and a complete-outcome |
|
validation and sensitivity methods. One bound is the second moment of one contrast’s Riesz representer, and the fit reports no such contrast. The five capability rows read |
|
|
validation and sensitivity methods. The short model would adjust for nothing, and no estimator here fits without a covariate. |
omitted-variable sensitivity when the doubly robust estimate of \(\nu^2\) is not positive |
|
validation and sensitivity methods. This row reads the data, not the fit class. The other omitted-variable rows refuse a whole class of fit before any computation, and their capability rows report |
the one-sided limits and the confidence-limit robustness value under |
|
validation and sensitivity methods. No derivation in a source this package cites gives the standard error of a bound built on the plug-in \(\nu^2\), which moves at first order with the fitted treatment mechanism. The bounds, |
omitted-variable limits when \(\sigma^2=0\) |
|
the bound’s standard error needs a positive maximal-bias scale. The point bounds remain, but the limit accessors refuse. An unreachable point robustness threshold is |
any E-value request on a fit with an intermediate variable and a discrete treatment |
|
the E-value paths. This package has no implemented bound for the fitted controlled direct effect and its confounding model. The ordinary conversion does not establish sensitivity for that target. The capability row reports |
the standardized Gaussian E-value on a fit with a response mechanism |
|
the E-value paths. The conversion needs the population outcome standard deviation, and the respondents’ one estimates a different quantity. A ratio E-value on the same fit stays available |
correction diagnostics after ordinary TMLE or collaborative TMLE |
|
those methods do not use the DR-TMLE correction system. A DR-TMLE fit with an empty guard also subtracts no correction term |
a binomial ATT or ATC E-value conversion |
|
the analysis needs a conditional baseline risk and a supported conditional risk-ratio target. The library implements neither |
a fixed-baseline E-value on a reference-arm mean that its own standard error does not separate from zero |
|
the E-value paths. The conversion divides by that mean, so the ratio has no stable denominator |
a fixed-baseline E-value whose risk difference is at or below the negative of the reference-arm mean |
|
the E-value paths. The difference implies a nonpositive risk in the contrast arm, and no risk ratio describes one |
a cached-nuisance risk ratio after CV evaluation |
|
the estimator refuses nonlinear ratio targets on this evaluation path. Post-fit retargeting keeps that boundary |
a controlled direct risk ratio from cached nuisances |
|
the fitted result can report a controlled direct risk ratio. The post-fit retarget path does not pass the fitted intermediate intervention level, so it cannot derive one from cached nuisances |
Which multi-arm surfaces are covered, and which are not, is tabulated in one place: where a multi-valued treatment is supported.
Several former gaps have landed. cleverly now supports multi-valued longitudinal treatment
nodes. It supports multi-valued selector-based C-TMLE, outcome-adaptive C-TMLE, DR-TMLE, ATT,
and ATC. LTMLE supports observation weights and a working model over regimens. IPSI fits
incremental interventions over time. Shift fits
support intermediate= and weights. They support delta= in sample. cleverly also supports multi-arm
omitted-variable and MNAR sensitivity analyses. TMLE supports the scalar missing-outcome
natural-course mean, and PAR and PAF, with ordinary fitting or one generated split of stacked
CV-TMLE. Missing-outcome natural-course contracts gives
the boundaries of each path.
TMLE also cross-fits arm-indexed means and contrasts with missing outcomes under one stacked CV-TMLE contract. Missing-outcome arm-indexed contract gives its boundaries.
The remaining shift gap is narrower than it was. The tilt itself is written. The missing derivation must establish whether the tilted parameter is still the shift parameter.
Targeting and fitting option refusals#
These options are Targeting(stopping=, step_bound=, step_search=, iterate=),
ModelSpec(outcome_by_arm=), CollaborativeTMLEMethod(patience=) and
PointTreatment(treatment_adjustment=). A refusal is raised before any learner fits, unless the
row says otherwise. A malformed value raises ValueError with no record. Every other row carries
the code shown.
tests/unit/test_stopping_rule.py, test_step_bound.py, test_best_iterate.py,
test_outcome_by_arm.py, test_patience.py, test_treatment_covariates.py,
test_treatment_covariates_collaborative.py and test_hazard_x22_refusal.py assert each text
with pytest.raises(match=re.escape(...)).
id |
refused |
class and code |
message |
|---|---|---|---|
R-a1 |
|
|
|
R-b1 |
|
|
|
R-c1 |
|
|
|
R-c1a |
|
|
|
R-c2 |
|
|
|
R-c3 |
|
|
|
R-abc1 |
|
|
|
R-abc2 |
a non-default |
|
|
R-abc3 |
the same settings on |
|
|
R-abc3a |
the same settings on |
|
|
R-abc4 |
|
|
|
R-f1 |
|
|
|
R-f2 |
an arm with no training row in a fold. It is raised at fit time |
|
|
R-f3 |
|
|
|
R-g1 |
|
|
|
R-g2 |
|
|
|
R-h1 |
|
|
|
R-h2 |
|
|
|
R-h3 |
|
|
|
R-h4 |
|
|
|
R-h5 |
a C-TMLE |
|
|
R-h6 |
a repeated name in |
|
|
TMLE.fit(treatment_covariates=) raises R-h1, R-h4 and R-h6 with the argument named
treatment_covariates in place of treatment_adjustment. The row R-abc4 names the
iterated-mean survival fit as its alternative, and the roadmap item X33 tracks the gap.
tests/unit/test_hazard_x22_refusal.py fits that alternative with each refused setting.
A declared stopping rule can leave a fit unconverged. That is a result and not a refusal. The
fit warns, reports converged=False, and score_check() fails it.
Stopping rules gives the
text.
Experimental compositions#
A composition that no study covers, and whose code computes the requested quantity, runs. The
fit returns its estimates under the experimental label. The old refusal text becomes the reason of
the label, and the roadmap id becomes its evidence. result.summary() prints the reason, and
estimate.support holds it as data.
The preflight collects each such condition and goes on. A later refusal therefore still fires, and no condition masks it. The fit reads no learner before the preflight ends.
tests/unit/test_group_a_experimental_fits.py checks each site below on four points.
The preflight collects the condition before any learner.
The fit carries the label, the reason, and the roadmap id.
The point lies within three standard errors of the truth, where the complete-case answer does not.
A deliberate mutation moves the point.
contract |
settings that run as experimental |
roadmap |
|---|---|---|
arm-indexed, stacked |
linear fluctuation, one-step update, weighted fluctuation |
X60 |
arm-indexed, stacked |
|
X60 |
arm-indexed, stacked |
fold evaluation, |
X53 |
arm-indexed, stacked |
|
F2 |
natural course, in sample |
a linear fluctuation, one-step targeting, weighted targeting, |
X60, X55 |
natural course, stacked |
the same settings, and |
X53, X60, F2 |
|
|
X62 |
The targeting and fitting options also run as experimental. The condition is collected in the
same preflight, and tests/unit/test_option_support.py checks each label against the fit
without the option.
option |
runs as experimental |
keeps the label of the fit |
roadmap |
|---|---|---|---|
|
|
|
X76 |
|
never on their own |
under |
none |
|
|
|
X76 |
|
any value |
|
X76 |
|
a declared subset |
the whole adjustment set |
X76 |
These requests stay refused, because the code returns a misleading number or has no path:
refused |
why |
|---|---|
a cross-fitted missing-outcome fit outside the two contracts ( |
the interval is misleading |
|
the interval is misleading |
a clustered |
the interval is misleading |
fold evaluation of the stacked natural-course mean |
the fold-evaluated report has no natural-course mean to read |
a continuous outcome without |
an invalid declaration, or an estimator with no written result |
Missing-outcome natural-course contracts#
NaturalCourseMean(), PopulationAttributableRisk() and PopulationAttributableFraction() with
missingness= and at least one missing outcome have two TMLE contracts. A request that reads the
natural course beside arm targets meets the same contracts.
CrossFitting(enabled=...) selects the contract. The table gives each requirement, the
compositions that each contract refuses, and the compositions that it runs with the
experimental label.
requirement |
ordinary TMLE, |
stacked CV-TMLE, |
|---|---|---|
target |
|
the same. ATT and ATC run as experimental under the arm-indexed contract |
estimator |
|
the same |
treatment |
two or more arms |
two or more arms. With three or more, the scalar |
outcome |
binary, or continuous with declared |
binary. A continuous outcome runs as experimental |
outer folds |
one fold |
package-generated folds with |
repeats |
|
|
targeting |
|
the same, with |
inference |
influence-curve Wald interval. |
influence-curve Wald interval. |
rows |
fixed |
unweighted iid rows. Weights and clusters run as experimental. |
response support |
no added check |
at least two respondents and two nonrespondents in the sample, and one of each in every training complement |
Three checks enforce the two contracts. Each check runs at a different time and raises a different error. A composition that runs as experimental raises nothing. The preflight collects its condition, and the fit carries the condition as the reason of the label.
check |
error |
code |
when it runs |
|---|---|---|---|
a cross-fitting declaration that is invalid by itself |
|
|
when |
a composition this target does not support |
|
|
when the fit starts, before any learner is fitted |
response support |
|
|
after fold generation, before either learner receives data |
The composition check refuses a CTMLE fit and a DRTMLE fit as an implementation_gap. The
message names X51 for CTMLE and
X50 for DRTMLE. On the stacked fit it also
refuses fold targeting and fold evaluation (X53),
and intermediate=
(X60). It refuses a
continuous outcome without q_bounds as an invalid_declaration. Fold targeting and the
cross-fitted off-contract regimes stay refused because the code returns a misleading interval
there. Fold evaluation stays refused because the fold-evaluated report has no natural-course mean
to read.
Every other row of the table above runs. The condition text of the old refusal becomes the reason of the label, and the roadmap id becomes its evidence. The check collects the condition and continues, so a condition never hides a later refusal.
The declaration check is not specific to this target. One function runs its steps in a fixed order,
and it returns the first reason it finds. The declaration and the engine therefore give one reason
for one input. A reason that names the cross-fitting switch names enabled from the declaration
and cross_fit from the engine.
tests/unit/test_split_plan.py::TestOneInputEarnsOneReasonAtBothLayers checks that both layers
give the same reason.
order |
the declaration check refuses |
code |
|---|---|---|
1 |
|
|
2 |
a |
|
3 |
a plan that records no generator. See reusable outer split plans for the two plans that carry the record |
|
4 |
a plan that the declared fold policy cannot use. A plan requires cross-fitting with at least two folds, and its fold and repeat counts must fit the declaration |
|
5 |
a plan together with |
|
6 |
|
|
7 |
cross-fitting declared with fewer than two folds |
|
8 |
a fold-stratification policy this package draws no split under. |
|
A split_plan can pass the declaration check and still fail the natural-course contract. Under
enabled=True, a valid plan constructs, and the stacked contract refuses it when the fit starts.
Under enabled=False, step 4 refuses any plan at construction.
response-support failure |
what the message tells you to do |
|---|---|
the sample holds fewer than two respondents or fewer than two nonrespondents |
use the in-sample estimator. No fold count or |
one training complement holds no respondent or no nonrespondent |
the message names the repeat and the fold. Use the in-sample estimator, or collect more observations at the rare level |
Both messages name the in-sample estimator as CrossFitting(enabled=False), or cross_fit=False
on the engine. One fold balances nothing, so no fold policy is part of the remedy. Neither message
names a redraw, because the split reads neither the response indicator nor the outcome.
stratify_by="none" is the default, and it is the only policy any fit that draws a split accepts.
It is not special to these two contracts. "treatment" and "treatment+outcome" are refused when
the fit draws a split, and at every setting for selector-based C-TMLE, whose search draws selection
folds without cross-fitting. An in-sample outcome-adaptive fit draws no split. It accepts the
declaration and applies none of it. The
fold and outcome-scale rules give the audit behind the
refusal and the message each layer raises.
Missing-outcome arm-indexed contract#
The contract covers the arm-indexed mean group ey, ey0, ey1, ate, rr, or, att, and
atc under cross-fitting with at least one missing outcome. It applies to an ordinary TMLE or
C-TMLE fit under three conditions. The parameters are indexed by the arms of a discrete treatment.
The design declares no intermediate. The request is not the scalar NaturalCourseMean.
A per-arm outcome-adaptive C-TMLE fit reports a point estimate only, so this contract does not govern it. The per-arm contract states its own preflight.
A request that reports ey_obs, PAR or PAF beside arm targets meets this contract and the
natural-course contract, and its admitted list includes ey_obs, par and paf.
Stacked CV-TMLE for arm-indexed targets
defines the estimator, its preflight, and its evidence.
requirement |
admitted |
refused |
|---|---|---|
estimator |
|
|
targets |
|
|
treatment |
two or more arms |
none |
outcome |
binary, or continuous with a fixed |
continuous with |
outer folds |
package-generated folds with |
|
repeats and targeting |
|
|
fluctuation |
|
a linear fluctuation, one-step targeting, weighted targeting, and fold evaluation run as experimental |
rows |
unweighted iid rows |
observation weights, clusters, and baseline strata run as experimental |
inference |
pointwise influence-curve Wald intervals and the simultaneous band |
|
content |
at least the minimum in the preflight table |
a sample or a training complement below that minimum. A sample below its minimum is refused with a remedy that does not repartition, because no partition can succeed |
Three checks enforce the contract. The table gives the error of each check and the time at which it runs.
check |
error |
code |
when it runs |
|---|---|---|---|
a composition outside the contract that stays refused |
|
|
when the fit starts, before fold generation and before any learner is fitted |
|
|
|
when the fit starts, before any learner is fitted |
minimum content |
|
|
after fold generation, before any learner is fitted |
A DRTMLE fit with missing outcomes and cross_fit=True raises the second CapabilityError in
the table at every guard, including guard=(), and under every fold policy.
tests/unit/test_drtmle_missing.py checks the guard=() case with zero learner calls.
The composition check runs its steps in a fixed order. A fit that breaks several rules meets the
first refusal. Each message names the missing result and the setting that the contract admits.
A setting message gives both the engine keyword and the public spelling. A step marked
experimental does not stop the fit. It adds a condition to the label and the check goes on, so a
later refusal still fires.
tests/unit/test_arm_indexed_stacked_mar.py checks each step through the engine and through
TMLEMethod, with zero learner calls.
order |
the composition check meets |
outcome |
code |
roadmap item |
|---|---|---|---|---|
1 |
C-TMLE |
refused |
|
|
2 |
|
experimental |
|
|
3 |
|
experimental |
|
X60 |
4 |
|
experimental |
|
X60 |
5 |
|
experimental |
|
|
6 |
|
experimental |
|
X60 |
7 |
a continuous outcome with |
refused |
|
none |
8 |
|
experimental |
|
X53 |
9 |
|
experimental |
|
X53 |
10 |
|
refused |
|
X53 |
11 |
observation weights |
experimental |
|
X60 |
12 |
clusters |
experimental |
|
X60 |
13 |
baseline strata |
experimental |
|
X53 |
14 |
|
experimental |
|
The list once held a stratified fold policy and a fold count below two. The declaration check
refuses both, so neither step can run. A copied or modified estimator can still carry the refused
policy. The fit then raises CapabilityError, a subclass of ValueError, with the declaration
check’s own reason. A result that holds such an estimator reads its refit_nuisances replay slot
false, with the code point_replay_refit_configuration.
A refusal that has an in-sample alternative names it as CrossFitting(enabled=False). The engine
form is cross_fit=False. An in-sample fit that draws no split keeps whatever fold policy the
declaration carries and applies none of it. Selector-based C-TMLE draws selection folds at every
setting, so it refuses a stratified policy in sample as well.
Hazard and isotonic survival refusals#
Hazard-based survival targeting and
the isotonic projection refuse each composition
below by name. Each refusal is raised before any learner. tests/unit/test_hazard_refusals.py pins
the type and the message of each one, with learners that raise if they are fitted. tests/unit/test_hazard_x22_refusal.py pins H13.
id |
trigger |
when |
type and code |
message |
|---|---|---|---|---|
H1 |
|
|
|
“survival_model=’hazard’ is built for one baseline treatment held over every node, and this design declares a treatment at {k} nodes. Hazard targeting for a treatment at several nodes is not built. X70 in docs/roadmap.md tracks this. LTMLE with survival_model=’iterated_mean’ estimates this curve, and isotonic=True makes it monotone. If one baseline treatment is held over every node, declare treatment=’A’.” |
H2 |
|
|
|
“survival_model=’hazard’ models the event and censoring hazards given the baseline treatment and the baseline covariates. This fit declares covariates measured after baseline ({names}), and hazard targeting with such covariates is not built: it needs a model and a fluctuation of each later covariate. X70 in docs/roadmap.md tracks this. LTMLE with survival_model=’iterated_mean’ adjusts for them, and isotonic=True makes its curve monotone.” |
H3 |
|
|
|
“survival_model=’hazard’ needs an event indicator at every node, and this fit declares one end-of-study outcome. A hazard is the chance of the event at a node among the units still at risk, so an outcome measured once has none.” |
H3 |
|
|
|
“isotonic=True makes a curve over the horizons monotone, and an end-of-study fit reports one horizon.” |
H4 |
|
construction |
|
“targeting=’one_step’ requires survival_model=’hazard’. The one-step walk moves one set of hazards toward the score equations of every horizon together (Cai and van der Laan 2020). A one-step update of the iterated-mean recursion is not built. Set survival_model=’hazard’ on a survival design with one baseline treatment, or keep targeting=’iterative’ and set isotonic=True for a monotone curve” |
H5 |
|
construction |
|
“msm= with {option} is not built. A working model reports coefficients over regimen and horizon cells. X70 in docs/roadmap.md tracks this. Fit msm= with survival_model=’iterated_mean’ and isotonic=False, or report the curve and take contrasts.” |
H6 |
|
construction |
|
“pseudo_learner= is not used with survival_model=’hazard’. A hazard fit regresses the event indicator at each node and fits no pseudo-outcome. Pass the hazard learner as outcome_learner=” |
H7 |
a modified treatment policy regimen with |
|
|
“regimen {label!r} is a modified treatment policy, and survival_model=’hazard’ does not support one. Its plug-in evaluates the event and censoring hazards at the shifted dose, and that construction is not built. X70 in docs/roadmap.md tracks this. Use survival_model=’iterated_mean’, which supports a modified treatment policy at baseline.” |
H8 |
an unknown |
construction |
|
“targeting must be ‘iterative’ or ‘one_step’; got {v!r}”, “survival_model must be ‘iterated_mean’ or ‘hazard’; got {v!r}”, “isotonic must be True or False; got {v!r}”, and “step_size must be a positive finite number; got {v!r}”. A |
H9 |
|
|
|
the binary-outcome refusal, “q_bounds does not apply to a binary outcome” |
H10 |
|
method configuration |
|
“survival_model= and isotonic= apply to a survival curve over time nodes, and a point-treatment functional has no curve. Remove them, or declare a longitudinal or TimeToEvent design” |
H11 |
|
|
|
“survival_model=’hazard’ needs a categorical baseline treatment, and treatment {name!r} is continuous. The hazard plug-in would need the treatment density at every dose, and that construction is not built. X70 in docs/roadmap.md tracks this. Use survival_model=’iterated_mean’.” A vector with a dose reads “is a vector with a dose” in place of “is continuous” |
H12 |
a non-default |
method configuration |
|
“survival_model=, a hazard targeting= algorithm, step_size= and isotonic= are settings of the targeted fit, method=’tmle’. The {method} estimator fits no hazard, and no registered study measures its projected curve. Fit method=’tmle’, or remove them”. The |
H13 |
|
construction |
|
“survival_model=’hazard’ walks its own path in steps of step_size, so iterate=, stopping= and step_bound= do not apply to it. X33 in docs/roadmap.md tracks this. survival_model=’iterated_mean’ with targeting=’iterative’ applies iterate=, stopping= and step_bound= at each node.” The list names the settings declared. |
The fit checks H3 first, then H1, H2, H11 and H7. So a continuous treatment meets H11 before a modified treatment policy meets H7.
H1 refuses an extension that qualifies and is not built. A static arm or a known rule at several nodes, without a covariate measured after baseline, reduces to an indicator. That indicator is the composite “uncensored and on the regimen through node \(k\)”. The hazard construction then applies to the composite. This package does not build that reduction.
H2 needs a model and a fluctuation of each later covariate. Stitelman, De Gruttola and van der Laan (2012) and Rytgaard, Gerds and van der Laan (2022) treat such covariates. Neither source was read for this contract.
X70 tracks H1, H2, H5, H7 and H11. Each message names the supported route. The roadmap carries no row for H4.
A different question#
These are well-posed parameters, but the selected estimator does not target them. No setting turns
one into the other. The code column holds different_estimand for most rows. Another code marks a
row where the raise site names a different remedy.
refused |
code |
the question it would answer instead |
|---|---|---|
|
|
the incidence of a cause if the competing events were removed. That intervenes on them rather than conditioning on the history. It needs a further factor per node in the denominator, and its own no-unmeasured-confounding and positivity assumptions for the competing event. What is reported instead is the incidence with the competing causes left alone |
|
|
C-TMLE selects or fits a treatment mechanism, and the declaration leaves nothing to select or fit. The question does not change. Fit |
a declared known treatment mechanism beside |
|
a missing treatment needs \(P(A=a\mid\Delta_A=1,W)\), the mechanism among the recorded rows. It equals the design mechanism only when recording is independent of treatment given \(W\), which the fit cannot check |
|
|
a controlled direct effect fixes a mediator at one time point. Over a sequence, with mediators that are themselves time-varying, that is a different identification rather than a further column |
|
|
each keyword declares what “counterfactual” means for the fit, or how the counterfactuals are summarised. One fluctuation solves one set of score equations, so a fit reporting parameters from two axes would put two of them under one heading. |
the per-arm propensity table on a continuous fit |
|
a per-arm table has no rows when there are no arms. |
|
|
each is part of the shared result contract. Stagewise support, scores, nuisance loss, and the descriptive full-recursion truncation grid are supported. Sensitivity operations without a longitudinal derivation report |
|
|
an end-of-study fit reports means and no survival curve. A repeated outcome can return to zero, so its mean at a horizon is not a cumulative risk. One minus one cause’s incidence is not all-cause survival. A coefficient is not a regimen mean. Functionals of a fitted result gives each message |
a plan that changes the treatment after baseline on a held design ( |
|
a time-varying treatment plan. Declare one treatment column per node on the wide layout. The held design raises |
|
|
R |
|
|
the term is defined for one logistic model on a covariate set. A |
|
|
the fit adds derivative columns to the fluctuation. R |
|
|
C-TMLE0 is R’s single logistic fit of the \(2K\) clever covariates. The one-step path targets one parameter along another path. |
|
|
No source defines a stratified derivative update. Fit the ordinary TMLE for stratum estimates. X62 part (e) tracks the gap |
|
none |
the partner equals the mechanism after |
|
|
the lasso pair, the declared learners and the arrays each fit the mechanism and its partner on every covariate. A screen would apply to neither. Fit |
a |
|
the pair is |
a single propensity array in |
|
an array fitted on every row would score a mechanism that saw the validation rows. Pass |
|
|
|
a |
|
the row is not a distribution over the arms, so the clever covariate weights the arms wrongly. Pass |
|
|
one penalty gives one candidate, and the fit selects nothing. Pass two or more penalties, or |
|
|
a declared split is fixed on the analysis rows. A repeat redraws it, and a resample changes the rows. |
a non-empty |
|
the validation battery owns that name and runs it argument-free. The battery presents one row per name, and it presents the validation row. A caller’s tolerance would be computed and then hidden, so a check that failed at that tolerance would never reach |
Wrong by construction#
These are worth reading even if you never reach for the keyword. Most are mistakes that are easy to
make by hand, in any framework, and none announces itself. They can produce a number for the wrong
target or an interval whose stated conditions are not met. The code column names the code of the
raise site. A row marked none has no raise site. The package cannot detect the mistake, or its
API does not offer the construction.
refused |
code |
what goes wrong if it is done anyway |
|---|---|---|
|
|
the odds of a mean outside \((0, 1)\) is not defined. The refusal applies the rule of the registered |
a population-mechanism odds tilt supplied as a |
|
the population-law target depends on |
a population-law incremental intervention built by hand as a |
|
the missing term has variance |
a user-written |
|
it can omit the same derivative while reporting an interval for the population-law target. On the RM25 witness law, a class that computes the sample-mechanism odds tilt reports 0.62 of that target’s exact standard error ( |
a |
|
the fit treats its rule as fixed. A learned policy can instead target a realized policy value or a population-indexed policy. Those targets need distinct inference conditions; an optimal rule can also be nonregular at ties. For a threshold at the sample mean of a continuous covariate, the curve omits a derivative through that mean. On one exact law, the reported standard error is 0.73 of the exact one at one node ( |
a shift’s inference taken from the regime inducing the same density |
none |
the means and the clever covariates agree entry for entry. The curves do not. The gap is `Var(Qbar(d(A,W),W) - E[Qbar(d(A,W),W) |
a shift fit run on the complete cases when outcomes are missing |
none |
it is an ordinary shift fit on a different joint law of |
a missingness or intermediate mechanism read at the observed dose rather than the assigned one |
none |
the fluctuation updates |
a “stabilised” MSM weighting |
|
the same argument once more. |
an MSM |
|
for the population-law target centred at \(E_P[W]\), the projection depends on \(P\) through the design. Its reported curve omits that derivative. On one exact law, the reported intercept standard error is 0.893 of the exact one ( |
|
|
|
a |
none |
the estimand becomes data-dependent. The interval conditions on an estimated boundary, and every bootstrap replicate targets a slightly different policy |
|
|
each candidate |
splitting a cluster across folds to buy more of them |
|
the out-of-fold predictions stop being independent of the rows they are used on, and the standard error shrinks in exactly the direction |
a supplied |
|
the labels could have been chosen by reading the outcome, the treatment, or a covariate, which is the leak cross-fitting exists to prevent. The fit draws each repeat again from the plan’s record and compares it label for label. Reusable outer split plans states what a plan records |
reusing a supplied |
|
a fold label is a row position. A reordering, a replaced column, or an added covariate leaves every label pointing at a different unit, while the row count still agrees. The out-of-fold guarantee is gone, and nothing in the output says so. Reusable outer split plans states how a plan binds to its rows |
joint covariance, post-fit contrasts, or simultaneous bands after |
|
coordinatewise medians do not preserve linear identities among estimates; a central-draw curve also does not represent the split-adjusted median estimator needed for a multiplier band |
a cross-validated variance for the curve a repeated fit retains |
none |
at equal fold sizes it collapses to the pooled uncentred second moment for every partition, so the partition carries no information. That rule was rejected, and the split-adjusted median variance replaced it |
a one-shot non-identity-link MSM |
none |
the derivative of the inverse link depends on the coefficient, so a single pass reports a standard error for an equation it did not solve. The link is supported. What is refused is skipping the alternation it needs |
a censoring time between two grid times, read as censored at a grid time |
|
the censoring cannot be ordered against the events of its interval, so either reading biases the risk. On one interval with exponential times the two readings give 0.280 and 0.236 against a truth of 0.259 ( |
an event value on a row whose censoring is 0, on a long panel or on the wide layout |
|
the order at a node is \(A_t, C_t, Y_t\), so the event is unobserved once the unit is censored. Reading past it drops an event that was meant as “had it and left”. |
a long-panel |
|
a unit-level column is measured before the first decision. A later value read as |
a long-panel covariate that equals the outcome, or the row’s event, on every row where it is recorded; one column in two roles |
|
a rule or a history frame could read the outcome under another name. |
a long panel whose time labels are not numbers, dates or durations, with no |
|
the sort order of such labels need not be their time order, and the node order is the time order. |
a long panel with a duplicated |
|
intermittent observation is not censoring, and a unit that left does not return. |
a long-panel time-varying column with no value at a node after its first recorded node |
|
leaving the column out there removes a confounder from that node’s adjustment set. A leading run of missing values is not refused. The column starts at its first recorded node, and |
a long-panel treatment that never changes within a unit |
|
one baseline decision held over every node, not a decision at each time. Build the wide frame and declare |
frequency (count) weights |
|
they assert a sample size the variance does not use. Expand the rows instead, which says the same thing where every part of the fit can see it |
an |
|
its probability of being observed is silently taken to be one. Encode it as a final censoring column so that it is estimated and enters the cumulative product |
a binary-only target on a multi-arm fit |
|
it would report a contrast of arms |
|
|
a model linear in the arm reads it as a dose to interpolate between, and the fallback coding is the sort order. That is a dose scale nobody chose |
The first three rows share one mechanism, and it generalises past this package. If an
intervention’s density is a functional of P, the influence function carries its pathwise
derivative. Omitting that term changes the variance. The direction depends on its covariance
with the retained curve. The MSM rows on a “stabilised” weighting and on a sample-statistic
design apply the same argument to a working-model function. The incremental and shift rows above give their own variance
gaps. The rule row shares the
mechanism when a population quantity indexes the rule, as in the threshold witness.
Two more share another. A nuisance that sits inside the estimand is not a knob. Truncating g
on an incremental fit, or fitting a shift’s cap from the data, moves the target rather than
regularising the estimator.
Three method entries carry their own detailed refusal tables, each with a code column:
marginal structural models,
longitudinal TMLE, and
the simulated common-cause surface.
The column holds the reason code of the raise site, as the code column of the tables above does.
The simulated common-cause surface refuses seven compositions for the whole fit. They are a
longitudinal fit, multi-arm treatment, a missing outcome, a missing treatment, an intermediate
variable, estimated observation weights, and a clustered fit. Each one is an implementation_gap
(_FIT_WIDE_RULES in src/cleverly/sensitivity/_simulated_confounding_request.py).
X47 and
X48 track the
laws a caller can declare. The
future investigations grid tracks each part that waits on
published theory, against the result a paper must supply.
The surface accepts fixed probability weights under ordinary TMLE, under binary complete-outcome collaborative TMLE, and under binary complete-outcome DR-TMLE. Read the surface’s own refusal table for every composition it refuses and the code of each one.
The simulated_confounding capability row resolves each request by the predicate that the call
raises from. A request for a refused parameter reads unavailable with the call’s sentence. The
natural-course mean and a zero-delta policy mean are such parameters. A categorical or a constant
benchmark covariate reads unavailable in the same way. The bare row of a natural-course fit reads
unavailable, because no reported parameter runs.
Where a multi-valued treatment is supported#
A multinomial treatment mechanism is the default construction here rather than a variant of a
binary one. CausalData carries treatment_levels, Propensity holds an (n, K) simplex, and
learners._fitting.predict_probabilities produces it. Two arms are the K = 2 case of that
construction and not a separate path, which is what the
bit-for-bit invariant is about.
So “does this estimator take more than two arms” usually has the answer “yes, through the same code
as two”. The informative entries are those that do not. The status column holds supported,
a reason code from How to read a refusal above, or waiting on published theory from the roadmap’s eligibility rules.
surface |
status |
why |
|---|---|---|
|
supported |
one counterfactual mean per arm and one contrast per non-reference arm. The oracle-law gate states that a target meant for more than two arms needs a branch on the three-armed law |
|
supported |
armwise one-vs-rest, with each reduction and correction indexed by a free level. The cited theorem is binary, so this is an implementation-backed armwise extension rather than a claim about that theorem’s literal scope |
|
supported |
Díaz and van der Laan’s missing-outcome construction, applied to each arm indicator and stacked. Each arm’s treatment, observation and outcome tilts are its own, so no fluctuation parameter is shared across arms. See the multi-arm contract |
|
supported |
the composite indicator, applied to each arm and stacked. Each arm’s column of the composite mechanism is tilted alone, two arms included. See the composite contract |
|
supported |
the selectors fit one shared |
|
supported |
treatment over time. Each node owns its level set, and the clever covariate selects the assigned label’s probability |
positivity, omitted-variable, E-value and MNAR sensitivity |
supported |
each is one parameter per contrast, and each reads its arms from the parameter’s structured index rather than assuming two |
simulated common-cause sensitivity |
supported for binary and continuous treatment; named theory stops cover the remaining fit-wide gaps |
The binary surface accepts marginal and baseline-stratum arm means, ATE, and ratios under ordinary TMLE and C-TMLE. Ordinary TMLE also accepts ATT and ATC with perturbed group membership. Ordinary TMLE alone accepts marginal and baseline-stratum PAR, PAF, fixed-regime parameters, and identity-link MSM coefficients. It also accepts marginal incremental targets and nonlinear MSM coefficients. Complete-outcome DR-TMLE supports marginal arm means, ATE, and ratios only. The continuous surface accepts marginal and baseline-stratum modified-policy means and contrasts under ordinary TMLE. A risk-ratio tilt fit is refused, because the replay declares no reading of its rule on a perturbed draw (X66 part (d)). Continuous MSMs require marginal fits. Every listed row accepts fixed probability weights. Longitudinal, multi-arm, missing-outcome, intermediate, estimated-weight, and clustered fits report unavailable before work begins. Each of those six waits on published theory, and so does logical categorical calibration. |
|
|
they name one of exactly two arms, so on five arms they would report a contrast of arms |
a scalar |
|
a scalar odds multiplier names two arms. The refusal runs before any learner, names the treatment or node and its levels, and shows the mapping form: “a scalar odds multiplier names two arms; node ‘A1’ has 3 levels ([‘high’, ‘low’, ‘standard’]). Declare one factor per level, for example Incremental({‘high’: 2.0}), which multiplies the odds of ‘high’ against every unnamed level.” Declare |
an |
|
the refusal runs before any learner and names the key, the treatment or node, and its levels. A key is a label as the data reports it, and a position is not a label. One |
|
supported |
known stochastic policies. A node holds a policy density fixed before the fit. The recursion carries the policy mean of the per-level predictions, and the clever covariate is the ratio of the policy to the bounded mechanism. Exact-law tests and the registered study cover it |
|
supported |
modified treatment policies at a node. A plan node holds a |
|
supported |
vector nodes with a continuous component. A mixed node holds a |
|
supported |
vector treatments. A vector of categorical columns is one categorical treatment over its tuples. A vector with a dose reports |
A source could close one entry as it stands. It is the multi-arm part of the simulated common-cause row.
The DRTMLE row needs no new source either. It needs the implementation that its roadmap row
holds.
Vector treatments#
A vector treatment holds several treatment columns. A vector of categorical columns is one
categorical treatment over the observed tuples. A vector with a continuous component has no
arms. TMLE(policies=[JointPolicy(...)]) fits it at a point treatment, and LTMLE and SDR fit
it at a node. The first table says what ships. The second table lists what a vector refuses. Each
refusal arrives before the nuisance fit.
surface |
status |
where |
|---|---|---|
point treatment, all columns categorical |
supported. Every estimand of a categorical treatment runs on the tuple levels |
|
point treatment, with a dose |
supported for |
vector treatments, |
longitudinal node, all columns categorical |
supported. The node is one categorical node over the tuples |
|
longitudinal node, with a dose |
supported on both ratio routes, for |
vector nodes with a continuous component, |
Each row of the next table gives the theoretical reason for its refusal.
refused |
raised |
reason |
test |
|---|---|---|---|
a |
|
the tilt is the |
|
an odds tilt, |
|
an odds tilt multiplies the odds of one categorical node (Kennedy 2019, Equation (1)). The joint density of a node with a dose has no odds to multiply. |
|
a dose map that reads the natural value of a later dose |
|
the Jacobian determinant of the joint map is the product of the declared derivatives only when each map reads earlier doses. Any acyclic read has an order that makes the map triangular. The message names it, and |
|
dose maps that read each other in a cycle, and any other joint map that is not triangular |
|
Equation (3) of Díaz et al. (2023) needs a map that is one-to-one on each piece. A determinant checked on a grid does not establish that a multivariate map is one-to-one, so no declaration of such a map is checkable |
|
a policy of one column on a point vector with a dose, and a static label, a rule, a known policy density or a policy of one column at a mixed node |
|
a static or dynamic value on a dose is not pathwise differentiable (Díaz et al. 2023, Section 4). A policy of one column does not say which component it moves. Declare a |
|
a known treatment mechanism, |
|
a known mechanism is a probability for each level. A vector with a dose has a conditional density and no levels, and a known density is outside this package |
|
a piece that sends an interval to an atom, |
|
the mass that such a piece sends to the atom is an integral of the factors of the later components over the interval. Each later factor conditions on the collapsed dose, so the integral is not a finite sum over preimages. At the last dose, in the column order and in the policy’s |
|
a declared missing treatment, |
|
the composite indicator divides by the probability that the treatment is recorded and that the arm is \(a\). A dose has no arm indicator |
|
|
|
the |
|
|
|
an |
|
an arm-indexed estimand, |
|
such a treatment has no arms, and each of these parameters is indexed by an arm. Say which values to compare with |
|
|
|
the setting would have no effect, and a silent no-op reads as a validated choice |
|
The refusal of a later-dose read is a scope limit and not a statement about the theory. A map that reads a later dose is valid after a reorder, and the message names the order that resolves the read. A cycle has no such order.