What each estimand’s correctness rests on#
tests/unit/test_registry.py’s test_every_target_has_an_oracle answers one question:
does this target have a branch in an oracle law? The answer is yes or no. That gate is
what stops an estimand shipping “on the strength of its author’s arithmetic alone”, and it
is not what this page is for.
This page is the other half: which instruments a target actually has, and which it does not. The distinction matters because the instruments go blind in different places, and How to read a refusal is not the only information to record. How to read a pass is also important. The trap that survives a green suite is:
an exact-law instrument goes blind wherever a quantity vanishes at the truth, which is where a parity check is blind too. At correct nuisances
Q_randg_{r,2}are zero row by row, so everytest_influence_gateaux*module passes against a flipped sign.
A row whose only evidence is a Gateaux comparison is therefore not the same claim as a row that also carries a remainder rate and an exact identity. A reader deciding how far to trust a number should be able to see which row they are looking at, without opening five test modules. That is the whole of what this table is.
It is a gate, not a note. tests/unit/test_registry.py::TestEvidenceManifest checks it
in both directions against TARGETS, and checks that every module named here exists. The part
that makes it more than a document is the third check: the oracle law column must agree with
the law whose functional really has the branch, read through the same oracle_for the coverage
gate uses. A row cannot claim an oracle the laws do not provide.
The instruments#
kind |
what it is |
what it cannot see |
|---|---|---|
oracle law |
the parameter written down longhand on an exactly representable discrete law, sharing no code with |
nothing about a term that is zero at the truth |
Gateaux |
the reported influence curve against a complex-step derivative of that functional, to ~1e-14 absolute with |
a sign on any block that vanishes at correct nuisances; a counterfactual block, at |
remainder |
the von Mises expansion’s second-order term, measured as a rate under one wrong nuisance |
a first-order error that cancels inside the remainder |
exact identity |
an algebraic relation that holds by definition and so must hold bit-for-bit |
anything symmetric in whatever the identity is symmetric in |
theorem |
a check against the source’s own theorem, at values where the quantity does not vanish |
this is the anchor the other checks need |
bounded implementation witness |
a frozen comparison with an independently maintained implementation, scoped to a named finite-sample choice that the scientific oracles cannot exercise |
the estimand’s derivation; any behavior outside the deliberately matched nuisance, bound, and targeting settings |
The table#
One row per registered target. none in a cell means there is no such instrument for this
target, which is a statement rather than an omission; the not covered column says what
follows from it.
target |
oracle law |
Gateaux |
remainder |
exact identity |
not covered |
|---|---|---|---|---|---|
|
|
|
|
|
no theorem anchor; the derivation is checked only where the oracle law can represent it |
|
|
|
|
relabelling the arms gives |
swapping the two conditioning populations outright is symmetric in the arms, so the symmetry test cannot see it; the closed-form comparison catches it |
|
|
|
|
relabelling the arms gives |
as |
|
|
|
|
none |
no identity of its own; it is the per-arm level the contrasts are built from, so its errors surface in them |
|
|
|
|
|
binary-only by declaration; the multi-arm path reports |
|
|
|
|
|
binary-only by declaration, as |
|
|
|
|
the complete-data curve is |
the MAR intervals cover the ordinary-TMLE fit, with fixed weights and clusters, and one binary, one-repeat stacked CV-TMLE configuration on unweighted iid rows. |
|
|
|
|
|
controlled intermediates and stacked missing-outcome strata are refused. The pointwise curve check sees each residual term at the truth, but the point and remainder checks at the truth do not, so the mutations run away from the truth |
|
|
|
the delta-method identity in |
its curve is the delta-method transform of |
defined only for a binary outcome with positive observed risk; small-sample coverage near zero risk is not established |
|
|
|
|
the ratio’s curve is the delta-method transform of the levels’ ( |
the log-scale interval’s small-sample coverage is a simulation claim, not an identity |
|
|
|
|
the odds ratio’s curve is the delta-method transform of the levels’ ( |
as with |
|
|
|
|
a degenerate regime equals the static arm it puts all its mass on ( |
a rule that is not deterministic; the density is evaluated once at fit time and the fit answers for that evaluation. No check detects a density, a rule, or a user-written class that reads the sample and is declared |
|
|
|
|
the contrast is the difference of the means ( |
a rule that is not deterministic, as for |
|
|
|
none |
the rule of each row comes from a fit that never saw the row ( |
the learning step: the oracle and the Gateaux check hold each fold’s rule fixed, so they cannot see how the rule was learned. The limiting-rule condition C3, the exceptional laws where it fails, the remainder along the fold rules and the quality of the rule are not covered here. The gated study measures coverage at two laws that meet C3. The interval under-covers at the exceptional and weak-blip laws that the boundary study reads |
|
|
|
|
|
this is the one estimand that is not doubly robust: every remainder term carries |
|
|
|
|
|
not doubly robust, as for |
|
|
|
|
the negative control: a shift’s mean equals the induced stochastic regime’s, and its curve does not ( |
there is no plain-shift remainder module. The rate is measured only with a third nuisance ( |
|
|
|
|
the same negative control as |
no plain-shift remainder module, as for |
|
|
|
none |
the curve carries a mechanism term: freezing |
no remainder module and no registered study. The tilt is refused by |
|
|
|
none |
the contrasts are checked on each side of one, where the tilt takes different branches ( |
no remainder module and no registered study, as for |
|
|
|
|
a saturated working model reproduces the per-arm report exactly, at the covariate and at the estimate ( |
the registered row is identity-link, ordinary, fixed-weight, two-arm evidence. A weight or a design computed from the sample and declared |
Where the repeated-sampling evidence lives#
The table above asks whether each parameter is implemented correctly. The registered studies ask the complementary question. Apply a complete estimator to samples from a known law, and does its bias and uncertainty behave as its source theory predicts?
Those studies are summarised in the implementation validation grid. Their test-by-test results are the implementation validation studies. The two halves are different instruments and neither one substitutes for the other.
Fixed observation weights are an estimator variant over the same registered point-treatment targets. Their registered weighted point-treatment study tests the tilted population target, exact weighted influence-curve efficiency, and the consequence of omitting the weights on identical selected samples. The separate learned-nuisance weighted study sends the weights through both regressions. Its learner-only control moves the untargeted plug-in to the selected target, while a correct treatment mechanism repairs the targeted estimate.
The registered ordinary weighted longitudinal study and cross-fitted weighted longitudinal study extend that instrument to two treatment nodes, censoring, static and dynamic plans, and correlated contrasts. Both use exact-size selected samples and fixed inverse-selection weights. Their independent property laws include separate controls for omitting weights from the full estimator and from the nuisance learners alone. These are reporting rows: their committed red cells remain limitations, not hidden successes.
Estimator variants over registered targets#
CTMLE and DRTMLE estimate the same registered ey and ate targets as TMLE, so they
do not add target-registry rows. Their multi-arm estimator constructions now have separate
registered repeated-sampling records for ordinary TMLE,
DR-TMLE,
outcome-adaptive C-TMLE, and
selector C-TMLE. The per-arm
outcome-adaptive design has its own record,
outcome-adaptive per-arm C-TMLE, for the
point estimate only. Its fits with strata and missing data have a second record,
per-arm C-TMLE with strata and missing data.
The per-arm design with ParametricOutcome and ParametricMechanism has a third record,
per-arm C-TMLE with a parametric outcome,
which measures its interval in sample. The cross-fitted and repeated forms of that interval have a
fourth record,
per-arm C-TMLE with a parametric outcome, cross-fitted.
Three arms, known observation weights, the targets ey_obs, par and paf, and the simultaneous
band of that fit have a fifth record,
per-arm C-TMLE with a parametric outcome, extensions.
The C-TMLE0 strategy has its own record,
C-TMLE0 derivative covariate, with R ctmle type 3 as its
declared comparator, whose runs are pending. Binary compatibility
remains covered by the existing C-TMLE and DR-TMLE rows, which continue down their original
branches.
A complete-data or randomized missing-outcome DRTMLE fit with one inconsistent nuisance
estimates the same targets. The case table
states when its interval holds. The table gives its instruments and what each one cannot see.
instrument |
what it checks |
what it cannot see |
|---|---|---|
|
on an exact law, a tie-splitting mechanism estimate gives a nonzero conditioning term that is linear in the split. Controls make it vanish |
the size of the term at a fitted nuisance |
the registered one-inconsistent-nuisance study |
valid intervals on a declared mechanism, conservative intervals with a correctly specified parametric mechanism, and a bias of the predicted sign that grows with n for a slow nuisance error, on law L, and a bias of the predicted sign on law M. Neither term vanishes on either law |
a flexible learner, the bivariate reduction, strata, clusters, weights and more arms |
A discrete C-TMLE fit with one declared candidate, equal to the full adjustment set, is the
ordinary TMLE. The TMLE studies are its evidence, and
tests/unit/test_ctmle.py::TestEquivalenceWithPlainTmle pins the identity to 1e-12.
Complete-outcome cross-validated DR-TMLE is a construction over the same targets, not a registry
addition. Its source audit maps the pinned R cvFolds path to
cross_fit=True, reduced_crossfit="pooled", targeting_scheme="pooled", cv_evaluation=False.
Primary and reduced predictions are out of fold. One global alternation follows, then a
whole-sample plug-in mean, then cov(IC) / n from the rowwise corrected curve.
A stratum-conditional parameter of strata= is the registered marginal parameter of the law
inside one baseline stratum. stratum_alias names it, so it adds no registry stem. The table gives
its instruments and what each one cannot see.
instrument |
what it checks |
what it cannot see |
|---|---|---|
|
each stratum curve equals \(I(V=s) D_s / P_n(V=s)\) at targeted predictions that moved, with within-stratum arm shares for the ATT and the ATC. Two mutations fail it: the marginal arm share, and a stratum mass of 1 |
the truth, because it checks an identity at fitted values |
|
the joint score blocks, the weighted stratum masses, and the refusals of a stratum parameter in a sensitivity analysis |
coverage |
bias, coverage, SE calibration and efficiency of each stratum parameter against an exact law, the band across strata, double robustness, and a stratum-targeting control. It pairs the arm means and the ATE with R |
ATT, ATC and PAR against a comparator, weights, clusters, missing outcomes, multi-arm treatment, and flexible learners |
|
bias, coverage and SE calibration of the stratified incremental, logit-MSM and stacked natural-course targets against exact laws, the incremental band, both targeting-necessity pairs, the robustness of the stratum MSM, and agreement with R |
cross-fitted incremental and MSM fits, and a calibrated dose MSM |
|
bias, coverage, SE calibration and efficiency of each stratum coefficient against an exact bounded-outcome law, and agreement with R |
cross-fitted fits and linked MSMs |
|
bias, coverage, SE calibration and efficiency of the marginal and stratum ATEs against exact quadrature truths, double robustness under four nuisance configurations, and agreement with R |
other guards, reductions and missing-data routes, which the exact-law tests cover |
|
|
each stratum’s outcome and mechanism blocks are zero, and each stratum curve is Kennedy’s Theorem 2 curve at the targeted pair; the pooled fit equals the subset fits. Four replacement mutations fail it: a marginal mechanism column, a marginal outcome column, no mass in the un-scaling, and no mass in the embedding |
the truth; a stratum curve away from the targeted pair |
|
linked stratum coefficients and curves equal the expanded-design MSM, the curve equals a complex-step Gateaux derivative of \(\beta_s\), the marginal record equals the unstratified fit bit for bit, and the stratum rank rule refuses before any learner. Two mutations fail it: blocks at the marginal coefficients, and the marginal fluctuation’s regression in the stratum estimates |
coverage; a nonuniform projection weight with strata |
|
identity and logit dose coefficients equal the expanded-design dose MSM; a marginal column fails it |
the dose truth, because the oracle is the shipped path |
|
the in-sample and stacked natural-course blocks are zero, each curve is the analytic one, each stratum equals its subset fit, each stacked standard error is the raw second moment of its curve, and the joint route’s PAR is the difference of its stratum curves |
the stacked arm-indexed targets, which refuse strata |
|
on every route and guard, each stratum estimate and curve equal a subset fit, and the marginal is the mixture. Reductions pooled across strata fail it |
the truth; more than one alternation round, whose global stop rule ends the strata together |
|
the replay of a stratum alias for each new composition |
interval validity of the replay |
A fit on data with a declared treatment mechanism estimates the registered targets. The declaration replaces the fitted mechanism, so the fit adds no registry stem. The table gives its instruments and what each one cannot see.
instrument |
what it checks |
what it cannot see |
|---|---|---|
|
on a finite law whose empirical law is the law, a wrong outcome regression with the declared mechanism returns the truth for the arm means, ATE, RR, OR, ATT and ATC at two and three arms, in every declaration form, under each CV-TMLE shape on replica folds, with population weights, and for complete-data |
the |
|
the declaration and a learner that returns it give the same fit, also nested, bootstrapped, at three arms and for a discrete modified treatment policy; the curve is \(D^*(\bar Q^*, g_0)\) written out; the incremental fit is the known stochastic regime; the declared values move the curve; the |
coverage |
|
a two-node SMART exact law returns the truth with every factor declared, with and without declared censoring, in both forms; a perturbed node moves the clever covariate from that node on; a dropped censoring declaration moves the estimate; a declared factor and a learner that returns it give one fit for a modified treatment policy, a held survival design, competing risks and a time-to-event container; a regimen that a declared zero leaves unidentified and the other refusals fire before any learner |
coverage |
the registered TMLE, DR-TMLE and LTMLE studies on declared mechanisms. Every gate passes |
accuracy, coverage and calibration at a wrong outcome regression, an estimated-mechanism control, the bootstrap, the variance direction and the rate, paired with R |
flexible learners; the ATT and ATC against R |
A cross-fitted clustered LTMLE fit estimates the registered longitudinal targets with the
cluster as the unit, so it adds no registry stem.
Longitudinal clusters states the step. The table gives its
instruments and what each one cannot see.
instrument |
what it checks |
what it cannot see |
|---|---|---|
|
whole-cluster folds, grouped inner folds, a held-out cluster unseen by a site-memorizing learner, the cluster-summed variance of every reported and derived estimate, the size term, the \(t\) reference, the band and the bootstrap kind. Each check has a mutation control that fails it |
coverage, and clustering of a competing-risk fit beyond the exact variance identity |
|
the study law keeps the unclustered truths exactly and carries a within-cluster correlation |
an informative cluster size, which the law does not carry |
the |
coverage, SE calibration and an IID control at 100 clusters, paired with R |
an informative cluster size, a competing-risk fit, and 4 to 19 clusters |
Repeated stacked point-treatment CV-TMLE likewise has a separate registered reporting-policy study. It reuses the shared CV-TMLE laws and structural checks under the median report. The point estimate is the median over complete fold draws. Its variance is the median of each draw’s variance plus squared displacement from that point. zEpid independently implements the same aggregation formula. Its fold training and targeting construct a different estimator, so the registered full-method equivalence artifact remains empty.
tests/unit/test_drtmle_crossfit.py::TestTheCanonicalSourceCVContract pins the last three choices
on 101 rows over folds of sizes 34, 34, and 33; both the equal-fold plug-in and cross-validated
variance are nonzero mutations there. The same module’s training-row and longhand cell-mean tests
pin the first choice for both pooled and nested reduced fits. This is structural implementation
provenance rather than R numerical parity. It does not establish a published cross-fitted theorem,
and it supplies no corrected fold-aggregation result for targeting_scheme="fold" or
cv_evaluation=True; both refusals remain pinned in tests/unit/test_drtmle_fit.py.
The complete-outcome alternation defaults to update_order="drtmle", the canonical R-package
sequence; "benkeser" names the published six-step recursion. The canonical round refits only
gr1/gr2 after equation (9) and only qr after equation (8). The fit-count mutation in
tests/unit/test_reduction_alternation.py requires six reduced-column fits per two-arm univariate
round rather than the former twelve, while the theorem, Gateaux, remainder, score, and correction
gates establish that the returned collection still satisfies the estimator identities. The same
module pins both source-specific solve orders. tests/unit/test_drtmle_fit.py separately requires
validate() to warn, and to preserve that warning through serialization, when equation (10)’s
historical ill_conditioned counter is positive despite a passing final score check.
The bivariate alternative is also a construction over these targets. Its acceptance chain
starts at the bivariate remainder representation of Benkeser, Carone, van der Laan and Gilbert
(2017), Appendix A, for the construction that their Section 3.1 cites as Theorem 3 of van der
Laan (2014), and then separately pins the
pinned R source’s two-column reduced probability and (gr-g)/(g*gr) outcome direction.
Three modules carry it. tests/unit/test_reduced_regressions.py uses a finite-support
joint-conditioning tie that fails if either generated design column is replaced by W.
tests/unit/test_reduced_submodel.py keeps a nonzero deliberate mutation omitting 1/g.
tests/unit/test_oracle_reductions.py injects the exact bivariate conditional expectations with
both primary nuisances wrong, recovers ey1, ey0, and ate, and requires every score and
correction identity to pass. The production cross-fitted fit
and serialization round trip are pinned in tests/unit/test_drtmle_fit.py. gr2 is NaN on this
path by design, so accidental use of the absent univariate-only regression cannot silently return
zero. For multiple treatment levels, the pinned R source applies those same branches once per
requested arm. tests/unit/test_reduced_regressions.py checks the three arm-specific joint
conditional probabilities against an exact finite-support law, and
tests/unit/test_multi_arm_collaborative.py carries a misspecified, nonzero end-to-end witness:
all armwise corrections are present and the score and correction gates pass. This is evidence for
the source’s armwise extension; it does not rewrite van der Laan’s binary theorem as a multi-arm
one.
Controlled direct effects now add registered repeated-sampling evidence to their exact-law,
Gateaux, remainder, and mutation chain. The
controlled direct-effect study fits both
intermediate levels, distinguishes the outcome-regression half of robustness from the joint
treatment-intermediate-observation mechanism half, and includes one nonzero control for each
mechanism. Its pinned R comparison recodes each requested level to tmle’s first result; a frozen
fixture separately records why the native second result is not a valid exact-nuisance comparator.
The stacked CV-TMLE of arm-indexed means and contrasts with missing outcomes is also an estimator
variant over registered targets. tests/unit/test_arm_indexed_stacked_mar.py holds its nonzero
witnesses. Each witness builds its value by hand, and a deliberate mutation makes it fail. The
mutations cover the clever covariate, the respondent mask, both inverse factors, the contrast
pairing, the same-row covariance, the band, and the held-out scale. The registered
stacked arm-indexed missing-outcome study
adds repeated-sampling evidence. It also compares R tmle 2.1.1 with the same stitched nuisance
predictions.
The randomized missing-outcome DR-TMLE surface is likewise an estimator variant over those
registered targets. Its acceptance evidence is Díaz & van der Laan (2017), §2.1, equation (6),
Theorems 1–2, and equations (11)–(13), plus tests/unit/test_drtmle_missing.py. That module keeps
all five reduced regressions and the separate D_A, D_Delta, and D_Y corrections, with a
nonzero finite-array witness that fails if the treatment correction is silently absorbed into
the observation correction. End-to-end fits require all three correction rows to agree with the
scores actually solved, exercise learned and known-randomization paths, refuse partial guards,
and round-trip the five reductions plus the targeted observation mechanism. A rowwise clever-
covariate identity verifies that treatment and observation are bounded separately before their
product is formed.
The registered randomized missing-outcome DR-TMLE study keeps the outcome-wrong cell as repeated-sampling evidence beyond score identities. Its observation mechanism is the exact law table, so the conditioning term is zero by construction. The cell measures the targeting, not one-wrong inference with an estimated mechanism.
The study
also contrasts the empirical correction scores before and after the five-reduction cycle. Its R drtmle comparison is limited
to the shared both-correct limit because the package exposes one joint treatment-response
mechanism; numeric agreement is not an acceptance gate for the separate reductions.
Above two arms, the same construction runs once per arm indicator, and the arm estimators are
stacked. The instruments are in the table. The column at the right says what each one cannot see.
instrument |
what it checks |
what it cannot see |
|---|---|---|
exact three-arm law, |
the oracle estimate equals the truth and the curve equals the EIF, to |
the three correction blocks, which vanish at the truth |
outcome and observation drifts on that law |
each block is nonzero at every arm, the estimate stays exact, and the stored scores equal the reported corrections |
a mutation on the treatment side, because the oracle |
mutation controls on a finite-sample fit |
a rolled arm mechanism, a swapped response, and rolled observation and outcome-drift columns each fail the correction check |
a sign error in the curve, because a solved block has mean zero either way |
independent one-indicator reference |
each arm’s estimate and curve equal a test-local implementation of Steps 1 to 5, under live drift |
the two-arm route, which takes one shared tilt |
repeated-sampling coverage, size, power and the simultaneous band at three arms |
the five-reduction construction in the R pairing, which uses the composite mechanism of Díaz and van der Laan’s page 25; and the estimated-mechanism route |
All of these are in tests/unit/test_drtmle_missing_multi_arm.py except the study. The R pairing
covers the both-correct limit only.
The composite indicator. An observational missing outcome and a declared missing treatment take the composite construction, for the DR-TMLE variants and for the ordinary TMLE. It is an estimator variant over the registered arm means and contrasts, and adds no target. The instruments are in the table.
instrument |
what it checks |
what it cannot see |
|---|---|---|
two exact laws, |
at the oracle factors the estimate equals the truth and the curve equals the EIF to |
the corrections, which vanish at the truth |
outcome and mechanism drifts on those laws |
the estimate stays exact, |
a defect that is exact on a saturated law |
nonzero witnesses and mutation controls (E5 to E12, E19) |
an omitted treatment or outcome observation factor, a renormalized or rolled composite, the two-arm complement form, a treatment factor fitted on every row, a view that masks by |
a defect the mutation list does not name |
exact reductions (E13, E14) |
on complete data at three arms the composite route is the shipped DR-TMLE bit for bit; with |
the two-arm composite, which is armwise where the complete-data route is not; the per-arm reference covers it |
independent per-arm reference (E18) |
each arm’s estimate and curve equal a test-local implementation of Benkeser et al.’s steps on |
each arm under both guards together under the outcome drift, where equation (10) has a near-zero covariate and its fixed point is not numerically identified |
the observational missing-data study, |
repeated-sampling coverage, drift robustness, the rate, calibration, size, power, the simultaneous band, the correction cycle, and a complete-case control, at two and three arms; a regime mean and an arm MSM slope on the composite TMLE; R |
partial guards, the bivariate reduction, weights and clusters, which have no coverage cell; the exact laws and the parent studies cover them |
No instrument can detect a violation of the treatment condition, Y(a) independent of Delta_A
given (A, W). No observed-data check can, because the data hold no outcome of a row whose
treatment is unrecorded under the other arm.
Cross-validated DR-TMLE missing-data compositions are not covered. The data carry a declared
known mechanism, so the bootstrap reads the declared values of each replicate’s own rows.
tests/unit/test_drtmle_missing.py checks that no replicate fits the treatment learner. An
unguarded delta= fit with known probabilities and cross_fit=False is a plain TMLE and is
accepted as one. _FailIfFit is the witness that the declaration reaches the fit. With
cross_fit=True, the same fit is refused at every guard=, including guard=().
Bounding the two mechanisms separately is what the scope label had to learn. contract
measured its truncation witnesses on the treatment mechanism alone. That is blind in exactly the
regime this construction is for. A randomized trial’s g is flat by design and cannot clip, so a
fit whose P(Delta=1|A,W) was pinned on a fifth of its rows was certified "theorem".
TestTheContractSeesTheObservationTruncations is now a pair of fits.
One is well-behaved. The second has its observation mechanism pinched while its treatment
mechanism demonstrably is not. It is asserted to leave every pre-existing column inactive, so a
bound-active verdict there can come only from the two new witnesses. The same fixture carries the positivity
report’s derived P(A=a,Delta=1|W) row, which counted its truncation against a product of floors
the estimator never applies: 1.1% reported against 20.1% actual, with the old rule kept beside it
as the control.
The exact law alone is not evidence for either construction, and this is worth writing
down rather than leaving to be rediscovered. Handed the oracle nuisances,
tests/discrete_law_multi.py makes every new term vanish: max|Qr| = 1.9e-17, gr2 = 0
exactly, the mechanism’s epsilon is [0, 0, 0] and the targeted mechanism equals the
initial one to 2.8e-17. A fit that recovers all five parameters to 2e-15 there has
therefore said nothing about equation (9), the corrections, or the outcome-adaptive design.
Reversing the columns of the targeted mechanism leaves the exact-law assertions,
score_check() and correction_check() all passing. So each construction carries its own
nonzero instrument:
construction |
instrument |
what fails without it |
|---|---|---|
armwise equation (9) |
|
any change to the response, offset, covariate or arm alignment; agreement is to |
the reported corrections |
|
Qr |
arm alignment of the exit state |
|
a per-arm quantity read at the wrong arm, which is invisible to any symmetric check |
|
|
the binary sign convention carried over, which the exact law cannot see |
the shared |
|
zeroing, permuting or substituting the design, none of which any exact-law or field-name assertion detects |
the per-arm |
|
a design column fed to the wrong arm, a complement taken off the simplex, the observed-arm prediction as the design, one shared coefficient, a block-diagonal covariance, the efficient curve reported in place of Theorem 1’s, dropped weights, and in each fold the wrong column, the first column, dropped weights or a stitched out-of-fold design. |
the per-arm |
|
factors pooled over strata, a dropped observation factor, an observation factor fitted on every recorded row, the full-covariate \(\pi(a, W)\) as the observation factor, observation factors fitted on the stitched out-of-fold \(\bar Q\), a treatment factor fitted on every row, the recording broadcast from arm 0, arm 0’s observation factor broadcast to every arm (which the natural course then reads), a marginal curve without the mass term, a learner fitted on a constant target, dropped weights, the tilt read from the design factor, the shared design treated as per-arm, and a status key that admits these fits. |
selector joint target |
|
scoring only the first contrast: the nonzero mutation changes the penalty by more than 100 |
The selector uses one shared categorical path. ey contributes all K arm curves; ate,
rr, and or contribute all K - 1 reference contrasts. Its pooled cross-validation array
therefore has shape (candidate, row, component), and the penalty sums every component’s
variance and squared mean. The finite-support armwise remainder identities in
test_remainder_multi.py cover the underlying mean vector and its reference-contrast map;
the ratio targets use the same independently tested delta-method curves as the final report.
The nonzero scientific instruments are now completed by
tests/unit/test_remainder_multi.py and
tests/unit/test_influence_gateaux_multi_collaborative.py.
The former evaluates every arm’s remainder at nuisances that are wrong on purpose,
and takes both DR-TMLE projections from the shipped reduced_correction_parts rather than
rebuilding them, against an exactly saturated ReducedSet this finite law admits. Its
longhand derivation is kept beside them as an independent oracle, and
test_the_library_corrections_are_the_longhand_ones is the assertion that reaches the
library: flipping the sign of either d_g or d_q, or zeroing one of them, fails the
module. Its OAT entry is a design-level boundary. It uses a coarsened Qbar whose generated
mechanism cannot be repaired independently of W. It is not a check on the shipped OAT
code, which is pinned by test_oat_fits_the_treatment_model_on_the_arm_specific_qbar_matrix
in the table above and by the OAT curve check named next.
The latter checks both multi-arm DR-TMLE union-model cells against the complex-step
derivative through real DRTMLE fits, and exercises CTMLE(strategy="oat") on the regular
exact law where its generated design identifies W.
The four registered rows publish replication accounting, truth and comparison verdicts, and nuisance-regime properties. They also cover root-n ladders and each selector path against one forced empty path. The reporting-policy rows keep red cells where the evidence does not support a stronger claim. Registration does not turn an observed limitation into an inherited binary theorem.
Targeting and fitting options#
The options stopping=, step_bound=, step_search=, iterate=, outcome_by_arm=,
patience= and treatment_adjustment= have no registered study. A fluctuation coefficient is
zero at the truth, so an exact-law check cannot see a stopping rule or a step. Each option
therefore rests on a hand computation, a nonzero witness and a deliberate-mutation control. The
Python tests below gate the change. The R comparators are pending, and the next table says which.
part |
exact test |
witness |
mutation controls |
|---|---|---|---|
|
|
\(k^*\) differs from the last iterate, and the estimates differ by more than \(10^{-6}\) |
report \(k^*+1\), rank by the relative statistic, include iterate 0, |
|
|
the one-step estimate under |
rows for \(n_u\), no span, uncapped \(\log n\) at \(n_u = 30{,}000\), a score column on an arm column, |
|
|
a fit that met its rule passes above the default bar. A fit that missed it fails below the bar |
|
|
|
the reported relative score exceeds |
an exit that reads the relative score or the solver’s ratio, an outcome entry that drops the ratio, a restatement that keeps old thresholds |
|
|
a crafted node where the shipped test takes 3 iterations and the rule takes 1 |
no cluster code, no span, no MSM tile |
|
|
the first bounded step is below the Newton step. A fit unconverged at |
un-normalised \(Hs\), reversed direction, |
|
|
an \(A \times W\) interaction law, where per-arm and pooled predictions differ by more than \(10^{-3}\) |
swapped arm models, a mask without the arm, the observed arm read for a shifted arm |
|
|
a law whose global minimum lies past the stop changes |
|
|
|
the arm-1 error exceeds \(10^{-3}\) when the restriction fails |
fit \(g\) on the full set, and in C-TMLE the pool, the preorder, the lasso design and the learners names |
|
|
a default hazard fit still runs when every targeting solver is replaced |
none |
The R comparators have scripts, a registry and a self-test in tests/canonical/option_fixtures/.
The local Study queue ran the four scripts in the pinned images. Each JSON file records the
checksum of its script. tests/unit/test_option_fixtures.py compares each file and ties the label
in the table to the registry.
fixture |
comparator |
parts |
status |
|---|---|---|---|
|
|
|
exact tests and R parity |
|
|
|
exact tests and R parity |
|
|
|
exact tests and R parity of the option |
|
|
|
exact tests and R parity of the option |
The two ctmle rows compare the option, not the whole fit. A transcription of ctmleDiscrete,
tests/canonical/option_fixtures/r_ctmle_discrete.py, must reproduce every R number in the two
files. The package must then stop where R’s counter stops on the package’s own scores, and keep
R’s candidate pool. R’s discrete path differs from the package in two ways that the options do not
touch. Candidate sequences lists them.
The declared differences from the comparators are D4, D7 and D7b. Stopping rules and patience state them.
Longitudinal estimands outside the target registry#
LTMLE parameters are indexed by regimen, horizon, and sometimes cause rather than by a
Target, so they do not belong in the registry-gated table above. They have their own
bidirectional oracle gates in the named Gateaux modules; this table makes the parallel
evidence structure explicit.
longitudinal variant |
parameter and EIF oracle |
nonzero or mutation witness |
canonical implementation witness |
not covered |
|---|---|---|---|---|
end-of-study, static and dynamic regimens |
|
dynamic-rule arm evaluation and dropped-censoring mutations in the Gateaux module; exact-fold and held-out-prediction checks; the pooled cross-fitted targeting longhand and its four mutations in |
registered ordinary R |
no observation weights or time-respecting splits, and |
categorical treatment nodes, static and dynamic regimens |
|
nonzero quadratic remainder, third-arm probability and dynamic-rule mutations, exact-fold support checks, and the arm-encoding witness in |
registered ordinary and five-fold studies against pinned R |
deterministic categorical regimens only; no censoring, missingness, continuous dose, weights, clusters, fold repeats, survival, or competing risks. The default band of each registered fit is red in the default-band study |
known stochastic categorical policies |
|
the policy witnesses and mutations M1 to M9 in the Gateaux module, including the targeting witness under a misspecified initial fit, the one-hot bit identity and the recorded-randomizer projection; the pooled cross-fitted longhand in |
the registered study |
a false |
absorbing survival curve |
|
the |
registered ordinary R |
no time-respecting splits, active truncation, observation weights, or competing events. The one-plan curve band passes, and the band over all ten parameters is red under |
repeated outcome means over horizons |
|
the reduction to the end-of-study fit of the truncated panel over 13 compositions in |
the registered study |
|
competing-risks cumulative incidence |
|
mutation from all-cause to cause-specific survival in the Gateaux module; a separate finite-sample all-cause risk-set control; one-cause reduction in |
registered ordinary and five-fold studies against pinned R |
two causes, two nodes, and one fixed split; no active truncation, learned-mechanism parity, weights, clustering, or eliminated competing event |
one baseline treatment held over survival nodes |
|
the held = copied-column identity at one fold, and a penalized learner whose copied factor is not one, in |
the registered study |
a continuous time model, hazard-based or monotone targeting (hazard-based survival targeting), and |
held end-of-study outcome after several censoring nodes |
|
the held = copied-column identity; the identity with point TMLE with |
the identities above carry the shipped end-of-study evidence; the |
|
a known stochastic policy or a modified treatment policy at a held node 1 |
|
the one-hot policy equals its rule bit for bit; the MTP equals identity policies on copied columns; a mutation that makes the identity nodes policy nodes fails the Gateaux check |
the registered study |
a continuous dose at a held node has fast-tier evidence only. |
the sequentially doubly robust estimator ( |
every exact longitudinal law through the sibling LTMLE fit helpers, with the engine swapped, in |
patterns P_A to P_D and the node-2 control on the exact two-node law in |
the registered study |
no multiply robust interval; a one-correct pattern reports its coverage only, because Theorem 4 needs the rate product |
incremental interventions over time ( |
|
|
the registered study |
not doubly robust, so the mechanism must be consistent at every tilt node. No band over a continuum of \(\delta\), no no-effect test and no TMLE for the tilt. See IPSI limitations. The registered study categorical incremental policies covers the rows |
sequential g-computation and IPW ( |
exact binary, survival, competing-risk, three-arm and policy laws in |
the weight-derivative and mutation checks of the stacked curve (W5, W6, W7) in the same module |
Python property evidence, study |
registered study |
structural nested mean model ( |
exact finite laws S2, S3, K, C, H, P and M of |
nonzero witnesses for the feature term and the level term on law H (M5, M6), the vector Jacobian on law K (M4), the later-blip adjustment, the reference level, the period-state nuisance and the product nuisance (M1 to M3, M7); finite differences of each curve in the observation weights (W3); a shared cluster effect for the cluster-summed variance, with a control that drops |
EconML 0.17.0 |
Python property evidence, with comparator runs pending: the reporting structural nested mean study gives the repeated-sampling evidence. See structural dynamic effects |
parametric g-formula ( |
exact binary, categorical, Gaussian, survival, competing-risk, tilt, policy, fixed-dose and rule-set-dose laws in |
the weight-derivative check of the stacked curve with the exact simulation terms (W3a), the production derivative against the exact one per draw kind (W3b), the simulation variance against the exact curve variance (W4) and against the variance across seeds, and the mutation controls listed on the technical page |
Python property evidence, study |
registered study |
working model over regimen/horizon cells |
|
non-saturated, nonuniform projection law plus exact pooled-design/loss-weight checks in |
registered ordinary study against a fixed projection of four correlated R |
survival, competing-risk, weighted and clustered projections have fast-tier exact identities only. R |
Descriptive longitudinal truncation replay#
The truncation grid adds no new parameter or influence-curve claim, so it has no registry row or
coverage study. tests/unit/test_longitudinal_truncation_refit.py is its fast implementation
instrument. It checks exact fitted-bound replay, active-bound movement, complete backward
recursion, the out-of-fold mechanism count, parameter-specific counts, replay omissions, persistence,
parallelism, dataframe backends, and combined-report gates. Separate active witnesses cover
end-of-study, survival, competing-risk, and MSM results. The composition cases include dynamic
categorical plans, weights, clusters, and an audit of the random state inside a nested Super
Learner.
Those tests do not establish interval coverage, a preferred bound, positivity clearance, or external implementation parity under active truncation. The registered rows above remain evidence for their fixed-bound point estimates and stated inference regimes only.
Post-fit functionals of reported estimates#
These outputs are computed from the influence curves of a fitted result. None is a Target, so
none enters the registry-gated table. Each is an exact delta-method or linear-functional
computation, and each row names the instruments that check it.
functional |
built from |
instruments |
nonzero or mutation witness |
not covered |
|---|---|---|---|---|
ratio contrasts, point treatment: |
two level estimates of a |
|
swapped arms give \(1/\psi\) and \(-IC\); an odds ratio built with the risk-ratio derivative fails; dropping the \(1/v\) factor of the ratio-scale contrast fails |
a ratio of two contrasts, and a numeric reference |
ratio contrasts, longitudinal: |
two |
|
a survival view that forgot the complement; a cause swap; the survival odds ratio as the reciprocal of the risk odds ratio |
a ratio as a fit-time estimand, and bands over derived ratios |
Wald test at a declared null: |
one estimate and its standard error |
|
a ratio-scale test that does not log the null |
one-sided tests, and a joint chi-square test |
transformed contrast: |
a smooth function of estimates and a monotone map |
|
a dropped slope; an interval mapped back with the forward map; a decreasing map without the sort; a band without the inverse map |
the coverage of a user transform. The user’s transform defines that estimand |
RMST and RMTL: |
the risks or cause-specific incidences below the horizon |
the enumeration truth of |
off-by-one horizon ranges; a dropped cause in the competing RMST; unit spacing on an unequal grid, in |
the RMST of a continuous event time on a grid, which is the RMST of its grid ceiling |
C-TMLE logistic plug-in diagnostic: |
a selector |
|
the term at a constant-outcome fit against the complex-step derivative; a flipped sign, a dropped inverse and a dropped intercept |
a coverage claim. The status stays |
|
a full refit of the estimator per resample |
|
a refit-count witness that fails when a replicate reuses the fit’s mechanism; a shrunken percentile interval that the study’s control must fail |
a data-adaptive nuisance, and any law or size the study does not run |
A simulated law is an instrument too, and it can be wrong the same way#
A coverage study is only evidence if the number it calls the truth is the number an adjusted
fit is estimating. Two of the generators shipped for clustered inference failed that: the
per-cluster latent drove the treatment mechanism as well as the outcome and was not emitted
as a covariate, so the declared ATE of 1.0 was not identified. The identified value was
1.83, and every interval missed by six to ten standard errors while the docstring claimed the
counterfactual means were unchanged. The longitudinal generator failed it twice, since its
shared effect also tilted the outcome on the logit scale, where
E_S[expit(eta + gamma S)] != expit(eta) moves the means whatever the mechanism does.
Both now put the sharing where it does not confound, and both assert it. clustered_dgp makes
the latent an effect modifier independent of treatment. The longitudinal generators share part
of L2’s own noise, and preserve its conditional law exactly, so a clustered draw’s truth is the
same number as an unclustered one’s. See tests/unit/test_datasets.py and
tests/unit/test_datasets_longitudinal.py.
The second half is the part that is easy to lose. Removing the confounding also removes the
clustering, because a shared additive residual reaches the influence curve only through
E[H | W], which is zero for a well-specified g. An additive shared residual measures a design
effect of 1.00. The arm-interacted form clustered_dgp ships measures about 1.95 at ten rows per
cluster, which its docstring records. So each generator carries a nonzero within-cluster witness
beside its identification test; without one, a correct-looking fix leaves the study measuring
nothing.
The registered clustered point-treatment study
adds repeated-sampling evidence for that witness. It runs on the binary law
clustered_dgp(family="binomial"), whose quadrature truths are ate = 0.10405 and rr = 1.2513,
and not on the Gaussian law above. The move is the outcome-scale rule: a cross-fitted continuous
outcome needs a declared support, and this law has none.
The study compares five-fold clustered TMLE against pinned R lmtp on one grouped partition that
cleverly draws and the R adapter retains. The partition balances nothing. Its IID control reuses
the same rows, estimates, and influence curves. Only cluster aggregation changes.
Two reporting-policy studies extend the clustered evidence. The
unequal-size study draws 200 clusters of size
uniform on 2 to 18, with the size in the outcome. It measures the row-weighted and the
cluster-average estimands, the fold-evaluated and DR-TMLE pairs, and the cluster-mean rule of R
tmle. The few-cluster study measures the
\(t\) reference at 10, 20 and 30 clusters on five fits (in-sample LTMLE from 20), and pairs in-sample LTMLE with R ltmle
1.3-0 at 20 clusters.
What this table says is missing#
Read down the not covered column and one thing recurs: the theorem column is empty
for every row. The tree contains one such instrument: tests/unit/test_theorem_drtmle.py,
which checks DRTMLE against Benkeser et al.’s Theorem 1 at values where the correction
does not vanish. This instrument exists because that variant’s corrections are zero row by row at
correct nuisances, so the exact-law instruments were blind exactly where it mattered.
The arm, regime, shift, tilt and MSM axes are not in that position: their influence curves do not vanish at the truth, so the Gateaux comparison is anchored where the quantity lives rather than where it disappears. That is the argument for the empty column, and it is an argument rather than a measurement. That is why it is written here, where the next person to add an estimand will read it, instead of being left to be re-derived.
The condition that would fill the column is a target whose curve contains a block that is zero at the truth. Registering one means the row above is no longer available, and the new row needs a check against its source’s own theorem, at a value where its block does not vanish, before the estimand is reported.