ordinary point-treatment TMLE |
R tmle3 0.2.0 at commit ed72f8a, with sl3 at 0e8f236, on a digest-pinned R 4.5.2 |
arm means, ATE, ATT, ATC, observed mean, PAR, PAF, RR, and OR; pointwise 95% Wald intervals, with RR and OR on the log scale |
34/34 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 17/17 paired similarity and non-inferiority tests pass using 99% confidence bounds |
12/12 cells pass: double robustness with its both-wrong control, bias, coverage, and SE checks at three sizes, two root-n rates, interval calibration, type-I error under a confounded null, and a power control |
1,600 replications x n=1,000 on two laws; worst standardized bias 0.1254, lowest coverage 0.9344 |
six declared limits, including what the standard-error band cannot resolve and the PAF scale qualification |
ordinary weighted point-treatment TMLE |
R tmle 2.1.1 on a digest-pinned R 4.5.2 image, with the identical exact nuisances and obsWeights= |
population arm means, ATE, RR, and OR from a biased selected sample; ordinary pointwise 95% Wald intervals, with ratios on the log scale |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired similarity and non-inferiority tests pass using 99% confidence bounds |
18/18 cells pass: double robustness and its control, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, targeting, and weight necessity against the exact selected-population target |
800 replications x n=2,000 on one law; worst standardized bias 0.0375, lowest coverage 0.9400; omitted-weight displacement 6.8141 SD |
five declared limits; fixed known weights, exact nuisances, ordinary pointwise inference, and no learned weights, missingness, clusters, strata, or longitudinal treatment |
ordinary learned weighted point-treatment TMLE |
R tmle 2.1.1 on the same pinned image, fitting identical weighted main-effects Gaussian and binomial GLMs |
population arm means and ATE from a continuous selected law; ordinary pointwise 95% Wald intervals |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
14/14 cells pass: three-size and root-n checks, interval calibration and controls, null and power, plus four learner-weight necessity cells |
800 replications x n=2,000 on one law; worst standardized bias 0.0432, lowest coverage 0.9413; unweighted plug-in displacement 7.9655 SD |
seven declared limits; known moderate weights, ordinary main-effects learners, randomized treatment, pointwise inference, no cross-fitted weighted nuisances, a learner-weight witness that resolves the outcome regression alone, and two calibration controls judged by one SE-ratio detector |
ordinary multi-arm point-treatment TMLE |
R tmle3 at commit ed72f8a, with sl3 at 0e8f236, on a digest-pinned R 4.5.2 |
three labelled treatment-arm means plus ATE, RR, and OR contrasts against the high arm; ordinary pointwise 95% Wald intervals, with ratios on the log scale |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
14/14 cells pass: double robustness and its both-wrong control, three-size and root-n checks, interval calibration, null size, power, and the default band over nine parameters with its pointwise control |
800 replications x n=1,500 on one law; worst standardized bias 0.0358, lowest coverage 0.9537 |
binary-outcome, non-cross-fitted GLM evidence with one reference arm and no conditional effects, missingness, weights, clusters, fold repeats, or longitudinal treatment |
ordinary point-treatment TMLE with baseline strata |
R tmle3 0.2.0 at commit ed72f8a through tmle_stratified(..., base_estimate = FALSE) in a treatment-specific-mean stage and an ATE stage, with sl3 at 0e8f236 |
the two arm means and the ATE inside each of three unequal baseline strata against tmle3, and the stratum ATT, ATC and PAR against the exact truth; pointwise 95% Wald intervals and the default band across strata |
16/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
80/83 cells pass: calibration and efficiency of every stratum parameter in sample and cross-fitted, double robustness, and a stratum-targeting control; the treated mean of the smallest stratum and the two bands that include it are red |
1,000 replications x n=2,000 on one exact law; worst standardized bias 0.0626, lowest coverage 0.9170 |
reporting, not gated; five declared limits: one law with three strata, binary treatment and outcome, logistic learners, and no weights, clusters, missing outcomes, repeats, or comparator for ATT, ATC and PAR |
stratified incremental and MSM targeting |
R npcausal ipsi at commit 56a5ac1, on a digest-pinned R 4.5.2 image, once per stratum subset and once on every row |
the incremental means at odds multipliers 1, 2 and 0.5 and their two contrasts, marginal and in each of three baseline strata; the logit and continuous-dose MSMs and the cross-fitted natural-course mean, marginal and in each stratum, against exact truths |
40/40 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 20/20 paired similarity and non-inferiority tests pass using 99% confidence bounds |
129/131 cells pass: calibration of the fifteen stratum incremental parameters, the logit MSM and the stacked natural-course mean with their controls, the twenty-parameter band, both targeting-necessity pairs, the stratum robustness family and the dose robustness cells; the logit-MSM slope calibration in strata 1 and 2 is red under X8-logit-small-stratum |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0634, lowest coverage 0.9350 |
five declared limits; reporting, not gated: finite-support laws with one three-level stratum, in-sample incremental and MSM fits, a binned dose density with no calibration claim, no stratified comparator for the MSM and natural-course targets, and fold targeting and fold evaluation refused |
identity-link MSM with baseline strata |
R tmle3 Param_MSM at commit ed72f8a and sl3 at 0e8f236, on a digest-pinned R 4.5.2 image, once per stratum subset |
the nine stratum coefficients of the identity-link MSM \((1, a, W)\) under a bounded outcome, with exact truths |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 8/9 paired similarity and non-inferiority tests pass using 99% confidence bounds; msm[W][V=2] is inconclusive on its calibration leg under X8-identity-small-stratum |
35/36 cells pass: exact-EIF efficiency and calibration of every marginal and stratum coefficient with their controls; the stratum-2 slope calibration is red under X8-identity-small-stratum |
1,000 replications x n=2,000 on one law; worst standardized bias 0.0706, lowest coverage 0.9360 |
three declared limits; reporting, not gated: one finite-support law with a bounded outcome, in-sample fits only, and the linked and dose MSMs measured in the incremental study |
DR-TMLE with baseline strata |
R drtmle 1.1.2 at commit 538a3a2, on a digest-pinned R 4.5.2 image, once per stratum subset with the subject’s initial arrays and folds |
the two arm means and the ATE in each of three baseline strata of the stratified paper law, cross-fitted at guard=("Q", "g"); the marginal ATE against its exact truth |
16/18 implementation-estimand tests against known truth pass using 99% confidence bounds; ey[1][V=2] under-covers in both implementations under X8-drtmle-small-stratum |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
23/28 cells pass: exact-bound efficiency and calibration of the marginal and stratum ATEs with their controls, and the both-correct, outcome-correct and both-wrong robustness cells; the four treatment-correct cells are red under X8-drtmle-one-sided-bias and the stratum-2 calibration under X8-drtmle-small-stratum |
800 replications x n=2,000 on one law; worst standardized bias 0.1375, lowest coverage 0.8925 |
four declared limits; reporting, not gated: one law with a complete binary outcome, the univariate reduction at one guard, no paired comparator for the marginal estimate, and one-correct cells on an H1 law |
stacked point-treatment CV-TMLE |
R tmle3 CV-TMLE at commit ed72f8a, with sl3 at 0e8f236 and identical realized folds |
arm means, ATE, ATT, ATC, observed mean, PAR, PAF, RR, and OR; ten-fold nuisance fitting, a stacked update, whole-sample plug-in evaluation, and pointwise 95% Wald intervals |
34/34 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 17/17 paired similarity and non-inferiority tests pass using 99% confidence bounds |
14/14 cells pass their own and family verdicts: the ordinary study’s twelve instruments on bounded laws, and flexible-learner cross-fit versus in-sample controls. Three fold-policy rows are reported rather than gated, and are counted separately |
1,600 replications x n=1,000 on two laws; worst standardized bias 0.1085, lowest coverage 0.9356 |
six declared limits; bounded to one unstratified ten-fold split, a known continuous outcome support and pointwise intervals, and PAF inference scales differ |
clustered point-treatment CV-TMLE |
R lmtp 1.5.4 at commit f04a2b4, with pinned ife 0.2.3, identical grouped folds, exact treatment mechanisms, and cluster identifiers |
untreated and treated means plus their joint ATE contrast; five grouped folds and pointwise cluster-robust 95% Wald intervals |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
2/2 cells pass their own and family verdicts: cluster-robust calibration and an IID variance control on identical influence curves |
800 replications x n=2,000 on one law; worst standardized bias 0.0748, lowest coverage 0.9425 |
nine declared limits; binary-outcome binary-treatment evidence with clusters of ten, one fixed grouped partition, exact treatment mechanisms, main-effects outcome learners, and no weights or simultaneous bands |
clustered point-treatment CV-TMLE at unequal cluster sizes |
none claimed. No pinned comparator cross-fits with cluster sums at unequal sizes, so a zero-row equivalence artifact records the absence |
untreated and treated means and the ATE at 200 clusters of size 2 to 18, non-informative and informative, and the cluster-average estimand through weights of one over the size; five grouped folds |
9/9 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
10/12 cells pass: the stacked, fold-evaluated and DR-TMLE cluster-robust pairs and their IID controls, and both estimand-weighting pairs; the fold-evaluated pair on the cluster-level covariate law at 40 clusters in 10 folds is red under F28 |
800 replications x n=2,000 on one law; worst standardized bias 0.0634, lowest coverage 0.9287 |
five declared limits; reporting, not gated: one size law, 200 clusters on the normal reference, a binary outcome with the exact propensity, main-terms learners and no comparator |
clustered TMLE intervals on a t reference at few clusters |
R ltmle 1.3-0 at commit 338c029 with id=, on the pinned ltmle image, with the identical exact propensity and the main-terms outcome formula |
in-sample LTMLE arm means and ATE at 20 clusters; the ATE of five fits at 10, 20 and 30 clusters of equal and informative sizes on a Student t reference with J - 2 degrees of freedom |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
49/56 cells pass: the t-reference arms of every fit but the fold-evaluated fit at informative sizes, and the IID controls of the TMLE and LTMLE fits; three fold-evaluated bias cells and four DR-TMLE IID controls are red under F28 |
1,000 replications x n=200 on one law; worst standardized bias 0.1057, lowest coverage 0.9310 |
four declared limits; reporting, not gated: 10 to 30 clusters, parametric learners, a binary outcome with the exact propensity, and one primary cluster count |
clustered cross-fitted end-of-study longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4 with ife 0.2.3, on the pinned lmtp image, with id=, the realized whole-cluster folds and the exact per-node density ratios |
three censored two-node regimen means and two contrasts at 100 clusters of 40 rows; five whole-cluster folds and a cluster-summed curve |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired similarity and non-inferiority tests pass using 99% confidence bounds |
9/10 cells pass: four cluster-robust pairs (static, dynamic, unequal sizes, survival) and their IID controls, and the band’s pointwise control; the cluster multiplier band is red under band-finite-sample |
1,600 replications x n=4,000 on one law; worst standardized bias 0.0155, lowest coverage 0.9381 |
five declared limits; reporting after the declared red-cell route: parametric learners, 100 clusters, equal sizes in the pair, no competing-risk cell and no informative size |
cross-fitted clustered longitudinal TMLE on a t reference at few clusters |
none claimed. No pinned comparator cross-fits a clustered longitudinal fit with a t reference, so a zero-row equivalence artifact records the absence |
the static contrast of the cross-fitted clustered LTMLE at 20 and 30 clusters of equal and unequal sizes, on a Student t reference with J - 2 degrees of freedom |
5/5 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
8/8 cells pass: the t-reference arms and the IID controls at 20 and 30 clusters of equal and unequal sizes; the normal-reference arms are reported only |
1,000 replications x n=800 on one law; worst standardized bias 0.0383, lowest coverage 0.9380 |
four declared limits; reporting, not gated: 20 and 30 clusters only, parametric learners, the scaled latent and no comparator |
fold-evaluated point-treatment CV-TMLE |
none claimed. No maintained package pairs a pooled targeting update with fold plug-in evaluation, so a zero-row equivalence artifact records the absence |
arm means, ATE, ATT, and ATC; ten-fold nuisance fitting, a pooled update, equal-fold plug-in evaluation, cross-validated variance, and pointwise 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
14/14 cells pass on bounded laws: double robustness and its control, bias, coverage, and SE checks at three sizes, two root-n rates, interval calibration, null and power controls, and flexible-learner cross-fit versus in-sample controls |
1,600 replications x n=1,000 on two laws; worst standardized bias 0.0393, lowest coverage 0.9425 |
four declared limits; bounded to one unstratified ten-fold split and a known continuous outcome support, and this is not parity evidence for stacked R CV-TMLE |
fold-targeted point-treatment CV-TMLE |
Python zEpid 0.9.1 at commit 16a0f96, in a digest-pinned Python 3.10 image |
binary-outcome ATE; one exact equal two-fold partition, complement-trained logistic nuisances, equal \(1/V\) fold targeting and evaluation, cross-validated variance, and pointwise 95% Wald intervals |
2/2 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 1/1 paired similarity and non-inferiority tests pass using 99% confidence bounds |
14/14 cells pass on bounded laws: double robustness and its control, rates, calibration, null, power, and overfitting controls |
1,600 replications x n=1,000 on one law; worst standardized bias 0.0256, lowest coverage 0.9569 |
seven declared limits; gated ATE-only parity at one fixed unstratified two-fold partition; the property cells need a known outcome support; zEpid’s ddof=1 variance differs from cleverly’s second moment, and RR and OR remain refused |
repeated point-treatment cross-fitted TMLE |
none claimed for the full estimator. zEpid at 16a0f96 corroborates the median point and split-adjusted median variance formula; its nuisance training and foldwise targeting differ, so a zero-row equivalence artifact records the absence of full-method parity |
arm means, ATE, ATT, ATC, observed mean, PAR, PAF, RR, and OR; five folds, three primary fold draws, median points, median within-draw variance plus squared split displacement, and pointwise 95% Wald intervals |
17/17 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 full-method external comparisons; the aggregation formula has pinned implementation corroboration |
14/14 declared repeated-sampling property cells pass on bounded laws, including paired one-versus-three-draw stability |
800 replications x n=1,000 on two primary laws; worst standardized bias 0.0557, lowest coverage 0.9337 |
reporting, not gated: every cell is green, and the policy means no test asserts that; eight declared limits, including no full-method comparator, no median-superiority claim, fixed-law stability evidence, one unstratified split per draw, a known continuous outcome support, and marginal-only inference |
selector-based point-treatment C-TMLE |
R ctmle 0.1.2 at commit 18de559, the maintained comparator for these selector entry points |
greedy, ordered, and discrete binary-treatment ATE selectors, reporting point estimates and a working-mechanism plug-in diagnostic. The package publishes no interval and no p-value for these paths, and F18 in the roadmap is the condition that reopens that |
6/6 implementation-strategy tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
11/14 cells pass their own and family verdicts: double robustness with its both-wrong control, bias, coverage, and SE checks at three sizes, two root-n rates, calibration, and power are green; the forced-selection cell and the null-size cell are red, and the empty-path control passes on its own but its selector family does not |
800 replications x n=2,000 on three selector strategies; worst standardized bias 0.0429, lowest coverage 0.9387 |
reporting, not gated: ten declared limits; parity is unpenalized, non-cross-fitted, binary ATE only; the forced-selection cell and the null-size cell are red; the coverage and interval cells measure a diagnostic that the package does not publish as inference |
selector-based multi-arm C-TMLE |
none claimed. R ctmle 0.1.2 is binary-treatment only, so the empty equivalence artifact avoids a surrogate comparison |
greedy, ordered, and discrete selectors for three labelled arm means plus ATE, RR, and OR contrasts against the high arm, reporting point estimates and a working-mechanism plug-in diagnostic. The package publishes no interval and no p-value for these paths, and F18 in the roadmap is the condition that reopens that |
27/27 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no canonical multi-arm selector was identified |
5/12 cells pass their own and family verdicts: the n=2,000 and n=8,000 size cells, both root-n rates, and power are green; calibration, null size, the n=500 coverage bound, and all three selector paths are red, while the empty-path control passes on its own but its selector family does not |
800 replications x n=1,500 on three selector strategies; worst standardized bias 0.0597, lowest coverage 0.9363 |
reporting, not gated: no external equivalence claim; the selector-necessity law is the only law here that leaves the declared truncation bounds, and the discrete ladder stops at the empty candidate on it; the n=500 coverage bound misses its floor under the unstratified split this row now declares; the coverage and interval cells measure a diagnostic that the package does not publish as inference, with no simultaneous intervals, conditional effects, missingness, weights, clusters, fold repeats, or longitudinal treatment |
C-TMLE candidate sequences |
R ctmle 0.1.2 at commit 18de559 (r-ctmle): Python property evidence, with comparator runs pending. The R runs wait for Study queue item 9 |
the binary-outcome ATE of strategy="lasso" and strategy="learners", reporting point estimates and a working-mechanism plug-in diagnostic. The package publishes no interval and no p-value for these paths, and F18 in the roadmap is the condition that reopens that |
1/2 implementation-strategy tests against known truth pass using 99% confidence bounds: the declared-learner row is green, and the lasso row’s coverage is red as declared, a plug-in variance shortfall that plain TMLE shares |
0/0 paired comparisons: Python property evidence, with comparator runs pending |
8/11 cells pass their own and family verdicts: double robustness and the multi-arm pair are green; the declared-learner necessity cell is the declared expected red on its bias clause, and its RMSE bound, the discriminating gate, passes, so the necessity family is red by its joint clause |
800 replications x n=2,000 on two candidate sequences; worst standardized bias 0.0076, lowest coverage 0.9237 |
reporting, not gated: nine declared limits; the R comparison and its replay gates are pending; the lasso coverage row is red under X16-plugin-variance; the first run declared an unordered learner sequence, which the fixture README records |
outcome-adaptive point-treatment C-TMLE |
archived tlverse ctmle3 0.1.0 at commit a4ea77b, with contemporaneous tmle3 at 3a61005 and sl3 at 821ca89 |
binary treatment-specific means, ATE, marginal RR, and marginal OR, reporting point estimates and a generated-design plug-in diagnostic on the identity or log scale. The study fits oat_design="shared". The package publishes no interval and no p-value for either design, and F19 in the roadmap is the condition that reopens that |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired similarity and non-inferiority tests pass using 99% confidence bounds |
14/14 cells pass on bounded laws: the robustness contract and its control, three-size and root-n checks, calibration, null size, power, cross-fit controls, and the design-estimation pair |
800 replications x n=1,000 on one law; worst standardized bias 0.0513, lowest coverage 0.9425 |
reporting, not gated: every cell is green, and the policy means no test asserts that; ten declared limits, including binary, two-arm, GLM and non-cross-fitted parity, a paper that directly proves one treatment-specific mean rather than the package’s joint derived-target vector, a known continuous outcome support, and one unstratified split; the coverage and interval cells measure a diagnostic that the package does not publish as inference |
outcome-adaptive multi-arm C-TMLE |
archived R ctmle3 0.1.0 at commit a4ea77b, with tmle3 at 3a61005 and sl3 at 821ca89 |
three labelled treatment-arm means plus ATE, RR, and OR contrasts against the high arm, reporting point estimates and a generated-design plug-in diagnostic. The study fits oat_design="shared". The package publishes no interval and no p-value for either design, and F19 in the roadmap is the condition that reopens that |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
10/12 cells pass: the robustness contract, three-size and root-n checks, calibration, null, and power are reported with a generated-design pair; the two generated-design cells are red |
800 replications x n=1,500 on one law; worst standardized bias 0.0376, lowest coverage 0.9325 |
reporting, not gated: binary-outcome ordinary-GLM evidence; the generated-design contrast did not resolve; one unstratified split does the cross-fitting, with no cross-fitted primary result, simultaneous intervals, missingness, weights, clusters, fold repeats, or longitudinal treatment; the coverage and interval cells measure a diagnostic that the package does not publish as inference |
outcome-adaptive per-arm C-TMLE |
R drtmle 1.1.2 at commit 538a3a2 with adapt_g = TRUE, called once per arm on the binary recode of that arm |
the two-arm means, ATE, RR, OR, PAR and PAF, and the three-arm means with the ATE, RR and OR against arm 0, reporting point estimates and a generated-design plug-in diagnostic. The study fits the default oat_design="per_arm". The package publishes no interval and no p-value for it, and F19 in the roadmap is the condition that reopens that |
2/32 implementation-estimand tests against known truth pass using 99% confidence bounds; both implementations miss coverage alike |
comparator-backed: 16/16 paired similarity and non-inferiority tests pass using 99% confidence bounds |
12/25 cells pass: the controls, the robustness contract, power, the root-n slopes and the overfitting pair’s SE rule pass, while the cross-fitted tree fit carries a bias of about one empirical SD; the calibration, generated-design, size, three-size and band cells are red, and the design pinned to the truth calibrates |
800 replications x n=1,500 on each of two laws; worst standardized bias 0.0269, lowest coverage 0.8588 |
reporting, not gated: the declared revert cells are red, so every oat fit withholds its interval; eight declared limits, including a diagnostic standard error that is too small on both laws, a bias under a flexible outcome learner, parametric binary-outcome laws only, no missing outcomes, strata or clusters, and one unstratified split |
C-TMLE0 derivative covariate |
R ctmle 0.1.2 at commit 18de559 (r-ctmle): Python property evidence, with comparator runs pending. The R runs wait for beta-plans TRACKER item 18 |
the binary-treatment ATE from one TMLE update that adds the derivative of each arm’s clever covariate, at a declared lasso penalty pair, at the package’s own penalty cross-validation, and with a constant outcome regression; ordinary pointwise 95% Wald intervals of the TMLE status |
3/3 implementation-scenario tests against known truth pass using 99% confidence bounds |
0/0 paired comparisons: Python property evidence, with comparator runs pending |
21/21 cells pass: interval calibration in sample, cross-fitted, with three arms and with missing outcomes; cluster-robust inference against its IID control; null size; power; three sizes and two root-n rates; six double-robustness cells, including the both-wrong control; and targeting necessity |
800 replications x n=2,000 on one law and three scenarios; worst standardized bias 0.0401, lowest coverage 0.9275 |
reporting, not gated: seven declared limits; the R comparison and its parity witness are pending; the pipeline scenario will compare two pipelines; parity is binary, two-arm, complete-outcome, non-cross-fitted ATE with a lasso pair; the interval is measured under a correct outcome regression |
outcome-adaptive per-arm C-TMLE with strata and missing data (point estimates) |
R drtmle 1.1.2 at commit 538a3a2 with adapt_g = TRUE, DeltaY, DeltaA and stratify = TRUE, called once per arm on the binary recode of that arm, and once per stratum on the strata law |
the arm means and ATE, marginal and in each of three baseline strata; the two-arm means, ATE, RR and OR with a missing outcome or a missing treatment; and the three-arm means with the ATE, RR and OR against arm 0 with a missing outcome. The study reports point estimates and a generated-design plug-in diagnostic. The package publishes no interval for these fits, and F19 in the roadmap is the condition that reopens that |
20/62 implementation-estimand tests against known truth pass using 99% confidence bounds; both implementations miss coverage alike, with a short diagnostic standard error and an equivalent bias |
comparator-backed: 31/31 paired similarity and non-inferiority tests pass using 99% confidence bounds |
34/37 cells pass: all 32 robustness-contract cells, the outcome-wrong control included, and both root-n slopes pass; the three ladder rungs at n = 2,000, 4,000 and 8,000 have an equivalent bias and a coverage interval below the floor, and F19 owns them |
800 replications x n=2,250 on each of four laws; worst standardized bias 0.0926, lowest coverage 0.8575 |
reporting, not gated: every oat fit withholds its interval; eight declared limits, including a diagnostic standard error that is too small on the primary laws, a heavy-tailed standard error in small factor cells, which moved the ladder to n = 2,000 and above, a paired strata law with complete data, parametric binary-outcome laws only, and no clusters |
outcome-adaptive per-arm C-TMLE with a parametric outcome |
none claimed. R drtmle with adapt_g = TRUE reports the shipped curve, so no R program measures the stacked curve, and a zero-row equivalence artifact records the absence of a comparator. Python property evidence, no comparator |
the arm means, ATE, RR and OR of a binary outcome, and the arm means and ATE of a bounded outcome, from CTMLE(strategy="oat", oat_design="per_arm", cross_fit=False) with ParametricOutcome and ParametricMechanism. The interval is the stacked-equation curve. It covers the arm mean under a correct outcome model and its limit psi* otherwise |
5/5 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator measures the stacked curve |
41/41 cells pass: SE ratio and coverage calibrate on six laws and from n = 500 to 8,000, the type-I and power cells pass, and both negative controls read red as declared |
800 replications x n=1,500 for the primary rows and 800 to 63,600 replications for each property group; worst standardized bias 0.0116, lowest coverage 0.9437 |
gated; nine declared limits, including in sample only, two arms, complete unweighted data, two parametric learners, a covered limit psi* under a wrong outcome model, the default alpha only, and a reported covariance that no band gates |
cross-fitted outcome-adaptive per-arm C-TMLE with a parametric outcome |
none claimed. R drtmle with adapt_g = TRUE reports the shipped curve, so no R program measures the stacked curve, and a zero-row equivalence artifact records the absence of a comparator. Python property evidence, no comparator |
the arm means, ATE, RR and OR of a binary outcome from CTMLE(strategy="oat", oat_design="per_arm", cross_fit=True, n_folds=5) with ParametricOutcome and ParametricMechanism, at one draw, and the arm means and ATE at the median of three draws. The interval is the V-fold stacked-equation curve. It covers the arm mean under a correct outcome model and its limit psi* otherwise |
5/5 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator measures the stacked curve |
32/32 cells pass: SE ratio and coverage calibrate at five folds from n = 500 to 8,000 and at the median of three draws, the generated-design pair and the sharp null pass, and both shipped-curve negative controls read red as declared |
1,200 replications x n=1,500 for the primary rows and 1,200 to 64,800 replications for each property group; worst standardized bias 0.0181, lowest coverage 0.9408 |
gated; ten declared limits, including five folds only, one and three draws, two arms with a binary outcome and complete unweighted data, two parametric learners, a covered limit psi* under a wrong outcome model, the default alpha only, balanced folds, a split-adjustment term that a unit test pins, and no band |
K-arm, weighted and population-target outcome-adaptive per-arm C-TMLE with a parametric outcome |
none claimed. R drtmle with adapt_g = TRUE reports the shipped curve, so no R program measures the stacked curve, and a zero-row equivalence artifact records the absence of a comparator. Python property evidence, no comparator |
the three-arm means, ATE, RR and OR, in sample and cross-fitted, the two-arm means and ATE under a known fixed weight, in sample and cross-fitted, the three-arm means and ATE under that weight cross-fitted, the population targets ey_obs, par and paf, and the simultaneous band over three declared estimate sets, from CTMLE(strategy="oat", oat_design="per_arm") with ParametricOutcome and ParametricMechanism. The interval is the stacked-equation curve. It covers the arm mean under a correct outcome model |
11/11 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator measures the stacked curve |
73/73 cells pass: SE ratio and coverage calibrate at three arms, under the known weight and for the population targets, all three bands cover inside the coverage band, the generated-design pair and the three-arm sharp null pass, and all nine negative controls read red as declared |
1,600 replications x n=1,500 per law for the primary rows and 1,600 to 80,000 replications for each property group; worst standardized bias 0.0173, lowest coverage 0.9481 |
gated; fourteen declared limits, including three arms only, five folds only, one draw and a refusal of repeats with the default band, a known fixed weight, a reference arm per group, the three measured band sets, a binary outcome with complete data, two parametric learners, a generated-design pair predicted at 0.99321 against a target of 0.99730, and no comparator |
DR-TMLE for binary complete data |
R drtmle 1.1.2 at commit 538a3a2, on a digest-pinned R 4.5.2 |
treatment-specific means and ATE under outcome-correct, treatment-correct, and both-correct nuisance regimes; one package-generated unstratified ten-fold draw supplied to both implementations, and pointwise 95% Wald intervals |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds; every cell concludes equivalent and none establishes coverage superiority |
19/22 cells pass their own and family verdicts: both-correct behavior, the both-wrong control, calibration, three-size behavior, two root-n rates, and both bias-contraction rates are green; both one-correct robustness cells fail at n=1,500, and so does one contraction rung’s coverage |
800 replications x n=3,000 on three regimes; worst standardized bias 0.1557, lowest coverage 0.9275 |
reporting, not gated: binary complete-data GLM evidence only, with one-correct cells on an H1 law; two robustness cells, one contraction rung, and fit-health audits are red |
multi-arm point-treatment DR-TMLE |
R drtmle 1.1.2 at commit 538a3a2, on a digest-pinned R 4.5.2, with one identical package-generated unstratified five-fold assignment and shared initial nuisance predictions |
three labelled treatment-arm means plus ATE, RR, and OR contrasts against the high arm; corrected pointwise 95% Wald intervals |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
18/22 cells pass their own and family verdicts: double robustness with its both-wrong control, the n=2,000 and n=8,000 size cells, both root-n rates, a three-size bias-contraction ladder with its own control, and both fitted contraction slopes at 73,000 replications per outer rung after the RM33 targeted seed repair, which read contracts (over the first 600 replications of each rung the n=2,000 bias interval covers zero, which would read contracts at the noise floor); the treatment-correct level cell, one contraction rung, the n=500 coverage bound, and the tight calibration band are red |
800 replications x n=2,000 on one law; worst standardized bias 0.0461, lowest coverage 0.9513 |
reporting, not gated: the source theorem is binary and this measures the packages’ armwise extension; two cleverly and four R score audits and four property cells are red, with one law, pooled reduced cross-fitting, pointwise intervals, a correct GLM mechanism in the treatment-correct cells, and no flexible learners, positivity stress, missingness, weights, clusters, fold repeats, or longitudinal treatment |
ordinary point-treatment TMLE with MAR outcomes |
R tmle 2.1.1 on a digest-pinned R 4.5.2 image |
treatment-specific means and ATE under an observational binary-treatment law with treatment- and covariate-dependent outcome observation; ordinary pointwise 95% Wald intervals |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
19/19 cells pass: three-nuisance MAR robustness with two single-mechanism controls, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, targeting, and a complete-case mutation |
800 replications x n=2,000 on one law; worst standardized bias 0.0877, lowest coverage 0.9425 |
five declared limits; binary observational MAR oracle evidence with ordinary fitting and pointwise intervals; no flexible learning, weights, clusters, simultaneous bands, missing treatment, MNAR, or DR-TMLE correction |
ordinary missing-outcome natural-course TMLE |
R tmle 2.1.1 population-mean path on a digest-pinned R 4.5.2 image, with the same supplied outcome and response predictions; for the continuous law, two planted nonrespondent rows set the R outcome scale, and a probe checks that workaround on every continuous replication |
observed outcome mean under the natural course on binary and bounded-continuous observational MAR laws; ordinary pointwise 95% Wald intervals |
4/4 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 2/2 paired comparisons pass using 99% confidence bounds; R tmle is equivalent on both laws, conditional on the supplied nuisance predictions |
16/16 cells pass: two-nuisance robustness and its both-wrong control, three-size and root-n checks, exact-EIF efficiency and calibration controls, targeting, and a complete-case mutation |
800 replications x n=2,000 on two laws; worst standardized bias 0.0653, lowest coverage 0.9350 |
six declared limits; binary and bounded-continuous MAR finite-law evidence with an exact binary outcome regression and a correctly specified affine continuous-outcome learner; a comparator conditional on supplied nuisances; no flexible learning, weights, clusters, simultaneous bands, missing treatment, MNAR, sensitivity analysis, or longitudinal data |
stacked missing-outcome natural-course CV-TMLE |
R tmle 2.1.1 population-mean path with the same supplied stitched out-of-fold outcome and response predictions; tmle3’s generic outcome fit can use the Delta = 0 pseudo-outcomes, zEpid targets inside each fold, and Newey and Robins (2018) is a different estimator |
observed outcome mean under one binary-outcome observational MAR law; one package-generated unstratified ten-fold draw, pooled targeting, whole-sample evaluation, and pointwise 95% Wald intervals |
2/2 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 1/1 paired comparison passes using 99% confidence bounds; R tmle is equivalent conditional on the supplied nuisance predictions |
7/7 cells pass: coverage and SE calibration with depth-five trees that fit the saturated six-cell model, a shrunken-SE control, a fully grown noise-covariate tree fitted cross-fit versus in sample, both union-model rescue directions, and a both-wrong bias control |
800 replications x n=2,000; worst standardized bias 0.0105, lowest coverage 0.9537; paired point difference -1.770e-11; fully grown tree coverage 0.9500 cross-fitted versus 0.6950 in sample |
seven declared limits; one binary finite law, one ten-fold draw, calibration rows with a correct saturated tree model, pointwise inference, comparator conditional on supplied nuisances with native inference conventions, and no alternative split, treatment, outcome, weighting, clustering, stratification, intermediate, MNAR, or longitudinal surface |
stacked arm-indexed missing-outcome CV-TMLE |
R tmle 2.1.1 with the same supplied stitched out-of-fold outcome, treatment, and response predictions: its two-arm path for two arms, and its population-mean path once per arm for three arms, with a probed planted-row scale workaround for bounded continuous outcomes; tmle3’s generic outcome fit can use the Delta = 0 pseudo-outcomes, zEpid targets inside each fold, and Newey and Robins (2018) is a different estimator |
arm means and reference-arm ATE under four MAR laws with two or three arms and a binary or bounded continuous outcome, plus RR and OR for the binary laws; one package-generated unstratified ten-fold draw, pooled targeting, whole-sample evaluation, pointwise 95% Wald intervals with ratios on the log scale, and the default simultaneous band |
44/44 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 22/22 paired comparisons pass using 99% confidence bounds; R tmle is equivalent conditional on the supplied nuisance predictions |
84/84 cells pass: coverage and SE calibration for all 22 estimands with depth-five trees, their shrunken-SE controls, joint coverage of the simultaneous band with a pointwise joint control, single-wrong-nuisance rescues with a Q-and-pi-wrong bias control, and a fully grown noise-covariate tree fitted cross-fit versus in sample on each law |
800 replications x n=2,000 on four laws; worst standardized bias 0.0636, lowest coverage 0.9300 |
nine declared limits; four finite laws, one ten-fold draw with one fold assignment for every replication, bound checks in the primary fits only, calibration rows with a correct saturated tree model, the default band only, comparator conditional on supplied nuisances with a scale workaround probed on one replication, and no alternative split, targeting, estimand, weighting, clustering, stratification, bootstrap, intermediate, MNAR, or C-TMLE and DR-TMLE surface |
missing-outcome attributable-effect TMLE |
R tmle 2.1.1, composed: its population-mean path once for the natural course and once per arm with the indicator 1{A = a} Delta, with the same supplied predictions, and PAR and PAF formed from R’s own influence curves; a probed planted-row scale workaround for the bounded continuous outcome; tmle3’s tmle_PAR has no response score, and zEpid targets inside each fold |
natural-course mean, arm means, PAR and PAF (PAR alone for the continuous outcome) under MAR on two- and three-arm laws, in sample and with one ten-fold stacked draw, with fixed weights and with 100 clusters; pointwise 95% Wald intervals and the default simultaneous band |
62/62 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 31/31 paired comparisons pass using 99% confidence bounds; the composed R tmle is equivalent conditional on the supplied nuisance predictions |
32/32 cells pass: union-model rescues with reference-path, natural-course and product-only controls, three-size and root-n checks, exact-EIF efficiency and calibration of PAR and PAF with shrunken, noise and cross-covariance-dropping controls, targeting and complete-case controls, and joint coverage of five default bands |
800 replications x n=2,000 on seven scenarios; worst standardized bias 0.0568, lowest coverage 0.9350 |
six declared limits; finite-support laws with the law’s nuisances in sample and saturating trees stacked, no cross-fitted continuous, clustered or weighted fit, no strata, bootstrap, DR-TMLE or C-TMLE, the product-only rescue measured as a control only, and no MAR-violation check |
ordinary controlled direct-effect TMLE |
R tmle 2.1.1 on a digest-pinned R 4.5.2 image, with each requested intermediate level recoded to the valid first result |
treatment-specific means, ATE, RR, and OR at both levels of a binary intermediate under MAR outcome observation; ordinary pointwise 95% Wald intervals |
20/20 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 10/10 paired similarity and non-inferiority tests pass using 99% confidence bounds |
36/36 cells pass: four-nuisance robustness with three single-mechanism controls at both levels, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, and targeting necessity |
3,200 replications x n=2,000 from one observed stochastic law, with both intervention levels evaluated on each sample; worst standardized bias 0.0685, lowest coverage 0.9216 |
seven declared limits, including exact nuisances, pointwise inference, the frozen native R second-result defect, and no claim for natural effects or general longitudinal mediation |
randomized missing-outcome DR-TMLE |
R drtmle 1.1.2 at commit 538a3a2, on a digest-pinned R 4.5.2 image; the comparison is limited to its joint-mechanism both-correct limit |
treatment-specific means and ATE in a randomized trial with treatment- and covariate-dependent outcome observation; corrected pointwise 95% Wald intervals |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
14/14 cells pass: both one-correct drift directions, the coarsened both-wrong control, exact-EIF efficiency and calibration controls, bias, coverage, and SE checks at three sizes, two root-n rates, and a direct five-reduction score mutation |
800 replications x n=2,000 on one law; worst standardized bias 0.0631, lowest coverage 0.9500 |
six declared limits; binary randomized MAR oracle evidence with pointwise intervals and no type-I, power, or targeting cell; R parity excludes the separate five-reduction cycle, the study excludes observational assignment, flexible learning, weights, clusters, simultaneous bands, missing treatment, MNAR, and other DR-TMLE compositions, and the consistent nuisance is an exact table |
randomized multi-arm missing-outcome DR-TMLE |
R drtmle 1.1.2 at commit 538a3a2, on a digest-pinned R 4.5.2 image; the comparison is limited to its joint-mechanism both-correct limit |
three labelled arm means plus ATE, RR, and OR contrasts against the high arm in a three-arm randomized trial with known W-stratified assignment and treatment- and covariate-dependent outcome observation; corrected pointwise 95% Wald intervals and the default simultaneous band |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
20/22 cells pass: both one-correct drift directions for both contrasts, the coarsened both-wrong controls, exact-EIF efficiency and calibration controls, size and power, three sizes, two root-n rates, the five-reduction score pair, and the pointwise joint control; the calibration cell and the simultaneous band, which read one draw set, are red under F4-calibration-draws |
800 replications x n=2,000 on one law; worst standardized bias 0.0467, lowest coverage 0.9300 |
six declared limits; reporting, not gated: the source theorem is stated for one arm indicator and this measures the armwise stack; known treatment probabilities only, oracle primaries, one law, no observational assignment, weights, clusters, missing treatment, MNAR, or cross-fitting, and the consistent nuisance an exact table |
observational missing-data DR-TMLE with a composite indicator |
R drtmle 1.1.2 at commit 538a3a2, on a digest-pinned R 4.5.2 image, with an NA treatment and the oracle outcome regressions and composite mechanism |
arm means plus ATE, RR, and OR contrasts at two and three arms, with an observational MAR outcome and with a MAR treatment whose recording depends on the treatment; the composite TMLE’s ATE; corrected pointwise 95% Wald intervals and the default simultaneous band |
40/42 implementation-estimand tests against known truth pass using 99% confidence bounds; ey[high] of the three-arm law is red in both implementations under composite-high-arm |
comparator-backed: 21/21 paired similarity and non-inferiority tests pass using 99% confidence bounds |
39/39 cells pass: both one-correct drifts and a both-wrong control for each contrast, the composite TMLE with a regime mean and an arm MSM slope, three sizes, two root-n rates, exact-EIF efficiency and calibration controls, the simultaneous bands, size and power, the correction cycle, and a complete-case control |
800 replications x n=2,000 on three laws; worst standardized bias 0.0565, lowest coverage 0.9225 |
five declared limits; reporting, not gated: one binary outcome and one discrete covariate with oracle primaries, no coverage cell for partial guards, the bivariate reduction, weights or clusters, no instrument for the treatment condition, and the consistent nuisance an exact table |
ordinary point-treatment MSM projection |
R tmle3 0.2.0 Param_MSM at commit ed72f8a, with sl3 at 0e8f236 and the same fixed custom projection measure |
identity-link intercept, treatment, and baseline coefficients from an unsaturated projection with fixed nonuniform weights |
6/6 implementation-coefficient tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
42/42 cells pass: coefficient-wise double robustness, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, targeting, and a uniform-weight projection control |
800 replications x n=2,000 on one law; worst standardized bias 0.0122, lowest coverage 0.9425 |
six declared limits; fixed identity-link ordinary fitting on two arms, with no continuous-dose or adaptive projection measure claim |
ordinary deterministic point-treatment regimes |
R lmtp 1.5.4 at commit f04a2b4, on the digest-pinned R 4.5.2 image, with mtp=FALSE |
the never-treated mean, a covariate-dependent dynamic-rule mean, and their contrast; ordinary pointwise 95% Wald intervals |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired similarity and non-inferiority tests pass using 99% confidence bounds |
20/20 cells pass: double robustness and its control, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, targeting, a wrong-rule control, and exact static reduction |
800 replications x n=2,000 on one law; worst standardized bias 0.0517, lowest coverage 0.9500 |
five declared limits; binary ordinary GLM evidence with one fixed rule, pointwise intervals, and no missingness, weights, clusters, cross-fitting, simultaneous bands, or longitudinal treatment |
CV-TMLE of the fold-local learned-rule value |
none claimed. R tmle3mopttx at 3c0e843 weights the folds by their size and centres the curve at the pooled estimate, and no other maintained package fits this fold-average target, so a zero-row equivalence artifact records the absence |
ey_learned_rule[learned rule], the average over ten outer folds of the value of the plug-in rule that each fold’s training rows learn; one pooled fluctuation, fold plug-ins weighted by \(1/V\), the cross-validated variance, and pointwise 95% Wald intervals, each read against the truth of its own replication
|
2/2 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
12/12 cells pass: interval calibration with shrunken-SE and noise controls, bias, coverage, and SE checks at three sizes, two root-n rates, targeting necessity, and a fold-locality mutation that learns each rule on its own validation rows |
6,000 replications x n=2,000 on two laws that meet the limiting-rule condition; worst standardized bias 0.0168, lowest coverage 0.9413 |
eight declared limits; two regular laws only, because the interval under-covers at the exceptional and weak-blip laws that the boundary study reads, one logistic learner class for coverage, binary treatment and outcome, equal folds, and no weights, clusters, missing outcomes, repeats, bootstrap, or strata |
ordinary known stochastic point-treatment regimes |
pinned R lmtp 1.5.4 at commit f04a2b4, reached through the shared point adapter because its shift interface reads the natural treatment value and this regime does not |
the never-treated mean, a nondegenerate known stochastic-regime mean, and their contrast; ordinary pointwise 95% Wald intervals |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 3/3 paired tests conclude equivalent on point similarity, RMSE, coverage, and SE calibration, all using 99% confidence bounds |
18/18 cells pass: double robustness and its control, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, targeting, and a uniform-density mutation |
800 replications x n=2,000 on one law; worst standardized bias 0.0588, lowest coverage 0.9375 |
eight declared limits; binary ordinary known-density evidence with pointwise intervals, a reference that draws once from the declared density rather than integrating over it, and no missingness, weights, clusters, cross-fitting, simultaneous bands, or longitudinal treatment |
ordinary continuous modified treatment policies |
R lmtp 1.5.4 at commit f04a2b4, on the digest-pinned R 4.5.2 image, with the analytic conditional-normal density ratio |
natural-course, uncapped +0.25-shift, and actively capped +0.5-shift means plus both natural-course contrasts; ordinary pointwise 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 4/5 paired similarity and non-inferiority tests pass using 99% confidence bounds; the uncapped contrast is similar but its relative SE-calibration bound is inconclusive |
21/21 cells pass: double robustness and its control, three-size and root-n checks, repeated-sampling calibration, null and power, targeting, density-ratio direction, the active cap, and exact natural-course identity |
800 replications x n=2,000 on one law; worst standardized bias 0.0564, lowest coverage 0.9437 |
nine declared limits; reporting, not gated, for one inconclusive paired comparison; ordinary quadratic-regression evidence with an uncapped support warning and no cross-fitting, simultaneous bands, missingness, weights, or clusters |
ordinary point modified treatment policies |
R lmtp 1.5.4 at commit f04a2b4, through a modified-treatment-policy adapter that reads the policy dose and the density ratio this package computed |
natural-course, Scale(1.25, cap=5.5), Piecewise and declared ModifiedPolicy means plus the three natural-course contrasts; ordinary pointwise 95% Wald intervals |
14/14 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 7/7 paired similarity and non-inferiority tests pass using 99% confidence bounds; the two fluctuations differ by construction, so the pairs read under the default margins |
26/27 cells pass: double robustness and its control, three-size and root-n checks, exact-EIF efficiency and calibration of four contrasts with their controls including the classifier ratio route, null and power, targeting and the declared inverse; the halve calibration cell is red under reporting, with a conservative standard error (SE ratio 1.043) |
1,000 replications x n=2,000 on one law; worst standardized bias 0.0722, lowest coverage 0.9490 |
six declared limits; an oracle density at 160 bins; no cell for a categorical treatment, a risk-ratio tilt or a randomized policy |
incremental propensity-score interventions |
R npcausal at commit 56a5ac1, the implementation Kennedy (2019) publishes; its influence values carry the treatment-mechanism derivative |
three incremental-odds means and two natural-course contrasts under fixed multipliers 0.5, 1, and 2; cleverly pointwise 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired tests conclude equivalent on point similarity, RMSE, coverage, and SE calibration, all using 99% confidence bounds |
21/21 cells pass: the mechanism requirement, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, both targeting equations, a treatment-score mutation, and natural-course identity |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0546, lowest coverage 0.9363 |
seven declared limits; binary ordinary evidence over a saturated covariate design, three fixed tilts, pointwise inference, and no missingness, weights, clusters, cleverly cross-fitting, simultaneous bands, multinomial treatment, or longitudinal intervention |
ordinary end-of-study longitudinal TMLE |
R ltmle 1.3-0 at source commit 338c029, with the tarball pinned by sha256 |
three censored two-time-point regimen means and two correlated contrasts, including a dynamic rule; ordinary fitting and pointwise identity-scale 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired similarity and non-inferiority tests pass using 99% confidence bounds |
32/32 cells pass: longitudinal double robustness and controls, targeting controls, static and dynamic root-n rates, exact-EIF efficiency and calibration controls, confounded-null type-I error, power, and the default band over five regimen parameters with its pointwise control |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0180, lowest coverage 0.9387 |
seven declared limits; this ordinary row excludes survival, competing risks, MSM, weights, clustering, flexible learning, and cross-fitting |
ordinary weighted end-of-study longitudinal TMLE |
R ltmle 1.3-0 at source commit 338c029, with fixed inverse-selection weights passed through observation.weights= |
three weighted censored two-time-point regimen means and two correlated contrasts, including a dynamic rule; exact-size selected samples, ordinary fitting, and pointwise identity-scale 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired tests conclude equivalent to numerical-solver precision using 99% confidence bounds |
30/36 cells pass: the complete reporting table covers longitudinal double robustness, rate, efficiency, calibration, null, power, targeting, target-weight, and learner-weight instruments |
3,200 replications x n=2,000 selected rows on one law; worst standardized bias 0.0295, lowest coverage 0.9347 |
seven declared limits; this is reporting evidence with six red or marginal property cells, known fixed weights, one selection law, ordinary weighted GLMs, pointwise inference, and no survival, competing risks, MSM, clustering, or cross-fitting |
ordinary longitudinal MSM projection |
four ordinary R ltmle 1.3-0 regimen fits at source commit 338c029, followed by the declared projection of their estimates and joint influence curves |
identity-link intercept and treatment-duration coefficients over four censored static and dynamic plans under fixed nonuniform weights |
4/4 implementation-coefficient tests against known truth pass using 99% confidence bounds |
comparator-backed: 2/2 paired similarity and non-inferiority tests pass using 99% confidence bounds |
30/30 cells pass: coefficient-wise double robustness, three-size and root-n checks, exact-EIF efficiency and calibration controls, null and power, an untargeted recursion, and a uniform-weight projection control |
800 replications x n=2,500 on one law; worst standardized bias 0.0376, lowest coverage 0.9337 |
seven declared limits; ordinary identity-link pointwise inference over four plans; the cross-fitted fit has its own row |
cross-fitted longitudinal MSM projection |
four cross-fitted R lmtp 1.5.4 regimen fits at commit f04a2b4 on the realized folds, followed by the declared projection of their estimates and joint influence curves; lmtp fluctuates on each training fold and targets each plan alone |
identity-link intercept and treatment-duration coefficients over four censored static and dynamic plans under fixed nonuniform weights, at five outer folds with one stacked fluctuation per node |
4/4 implementation-coefficient tests against known truth pass using 99% confidence bounds |
comparator-backed: 2/2 paired similarity and non-inferiority tests pass using 99% confidence bounds |
37/37 cells pass: coefficient-wise double robustness, three-size and root-n checks, exact-EIF efficiency and calibration on the identity and logit links, null and power, targeting and projection-measure controls, a fully grown tree pair, and a simultaneous band. Two in-sample distance rows are reported rather than gated |
800 replications x n=2,500 on one law; worst standardized bias 0.0696, lowest coverage 0.9375 |
seven declared limits; one law, with survival, competing-risk, weighted and clustered projections covered by fast-tier exact identities only |
cross-fitted end-of-study longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4, the maintained package implementing a cross-fitted sequential regression; R ltmle has no cross-fitting and cannot witness this construction |
three censored two-time-point regimen means and two correlated contrasts, including a dynamic rule; one exact five-fold assignment and pointwise identity-scale 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired similarity and non-inferiority tests pass using 99% confidence bounds |
32/32 cells pass their own and family verdicts: longitudinal double robustness and controls, targeting controls, static and dynamic root-n rates, exact-EIF efficiency, calibration, null, power, and the flexible-tree cross-fit pair under the pooled update |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0324, lowest coverage 0.9431 |
reporting, not gated: seven declared limits; the pinned comparator runs a training-fold fluctuation rather than the pooled update, agreement is distributional, one fixed split, pointwise inference, the mechanism is supplied rather than estimated, and no claim for survival, competing risks, MSMs, weights, or clustering |
cross-fitted weighted end-of-study longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4 with pinned ife 0.2.3, fixed inverse-selection task weights, an explicit weighted-GLM learner adapter, exact density ratios, and identical rowwise five-fold assignments |
three weighted censored two-time-point regimen means and two correlated contrasts, including a dynamic rule; exact-size selected samples and pointwise identity-scale 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 2/5 paired tests conclude equivalent using 99% confidence bounds; the three regimen-mean gates conclude underpowered. Two compare lmtp’s native Horvitz-Thompson inference with cleverly’s centered Hájek convention, and ey_regimen[never] went red under the pooled update |
34/36 cells pass their own and family verdicts: the complete reporting table covers longitudinal double robustness, rate, efficiency, calibration, null, power, targeting, target-weight, and learner-weight instruments; the static calibration cell went red under the pooled update |
800 replications x n=2,000 selected rows on one law; worst standardized bias 0.0572, lowest coverage 0.9300 across both implementations |
eight declared limits; this is reporting evidence with a marginal failed both-wrong control, a static calibration cell that misses its efficiency band, and three paired gates that conclude underpowered, known fixed weights, one selection law and split, weighted GLMs, pointwise inference, and no survival, competing risks, MSM, or clustering |
ordinary categorical longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4, with exact categorical mechanisms and one all-row fold |
five deterministic regimen means and four contrasts across three labelled treatment levels at two nodes; ordinary pointwise 95% Wald intervals |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
34/34 cells pass: sequential double robustness, three-size and root-n checks, exact-EIF efficiency and calibration, null, power, targeting, third-arm probability, and dynamic-rule controls |
2,000 replications x n=2,000 on one law; worst standardized bias 0.0399, lowest coverage 0.9300 |
eight declared limits; complete two-node, three-level GLM evidence for deterministic plans; no censoring, missingness, stochastic policies, continuous dose, survival, competing risks, weights, clusters, fold repeats, simultaneous bands, or flexible primary learner parity |
ordinary known stochastic categorical longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4, through a policy adapter that copies each unit four times, with exact policy ratios and one all-row fold |
two regimen means and their contrasts against low under known stochastic policies, plus the low mean, across three labelled levels at two nodes; ordinary pointwise 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 5/5 paired similarity and non-inferiority tests pass using 99% confidence bounds; the policy mean has a mean paired difference of 4.9e-11 and a largest of 6.3e-10; the low and contrast rows have a mean absolute difference of about 9e-3 and a largest of 0.042 |
29/30 cells pass: double robustness, three-size and root-n checks, exact-EIF efficiency and calibration of the policy contrast, a survival policy and a working model over policy cells, null, targeting, policy and randomizer-projection controls, a fully grown tree pair and a simultaneous band; the power cell is red under reporting, underpowered at its declared size (exact power 0.5365) |
2,000 replications x n=2,000 on one law; worst standardized bias 0.0603, lowest coverage 0.9355 |
eight declared limits; one law with policies on a quarter grid; no study cell for weights, clusters, competing risks, a continuous outcome or fold repeats |
longitudinal modified treatment policies |
R lmtp 1.5.4 at commit f04a2b4, through a modified-treatment-policy adapter that reads the policy values and the per-node density ratio this package computed, with identical folds |
regimen means and contrasts under continuous-dose shifts, a scale and a history-reading policy at two nodes, categorical policies at six levels, and a risk-ratio tilt of 0.25; ordinary pointwise 95% Wald intervals |
44/44 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 22/22 paired similarity and non-inferiority tests pass using 99% confidence bounds; the one-fold continuous, categorical and tilt pairs are exactness pairs, and the cross-fitted pairs compare two fluctuation constructions |
39/40 cells pass: double robustness, the root-n ladder, exact-EIF efficiency and calibration of continuous, classifier-route, vector, randomized, survival and working-model cells, null, power, targeting, the declared inverse, the cross-fit overfitting pair and a simultaneous band; the categorical calibration cell is red as a finite-sample limit under reporting |
1,000 replications x n=2,000 per scenario on four laws; worst standardized bias 0.0731, lowest coverage 0.9380 |
six declared limits; an oracle density whose bin count grows with n |
vector modified treatment policies |
Python property evidence, with comparator runs pending. The pending comparator is R lmtp 1.5.4 at commit f04a2b4, with vector trt nodes and mtp = TRUE, through tests/canonical/lmtp_vector_adapter.R. A later change registers the runs |
policy means and contrasts on a vector of a binary flag and one or two doses: the point TMLE and the two-node LTMLE, on the joint induced density route and the stacked classifier route; ordinary pointwise 95% Wald intervals |
20/20 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 paired tests: Python property evidence, with comparator runs pending |
36/36 cells pass: exact-EIF efficiency and interval calibration on both routes with shrunken-SE and noise controls, double robustness in four nuisance configurations, and three necessity pairs that cut the preimage sum, drop the Jacobian and read the flag map at the observed dose |
1,000 replications x n=2,000 per scenario on four laws; worst standardized bias 0.0464, lowest coverage 0.9320 |
seven declared limits; the lmtp runs are pending; an oracle density and a one-feature oracle classifier |
modified treatment policies at declared boundaries |
Python property evidence, with comparator runs pending. The pending comparator is R lmtp 1.5.4 at commit f04a2b4, through tests/canonical/lmtp_mtp_adapter.R and tests/canonical/lmtp_sdr_adapter.R, reading the policy values and the per-node ratio this package wrote. A later change registers the runs |
regimen means and contrasts under continuous-dose policies at two nodes whose declared support depends on the history: a map that never binds, an upper boundary that binds, a varying Jacobian, two inverse branches, and an interval of doses sent to a declared atom; LTMLE and SDR on the oracle density route; ordinary pointwise 95% Wald intervals |
24/24 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 paired tests: Python property evidence, with comparator runs pending |
28/29 cells pass: interval calibration with shrunken-SE and noise controls, and four necessity pairs that drop the Jacobian, sum one inverse branch, omit the atom mass and set the ratio to one; the p4_sdr calibration cell is red under reporting |
1,000 replications x n=5,000 per scenario on two laws; worst standardized bias 0.0579, lowest coverage 0.9290 |
eight declared limits; the lmtp runs are pending; an oracle density, a declared support that is the true support, and two nodes |
boundary-binding price plans over follow-up lengths |
Python property evidence, with comparator runs pending. The pending comparator is R lmtp 1.5.4 at commit f04a2b4, through tests/canonical/lmtp_mtp_adapter.R and tests/canonical/lmtp_sdr_adapter.R. The runs need R in the lmtp container, and a later change registers them |
regimen means and contrasts for a price raised by 12% (upper boundary binds) and lowered by 15% (lower boundary binds) at 2, 3 and 5 nodes, on a support that depends on engagement; LTMLE and SDR on the exact classifier ratio; ordinary pointwise 95% Wald intervals |
26/30 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 paired tests: Python property evidence, with comparator runs pending |
39/42 cells pass: interval calibration with shrunken-SE and noise controls on the exact ratio, a learned density and a screened classifier, a revenue outcome, cross-fitted cells, a double-robustness pair and two ratio necessity pairs; the learned-density cells are red as conservative intervals under reporting |
1,000 replications x n=5,000 per scenario on one law; worst standardized bias 0.0444, lowest coverage 0.9160 |
six declared limits; the lmtp runs are pending; the tau = 5 price_down rows are experimental, the learned-density cells report, and the cross-fold leak pair is not registered |
tuned caps and their population and fold-average targets |
none claimed. No maintained package fits the tuning term of a tuned cap, so a zero-row equivalence artifact records the absence |
the mean under a one-unit shift capped at a dose quantile or a linear quantile regression of the dose, at two nodes, for the population target and the fold-average target; LTMLE and SDR; ordinary pointwise 95% Wald intervals |
2/2 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
24/24 cells pass: interval calibration of six tuned-cap positives with shrunken-SE and noise controls, and the tuning-term and target controls that leave the band |
1,000 replications x n=2,000 per scenario on one law; worst standardized bias 0.0208, lowest coverage 0.9470 |
five declared limits; reporting, not gated: one law, the oracle density on one-fold cells, two tuners and no comparator |
sequentially doubly robust longitudinal estimation |
R lmtp 1.5.4 at commit f04a2b4: cf_sdr and theta_dr(is_sdr = TRUE), reading the per-node ratios and the fold assignment this package wrote |
regimen means and contrasts of static and dynamic plans with censoring, survival risks at two horizons, continuous-dose policies and known stochastic categorical policies; pointwise 95% Wald intervals |
60/60 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 30/30 paired similarity and non-inferiority tests pass using 99% confidence bounds; the continuous-dose and mix pairs are exactness pairs, and the cross-fitted pairs compare two constructions |
54/56 cells pass: sequential double robustness in four patterns with two controls, the root-n ladder, exact-EIF efficiency and calibration of end-of-study, survival, competing-risk, policy, modified-treatment-policy and MSM cells, calibration of weighted and clustered cells, null, power, the cross-fit overfitting pair and a simultaneous band; the both_correct cell at n = 2,000 and the band are red as finite-sample limits under reporting |
1,000 replications x n=2,000 per scenario on four laws; worst standardized bias 0.0871, lowest coverage 0.9340 |
eight declared limits; the known mechanism in every primary scenario; no multiply robust interval |
longitudinal incremental propensity-score interventions |
Python property evidence, with comparator runs pending. The declared comparator is R npcausal 0.1.0 ipsi() at commit 56a5ac11, and its runs have not registered |
three incremental-odds means and two natural-course contrasts under multipliers 0.5, 1 and 2, on a three-node law with a continuous end-of-study outcome, at two and five folds; cleverly pointwise 95% Wald intervals |
10/10 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; Python property evidence, with comparator runs pending |
36/36 cells pass: exact-efficiency calibration with shrunken-SE and noise controls at seven labels, the root-n ladder and rate, null and power, a simultaneous band, the mechanism-term mutation, single robustness and targeting necessity |
1,000 replications x n=96,000 per scenario on one law; worst standardized bias 0.1311, lowest coverage 0.9350 |
eight declared limits; one law family, saturated cell means, no npcausal verdict yet, two and five folds, and no dropout, survival, weights or clusters |
longitudinal incremental propensity-score interventions under dropout |
none claimed. No maintained package ships the dropout construction, so a zero-row equivalence artifact records the absence |
end-of-study plan means and natural-course contrasts under dropout, and survival risks at horizons 1 to 3 with their contrasts, at one and five folds; cause-1 incidence, a working-model slope, an informative weight, clusters and a simultaneous band in the property cells; cleverly pointwise 95% Wald intervals |
40/40 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator ships this construction |
24/25 cells pass: exact-efficiency calibration with shrunken-SE and noise controls at seven labels, a simultaneous band, cluster-robust inference against an independent-unit control; the complete-case cell is reported only; the competing-risk calibration cell is red on the conservative side under reporting |
1,000 replications x n=80,000 per scenario on four scenarios of two laws; worst standardized bias 0.0843, lowest coverage 0.9350 |
eight declared limits; dropout at random given the history, saturated cell means, three nodes, and no comparator |
categorical incremental policies |
none claimed. No maintained package ships a categorical odds tilt with its influence curve, so a zero-row equivalence artifact records the absence |
three-level odds-tilt plan means and natural-course contrasts for point TMLE, in sample and at five folds, and for two-node IPSI with and without dropout at five folds; cleverly pointwise 95% Wald intervals |
20/20 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator ships this construction |
28/28 cells pass: exact-efficiency calibration with shrunken-SE and noise controls at six labels, necessity controls for the mechanism term, softmax-update targeting against an outcome-only control, type I error and power, single robustness; the one-fold, constant-mechanism and complete-case cells are reported only |
1,000 replications x n=8,000 per scenario on four scenarios of two laws; worst standardized bias 0.1422, lowest coverage 0.9350 |
eight declared limits; three levels per node, saturated learners, two nodes, dropout at random given the history, and no comparator |
cross-fitted categorical longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4, with exact categorical mechanisms and the identical rowwise five-fold assignment |
five deterministic regimen means and four contrasts across three labelled treatment levels at two nodes; pointwise 95% Wald intervals |
18/18 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 9/9 paired similarity and non-inferiority tests pass using 99% confidence bounds |
36/36 cells pass their own and family verdicts: sequential double robustness, three-size and root-n checks, exact-EIF efficiency and calibration, null, power, targeting, third-arm probability, dynamic-rule, and flexible-tree cross-fit controls |
4,000 replications x n=2,000 on one law; worst standardized bias 0.0525, lowest coverage 0.9407; tree coverage 0.9422 cross-fitted versus 0.4037 in sample |
eight declared limits; complete two-node, three-level evidence with one fixed split and deterministic plans; no censoring, missingness, stochastic policies, continuous dose, survival, competing risks, weights, clusters, fold repeats, simultaneous bands, or flexible primary learner parity |
ordinary survival-curve longitudinal TMLE |
R ltmle 1.3-0 at source commit 338c029, with the tarball pinned by sha256 and survivalOutcome=TRUE |
five unique cumulative risks and three correlated risk differences across two horizons, static and dynamic plans, censoring, ordinary fitting, and pointwise 95% Wald intervals |
16/16 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 8/8 paired similarity and non-inferiority tests pass using 99% confidence bounds |
53/54 cells pass: horizon-specific double robustness, targeting and survival-recursion controls, root-n rates, exact-EIF efficiency, calibration, confounded-null type-I error, power, and the one-plan curve band; the band over all ten parameters is red under band-finite-sample |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0638, lowest coverage 0.9387 |
ten declared limits; reporting, not gated; this ordinary row excludes cross-fitting, a band over more than two horizons, competing risks, active truncation, and learned-mechanism parity |
cross-fitted survival-curve longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4, with one fitted prefix per reported horizon; R ltmle has no cross-fitting |
five unique cumulative risks and three correlated differences across two horizons, static and dynamic plans, one exact five-fold assignment, and pointwise 95% Wald intervals |
16/16 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 8/8 paired similarity and non-inferiority tests pass using 99% confidence bounds |
52/52 cells pass their own and family verdicts: horizon-specific double robustness, targeting and survivor-only controls, root-n rates, exact-EIF efficiency, calibration, null, power, and flexible-tree cross-fit controls |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0479, lowest coverage 0.9437 |
seven declared limits; agreement is distributional, one fixed split, pointwise inference, two horizons, one cause, nonbinding primary mechanism bounds, and the mechanism is supplied rather than estimated |
ordinary competing-risk longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4, with the other cause supplied through compete= and exact mechanisms supplied to both implementations |
ten unique cause-specific incidences and six correlated differences across two causes, two horizons, and static and dynamic plans; pointwise 95% Wald intervals |
32/32 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 16/16 paired similarity and non-inferiority tests pass using 99% confidence bounds |
40/40 cells pass: two-cause double robustness, targeting and all-cause risk-set controls, root-n rates, exact-EIF efficiency, calibration, confounded-null type-I error, power, and two default bands with their pointwise controls |
1,600 replications x n=4,000 on one law; worst standardized bias 0.0432, lowest coverage 0.9325 |
seven declared limits; this ordinary row keeps competing events natural, supplies the mechanisms, and excludes weights and clustering |
cross-fitted competing-risk longitudinal TMLE |
R lmtp 1.5.4 at commit f04a2b4, with the other cause in compete=, exact mechanisms, and the identical realized five-fold assignment |
ten unique cause-specific incidences and six correlated differences across two causes, two horizons, and static and dynamic plans; pointwise 95% Wald intervals |
32/32 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 16/16 paired similarity and non-inferiority tests pass using 99% confidence bounds |
38/38 cells pass their own and family verdicts: two-cause robustness, targeting and recursion controls, root-n rates, exact-EIF efficiency, calibration, null, power, and a balanced flexible-tree cross-fit pair |
1,600 replications x n=4,000 on one law; worst standardized bias 0.0595, lowest coverage 0.9363; tree coverage 0.9460 cross-fitted versus 0.5650 in sample |
eight declared limits; one fixed split, pointwise inference, two causes and horizons, supplied mechanisms, and no weights, clustering, or eliminated competing event |
point-treatment survival |
R survtmle 1.1.1, method = "mean", from its digest-pinned CRAN archive tarball on the digest-pinned R 4.5.2 image |
the risks of two arms and their difference at visits 1, 3 and 5, and two causes’ incidences and their differences at visits 2 and 4, on a held baseline treatment read from one time and one event column; pointwise 95% Wald intervals |
42/42 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 21/21 paired similarity and non-inferiority tests pass using 99% confidence bounds; the targeting submodels differ by construction, so the pairs read under the default margins |
34/35 cells pass: double robustness and its control, three-size and root-n checks, exact-EIF efficiency and calibration of seven labels with their controls (RMST, competing risks, three arms, a binned continuous time, a held end-of-study outcome and weights), power, cluster-robust inference with its IID control, and the default band; the weighted calibration cell is red under reporting on its SE-ratio band (SE ratio 1.027) |
1,600 replications x n=2,000 on two laws; worst standardized bias 0.0538, lowest coverage 0.9287 |
six declared limits; finite laws with discrete covariates, per-horizon targeting, discrete time, and survtmle’s bounds= not paired |
cross-fitted point-treatment survival |
none claimed. No maintained package fits five-fold cross-fitting with pooled targeting on a held baseline treatment, so a zero-row equivalence artifact records the absence |
the risks of two arms and their difference at visits 1, 3 and 5, and three arms’ risks and differences at visits 1 and 3; five outer folds, one pooled fluctuation per node, and pointwise 95% Wald intervals |
17/17 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
11/12 cells pass: double robustness and its control, exact-EIF efficiency and calibration of two labels with their controls, and the default band; the three-arm calibration cell is red under reporting on its empirical efficiency band, a finite-sample cost of cross-fitting |
1,600 replications x n=2,000 on two laws; worst standardized bias 0.0559, lowest coverage 0.9394 |
four declared limits; no comparator, saturated cell means only, and five fixed folds |
point-treatment survival, per-horizon hazard targeting |
Python property evidence, with comparator runs pending: R survtmle 1.1.1 with method = "hazard" (Study queue row 20) |
the risks of two arms and their difference at visits 1, 3 and 5, and two causes’ incidences and their differences at visits 2 and 4, on a held baseline treatment read from one time and one event column, with hazards targeted per horizon; pointwise 95% Wald intervals |
21/21 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 paired tests; Python property evidence, with comparator runs pending |
18/18 cells pass: double robustness and its control, targeting necessity and its control, three-size and root-n checks, exact-EIF efficiency and calibration of two labels with their controls, power, and the simultaneous band with its pointwise control |
1,600 replications x n=2,000 on two laws; worst standardized bias 0.0543, lowest coverage 0.9431 |
five declared limits; comparator runs pending, per-horizon targeting only, finite laws with discrete covariates, unmatched survtmle targeting, and no cross-fitting, weights or clusters |
point-treatment survival, one-step hazard targeting of the curve |
Python property evidence, with comparator runs pending: R tmle3 at commit ed72f8a with tmle_survival (Study queue row 21) |
the risks of two arms and their difference at visits 1 to 5, on a held baseline treatment read from one time and one event column, with one set of hazards targeted for every visit at once; pointwise 95% Wald intervals |
15/15 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 paired tests; Python property evidence, with comparator runs pending |
31/31 cells pass: double robustness and its control, targeting necessity and its control, three-size and root-n checks, exact-EIF efficiency and calibration of rule, policy, weighted, competing-cause and isotonic labels with their controls, cluster-robust inference with its independent-error control, power, and the simultaneous bands with their pointwise controls |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0370, lowest coverage 0.9425 |
seven declared limits; comparator runs pending, unmatched tmle3 settings, one primary scenario, learners that contain the truth, an isotonic cell that does not bind, per-horizon targeting in three cells, and no cross-fitting |
cross-fitted point-treatment survival, hazard targeting |
none claimed. No maintained package fits hazard targeting with cross-fitted nuisances on a held baseline treatment, so a zero-row equivalence artifact records the absence |
the risks of two arms and their difference at visits 1, 3 and 5, on a held baseline treatment read from one time and one event column; five outer folds, one pooled one-step hazard targeting step, and pointwise 95% Wald intervals |
9/9 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
15/15 cells pass: double robustness and its control, exact-EIF efficiency and calibration of the one-step difference with its controls, the per-horizon difference and a two-cause incidence difference, and the simultaneous bands over each arm’s curve and over all reported parameters with their pointwise controls |
1,600 replications x n=2,000 on one law; worst standardized bias 0.0448, lowest coverage 0.9375 |
seven declared limits; no comparator, one primary scenario, learners that contain the truth, one per-horizon cell, five fixed folds, no targeting-necessity, power or root-n cell, and no weights, clusters, policies or three arms |
policies at a held baseline treatment |
R lmtp 1.5.4 at commit f04a2b4, with K copied treatment columns and the identity policy after node 1, reading the density ratio from the law’s own probabilities |
the natural-course risk, a known stochastic policy’s risk and a modified treatment policy’s risk at visits 1, 3 and 5, with each contrast against the natural course, on a held baseline treatment; pointwise 95% Wald intervals |
36/36 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 18/18 paired similarity and non-inferiority tests pass using 99% confidence bounds; the two fluctuations differ by construction, so the pairs read under the default margins |
6/6 cells pass: exact-EIF efficiency and calibration of the policy and the modified treatment policy contrasts, with their controls |
1,600 replications x n=2,000 on two laws; worst standardized bias 0.0472, lowest coverage 0.9400 |
five declared limits; a policy at node 1 only, the law’s own mechanism in the paired fits, one categorical dose law, and calibration cells only |
TMLE on a declared known treatment mechanism |
R tmle 2.1.1 on a digest-pinned R 4.5.2 image, with g1W set to the declared mechanism and this package’s initial outcome regression as Q |
arm means, ATE, RR and OR at a correct and a wrong outcome regression, and the ATT and ATC at the correct one, on a mechanism declared by treatment_probabilities=; pointwise 95% Wald intervals, with RR and OR on the log scale |
24/24 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 12/12 paired similarity and non-inferiority tests pass using 99% confidence bounds |
21/21 cells pass: accuracy of the ATE, ATT, ATC, CV-TMLE, incremental and three-arm fits at a wrong outcome regression with an estimated-mechanism control, percentile-bootstrap coverage, interval calibration with its controls, and the root-n rate; the variance direction is reported |
1,000 replications x n=2,000 on one law; worst standardized bias 0.0955, lowest coverage 0.9330 |
four declared limits; parametric outcome regressions, the ATT and ATC paired at the correct outcome regression only, one binary and one three-arm law, and a reported variance direction |
DR-TMLE on a declared known treatment mechanism |
R drtmle 1.1.2 at commit 538a3a2, on a digest-pinned R 4.5.2 image, with gn set to the declared mechanism and this package’s initial outcome regression as Qn |
the two arm means and the ATE at each of the four guards with a wrong outcome regression, and the three arm means and two contrasts at guard=(), on a declared mechanism; pointwise 95% Wald intervals |
34/34 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 17/17 paired similarity and non-inferiority tests pass using 99% confidence bounds |
4/4 cells pass: accuracy of the ATE at each guard with a wrong outcome regression |
1,000 replications x n=2,000 on two laws; worst standardized bias 0.0562, lowest coverage 0.9400 |
four declared limits; a declared mechanism only, complete data in sample with the univariate reduction, one wrong outcome regression, and no missing-outcome path, whose interval the theory does not establish |
DR-TMLE with one inconsistent nuisance |
none claimed. No pinned comparator constructs a slow nuisance error, so a zero-row equivalence artifact records the absence |
the two arm means and the ATE with a wrong outcome regression and a declared, a correctly specified parametric (in sample and cross-fitted) or a slow mechanism; the ATE with a wrong mechanism and a slow outcome regression; the arm means of the randomized missing-outcome route with a slow observation mechanism; pointwise 95% Wald intervals |
3/3 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
19/19 cells pass: valid on the declared mechanism, conservative with a correct parametric mechanism, and first-order and growth controls with a slow nuisance error |
4,000 replications x n=4,000 on law L; worst standardized bias 0.0246, lowest coverage 0.9505 |
four declared limits; a binary outcome with two Gaussian covariates, the univariate reduction with guard Q and g, constructed errors rather than flexible learners, and no strata, clusters, weights or more arms |
LTMLE on declared known node mechanisms |
R ltmle 1.3-0 at commit 338c029, on the pinned ltmle image, with the declared treatment and censoring factors as a numeric gform |
the means of two static plans and their contrast on a two-node SMART with censoring, at a correct and a wrong set of sequential regressions; pointwise 95% Wald intervals |
12/12 implementation-estimand tests against known truth pass using 99% confidence bounds |
comparator-backed: 6/6 paired similarity and non-inferiority tests pass using 99% confidence bounds |
2/2 cells pass: accuracy of the contrast at wrong sequential regressions with every factor declared, and with the censoring factor estimated |
1,000 replications x n=2,000 on one law; worst standardized bias 0.0242, lowest coverage 0.9350 |
three declared limits; one SMART with static plans, in-sample parametric regressions, and no estimated-mechanism control |
sequential g-computation and IPW |
none claimed. This row is Python property evidence. The R ltmle 1.3-0 and R stremr agreements are fixture-level tests of the estimate and not a registered comparator study, so a zero-row equivalence artifact records the absence |
the means of static, dynamic and known-policy regimens and their contrasts, for SequentialGComputation and for SequentialIPW with Hajek and Horvitz-Thompson normalization, on binary, three-arm, survival, competing-risk and continuous laws with GLM nuisances; pointwise 95% Wald intervals |
27/27 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator study is registered |
159/162 declared repeated-sampling property cells pass; the three failures are the declared misspecification control on law X, and 17 cells report without a verdict |
1,000 replications x n=8,000 on the primary laws, and 8,800 replications x n=2,000 for each calibration cell; worst standardized bias 0.0331, lowest coverage 0.9410 |
reporting, not gated: the three red cells are a declared negative control with an owner, and the nine declared limits include no comparator study, parametric nuisances only, no cross-fitting, and a non-saturated first node without coverage evidence |
structural nested mean model |
Python property evidence, with comparator runs pending: EconML 0.17.0 DynamicDML (econml) in a pinned container, declared in tests/canonical/econml_dynamic_dml. A fast-tier parity fixture checks the point estimates on one panel |
the blip coefficients, the reference mean ey_reference, and the levels and contrasts of static regimens, on binary, three-level and dose nodes at two and three nodes, with features that read baseline columns, earlier treatments and a time-varying covariate, known weights and clusters; cross-fitted GLM nuisances and one early-stopped booster; pointwise 95% Wald intervals from the influence curve |
61/61 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 paired tests; Python property evidence, with comparator runs pending |
237/237 cells pass: interval calibration of 61 labels with shrunken-SE and noise controls, three-size and root-n checks on seven labels, the size and power of the model check, and the detection of too few blip features; 14 diagnostic cells reported: weak positivity, and the clustered law fitted without its clusters |
1,000 replications x n=2,000 on 11 laws, and 8,000 replications x n=2,000 for each calibration cell; worst standardized bias 0.0883, lowest coverage 0.9360 |
reporting, not gated: nine declared limits; pending comparator runs, laws that satisfy the model, one calibration size, GLM nuisances and one booster, low model-check power under weak positivity, ungated weak-positivity cells, six lost property replicates, no censoring or rules, and one setting each for clusters and weights |
parametric g-formula |
none claimed on the study laws. pygformula 1.2.0 fits pooled models, so it runs on four separate fixtures, and all 84 fixture rows pass |
the means of static, dynamic, stochastic, odds-tilt, shift, fixed-dose and rule-set-dose regimens, their contrasts against another regimen or the natural course, and competing-risk cumulative incidences, on binary, Gaussian, categorical-covariate, competing-risk and continuous-dose laws with node-specific main-terms GLM models; the stacked-equation interval with the simulation term, pointwise 95% Wald |
21/21 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
95/96 cells with a verdict pass, and 23 cells report without one: interval calibration of 21 labels with shrunken-SE and added-noise controls, three-size and root-n checks, Monte Carlo accounting at one simulated history per unit, and the necessity of the parametric g-formula against a main-terms ICE fit; the G1 treat-all calibration cell is red on its SE-ratio band (SE ratio 0.9619) and reads as finite-sample |
1,000 replications x n=8,000 on five laws, and 2,400 replications x n=2,000 for each calibration cell; worst standardized bias 0.0782, lowest coverage 0.9360 |
reporting, not gated: seven declared limits; no comparator on the study laws, correctly specified laws, one red cell, GLM learners only, node-specific models that pygformula cannot express, no published variance for the simulation form, and ten lost property replicates |
repeated-outcome longitudinal TMLE |
Python property evidence, with comparator runs pending: R ltmle 1.3-0 at source commit 338c029 with survivalOutcome=FALSE, fitted on the prefix through each horizon, runs as TRACKER Study queue item 14(c) |
the means of three plans at three horizons and the contrasts of two plans against never, on a censored three-node panel with a repeated binary outcome; pointwise 95% Wald intervals |
13/13 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 paired tests; Python property evidence, with comparator runs pending |
97/97 cells pass: interval calibration at every horizon, with cross-fitted GLM, cross-fitted booster and continuous arms, shrunken-SE and noise controls, double robustness, targeting necessity, full-history necessity, the simultaneous band, rule necessity, the absorbing-reading and history-free controls; 10 small_sample_calibration cells reported but not gated |
2,400 replications x n=3,000 on one law; worst standardized bias 0.0426, lowest coverage 0.9325 |
nine declared limits; in-sample binary evidence on one three-node law, pending comparator runs, fixed mechanisms in the declared comparison, and no learned-mechanism parity |
repeated-outcome longitudinal IPSI |
none claimed. No maintained implementation fits a repeated outcome under an incremental propensity intervention, so a zero-row equivalence artifact records the absence. Python property evidence, no comparator |
the means of three odds tilts at three horizons and the contrasts of two tilts against the natural course, on a censored three-node panel with a repeated binary outcome, in sample and at five folds; pointwise 95% Wald intervals and the default simultaneous band |
30/30 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; the row rests on the other two columns |
81/81 cells pass: interval calibration at every horizon in sample and at the last horizon with five folds and a continuous outcome, shrunken-SE and noise controls, the simultaneous band, single robustness, and the absorbing-reading and history-free necessity controls; 15 cells reported but not gated, the cross-fitted booster arm and the small_sample_calibration arms |
2,400 replications x n=8,000 per scenario; worst standardized bias 0.0625, lowest coverage 0.9358 |
nine declared limits; in-sample and five-fold evidence on one three-node law, no comparator, correctly specified mechanisms, and cell-mean outcome regressions |
omitted-variable bound standard error |
none claimed. DoubleML omits the conditioning-share term, and dml.sensemakr estimates the share out of fold and, at 58ac44d, divides its point estimate, so a zero-row equivalence artifact records the absence of a comparator |
the lower and upper omitted-variable bounds of the ATT, the ATC and the ATE at \(c_Y = 0.5\), \(c_D = 0.3\), \(\rho = 1\), with the doubly robust \(\nu^2\); each bound’s one-sided limit read back as a standard error and a pointwise 95% Wald interval |
6/6 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no maintained implementation computes these limits with this curve |
8/8 declared repeated-sampling property cells pass, including two inflated-SE controls that read the curve without the share term |
10,000 replications x n=1,000 on one primary law; worst standardized bias 0.0475, lowest coverage 0.9478 |
reporting, not gated: every cell is green, and the policy means no test asserts that; eleven declared limits, including no comparator, an in-sample fit with correct GLMs, positive residual variance, one law, size and strength, two arms without weights or clusters, the doubly robust estimator only, no ATC control, a narrow pass of the att_upper control, and a symmetric law |
calibration-slope warning |
none claimed. No maintained implementation computes this rule, so a zero-row equivalence artifact records the absence of a comparator |
the calibration slope of the propensity report, a logistic recalibration with one intercept per validation fold, and its sandwich standard error; the rule that warns when the Bonferroni interval over the tested models lies above 0 and excludes 1 |
2/2 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no maintained implementation computes this rule |
11/11 declared repeated-sampling property cells pass: five false-warning rates, four fixed-band controls on the same fits, and two detection cells |
10,000 replications x n=2,000 on two primary laws; worst standardized bias 0.0572, lowest coverage 0.9484 |
reporting, not gated: every cell is green, and the policy means no test asserts that; ten declared limits, including no comparator, no weights or clusters, a binary propensity only, three folds and one size, known and logistic learners only, a standard error that treats the fold models as fixed, and no prediction at 0 or 1 |
default simultaneous bands across shipped fit shapes |
none claimed. The pinned comparators publish no multiplier band over these families, so a zero-row equivalence artifact records the absence |
the default max-t multiplier band of 25 shipped fit shapes, each the subject fit of a registered source study at its own law and size, and the pointwise intervals of the same fits read jointly |
7/7 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator publishes these bands |
47/50 cells pass: 22 of 25 bands and all 25 pointwise controls; the categorical, cross-fitted categorical and controlled-direct-effect bands at Z = 0 are red |
1,000 replications x n=1,000 of the shipped TMLE() default; worst standardized bias 0.0320, lowest coverage 0.9380 |
reporting, not gated; four declared limits: one law and size per shape, one multiplier seed per fit, no comparator, and design correlations that are estimates |
full-refit bootstrap and derived contrasts |
none claimed. No pinned comparator ships these outputs, so a zero-row equivalence artifact records the absence |
the log-scale ratio of two regimen levels, the RMST and its contrast, the LTMLE full-refit bootstrap percentile interval in sample, cross-fitted and clustered, and the point TMLE bootstrap percentile interval |
2/2 implementation-estimand tests against known truth pass using 99% confidence bounds |
0/0 external comparisons; no comparator ships these outputs |
20/23 cells pass: every ratio, RMST and control cell, and the in-sample, survival and point bootstrap cells; the two cross-fitted bootstrap cells over-cover and the cluster contrast under-covers |
1,000 replications x n=2,000 of the end-of-study ratios; worst standardized bias 0.0219, lowest coverage 0.9640 |
reporting, not gated; six declared limits: correctly specified cell-mean nuisances only, one size per cell, finite binary laws, static regimens, node units, and unmeasured compositions; the red kinds stay diagnostic |