# Technical reference

This reference accounts for every statistical method `cleverly` ships, and for the evidence that
each one is implemented correctly.

The project validates a method in two independent ways. It compares `cleverly` against a canonical
implementation, where a maintained one exists. It also measures `cleverly` against the
repeated-sampling properties the method's own source theory predicts. Neither one replaces the
scientific instruments that check the derivation itself: the exact-law, Gateaux, remainder,
identity, and deliberate-mutation checks recorded in the [evidence manifest](evidence.md).

## Implementation matrix

The [implementation validation grid](method-evidence/validation-grid.md) covers the methods with a
registered validation study, which has property cells. This matrix covers every implementation
family `cleverly` ships, and names the correctness evidence for each.

| implementation family | theory and citation | `cleverly` implementation | external provenance | correctness evidence |
| --- | --- | --- | --- | --- |
| Counterfactual means; ATE; risk and odds ratios; multi-arm contrasts | [Point-treatment TMLE](point-treatment-tmle.md#counterfactual-means-and-arm-contrasts); van der Laan & Rubin (2006) | [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py), [`targets/builtin.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/targets/builtin.py) | R `tmle`; pinned `tmle3` parameter/update source | [`ate`, `ey1`, `ey0`, `rr`, `or`](evidence.md#the-table) oracle, Gateaux, remainder, and identity rows; gated [fixed-weight](method-evidence/weighted-point-treatment-tmle.md) and [learned-nuisance weighted](method-evidence/learned-weighted-point-treatment-tmle.md) studies |
| ATT and ATC | [Conditional-population effects](point-treatment-tmle.md#conditional-population-effects); van der Laan (2010) | [`inference/influence.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/inference/influence.py) | no numeric external oracle claimed | [`att`, `atc`](evidence.md#the-table), including conditioning-event influence terms |
| Known treatment mechanism | [Known treatment mechanism](point-treatment-tmle.md#known-treatment-mechanism), [known node mechanisms](longitudinal-tmle.md#known-node-mechanisms) and the [DR-TMLE contract](dr-tmle/theorem.md#a-known-treatment-mechanism); Moore & van der Laan (2009), van der Laan & Gruber (2012), Benkeser et al. (2017) | [`data/known_mechanism.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/data/known_mechanism.py), [`estimators/_nuisance.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/_nuisance.py), [`longitudinal/sequential.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sequential.py) | R `tmle` 2.1.1 with `g1W`, R `drtmle` 1.1.2 with `gn`, and R `ltmle` 1.3-0 with a numeric `gform` | reporting-policy [TMLE study](method-evidence/known-treatment-mechanism.md), [DR-TMLE study](method-evidence/known-mechanism-dr-tmle.md) and [LTMLE study](method-evidence/known-node-mechanisms-ltmle.md); exact laws, witnesses, and mutations remain in [estimator variants evidence](evidence.md#estimator-variants-over-registered-targets) |
| Natural-course mean, PAR, and PAF | [Population interventions](point-treatment-tmle.md#population-interventions); Díaz Muñoz & van der Laan (2012) for complete outcomes; Díaz, Carone & van der Laan (2016) for the MAR mean; Zheng & van der Laan (2011) and Levy (2018) for its stacked CV-TMLE; the [MAR stack](point-treatment-tmle.md#par-and-paf-with-missing-outcomes) for PAR and PAF | [`targets/builtin.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/targets/builtin.py), [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py) | R `tmle` 2.1.1 population-mean path with supplied predictions for the ordinary and stacked MAR targets, composed per arm for PAR and PAF; stochastic-intervention literature | [`ey_obs`, `par`, `paf`](evidence.md#the-table) exact-law and transformation checks; gated [ordinary](method-evidence/ordinary-missing-outcome-natural-course-tmle.md) and [stacked](method-evidence/stacked-missing-outcome-natural-course-cvtmle.md) missing-outcome natural-course studies, and the gated [missing-outcome attributable-effect study](method-evidence/missing-outcome-attributable-effects-tmle.md) |
| Missing outcomes and controlled direct effects | [Observed-data extensions](point-treatment-tmle.md#missing-outcomes-and-controlled-direct-effects); Díaz & van der Laan (2017) for the randomized arm-indexed missing-outcome construction | [`estimators/direct_effect.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/direct_effect.py), [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py) | R `tmle` 2.1.1 for ordinary and stacked arm-indexed MAR and controlled direct effects; pinned R `drtmle` 1.1.2 for the randomized both-correct limit | gated [ordinary arm-indexed MAR](method-evidence/ordinary-missing-outcome-tmle.md), [stacked arm-indexed MAR](method-evidence/stacked-arm-indexed-missing-outcome-cvtmle.md), [controlled direct-effect](method-evidence/controlled-direct-effect-tmle.md), and [randomized DR-TMLE](method-evidence/randomized-missing-outcome-dr-tmle.md) studies, plus the exact MAR and CDE laws, Gateaux checks, remainder identities, and deliberate mutations |
| Deterministic and dynamic regimes | [Known regimes](point-treatment-tmle.md#known-regimes); Robins (2004), Díaz Muñoz & van der Laan (2012) | [`interventions/base.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/base.py), [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py) | pinned R `lmtp` 1.5.4 at `f04a2b4` | gated [deterministic-regime study](method-evidence/deterministic-point-treatment-regimes.md), plus [`ey_regime`, `ate_regime`](evidence.md#the-table), dynamic-rule and support mutations |
| Known stochastic regimes | [Known regimes](point-treatment-tmle.md#known-regimes); Díaz & van der Laan (2013) | [`interventions/base.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/base.py), [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py) | pinned R `lmtp` 1.5.4 at `f04a2b4`, reached through the point adapter | gated [stochastic-regime study](method-evidence/stochastic-point-treatment-regimes.md), including density replacement and targeting controls |
| Value of a learned treatment rule | [Learned rules](point-treatment-tmle.md#learned-rules); van der Laan & Luedtke (2015), Section 7 and Appendix B; Montoya, van der Laan, Skeem & Petersen (2023) | [`interventions/learned.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/learned.py), [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py) | no numeric external oracle claimed. R `tmle3mopttx` weights the folds by their size and centres the curve at the pooled estimate | gated [learned-rule study](method-evidence/learned-rule-cvtmle.md) at two laws that meet the limiting-rule condition, with the declared logistic outcome learner, and a reporting [boundary study](method-evidence/learned-rule-cvtmle-boundary.md): the interval under-covers at the exceptional and weak-blip laws that it reads; plus the [`ey_learned_rule`](evidence.md#the-table) oracle, Gateaux and identity rows |
| Continuous modified treatment policies | [Modified treatment policies](point-treatment-tmle.md#modified-treatment-policies); Díaz Muñoz & van der Laan (2012), Haneuse & Rotnitzky (2013), Díaz et al. (2023) | [`interventions/policy.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/policy.py), [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py) | pinned R `lmtp` 1.5.4 at `f04a2b4` | reporting-policy [continuous-policy study](method-evidence/continuous-modified-treatment-policies.md) and [point-policy study](method-evidence/point-modified-treatment-policies.md), plus [`ey_policy`, `ate_policy`](evidence.md#the-table), continuous-law Gateaux and remainder checks |
| Modified treatment policies on a vector treatment | [Vector treatments](point-treatment-tmle.md#vector-treatments) and [vector nodes with a continuous component](longitudinal-tmle.md#vector-nodes-with-a-continuous-component). Díaz, Williams, Hoffman & Schenck (2023), Section 2, Theorem 1, Equation (3), Section 5.4 and Theorem 3. Hoffman et al. (2024) | [`interventions/policy.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/policy.py), [`interventions/joint_density.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/joint_density.py), [`longitudinal/sequential.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sequential.py) | pinned R `lmtp` 1.5.4 at `f04a2b4` with vector `trt` nodes. The comparator runs are pending | a reporting [vector study](method-evidence/vector-modified-treatment-policies.md), whose Python property cells are declared and not yet run, plus [`ey_policy`, `ate_policy`](evidence.md#the-table), the mixed-law Gateaux checks and the mutation controls of the vector witnesses |
| Continuous policies at declared boundaries, with learned and tuned inputs | [Modified treatment policies at a node](longitudinal-tmle.md#modified-treatment-policies-at-a-node) and [limits of the continuous-policy evidence](longitudinal-tmle.md#limits-of-the-continuous-policy-evidence). Díaz, Williams, Hoffman & Schenck (2023) | [`longitudinal/sequential.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sequential.py), [`interventions/policy.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/policy.py) | pinned R `lmtp` 1.5.4 at `f04a2b4` for the boundary and pricing studies, with comparator runs pending. No package fits a tuned cap | reporting [boundary study](method-evidence/mtp-boundaries.md), [pricing study](method-evidence/mtp-pricing.md) and [tuned-cap study](method-evidence/mtp-tuning.md), each with Python property evidence |
| Incremental propensity-score interventions | [Incremental interventions](point-treatment-tmle.md#incremental-propensity-score-interventions); Kennedy (2019) | [`interventions/incremental.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/incremental.py), [`estimators/tmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/tmle.py) | R `npcausal` at `56a5ac1`, the implementation Kennedy (2019) publishes | gated [incremental-intervention study](method-evidence/incremental-propensity-interventions.md), plus [`ey_ipsi`, `ate_ipsi`](evidence.md#the-table), including nonzero treatment-score and one-sided remainder witnesses |
| Incremental interventions over time | [Incremental interventions over time](longitudinal-tmle.md#incremental-interventions-over-time); Kennedy (2019), Kim, Kennedy & Naimi (2021) | [`longitudinal/ipsi.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/ipsi.py), [`longitudinal/estimator.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/estimator.py) | R `npcausal` 0.1.0 at `56a5ac11` for the complete-data estimator, with comparator runs pending. No maintained package ships the dropout construction | gated [complete-data study](method-evidence/longitudinal-incremental-interventions.md) with Python property evidence, and a reporting [dropout study](method-evidence/longitudinal-incremental-interventions-dropout.md) with no comparator; exact laws, Gateaux checks and the `npcausal` transcription in [longitudinal evidence](evidence.md#longitudinal-estimands-outside-the-target-registry) |
| Categorical incremental interventions | [Incremental interventions](point-treatment-tmle.md#incremental-propensity-score-interventions) and [over time](longitudinal-tmle.md#incremental-interventions-over-time); Kennedy (2019), Equation (1) and Theorems 1 and 2, with the general-level extension of the theory pages | [`interventions/incremental.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/interventions/incremental.py), [`longitudinal/ipsi.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/ipsi.py) | none. No maintained package ships a categorical odds tilt with its influence curve | gated [categorical incremental study](method-evidence/categorical-incremental.md) with Python property evidence, plus [`ey_ipsi`, `ate_ipsi`](evidence.md#the-table) and the mutation controls of `tests/unit/test_categorical_incremental_mutations.py` |
| Point and longitudinal MSM projections | [Marginal structural models](msm-projections.md); Neugebauer & van der Laan (2007), Rosenblum & van der Laan (2010), Petersen et al. (2014) | [`msm.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/msm.py), [`longitudinal/msm.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/msm.py) | pinned `tmle3` `Param_MSM`; projected ordinary R `ltmle` and cross-fitted R `lmtp` regimen fits; raw R `ltmleMSM` differs in projection scale | gated [point](method-evidence/point-treatment-msm-projection.md), [ordinary longitudinal](method-evidence/ordinary-longitudinal-msm-projection.md) and [cross-fitted longitudinal](method-evidence/cross-fitted-longitudinal-msm-projection.md) studies, plus the point [`msm`](evidence.md#the-table) and [longitudinal MSM](evidence.md#longitudinal-estimands-outside-the-target-registry) oracle rows |
| Longitudinal end-of-study TMLE | [Sequential regression](longitudinal-tmle.md#end-of-study-regimen-means); Bang & Robins (2005), van der Laan & Gruber (2012); [cross-fitting](longitudinal-tmle.md#cross-fitting-the-recursion), Díaz et al. (2023) | [`longitudinal/estimator.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/estimator.py), [`longitudinal/sequential.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sequential.py) | R `ltmle` 1.3-0 for ordinary binary fitting; pinned `lmtp` 1.5.4 for cross-fitting and categorical treatment | a gated [ordinary](method-evidence/ordinary-end-of-study-longitudinal-tmle.md) and a reporting [cross-fitted](method-evidence/cross-fitted-end-of-study-longitudinal-tmle.md) binary study, plus gated [ordinary](method-evidence/ordinary-categorical-longitudinal-tmle.md) and [cross-fitted](method-evidence/cross-fitted-categorical-longitudinal-tmle.md) categorical studies, a reporting [known-policy](method-evidence/stochastic-categorical-longitudinal-tmle.md) study, and a reporting [modified-treatment-policy](method-evidence/longitudinal-modified-treatment-policies.md) study; exact laws, Gateaux checks, and mutations remain in [longitudinal evidence](evidence.md#longitudinal-estimands-outside-the-target-registry) |
| Sequentially doubly robust longitudinal estimation | [The sequentially doubly robust estimator](longitudinal-tmle.md#the-sequentially-doubly-robust-estimator); Díaz, Williams, Hoffman & Schenck (2023), Equation (4) and Section 5.3 | [`longitudinal/sdr.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sdr.py) | pinned R `lmtp` 1.5.4 at `f04a2b4`, `cf_sdr` | a reporting [SDR study](method-evidence/sequentially-doubly-robust-estimation.md); exact laws, the longhand recursion and mutations remain in [longitudinal evidence](evidence.md#longitudinal-estimands-outside-the-target-registry) |
| Sequential g-computation and IPW over time | [Sequential g-computation and IPW](sequential-baselines.md); Bang & Robins (2005), Section 4; Stefanski & Boos (2002), Section 2 | [`longitudinal/estimator.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/estimator.py), [`longitudinal/sequential.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sequential.py) | R `ltmle` 1.3-0 and R `stremr`, as parity tests | Python property evidence, a reporting [`sequential-baselines`](method-evidence/sequential-baselines.md) study; R comparator parity: ltmle (W9, `tests/unit/test_sequential_baselines_ltmle_r.py`) and stremr (`tests/unit/test_sequential_baselines_stremr.py`); the [technical page](sequential-baselines.md#validation) lists the witnesses |
| Structural nested mean model | [Structural dynamic effects](structural-dynamic-effects.md); Lewis & Syrgkanis (2021), supplement Theorem 8 and arXiv Theorem 4 | [`longitudinal/structural.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/structural.py) | Python property evidence, with comparator runs pending: EconML 0.17.0 `DynamicDML` in a pinned container. A fixture-level parity test checks the point estimates | reporting [structural nested mean study](method-evidence/structural-nested-mean.md); exact finite laws, influence-curve derivative checks and mutation controls on the [technical page](structural-dynamic-effects.md#validation); parity with `DynamicDML` in `tests/unit/test_structural_econml_parity.py` |
| Parametric g-formula over time | [Parametric g-formula](parametric-gformula.md); Robins (1986), eq. (4.7); McGrath et al. (2020); Pakes & Pollard (1989) and Newey & McFadden (1994) for the simulated-estimator step | [`longitudinal/gformula.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/gformula.py), [`longitudinal/estimator.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/estimator.py) | `pygformula` 1.2.0 on four fixtures where pooled and node-specific models are equal. `gfoRmula` 1.1.1 and `pygformula` fit pooled models, so they are not a comparator on the study laws | Python property evidence, a registered `parametric-gformula` study (page `docs/technical-reference/method-evidence/parametric-gformula.md`); the [technical page](parametric-gformula.md#validation) lists the witnesses |
| Longitudinal survival and competing risks | [Event-process TMLE](longitudinal-tmle.md#survival-and-competing-risks); Stitelman et al. (2012) | [`longitudinal/estimator.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/estimator.py) | R `ltmle` 1.3-0 for ordinary survival; pinned `lmtp` 1.5.4 for cross-fitted survival and both competing-risk rows | a reporting ordinary [survival](method-evidence/ordinary-survival-curve-longitudinal-tmle.md) study, and gated cross-fitted survival and ordinary and cross-fitted [competing-risk](method-evidence/ordinary-competing-risk-longitudinal-tmle.md) studies, with a separate cross-fitted page for each; exact laws, Gateaux checks, and deliberate mutations remain in [longitudinal evidence](evidence.md#longitudinal-estimands-outside-the-target-registry) |
| Longitudinal repeated outcomes, `IPSI` | [IPSI on a repeated outcome](longitudinal-tmle.md#ipsi-on-a-repeated-outcome); Kim, Kennedy and Naimi (2021) | [`longitudinal/estimator.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/estimator.py), [`longitudinal/ipsi.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/ipsi.py) | none claimed. Python property evidence | gated [repeated-outcome IPSI study](method-evidence/repeated-outcome-ipsi.md); exact laws and Gateaux checks in [longitudinal evidence](evidence.md#longitudinal-estimands-outside-the-target-registry) |
| Longitudinal repeated outcomes | [Repeated outcomes](longitudinal-tmle.md#repeated-outcomes); Bang & Robins (2005), van der Laan & Gruber (2012) | [`longitudinal/estimator.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/estimator.py), [`longitudinal/sequential.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sequential.py) | Python property evidence, with comparator runs pending: R `ltmle` 1.3-0, fitted on the prefix through each horizon | gated [repeated-outcome LTMLE study](method-evidence/repeated-outcome-longitudinal-tmle.md); exact laws, Gateaux checks, and mutations remain in [longitudinal evidence](evidence.md#longitudinal-estimands-outside-the-target-registry) |
| Point-treatment survival | [One baseline treatment held over the nodes](longitudinal-tmle.md#one-baseline-treatment-held-over-the-nodes) and [time-to-event input](longitudinal-tmle.md#time-to-event-input); Benkeser, Carone & Gilbert (2018) | [`longitudinal/data.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/data.py), [`longitudinal/sequential.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/longitudinal/sequential.py) | R `survtmle` 1.1.1 for the arm risks and incidences; pinned `lmtp` 1.5.4 for a policy at baseline | reporting-policy [survival study](method-evidence/point-treatment-survival.md), [cross-fitted survival study](method-evidence/cross-fitted-point-treatment-survival.md), [policy study](method-evidence/point-treatment-survival-policies.md), [hazard targeting study](method-evidence/point-survival-hazard.md), whose `survtmle` comparator runs are pending, [one-step curve study](method-evidence/point-survival-curve.md), whose `tmle3` comparator runs are pending, and [cross-fitted hazard targeting study](method-evidence/point-survival-curve-crossfit.md), which has no comparator; exact laws, Gateaux checks, and mutations remain in [longitudinal evidence](evidence.md#longitudinal-estimands-outside-the-target-registry) |
| Collaborative TMLE | [Collaborative selection](collaborative-tmle.md); van der Laan & Gruber (2010), Ju, Gruber et al. (2019) for the selector paths; Ju, Wyss et al. (2019) for the lasso path and C-TMLE0; Benkeser, Cai & van der Laan (2020), Theorem 1, arm by arm, for the per-arm outcome-adaptive design | [`estimators/ctmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/ctmle.py) | pinned R `ctmle` `ctmleGlmnet` and `ctmleGeneral` for the lasso and learners paths and for C-TMLE0 (`ctmletype = 3`); pinned `ctmle3` outcome-adaptive source; pinned `tmle3` / `sl3` fold semantics. Neither `ctmle` nor `ctmle3` supplies weighted parity | [C-TMLE estimator-variant evidence](evidence.md#estimator-variants-over-registered-targets), the reporting-policy [selector](method-evidence/selector-based-point-treatment-c-tmle.md) and [outcome-adaptive](method-evidence/outcome-adaptive-point-treatment-c-tmle.md) studies, the [per-arm outcome-adaptive](method-evidence/outcome-adaptive-per-arm-c-tmle.md) study and its [strata and missing-data](method-evidence/outcome-adaptive-per-arm-c-tmle-strata-missing.md) study, the [candidate-sequence](method-evidence/c-tmle-candidate-sequences.md) study and its R witness fixture, the [C-TMLE0 derivative](method-evidence/ctmle-derivative.md) study and its R witness fixture, selector mutations, support path tests, and the fixed-weight refit controls in the simulated common-cause surface |
| DR-TMLE | [Doubly-robust inference](dr-tmle/index.md); van der Laan (2014), Benkeser et al. (2016/2017), Benkeser & Hejazi (2023) | [`estimators/drtmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/drtmle.py), [`estimators/reduced.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/reduced.py) | pinned R `drtmle` 1.1.2 source and bounded numerical comparators for complete outcomes, the randomized MAR both-correct limit, and the observational composite indicator | reporting-policy [complete-outcome](method-evidence/canonical-dr-tmle.md), [three-arm missing-outcome](method-evidence/randomized-multi-arm-missing-outcome-dr-tmle.md) and [composite missing-data](method-evidence/observational-missing-data-dr-tmle.md) studies, a gated [randomized missing-outcome](method-evidence/randomized-missing-outcome-dr-tmle.md) study, a reporting [one-inconsistent-nuisance](method-evidence/dr-tmle-one-inconsistent-nuisance.md) witness study, [DR-TMLE contract](dr-tmle/index.md), theorem identity, Gateaux, remainder, score, cross-fit, and missing-data mutations |
| Cross-fitting and CV-TMLE | [CV-TMLE and cross-fitting](cv-tmle.md); Zheng & van der Laan (2011), Levy (2018), Chernozhukov et al. (2018) | [`learners/crossfit.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/learners/crossfit.py) | pinned `tmle3` stacked validation likelihood; pinned `lmtp` and pinned `ife` for cluster-robust inference on the package's own grouped fold draw; R `tmle` 2.1.1 population mean with supplied stitched nuisances; no parity implementation for the complete repeat rule | gated [stacked CV-TMLE](method-evidence/stacked-point-treatment-cv-tmle.md), [fold-evaluated CV-TMLE](method-evidence/fold-evaluated-point-treatment-cv-tmle.md), [fold-targeted CV-TMLE](method-evidence/fold-targeted-point-treatment-cv-tmle.md), and binary-outcome [clustered CV-TMLE](method-evidence/clustered-point-treatment-cv-tmle.md) rows; reporting-policy [unequal-size clustered CV-TMLE](method-evidence/clustered-unequal-cv-tmle.md) and [few-cluster t reference](method-evidence/clustered-few-cluster-tmle.md) rows; the [fold and outcome-scale rules](cv-tmle.md#fold-and-outcome-scale-rules) with their witnesses; gated [missing-outcome natural-course CV-TMLE](method-evidence/stacked-missing-outcome-natural-course-cvtmle.md) and [arm-indexed missing-outcome CV-TMLE](method-evidence/stacked-arm-indexed-missing-outcome-cvtmle.md); reporting-policy [repeated CV-TMLE](method-evidence/repeated-cross-fitting.md); leakage and repeated-fold mutations |
| Targeting and fitting options | [Stopping rules, step bound and best iterate](point-treatment-tmle.md#stopping-rules-step-bound-and-best-iterate), [per-arm outcome regression](point-treatment-tmle.md#per-arm-outcome-regression), [treatment-mechanism covariates](point-treatment-tmle.md#treatment-mechanism-covariates) and [patience](collaborative-tmle.md#patience). No option adds an estimand | [`fluctuation/_score.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/fluctuation/_score.py), [`fluctuation/iterative.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/fluctuation/iterative.py), [`estimators/_nuisance.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/_nuisance.py), [`estimators/ctmle.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/estimators/ctmle.py), [`data/treatment_covariates.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/data/treatment_covariates.py) | pinned `tmle3` updater, R `drtmle` `stratify`, and R `ctmle` `patience` and `Wg`, as comparator fixtures: `tmle3_updater.json`, `drtmle_stratify.json`, `ctmle_patience.json` and `ctmle_wg.json` | Python exact tests, with witnesses and mutation controls, in the [options table](evidence.md#targeting-and-fitting-options); exact tests and R parity, and for `ctmle` the parity of the option |
| Inference, diagnostics, and sensitivity | [How every method reports uncertainty](inference.md) and [sensitivity and validation methods](validation-methods.md); [sensitivity references](../references.md#sensitivity-analysis) | [`inference/`](https://github.com/esbraun/cleverly-tmle/tree/main/src/cleverly/inference), [`sensitivity/`](https://github.com/esbraun/cleverly-tmle/tree/main/src/cleverly/sensitivity), [`validation/`](https://github.com/esbraun/cleverly-tmle/tree/main/src/cleverly/validation), [`assessment.py`](https://github.com/esbraun/cleverly-tmle/blob/main/src/cleverly/assessment.py) | no external implementation used as acceptance evidence; three pinned DoWhy sources at revision `2116d5c` supply secondary control-flow evidence: the [dummy outcome refuter](https://github.com/py-why/dowhy/blob/2116d5cbace5a057937e03b2efba95c13140cc4c/dowhy/causal_refuters/dummy_outcome_refuter.py), [bootstrap refuter](https://github.com/py-why/dowhy/blob/2116d5cbace5a057937e03b2efba95c13140cc4c/dowhy/causal_refuters/bootstrap_refuter.py), and [simulated common-cause refuter](https://github.com/py-why/dowhy/blob/2116d5cbace5a057937e03b2efba95c13140cc4c/dowhy/causal_refuters/add_unobserved_common_cause.py) | inference, assessment, sensitivity, and serialization mutations |

"Existing implementation" has two meanings in this matrix. One is the local module that executes
the method. The other is a pinned external implementation, where the source was audited. External
parity is acceptance evidence for the claims it covers. It is never the derivation, and it is
never a required gate. Each comparator-backed study declares one comparison kind: a matching
implementation, a different estimator, or a related target.

The [natural-extension verdicts](natural-extension-verdicts.md) record which refused compositions
qualify for implementation as natural extensions, and the roadmap row that owns each one. The
[hard-stop review](hard-stop-review.md) records the verdict on each inherited hard stop of the
roadmap, and the row that now owns each part.

## How to read the evidence column

The [evidence manifest](evidence.md) distinguishes independent scientific checks from bounded
implementation witnesses. An exact finite-support law can verify a complete functional but miss a
term that vanishes at the truth. A Gateaux comparison can verify the influence curve but share a
mistake with the law. Remainder-rate and deliberate-mutation tests make those blind spots visible.
An external comparison does not substitute for these checks on a changed term.

```{toctree}
:maxdepth: 2

method-evidence/index
evidence
validation-methods
inference
point-treatment-tmle
cv-tmle
collaborative-tmle
dr-tmle/index
longitudinal-tmle
sequential-baselines
structural-dynamic-effects
parametric-gformula
msm-projections
scope-and-refusals
natural-extension-verdicts
hard-stop-review
../references
```