# Outcome-adaptive multi-arm C-TMLE

This study compares outcome-adaptive C-TMLE for a labelled three-arm treatment. The canonical
comparison uses archived R [`ctmle3`](https://github.com/tlverse/ctmle3) at pinned commit
`a4ea77b`. Both implementations receive identical draws and start from corresponding empirical
treatment mechanisms and correctly specified outcome regressions.

The study uses the family-wide three-arm binary law and reports arm means, ATEs, risk ratios,
and odds ratios against the `high` arm. Its property design separately checks the method's
outcome-regression robustness contract and the precision cost of estimating the generated
outcome-adaptive design.

This study fits the shared design, `oat_design="shared"`. Every shared fit has the
`generated_design_plugin` status. The coverage and
standard-error ratios below measure `plugin_interval` and `plugin_std_error` as diagnostics;
their pass labels do not authorize `ci`, `pvalue`, or `std_error`. The two red generated-design
property cells remain open under [F19](../../roadmap.md#f19-outcome-adaptive-c-tmle-generated-design-inference).

The primary fit draws no split at all. It is not cross-fitted, and the outcome-adaptive strategy
scores no candidate path, so it needs no selection folds. Every property cell is cross-fitted and
declares `stratify_folds="none"`, which reaches its outer folds and the Super Learner's inner
folds. No cell declares `q_bounds`, because every law in this row has a binary outcome, whose
outcome scaler is already the identity.

## Accuracy against known truth

<!-- generated: accuracy -->
| law | estimand | what was tested | implementation | bias (99% interval) | coverage | SE ratio | result |
| --- | --- | --- | --- | --- | --- | --- | --- |
| three-arm binary-outcome law, outcome-adaptive selector | `ate[low vs high]` | difference in counterfactual means, low versus high | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0020 to 0.0036 | 0.9525 | 0.9675 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ate[low vs high]` | difference in counterfactual means, low versus high | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0019 to 0.0036 | 0.9525 | 0.9700 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ate[medium vs high]` | difference in counterfactual means, medium versus high | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0034 to 0.0020 | 0.9350 | 0.9527 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ate[medium vs high]` | difference in counterfactual means, medium versus high | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0033 to 0.0021 | 0.9350 | 0.9527 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[high]` | counterfactual mean under treatment arm 'high' | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0020 to 0.0018 | 0.9450 | 0.9780 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[high]` | counterfactual mean under treatment arm 'high' | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0021 to 0.0018 | 0.9463 | 0.9786 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[low]` | counterfactual mean under treatment arm 'low' | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0013 to 0.0027 | 0.9413 | 0.9822 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[low]` | counterfactual mean under treatment arm 'low' | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0013 to 0.0027 | 0.9413 | 0.9844 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[medium]` | counterfactual mean under treatment arm 'medium' | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0027 to 0.0011 | 0.9325 | 0.9386 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[medium]` | counterfactual mean under treatment arm 'medium' | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0027 to 0.0012 | 0.9363 | 0.9391 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `or[low vs high]` | marginal odds ratio, low versus high, reported on the log scale | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0071 to 0.0158 | 0.9487 | 0.9678 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `or[low vs high]` | marginal odds ratio, low versus high, reported on the log scale | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0070 to 0.0159 | 0.9487 | 0.9703 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `or[medium vs high]` | marginal odds ratio, medium versus high, reported on the log scale | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0125 to 0.0108 | 0.9350 | 0.9529 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `or[medium vs high]` | marginal odds ratio, medium versus high, reported on the log scale | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0123 to 0.0111 | 0.9350 | 0.9529 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `rr[low vs high]` | marginal risk ratio, low versus high, reported on the log scale | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0041 to 0.0085 | 0.9525 | 0.9687 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `rr[low vs high]` | marginal risk ratio, low versus high, reported on the log scale | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0040 to 0.0085 | 0.9525 | 0.9710 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `rr[medium vs high]` | marginal risk ratio, medium versus high, reported on the log scale | `cleverly` multi-arm outcome-adaptive C-TMLE | -0.0057 to 0.0057 | 0.9413 | 0.9645 | pass |
| three-arm binary-outcome law, outcome-adaptive selector | `rr[medium vs high]` | marginal risk ratio, medium versus high, reported on the log scale | R `ctmle3` multi-arm outcome-adaptive TMLE | -0.0056 to 0.0058 | 0.9413 | 0.9646 | pass |
<!-- /generated -->

## Agreement with the canonical implementation

<!-- generated: agreement -->
Comparator-backed: a paired comparison with `ctmle3-multi-arm-oat` on the same samples. Each row reports estimate equivalence within the declared margin, the RMSE ratio and the coverage difference, and names its comparison kind.

| law | estimand | what was compared | paired difference | share of margin used | RMSE ratio bound | coverage difference | comparison kind | calibration resolution | result |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| three-arm binary-outcome law, outcome-adaptive selector | `ate[low vs high]` | difference in counterfactual means, low versus high | -0.000020 | 0.0043 | 1.0031 | 0 | matching implementation | 0.000452 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `ate[medium vs high]` | difference in counterfactual means, medium versus high | -0.000056 | 0.0125 | 1.0005 | 0 | matching implementation | 0.000446 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[high]` | counterfactual mean under treatment arm 'high' | 0.000025 | 0.0080 | 1.0010 | -0.0013 | matching implementation | 0.000390 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[low]` | counterfactual mean under treatment arm 'low' | 0.000006 | 0.0017 | 1.0025 | 0 | matching implementation | 0.000227 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `ey[medium]` | counterfactual mean under treatment arm 'medium' | -0.000030 | 0.0094 | 1.0010 | -0.0037 | matching implementation | 0.000366 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `or[low vs high]` | marginal odds ratio, low versus high, reported on the log scale | -0.000074 | 0.0023 | 1.0030 | 0 | matching implementation | 0.000454 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `or[medium vs high]` | marginal odds ratio, medium versus high, reported on the log scale | -0.000709 | 0.0124 | 1.0002 | 0 | matching implementation | 0.000438 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `rr[low vs high]` | marginal risk ratio, low versus high, reported on the log scale | -0.000063 | 0.0045 | 1.0027 | 0 | matching implementation | 0.000477 vs 0.0500 | equivalent |
| three-arm binary-outcome law, outcome-adaptive selector | `rr[medium vs high]` | marginal risk ratio, medium versus high, reported on the log scale | -0.000186 | 0.0117 | 1.0004 | 0 | matching implementation | 0.000456 vs 0.0500 | equivalent |
<!-- /generated -->

## Theory properties

<!-- generated: properties -->
| property | cell | role | what was tested | what must hold | measured | result |
| --- | --- | --- | --- | --- | --- | --- |
| `generated_design` | `estimated` | positive | the same design is estimated from the data, as a real fit does | SE ratio and coverage intervals both inside their calibration bands; the paired deficit is reported only | coverage 0.9267 to 0.9677, SE ratio 0.9694 to 1.1026 | **fail** |
| `generated_design` | `oracle_design` | positive | the outcome-adaptive design is supplied rather than estimated | SE ratio and coverage intervals both inside their calibration bands | coverage 0.9282 to 0.9688, SE ratio 0.9797 to 1.1151 | **fail** |
| `interval_calibration` | `correctly_specified` | positive | both nuisances are correctly specified | SE ratio and coverage intervals both inside their calibration bands | coverage 0.9357 to 0.9641, SE ratio 0.9668 to 1.0575 | pass |
| `power` | `alternative` | positive | the same test applied to a law with a real effect | rejection lower bound clears the minimum power | rejection 1, 0.9868 to 1 | pass |
| `robustness_contract` | `outcome_correct` | positive | the outcome regression is correct and the mechanism is not | bias interval inside the equivalence margin, SE ratio must remain between 0.1 and 10.0 | bias -0.0022 to 0.0022, margin 0.0052, SE ratio 0.9641 | pass |
| `robustness_contract` | `outcome_wrong` | control | the outcome regression is misspecified | bias interval must fall entirely outside the margin, SE ratio must remain between 0.1 and 10.0 | bias 0.0659 to 0.0733, margin 0.0088, SE ratio 1.0382 | pass |
| `root_n_and_efficiency` | `n_2000` | positive | bias, coverage and SE calibration at n = 2,000 | bias inside the margin, coverage clears the floor, SE ratio inside the sanity band | bias 0.0020, coverage 0.9149 to 0.9738, SE ratio 0.9825 | pass |
| `root_n_and_efficiency` | `n_500` | positive | bias, coverage and SE calibration at n = 500 | bias inside the margin, coverage clears the floor, SE ratio inside the band | bias -0.0041, coverage 0.9057 to 0.9683, SE ratio 1.0097 | pass |
| `root_n_and_efficiency` | `n_8000` | positive | bias, coverage and SE calibration at n = 8,000 | bias inside the margin, coverage clears the floor, SE ratio inside the sanity band | bias -0.000055, coverage 0.9180 to 0.9756, SE ratio 1.0294 | pass |
| `root_n_rate` | `empirical_sd` | positive | log empirical spread of the estimates regressed on log n across three sizes | slope interval inside the root-n band and excluding -1/4 | slope -0.5645 to -0.4715 | pass |
| `root_n_rate` | `reported_se` | positive | the same regression applied to the mean reported standard error | slope interval inside the root-n band and excluding -1/4 | slope -0.5136 to -0.5090 | pass |
| `type_i_error` | `sharp_null` | positive | a confounded law whose true contrast is exactly zero | one-sided rejection bound stays under the declared type-I ceiling | rejection 0.0575, 0.0317 to 0.0943 | pass |
<!-- /generated -->

## Measured values

| quantity | value | source |
| --- | --- | --- |
| `replicates` | 800 | primary replications |
| `n` | 1500 | observations per primary replication |
| `independent_tests_total` | 18 | implementation-estimand truth tests |
| `independent_tests_passed` | 18 | truth tests passing |
| `paired_tests_total` | 9 | paired comparison tests |
| `paired_tests_passed` | 9 | paired tests passing |
| `property_cells_total` | 12 | repeated-sampling property cells |
| `property_cells_passed` | 10 | property cells passing |
| `margin:confidence_level` | 0.9900 | Monte Carlo confidence level |
| `margin:alpha` | 0.0500 | nominal estimator size |
| `margin:nominal_coverage` | 0.9500 | nominal estimator coverage |
| `margin:bootstrap_replicates` | 10000 | resamples for bootstrap intervals |
| `margin:standardized_bias` | 0.2500 | bias equivalence margin in empirical SDs |
| `margin:union_model_se_lower` | 0.1000 | union-model SE-ratio screen, lower limit |
| `margin:union_model_se_upper` | 10 | union-model SE-ratio screen, upper limit |
| `margin:coverage_floor` | 0.9000 | lower coverage gate |
| `margin:over_coverage_ceiling` | 0.9900 | reported conservative-coverage ceiling |
| `margin:se_ratio_sanity_lower` | 0.8000 | lower SE-ratio screen |
| `margin:se_ratio_sanity_upper` | 1.2000 | upper SE-ratio screen |
| `margin:calibration_se_ratio_lower` | 0.9300 | lower calibration band |
| `margin:calibration_se_ratio_upper` | 1.0700 | upper calibration band |
| `margin:calibration_coverage_lower` | 0.9200 | lower calibration-coverage band |
| `margin:calibration_coverage_upper` | 0.9800 | upper calibration-coverage band |
| `margin:type_i_ceiling` | 0.1000 | largest supported type-I rate |
| `margin:paired_difference` | 0.1500 | paired similarity margin in empirical SDs |
| `margin:rmse_noninferiority` | 1.1000 | RMSE-ratio non-inferiority margin |
| `margin:coverage_noninferiority` | -0.0250 | coverage-difference non-inferiority margin |
| `margin:calibration_noninferiority` | 0.0500 | SE-calibration non-inferiority margin |
| `margin:minimum_power` | 0.8000 | lower power gate |
| `margin:root_n_slope` | -0.5000 | root-n contraction target |
| `margin:root_n_slope_lower` | -0.6250 | lower contraction band |
| `margin:root_n_slope_upper` | -0.3750 | upper contraction band |
| `margin:excluded_slope` | -0.2500 | slower rate the interval must exclude |

## Limitations

These cells now measure a diagnostic that the package does not publish as inference. Every shared
`strategy="oat"` fit takes the `"generated_design_plugin"` status, which
[RM20](https://github.com/esbraun/cleverly-tmle/blob/4ce96cda2bda93ba9233026977e3ff63ea3e0003/docs/roadmap.md#rm20-intervals-outside-every-claimed-contract) decides. `ci`, `pvalue`,
and `std_error` refuse, and the fit reports `plugin_std_error` and `plugin_interval` instead.

The `std_error`, `ci_lower`, `ci_upper`, and `covered` columns of this study read those two
accessors. The numbers are the ones the committed artifacts carry. Each accessor calls the same
body as the refused one, so the reframing changed no arithmetic. No regeneration was run. The
coverage, calibration, null-size, and power cells therefore describe that diagnostic.

This row has reporting policy, so its red cells publish. Both generated-design SE-ratio intervals
leave the calibration band. Their paired difference includes zero, which is reported but is not a
failure criterion: the relevant question is whether each reported standard error calibrates to
its own sampling spread.
[F19](../../roadmap.md#f19-outcome-adaptive-c-tmle-generated-design-inference) records what a
source must settle before that changes. The [red-cell ledger](red-cells.md) lists each red cell
of this study with the roadmap ask that owns it.

The n = 500 coverage bound was red on the treatment-stratified split this row used before, and it
clears the floor on the unstratified split it now declares. The budget, the size, the seed and the
margins did not move, so read that cell as one Monte Carlo resolution at 400 replications rather
than as a demonstrated gain.

The row draws every cross-fitted split without reading a label. It does not establish a stratified
split, a grouped split, or a supplied plan. It says nothing about a continuous outcome whose
support the analyst does not know, because every law here has a binary outcome.

The archived R comparison requires numeric arm codes because of its counterfactual-prediction
adapter. The study covers one binary-outcome law, ordinary GLM nuisance fits, pointwise intervals,
and one outcome-adaptive strategy. It does not transfer the result to selector-based C-TMLE,
cross-fitted primary fitting, flexible learners, simultaneous inference, or severe
practical-positivity violations. It excludes missing outcomes, weights, clusters, fold repeats,
and longitudinal treatment.