Outcome-adaptive multi-arm C-TMLE

This study compares outcome-adaptive C-TMLE for a labelled three-arm treatment. The canonical comparison uses archived R ctmle3 at pinned commit a4ea77b. Both implementations receive identical draws and start from corresponding empirical treatment mechanisms and correctly specified outcome regressions.

The study uses the family-wide three-arm binary law and reports arm means, ATEs, risk ratios, and odds ratios against the high arm. Its property design separately checks the method’s outcome-regression robustness contract and the precision cost of estimating the generated outcome-adaptive design.

This study fits the shared design, oat_design="shared". Every shared fit has the generated_design_plugin status. The coverage and standard-error ratios below measure plugin_interval and plugin_std_error as diagnostics; their pass labels do not authorize ci, pvalue, or std_error. The two red generated-design property cells remain open under F19.

The primary fit draws no split at all. It is not cross-fitted, and the outcome-adaptive strategy scores no candidate path, so it needs no selection folds. Every property cell is cross-fitted and declares stratify_folds="none", which reaches its outer folds and the Super Learner’s inner folds. No cell declares q_bounds, because every law in this row has a binary outcome, whose outcome scaler is already the identity.

Accuracy against known truth

law	estimand	what was tested	implementation	bias (99% interval)	coverage	SE ratio	result
three-arm binary-outcome law, outcome-adaptive selector	ate[low vs high]	difference in counterfactual means, low versus high	cleverly multi-arm outcome-adaptive C-TMLE	-0.0020 to 0.0036	0.9525	0.9675	pass
three-arm binary-outcome law, outcome-adaptive selector	ate[low vs high]	difference in counterfactual means, low versus high	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0019 to 0.0036	0.9525	0.9700	pass
three-arm binary-outcome law, outcome-adaptive selector	ate[medium vs high]	difference in counterfactual means, medium versus high	cleverly multi-arm outcome-adaptive C-TMLE	-0.0034 to 0.0020	0.9350	0.9527	pass
three-arm binary-outcome law, outcome-adaptive selector	ate[medium vs high]	difference in counterfactual means, medium versus high	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0033 to 0.0021	0.9350	0.9527	pass
three-arm binary-outcome law, outcome-adaptive selector	ey[high]	counterfactual mean under treatment arm ‘high’	cleverly multi-arm outcome-adaptive C-TMLE	-0.0020 to 0.0018	0.9450	0.9780	pass
three-arm binary-outcome law, outcome-adaptive selector	ey[high]	counterfactual mean under treatment arm ‘high’	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0021 to 0.0018	0.9463	0.9786	pass
three-arm binary-outcome law, outcome-adaptive selector	ey[low]	counterfactual mean under treatment arm ‘low’	cleverly multi-arm outcome-adaptive C-TMLE	-0.0013 to 0.0027	0.9413	0.9822	pass
three-arm binary-outcome law, outcome-adaptive selector	ey[low]	counterfactual mean under treatment arm ‘low’	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0013 to 0.0027	0.9413	0.9844	pass
three-arm binary-outcome law, outcome-adaptive selector	ey[medium]	counterfactual mean under treatment arm ‘medium’	cleverly multi-arm outcome-adaptive C-TMLE	-0.0027 to 0.0011	0.9325	0.9386	pass
three-arm binary-outcome law, outcome-adaptive selector	ey[medium]	counterfactual mean under treatment arm ‘medium’	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0027 to 0.0012	0.9363	0.9391	pass
three-arm binary-outcome law, outcome-adaptive selector	or[low vs high]	marginal odds ratio, low versus high, reported on the log scale	cleverly multi-arm outcome-adaptive C-TMLE	-0.0071 to 0.0158	0.9487	0.9678	pass
three-arm binary-outcome law, outcome-adaptive selector	or[low vs high]	marginal odds ratio, low versus high, reported on the log scale	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0070 to 0.0159	0.9487	0.9703	pass
three-arm binary-outcome law, outcome-adaptive selector	or[medium vs high]	marginal odds ratio, medium versus high, reported on the log scale	cleverly multi-arm outcome-adaptive C-TMLE	-0.0125 to 0.0108	0.9350	0.9529	pass
three-arm binary-outcome law, outcome-adaptive selector	or[medium vs high]	marginal odds ratio, medium versus high, reported on the log scale	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0123 to 0.0111	0.9350	0.9529	pass
three-arm binary-outcome law, outcome-adaptive selector	rr[low vs high]	marginal risk ratio, low versus high, reported on the log scale	cleverly multi-arm outcome-adaptive C-TMLE	-0.0041 to 0.0085	0.9525	0.9687	pass
three-arm binary-outcome law, outcome-adaptive selector	rr[low vs high]	marginal risk ratio, low versus high, reported on the log scale	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0040 to 0.0085	0.9525	0.9710	pass
three-arm binary-outcome law, outcome-adaptive selector	rr[medium vs high]	marginal risk ratio, medium versus high, reported on the log scale	cleverly multi-arm outcome-adaptive C-TMLE	-0.0057 to 0.0057	0.9413	0.9645	pass
three-arm binary-outcome law, outcome-adaptive selector	rr[medium vs high]	marginal risk ratio, medium versus high, reported on the log scale	R ctmle3 multi-arm outcome-adaptive TMLE	-0.0056 to 0.0058	0.9413	0.9646	pass

Agreement with the canonical implementation

Comparator-backed: a paired comparison with ctmle3-multi-arm-oat on the same samples. Each row reports estimate equivalence within the declared margin, the RMSE ratio and the coverage difference, and names its comparison kind.

law	estimand	what was compared	paired difference	share of margin used	RMSE ratio bound	coverage difference	comparison kind	calibration resolution	result
three-arm binary-outcome law, outcome-adaptive selector	ate[low vs high]	difference in counterfactual means, low versus high	-0.000020	0.0043	1.0031	0	matching implementation	0.000452 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	ate[medium vs high]	difference in counterfactual means, medium versus high	-0.000056	0.0125	1.0005	0	matching implementation	0.000446 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	ey[high]	counterfactual mean under treatment arm ‘high’	0.000025	0.0080	1.0010	-0.0013	matching implementation	0.000390 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	ey[low]	counterfactual mean under treatment arm ‘low’	0.000006	0.0017	1.0025	0	matching implementation	0.000227 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	ey[medium]	counterfactual mean under treatment arm ‘medium’	-0.000030	0.0094	1.0010	-0.0037	matching implementation	0.000366 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	or[low vs high]	marginal odds ratio, low versus high, reported on the log scale	-0.000074	0.0023	1.0030	0	matching implementation	0.000454 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	or[medium vs high]	marginal odds ratio, medium versus high, reported on the log scale	-0.000709	0.0124	1.0002	0	matching implementation	0.000438 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	rr[low vs high]	marginal risk ratio, low versus high, reported on the log scale	-0.000063	0.0045	1.0027	0	matching implementation	0.000477 vs 0.0500	equivalent
three-arm binary-outcome law, outcome-adaptive selector	rr[medium vs high]	marginal risk ratio, medium versus high, reported on the log scale	-0.000186	0.0117	1.0004	0	matching implementation	0.000456 vs 0.0500	equivalent

Theory properties

property	cell	role	what was tested	what must hold	measured	result
generated_design	estimated	positive	the same design is estimated from the data, as a real fit does	SE ratio and coverage intervals both inside their calibration bands; the paired deficit is reported only	coverage 0.9267 to 0.9677, SE ratio 0.9694 to 1.1026	fail
generated_design	oracle_design	positive	the outcome-adaptive design is supplied rather than estimated	SE ratio and coverage intervals both inside their calibration bands	coverage 0.9282 to 0.9688, SE ratio 0.9797 to 1.1151	fail
interval_calibration	correctly_specified	positive	both nuisances are correctly specified	SE ratio and coverage intervals both inside their calibration bands	coverage 0.9357 to 0.9641, SE ratio 0.9668 to 1.0575	pass
power	alternative	positive	the same test applied to a law with a real effect	rejection lower bound clears the minimum power	rejection 1, 0.9868 to 1	pass
robustness_contract	outcome_correct	positive	the outcome regression is correct and the mechanism is not	bias interval inside the equivalence margin, SE ratio must remain between 0.1 and 10.0	bias -0.0022 to 0.0022, margin 0.0052, SE ratio 0.9641	pass
robustness_contract	outcome_wrong	control	the outcome regression is misspecified	bias interval must fall entirely outside the margin, SE ratio must remain between 0.1 and 10.0	bias 0.0659 to 0.0733, margin 0.0088, SE ratio 1.0382	pass
root_n_and_efficiency	n_2000	positive	bias, coverage and SE calibration at n = 2,000	bias inside the margin, coverage clears the floor, SE ratio inside the sanity band	bias 0.0020, coverage 0.9149 to 0.9738, SE ratio 0.9825	pass
root_n_and_efficiency	n_500	positive	bias, coverage and SE calibration at n = 500	bias inside the margin, coverage clears the floor, SE ratio inside the band	bias -0.0041, coverage 0.9057 to 0.9683, SE ratio 1.0097	pass
root_n_and_efficiency	n_8000	positive	bias, coverage and SE calibration at n = 8,000	bias inside the margin, coverage clears the floor, SE ratio inside the sanity band	bias -0.000055, coverage 0.9180 to 0.9756, SE ratio 1.0294	pass
root_n_rate	empirical_sd	positive	log empirical spread of the estimates regressed on log n across three sizes	slope interval inside the root-n band and excluding -1/4	slope -0.5645 to -0.4715	pass
root_n_rate	reported_se	positive	the same regression applied to the mean reported standard error	slope interval inside the root-n band and excluding -1/4	slope -0.5136 to -0.5090	pass
type_i_error	sharp_null	positive	a confounded law whose true contrast is exactly zero	one-sided rejection bound stays under the declared type-I ceiling	rejection 0.0575, 0.0317 to 0.0943	pass

Measured values

quantity	value	source
replicates	800	primary replications
n	1500	observations per primary replication
independent_tests_total	18	implementation-estimand truth tests
independent_tests_passed	18	truth tests passing
paired_tests_total	9	paired comparison tests
paired_tests_passed	9	paired tests passing
property_cells_total	12	repeated-sampling property cells
property_cells_passed	10	property cells passing
margin:confidence_level	0.9900	Monte Carlo confidence level
margin:alpha	0.0500	nominal estimator size
margin:nominal_coverage	0.9500	nominal estimator coverage
margin:bootstrap_replicates	10000	resamples for bootstrap intervals
margin:standardized_bias	0.2500	bias equivalence margin in empirical SDs
margin:union_model_se_lower	0.1000	union-model SE-ratio screen, lower limit
margin:union_model_se_upper	10	union-model SE-ratio screen, upper limit
margin:coverage_floor	0.9000	lower coverage gate
margin:over_coverage_ceiling	0.9900	reported conservative-coverage ceiling
margin:se_ratio_sanity_lower	0.8000	lower SE-ratio screen
margin:se_ratio_sanity_upper	1.2000	upper SE-ratio screen
margin:calibration_se_ratio_lower	0.9300	lower calibration band
margin:calibration_se_ratio_upper	1.0700	upper calibration band
margin:calibration_coverage_lower	0.9200	lower calibration-coverage band
margin:calibration_coverage_upper	0.9800	upper calibration-coverage band
margin:type_i_ceiling	0.1000	largest supported type-I rate
margin:paired_difference	0.1500	paired similarity margin in empirical SDs
margin:rmse_noninferiority	1.1000	RMSE-ratio non-inferiority margin
margin:coverage_noninferiority	-0.0250	coverage-difference non-inferiority margin
margin:calibration_noninferiority	0.0500	SE-calibration non-inferiority margin
margin:minimum_power	0.8000	lower power gate
margin:root_n_slope	-0.5000	root-n contraction target
margin:root_n_slope_lower	-0.6250	lower contraction band
margin:root_n_slope_upper	-0.3750	upper contraction band
margin:excluded_slope	-0.2500	slower rate the interval must exclude

Limitations

These cells now measure a diagnostic that the package does not publish as inference. Every shared strategy="oat" fit takes the "generated_design_plugin" status, which RM20 decides. ci, pvalue, and std_error refuse, and the fit reports plugin_std_error and plugin_interval instead.

The std_error, ci_lower, ci_upper, and covered columns of this study read those two accessors. The numbers are the ones the committed artifacts carry. Each accessor calls the same body as the refused one, so the reframing changed no arithmetic. No regeneration was run. The coverage, calibration, null-size, and power cells therefore describe that diagnostic.

This row has reporting policy, so its red cells publish. Both generated-design SE-ratio intervals leave the calibration band. Their paired difference includes zero, which is reported but is not a failure criterion: the relevant question is whether each reported standard error calibrates to its own sampling spread. F19 records what a source must settle before that changes. The red-cell ledger lists each red cell of this study with the roadmap ask that owns it.

The n = 500 coverage bound was red on the treatment-stratified split this row used before, and it clears the floor on the unstratified split it now declares. The budget, the size, the seed and the margins did not move, so read that cell as one Monte Carlo resolution at 400 replications rather than as a demonstrated gain.

The row draws every cross-fitted split without reading a label. It does not establish a stratified split, a grouped split, or a supplied plan. It says nothing about a continuous outcome whose support the analyst does not know, because every law here has a binary outcome.

The archived R comparison requires numeric arm codes because of its counterfactual-prediction adapter. The study covers one binary-outcome law, ordinary GLM nuisance fits, pointwise intervals, and one outcome-adaptive strategy. It does not transfer the result to selector-based C-TMLE, cross-fitted primary fitting, flexible learners, simultaneous inference, or severe practical-positivity violations. It excludes missing outcomes, weights, clusters, fold repeats, and longitudinal treatment.