Outcome-adaptive per-arm C-TMLE with strata and missing data

This study measures CTMLE(strategy="oat") with the default oat_design="per_arm" on fits with baseline strata (strata=), a missing outcome (delta=) or a missing treatment (treatment_delta=). For arm \(a\) the design is the product of three factors, each fitted on the one column \(\bar Q_n(a, W)\) inside each stratum. The collaborative TMLE reference states the factors and the routes. The base result is Theorem 1 of Benkeser, Cai and van der Laan (2020), applied to the composite indicator \(C_a = \Delta_A \Delta 1\{A = a\}\).

The study pairs the fit with R drtmle 1.1.2 at commit 538a3a2, with adapt_g = TRUE, DeltaY, DeltaA and stratify = TRUE. The runner calls drtmle once per arm on the binary recode \(1\{A = a\}\), as the per-arm study does. On the strata law it calls drtmle on the rows of each stratum. Both sides read this package’s initial outcome regression.

The study publishes under the reporting policy, and it claims the point estimate only. Every oat fit takes the generated_design_plugin status and reports no interval. The coverage and standard-error columns below measure plugin_interval and plugin_std_error as diagnostics. Their pass labels do not authorize ci, pvalue or std_error.

F19 owns the red truth rows and the three red rungs of the root_n_and_efficiency ladder. The study re-declared the ladder after the first run, and a rerun of the property phase measured it. The re-declared ladder gives the reason.

What was compared

setting	cleverly	R drtmle
datasets	800 samples of 2,250 rows for each law	the identical rows
strata law	\(W = (W_1, W_2, W_3)\) is independent standard normal, and the stratum \(S \in \{0, 1, 2\}\) has the masses 0.5, 0.3 and 0.2. The binary outcome reads the per-arm study’s two-arm index plus a stratum term. \(W_3\) enters only the treatment and observation laws, so it is an instrument. No row is missing	the same
missing_outcome law	the strata law at \(S = 0\), with an outcome that is missing at random given \(A\), \(W_1\) and \(W_3\)	the same
three_arm_missing law	the per-arm study’s three-arm law, with an outcome that is missing at random	the same
missing_treatment law	the missing_outcome law, with a treatment that is missing at random given \(A\) and \(W_3\)	the same
reported parameters	strata: the arm means and the ATE, marginal and in each stratum. Two arms: the arm means, ATE, RR and OR. Three arms: the arm means and the ATE, RR and OR of arms 1 and 2 against arm 0	the same
outcome regression	an unpenalized logistic regression on the arm indicators, \(W\), the stratum indicators, \(A \times W\) and \(A \times S\). It is correctly specified	this package’s initial fit, passed as Qn
design	for each arm, factor and stratum, an unpenalized logistic regression of the factor’s target on \((l, l^2, l^3)\), \(l = \operatorname{logit} \bar Q_n(a, W)\). g_bounds=(0.01, 0.99) and nuisance_bound=0.01	glm_g for A, DeltaY and DeltaA with the same cubic
targeting	one logistic fluctuation for each arm, and for each stratum on the strata law	the TMLE stage of drtmle
intervals	pointwise 95% Wald from the per-arm curve, with RR and OR on the log scale. They are diagnostics	the same, from tmle$cov

The runner rebuilds each arm’s TMLE curve on its logistic submodel. It refuses a fit unless the rebuilt variance equals tmle$cov[1, 1] to 1e-8. On the strata law it embeds each stratum curve. The marginal curve is the \(P_n(S = s)\) mixture with its mass term. tests/studies/oat_per_arm_extension_laws.py declares the laws and computes each truth by quadrature.

Accuracy against known truth

law	estimand	what was tested	implementation	bias (99% interval)	coverage	SE ratio	result
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate	average treatment effect	cleverly per-arm outcome-adaptive C-TMLE	-0.0040 to 0.000874	0.8750	0.8249	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate	average treatment effect	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0040 to 0.000874	0.8750	0.8248	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey0	counterfactual mean under no treatment	cleverly per-arm outcome-adaptive C-TMLE	-0.000251 to 0.0033	0.8975	0.8426	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey0	counterfactual mean under no treatment	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.000251 to 0.0033	0.8975	0.8426	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey1	counterfactual mean under treatment	cleverly per-arm outcome-adaptive C-TMLE	-0.0016 to 0.0016	0.8738	0.8094	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey1	counterfactual mean under treatment	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0016 to 0.0016	0.8738	0.8094	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or	marginal odds ratio, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0152 to 0.0056	0.8875	0.8237	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or	marginal odds ratio, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0152 to 0.0056	0.8875	0.8237	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr	marginal risk ratio, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0076 to 0.0019	0.8862	0.8323	fail
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr	marginal risk ratio, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0076 to 0.0019	0.8862	0.8323	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ate	average treatment effect	cleverly per-arm outcome-adaptive C-TMLE	-0.0023 to 0.0031	0.9012	0.8231	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ate	average treatment effect	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0023 to 0.0032	0.9012	0.8231	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ey0	counterfactual mean under no treatment	cleverly per-arm outcome-adaptive C-TMLE	-0.0017 to 0.0021	0.9125	0.8751	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ey0	counterfactual mean under no treatment	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0017 to 0.0021	0.9125	0.8750	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ey1	counterfactual mean under treatment	cleverly per-arm outcome-adaptive C-TMLE	-0.0013 to 0.0026	0.8675	0.7576	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ey1	counterfactual mean under treatment	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0013 to 0.0026	0.8675	0.7576	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	or	marginal odds ratio, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0076 to 0.0158	0.8950	0.8163	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	or	marginal odds ratio, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0076 to 0.0158	0.8950	0.8163	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	rr	marginal risk ratio, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0041 to 0.0064	0.9062	0.8453	fail
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	rr	marginal risk ratio, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0041 to 0.0064	0.9062	0.8452	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate	average treatment effect	cleverly per-arm outcome-adaptive C-TMLE	-0.000778 to 0.0033	0.9187	0.9014	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate	average treatment effect	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.000772 to 0.0033	0.9187	0.9013	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=0]	average treatment effect, in stratum S=0	cleverly per-arm outcome-adaptive C-TMLE	-0.0013 to 0.0043	0.9250	0.9175	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=0]	average treatment effect, in stratum S=0	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0013 to 0.0043	0.9250	0.9175	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=1]	average treatment effect, in stratum S=1	cleverly per-arm outcome-adaptive C-TMLE	-0.0024 to 0.0043	0.9487	0.9916	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=1]	average treatment effect, in stratum S=1	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0024 to 0.0043	0.9487	0.9917	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=2]	average treatment effect, in stratum S=2	cleverly per-arm outcome-adaptive C-TMLE	-0.0032 to 0.0054	0.9437	0.9897	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=2]	average treatment effect, in stratum S=2	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0032 to 0.0054	0.9437	0.9897	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0	counterfactual mean under no treatment	cleverly per-arm outcome-adaptive C-TMLE	-0.0029 to 0.000181	0.9100	0.8733	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0	counterfactual mean under no treatment	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0029 to 0.000176	0.9100	0.8733	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=0]	counterfactual mean under no treatment, in stratum S=0	cleverly per-arm outcome-adaptive C-TMLE	-0.0040 to -0.000028	0.9313	0.9131	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=0]	counterfactual mean under no treatment, in stratum S=0	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0040 to -0.000028	0.9313	0.9131	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=1]	counterfactual mean under no treatment, in stratum S=1	cleverly per-arm outcome-adaptive C-TMLE	-0.0037 to 0.0015	0.9437	0.9663	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=1]	counterfactual mean under no treatment, in stratum S=1	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0037 to 0.0015	0.9437	0.9663	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=2]	counterfactual mean under no treatment, in stratum S=2	cleverly per-arm outcome-adaptive C-TMLE	-0.0035 to 0.0032	0.9413	0.9841	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=2]	counterfactual mean under no treatment, in stratum S=2	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0035 to 0.0032	0.9413	0.9841	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1	counterfactual mean under treatment	cleverly per-arm outcome-adaptive C-TMLE	-0.0015 to 0.0012	0.9038	0.9006	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1	counterfactual mean under treatment	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0015 to 0.0012	0.9038	0.9006	fail
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=0]	counterfactual mean under treatment, in stratum S=0	cleverly per-arm outcome-adaptive C-TMLE	-0.0024 to 0.0014	0.9363	0.9241	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=0]	counterfactual mean under treatment, in stratum S=0	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0024 to 0.0014	0.9363	0.9241	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=1]	counterfactual mean under treatment, in stratum S=1	cleverly per-arm outcome-adaptive C-TMLE	-0.0024 to 0.0021	0.9450	0.9747	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=1]	counterfactual mean under treatment, in stratum S=1	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0024 to 0.0021	0.9450	0.9746	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=2]	counterfactual mean under treatment, in stratum S=2	cleverly per-arm outcome-adaptive C-TMLE	-0.0017 to 0.0036	0.9525	0.9923	pass
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=2]	counterfactual mean under treatment, in stratum S=2	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0017 to 0.0036	0.9525	0.9923	pass
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate[1 vs 0]	difference in counterfactual means, 1 versus 0	cleverly per-arm outcome-adaptive C-TMLE	-0.0024 to 0.0038	0.9062	0.8510	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate[1 vs 0]	difference in counterfactual means, 1 versus 0	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0024 to 0.0038	0.9062	0.8510	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate[2 vs 0]	difference in counterfactual means, 2 versus 0	cleverly per-arm outcome-adaptive C-TMLE	-0.0025 to 0.0037	0.9187	0.9054	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate[2 vs 0]	difference in counterfactual means, 2 versus 0	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0025 to 0.0037	0.9187	0.9054	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[0]	counterfactual mean under treatment arm ‘0’	cleverly per-arm outcome-adaptive C-TMLE	-0.0018 to 0.0024	0.9337	0.9553	pass
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[0]	counterfactual mean under treatment arm ‘0’	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0018 to 0.0024	0.9337	0.9553	pass
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[1]	counterfactual mean under treatment arm ‘1’	cleverly per-arm outcome-adaptive C-TMLE	-0.0012 to 0.0033	0.8575	0.7364	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[1]	counterfactual mean under treatment arm ‘1’	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0012 to 0.0033	0.8575	0.7364	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[2]	counterfactual mean under treatment arm ‘2’	cleverly per-arm outcome-adaptive C-TMLE	-0.0015 to 0.0034	0.8800	0.8012	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[2]	counterfactual mean under treatment arm ‘2’	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0015 to 0.0034	0.8800	0.8012	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or[1 vs 0]	marginal odds ratio, 1 versus 0, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0089 to 0.0160	0.9100	0.8537	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or[1 vs 0]	marginal odds ratio, 1 versus 0, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0089 to 0.0160	0.9100	0.8538	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or[2 vs 0]	marginal odds ratio, 2 versus 0, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0086 to 0.0172	0.9187	0.9047	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or[2 vs 0]	marginal odds ratio, 2 versus 0, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0086 to 0.0172	0.9187	0.9046	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr[1 vs 0]	marginal risk ratio, 1 versus 0, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0051 to 0.0084	0.9137	0.8704	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr[1 vs 0]	marginal risk ratio, 1 versus 0, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0051 to 0.0084	0.9137	0.8704	fail
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr[2 vs 0]	marginal risk ratio, 2 versus 0, reported on the log scale	cleverly per-arm outcome-adaptive C-TMLE	-0.0051 to 0.0077	0.9300	0.9288	pass
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr[2 vs 0]	marginal risk ratio, 2 versus 0, reported on the log scale	R drtmle with adapt_g, by a binary recode per arm, with DeltaY, DeltaA and one fit per stratum	-0.0050 to 0.0077	0.9300	0.9288	pass

Agreement with the canonical implementation

Comparator-backed: a paired comparison with drtmle-r-oat-per-arm-strata-missing on the same samples. Each row reports estimate equivalence within the declared margin, the RMSE ratio and the coverage difference, and names its comparison kind.

law	estimand	what was compared	paired difference	share of margin used	RMSE ratio bound	coverage difference	comparison kind	calibration resolution	result
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate	average treatment effect	-1.928e-07	0.000048	1.0000	0	matching implementation	0.000003 vs 0.0500	equivalent
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey0	counterfactual mean under no treatment	1.930e-07	0.000065	1.0000	0	matching implementation	0.000003 vs 0.0500	equivalent
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey1	counterfactual mean under treatment	1.686e-10	6.352e-08	1.0000	0	matching implementation	2.623e-09 vs 0.0500	equivalent
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or	marginal odds ratio, reported on the log scale	-0.000002	0.000049	1.0000	0	matching implementation	0.000003 vs 0.0500	equivalent
two arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr	marginal risk ratio, reported on the log scale	-6.895e-07	0.000058	1.0000	0	matching implementation	0.000003 vs 0.0500	equivalent
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ate	average treatment effect	-0.000001	0.000293	1.0000	0	matching implementation	0.000024 vs 0.0500	equivalent
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ey0	counterfactual mean under no treatment	0.000001	0.000405	1.0000	0	matching implementation	0.000027 vs 0.0500	equivalent
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	ey1	counterfactual mean under treatment	-2.499e-08	0.000008	1.0000	0	matching implementation	0.000002 vs 0.0500	equivalent
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	or	marginal odds ratio, reported on the log scale	-0.000014	0.000281	1.0000	0	matching implementation	0.000023 vs 0.0500	equivalent
two arms, treatment unrecorded at random given W3 and the arm, outcomes missing at random	rr	marginal risk ratio, reported on the log scale	-0.000005	0.000350	1.0000	0	matching implementation	0.000023 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate	average treatment effect	-0.000007	0.0020	1.0001	0	related target	0.000196 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=0]	average treatment effect, in stratum S=0	-1.936e-08	0.000004	1.0000	0	related target	0.000002 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=1]	average treatment effect, in stratum S=1	-0.000002	0.000408	1.0001	0	related target	0.000042 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ate[S=2]	average treatment effect, in stratum S=2	-0.000030	0.0043	1.0005	0	related target	0.000727 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0	counterfactual mean under no treatment	0.000006	0.0022	1.0002	0	related target	0.000237 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=0]	counterfactual mean under no treatment, in stratum S=0	-3.448e-09	0.000001	1.0000	0	related target	3.806e-07 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=1]	counterfactual mean under no treatment, in stratum S=1	3.434e-07	0.000080	1.0000	0	related target	0.000014 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey0[S=2]	counterfactual mean under no treatment, in stratum S=2	0.000028	0.0050	1.0007	0	related target	0.000838 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1	counterfactual mean under treatment	-0.000001	0.000485	1.0000	0	related target	0.000029 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=0]	counterfactual mean under treatment, in stratum S=0	-2.281e-08	0.000007	1.0000	0	related target	0.000004 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=1]	counterfactual mean under treatment, in stratum S=1	-0.000002	0.000513	1.0000	0	related target	0.000285 vs 0.0500	equivalent
two arms, three baseline strata, an instrument, complete data, correct outcome regression	ey1[S=2]	counterfactual mean under treatment, in stratum S=2	-0.000003	0.000628	1.0001	0	related target	0.000165 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate[1 vs 0]	difference in counterfactual means, 1 versus 0	-0.000004	0.000847	1.0002	0	matching implementation	0.000102 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ate[2 vs 0]	difference in counterfactual means, 2 versus 0	-0.000004	0.000850	1.0000	0	matching implementation	0.000066 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[0]	counterfactual mean under treatment arm ‘0’	0.000002	0.000561	1.0001	0	matching implementation	0.000062 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[1]	counterfactual mean under treatment arm ‘1’	-0.000002	0.000632	1.0002	0	matching implementation	0.000106 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	ey[2]	counterfactual mean under treatment arm ‘2’	-0.000002	0.000601	1.0001	0	matching implementation	0.000085 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or[1 vs 0]	marginal odds ratio, 1 versus 0, reported on the log scale	-0.000023	0.000783	1.0001	0	matching implementation	0.000105 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	or[2 vs 0]	marginal odds ratio, 2 versus 0, reported on the log scale	-0.000038	0.000895	1.0000	0	matching implementation	0.000065 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr[1 vs 0]	marginal risk ratio, 1 versus 0, reported on the log scale	-0.000011	0.000821	1.0001	0	matching implementation	0.000102 vs 0.0500	equivalent
three arms, outcomes missing at random given W1, W3 and the arm, correct outcome regression	rr[2 vs 0]	marginal risk ratio, 2 versus 0, reported on the log scale	-0.000013	0.000855	1.0000	0	matching implementation	0.000069 vs 0.0500	equivalent

The paired margin of 0.15 empirical SDs in this comparator-backed study cannot detect the design change. The shipped construction differs from this design by 0.024 to 0.079 empirical SDs on average. The file design-gap.csv reports these gaps for the strata and missing_outcome laws. Each gap is inside the margin. The RMSE ratio of the two designs is 0.996 to 1.005 on those rows. Thus the RMSE ratio carries no verdict here.

test_the_pairing_reads_the_design_change in tests/unit/test_ctmle_oat_per_arm_strata_missing_study_design.py reads the gap itself. On every paired row, the mean of \(|\hat\psi_{\text{cleverly}} - \hat\psi_{\text{drtmle}}|\) must be at most 0.005 empirical SDs. The largest measured value is 0.0008, for ey0[S=2] on the strata law. The smallest mean_design_gap in design-gap.csv is 0.024. Thus the shipped design would fail this check on every reported row.

Theory properties

property	cell	role	what was tested	what must hold	measured	result
robustness_contract	ate__missing_outcome_crossfit	positive	average treatment effect: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0016 to 0.0034, margin 0.0069, SE ratio 0.8488	pass
robustness_contract	ate__strata_missing	positive	average treatment effect: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0020 to 0.0020, margin 0.0055, SE ratio 0.9447	pass
robustness_contract	ate__strata_missing_outcome_wrong	control	average treatment effect: the strata law with the outcome regression without W1, n = 3,000	bias interval must fall entirely outside the margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias 0.0685 to 0.0731, margin 0.0062, SE ratio 0.9692	pass
robustness_contract	ate__weighted_strata_missing	positive	average treatment effect: the strata law under the declared known weight 0.5 + 1{W1 > 0}, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0025 to 0.0021, margin 0.0064, SE ratio 0.9112	pass
robustness_contract	ate_s0__strata_missing	positive	average treatment effect in baseline stratum S = 0: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0027 to 0.0025, margin 0.0071, SE ratio 0.9556	pass
robustness_contract	ate_s0__weighted_strata_missing	positive	average treatment effect in baseline stratum S = 0: the strata law under the declared known weight 0.5 + 1{W1 > 0}, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0029 to 0.0030, margin 0.0081, SE ratio 0.9446	pass
robustness_contract	ate_s1__strata_missing	positive	average treatment effect in baseline stratum S = 1: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0025 to 0.0050, margin 0.0102, SE ratio 0.9158	pass
robustness_contract	ate_s1__weighted_strata_missing	positive	average treatment effect in baseline stratum S = 1: the strata law under the declared known weight 0.5 + 1{W1 > 0}, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0044 to 0.0038, margin 0.0112, SE ratio 0.9659	pass
robustness_contract	ate_s2__strata_missing	positive	average treatment effect in baseline stratum S = 2: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0064 to 0.0030, margin 0.0129, SE ratio 0.9986	pass
robustness_contract	ate_s2__weighted_strata_missing	positive	average treatment effect in baseline stratum S = 2: the strata law under the declared known weight 0.5 + 1{W1 > 0}, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0060 to 0.0043, margin 0.0141, SE ratio 1.0161	pass
robustness_contract	ey0__missing_outcome_crossfit	positive	arm-0 mean: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0026 to 0.0012, margin 0.0052, SE ratio 0.8555	pass
robustness_contract	ey1__missing_outcome_crossfit	positive	arm-1 mean: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0015 to 0.0019, margin 0.0046, SE ratio 0.8361	pass
robustness_contract	ey_obs__missing_outcome_crossfit	positive	observed outcome mean under the natural course: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0011 to 0.0011, margin 0.0031, SE ratio 0.9920	pass
robustness_contract	ey_obs__natural_course_insample	positive	observed outcome mean under the natural course: missing outcomes, in-sample per-arm design, n = 2,250, natural-course estimands	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.000289 to 0.0019, margin 0.0030, SE ratio 0.9682	pass
robustness_contract	ey_obs__strata_missing	positive	observed outcome mean under the natural course: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.000866 to 0.000975, margin 0.0025, SE ratio 1.0171	pass
robustness_contract	ey_obs_s0__strata_missing	positive	observed outcome mean in baseline stratum S = 0: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0012 to 0.0014, margin 0.0036, SE ratio 0.9926	pass
robustness_contract	ey_obs_s1__strata_missing	positive	observed outcome mean in baseline stratum S = 1: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0015 to 0.0019, margin 0.0046, SE ratio 1.0114	pass
robustness_contract	ey_obs_s2__strata_missing	positive	observed outcome mean in baseline stratum S = 2: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0025 to 0.0018, margin 0.0059, SE ratio 1.0225	pass
robustness_contract	or__missing_outcome_crossfit	positive	marginal odds ratio: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias 0.0037 to 0.0606, margin 0.0780, SE ratio 0.8489	pass
robustness_contract	paf__missing_outcome_crossfit	positive	population attributable fraction: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0014 to 0.0041, margin 0.0075, SE ratio 0.7722	pass
robustness_contract	paf__natural_course_insample	positive	population attributable fraction: missing outcomes, in-sample per-arm design, n = 2,250, natural-course estimands	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0031 to 0.0020, margin 0.0069, SE ratio 0.7832	pass
robustness_contract	paf__strata_missing	positive	population attributable fraction: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0025 to 0.0020, margin 0.0061, SE ratio 0.9121	pass
robustness_contract	paf_s0__strata_missing	positive	population attributable fraction in baseline stratum S = 0: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0028 to 0.0027, margin 0.0075, SE ratio 0.8943	pass
robustness_contract	paf_s1__strata_missing	positive	population attributable fraction in baseline stratum S = 1: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0032 to 0.0048, margin 0.0110, SE ratio 0.9162	pass
robustness_contract	paf_s2__strata_missing	positive	population attributable fraction in baseline stratum S = 2: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0083 to 0.0034, margin 0.0160, SE ratio 1.0113	pass
robustness_contract	par__missing_outcome_crossfit	positive	population attributable risk: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.000810 to 0.0022, margin 0.0041, SE ratio 0.7632	pass
robustness_contract	par__natural_course_insample	positive	population attributable risk: missing outcomes, in-sample per-arm design, n = 2,250, natural-course estimands	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0016 to 0.0012, margin 0.0038, SE ratio 0.7746	pass
robustness_contract	par__strata_missing	positive	population attributable risk: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0014 to 0.0011, margin 0.0034, SE ratio 0.9067	pass
robustness_contract	par_s0__strata_missing	positive	population attributable risk in baseline stratum S = 0: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0016 to 0.0014, margin 0.0041, SE ratio 0.8909	pass
robustness_contract	par_s1__strata_missing	positive	population attributable risk in baseline stratum S = 1: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0018 to 0.0026, margin 0.0060, SE ratio 0.9166	pass
robustness_contract	par_s2__strata_missing	positive	population attributable risk in baseline stratum S = 2: three strata with missing outcomes, cross-fitted per-arm design per stratum, n = 3,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0043 to 0.0019, margin 0.0085, SE ratio 1.0132	pass
robustness_contract	rr__missing_outcome_crossfit	positive	marginal risk ratio: missing outcomes, five-fold cross-fitted per-arm design, n = 2,000	bias interval inside the equivalence margin, SE ratio inside (0.1, 10), SE ratio must remain between 0.1 and 10.0	bias -0.0014 to 0.0138, margin 0.0208, SE ratio 0.8567	pass
root_n_and_efficiency	n_2000	positive	bias, coverage and SE calibration at n = 2,000	bias inside the margin, coverage clears the floor, SE ratio inside the sanity band	bias 0.000923, coverage 0.8963 to 0.9460, SE ratio 0.9292	fail
root_n_and_efficiency	n_4000	positive	bias, coverage and SE calibration at n = 4,000	bias inside the margin, coverage clears the floor, SE ratio inside the sanity band	bias 0.000466, coverage 0.8992 to 0.9481, SE ratio 0.8955	fail
root_n_and_efficiency	n_8000	positive	bias, coverage and SE calibration at n = 8,000	bias inside the margin, coverage clears the floor, SE ratio inside the sanity band	bias 0.000447, coverage 0.8949 to 0.9449, SE ratio 0.8812	fail
root_n_rate	empirical_sd	positive	log empirical spread of the estimates regressed on log n across three sizes	slope interval inside the root-n band and excluding -1/4	slope -0.5636 to -0.4362	pass
root_n_rate	reported_se	positive	the same regression applied to the mean reported standard error	slope interval inside the root-n band and excluding -1/4	slope -0.5700 to -0.5183	pass

The robustness_contract cells measure the bias of the point estimate. Each positive cell passes when its bias interval is inside the equivalence margin. The control fits an outcome model without \(W_1\) and passes when its bias is outside the margin. The root_n_and_efficiency ladder fits the strata law with a missing outcome and five folds, and reads the marginal ATE.

The three rungs are n = 2,000, 4,000 and 8,000. Each rung has an equivalent bias, and each coverage interval reaches below the 0.90 floor. The root_n_rate rows read the slopes of the log empirical SD and of the log mean standard error on log n. Both slopes are inside the root-n band.

Each rr and or row records the standard error of \(\log \hat\psi\). The summary multiplies it by \(\hat\psi\), the delta-method standard error of \(\hat\psi\), before it compares it with the empirical SD of \(\hat\psi\). The first summary of the rerun did not, and it published SE ratios of 0.559 for rr and 0.322 for or. The current properties.csv was summarized again from the committed property-replicates.csv.gz, with no new fit. No verdict changed. test_the_ratio_cells_compare_one_scale checks both readings.

A mean SE ratio is sensitive to a few fits with a large standard error. The table gives the median beside the mean for the ladder and for the cells that have a tail fit. A tail fit has a standard error above twice the empirical SD of its cell. se_ratio_spread in tests/studies/ctmle_oat_per_arm_strata_missing_properties.py computes the table from property-replicates.csv.gz, and test_the_tail_fits_are_rare checks it.

cell	mean SE ratio	median SE ratio	tail fits of 800
n_2000	0.929	0.898	3
n_4000	0.896	0.893	0
n_8000	0.881	0.880	0
ate__strata_missing	0.945	0.925	2
ate_s2__strata_missing	0.999	0.946	2
ey_obs__strata_missing	1.017	1.011	2
ey_obs_s2__strata_missing	1.023	1.006	2
par__strata_missing	0.907	0.877	1
par_s2__strata_missing	1.013	0.948	2
paf__strata_missing	0.912	0.885	1
paf_s2__strata_missing	1.011	0.950	2
ate_s2__weighted_strata_missing	1.016	0.997	1
ate__strata_missing_outcome_wrong	0.969	0.907	5

In the strata_missing cells, the two tail fits are replications 271 and 605. They lift the mean SE ratio of the \(S = 2\) ate, par and paf cells 5% to 7% above its median. No cell outside the table and no primary row has a tail fit.

Measured values

Names beginning margin: are thresholds declared before the run. Everything else is measured from the committed results and checked at the precision printed.

quantity	value	source
replicates	800	primary replications per law
n	2250	observations per primary replication
independent_tests_total	62	implementation-estimand tests against the truth
independent_tests_passed	20	of those, passing
paired_tests_total	31	paired comparisons with R drtmle
paired_tests_passed	31	of those, passing
max_standardized_bias	0.0926	largest absolute primary bias in empirical standard deviations
min_coverage	0.8575	lowest primary coverage of the diagnostic interval
min_se_ratio_ci_lower	0.6898	lowest bootstrap primary SE-ratio endpoint
max_se_ratio_ci_upper	1.0584	highest bootstrap primary SE-ratio endpoint
max_margin_utilization	0.0050	largest share of the paired margin used
mechanism_fits_with_rows_at_bound	341	primary fits with one or more rows of a factor at a declared bound
mechanism_max_gap_off_bound	4.491e-07	largest gap between the two designs in a fit with no row at a bound
mechanism_max_gap	0.0100	largest gap between the two designs in any fit
properties[robustness_contract/ate__strata_missing]:se_ratio	0.9447	SE ratio of the marginal ATE on the strata law at n = 3,000
properties[robustness_contract/ate__strata_missing_outcome_wrong]:standardized_bias	2.8427	bias of the outcome-wrong control, in empirical standard deviations
margin:alpha	0.0500	nominal size of the reported intervals
margin:bootstrap_replicates	10000	resamples behind every bootstrap interval
margin:confidence_level	0.9900	confidence level of every Monte Carlo interval
margin:coverage_floor	0.9000	validity floor the coverage lower endpoint must clear
margin:coverage_noninferiority	-0.0250	smallest external-comparison coverage difference bound
margin:calibration_noninferiority	0.0500	largest external-comparison calibration excess bound
margin:calibration_coverage_lower	0.9200	calibration-cell coverage band, lower limit
margin:calibration_coverage_upper	0.9800	calibration-cell coverage band, upper limit
margin:calibration_se_ratio_lower	0.9300	calibration-cell SE-ratio band, lower limit
margin:calibration_se_ratio_upper	1.0700	calibration-cell SE-ratio band, upper limit
margin:minimum_power	0.8000	rejection lower bound the power cell must clear
margin:type_i_ceiling	0.1000	the rate the null cell must bound
margin:excluded_slope	-0.2500	slower rate a root-n interval must exclude
margin:nominal_coverage	0.9500	nominal coverage those intervals claim
margin:over_coverage_ceiling	0.9900	above this, coverage is conservative rather than invalid
margin:paired_difference	0.1500	paired similarity margin, in pooled empirical standard deviations
margin:rmse_noninferiority	1.1000	largest external-comparison RMSE ratio bound
margin:root_n_slope	-0.5000	contraction rate root-n asymptotics predict
margin:root_n_slope_lower	-0.6250	accepted root-n slope band, lower limit
margin:root_n_slope_upper	-0.3750	accepted root-n slope band, upper limit
margin:se_ratio_sanity_lower	0.8000	SE-ratio screen, lower limit
margin:se_ratio_sanity_upper	1.2000	SE-ratio screen, upper limit
margin:standardized_bias	0.2500	bias equivalence margin, in empirical standard deviations
margin:union_model_se_lower	0.1000	robustness-contract SE-ratio screen, lower limit
margin:union_model_se_upper	10	robustness-contract SE-ratio screen, upper limit

The reading rule and the red rows

The study declared its reading rule before the run, in READING_RULE of tests/studies/canonical_ctmle_oat_per_arm_strata_missing.py. A red row goes to F19 when its bias is equivalent and its SE ratio is below 1. The upper end of its 99% SE-ratio interval must also be below 1.07. For a ladder rung, a bootstrap interval of the SE ratio gives that upper end. Every other red row is a defect suspect.

A floor was added to the rule after the run. A routed row must also have a point SE ratio of 0.70 or more. A shorter standard error is a defect suspect, because it is too short for the generated-design term to explain. The lowest routed row is ey[1] on the three_arm_missing law, at 0.736 for both implementations. Thus the floor changes no routing of this run.

red rows	count	reading	owner
truth rows of both implementations	42	short diagnostic standard error, equivalent bias. Both implementations read the same rows alike	F19
root_n_and_efficiency rungs	3	coverage interval reaches below 0.90, equivalent bias, SE ratios 0.929, 0.896 and 0.881, bootstrap upper ends 1.0065, 0.952 and 0.940	F19

No paired row, robustness_contract cell or control is red, and no replication failed. Both root_n_rate rows pass. The empirical SD contracts at a slope of -0.499, and the mean standard error at -0.537.

The re-declared ladder

The first run declared the ladder at n = 500, 2,000 and 8,000. The n_500 rung measured the stability of the factor learner in small cells and not the rate. After the run, the study dropped that rung and re-declared the ladder at n = 2,000, 4,000 and 8,000. The rerun of the property phase replaced the first run’s property artifacts. The first run is commit ae53bbb.

The table gives what the n_500 rung measured in the first run.

quantity	value
mean diagnostic standard error	0.0884
empirical SD of the marginal ATE	0.0561
mean SE ratio	1.575
median standard error over the empirical SD	1.00
fits with a standard error above twice the empirical SD	40 of 800 (5%)
mean SE ratio without those fits	1.035
coverage of the diagnostic interval	0.954
root_n_rate/reported_se slope (99% interval)	-0.71 (-0.79 to -0.63)
root_n_rate/empirical_sd slope	-0.50
slope of the median standard error	-0.55

The reading rule made n_500 and root_n_rate/reported_se defect suspects. A diagnosis refitted the ladder replications and found no defect in the code.

finding	value
rows in the stratum, arm and fold cell that fits each factor at n = 500	about 20 to 35; 29.3 expected in the smallest cell
estimated design at the row that dominates a tail fit	far below 0.01
true design at those rows	0.11 or more
largest gap between the fit’s curve and the textbook curve	1e-13
tail fits at n = 2,000 and at n = 8,000	3 of 800, and none

In a small cell the unpenalized cubic factor learner extrapolates. One or two observed rows then get a very small design, and their term \(C_a / G\, (Y - \bar Q^*)\) dominates the plug-in variance. The point estimate does not change, so its bias stays inside the margin.

The re-declaration added condition (t). As declared, it required the smallest factor training cell to hold 100 rows or more in expectation at the smallest rung. That cell is arm 0 in the stratum \(S = 2\), with a share of 0.0732 of the rows. Four fifths of the rows train each fold, so the cell holds 117.1 rows at n = 2,000.

The row count is not the quantity that limits a factor fit. Each factor is a logistic regression on a binary target, and its smaller class limits it. On the strata law the smaller class is the nonrespondents of arm 0 in \(S = 2\). After the rerun, the study restated condition (t) as the expected size of the smallest class for each stratum, arm, fold and factor. The restated condition sets no floor, because the declared ladder does not hold 100 such rows. smallest_minority_class in tests/studies/oat_per_arm_extension_laws.py computes the count by quadrature.

configuration	n	rows that train each fit	smallest class	expected count	tail fits of 800
dropped n_500 rung of the first run	500	four fifths	nonrespondents, arm 0, \(S = 2\)	9.9	40
n_2000	2,000	four fifths	nonrespondents, arm 0, \(S = 2\)	39.5	3
strata_missing cells	3,000	four fifths	nonrespondents, arm 0, \(S = 2\)	59.2	2
n_4000	4,000	four fifths	nonrespondents, arm 0, \(S = 2\)	79.0	0
missing_outcome_crossfit	2,000	four fifths	nonrespondents, arm 1	107.0	0
primary missing_treatment	2,250	all	nonrespondents, arm 1	110.5	0
primary three_arm_missing	2,250	all	nonrespondents, arm 0	116.4	0
primary missing_outcome and natural_course_insample	2,250	all	nonrespondents, arm 1	150.5	0
n_8000	8,000	four fifths	nonrespondents, arm 0, \(S = 2\)	157.9	0
primary strata	2,250	all	arm 0 in \(S = 2\)	164.7	0

The weighted and the outcome-wrong cells sample the strata law at n = 3,000, so their count is 59.2 too. They hold 1 and 5 tail fits. The measured consequence of a small class is a rare tail fit. It moves the mean SE ratio of a cell and not its bias. test_condition_t_the_smallest_minority_class_of_each_configuration checks each count.

Condition (r) does not read n, so each rung keeps 800 replications, with a pass probability of 0.950. The primary laws, the pairing, the robustness cells and the seeds do not change. tests/studies/ctmle_oat_per_arm_strata_missing_properties.py records the change.

Limitations

limitation	what it means for use
No cell measures published inference	Every oat fit withholds ci, pvalue and std_error. The std_error, ci_lower, ci_upper and covered columns read plugin_std_error and plugin_interval
The diagnostic standard error is too small on the primary laws	The primary SE ratios run from min_se_ratio_ci_lower to max_se_ratio_ci_upper. The truth rows of both implementations are red alike, and F19 owns them. R drtmle reports the same standard errors, so the pairing cannot detect the gap
The diagnostic standard error is heavy-tailed in small factor cells	In the first run, at n = 500, the smallest class of a factor target held about 10 rows, and the mean standard error was 1.575 times the empirical SD. The ladder now starts at n = 2,000, where that class holds 39.5 rows and 3 of 800 fits have a tail. The re-declared ladder gives the measurement
The paired strata law has complete data	The strata primary samples strata_complete, so R drtmle never meets a stratum-local observation factor. Only the fast tier, tests/unit/test_outcome_adaptive_per_arm_extensions.py, checks that factor against a longhand. The strata_missing property cells measure its bias, unpaired
The points are paired, not the intervals	All 31 paired rows are equivalent, with max_margin_utilization of the margin used
The designs agree only off the bounds	Where no factor row sits at a bound, the two designs agree to mechanism_max_gap_off_bound. In mechanism_fits_with_rows_at_bound fits some rows sit at a bound, and the gap reaches mechanism_max_gap. The rows at a bound are reported, not gated
Parametric laws with a binary outcome	Each law has a normal index for each arm and one instrument. A continuous outcome and more than three arms have fast-tier evidence only
Clusters stay refused	A per-arm fit with id= refuses. A cross-fitted fit with a missing treatment refuses too

Reproduction

The manifest records the seeds, the margins, the reading rule, the estimator configuration and the hashes. Run python -m tests.canonical.drtmle_oat_per_arm_strata_missing.regenerate to regenerate the artifacts. The replications, paired decisions, design gap and property results carry every published row. The post-run gate in tests/unit/test_ctmle_oat_per_arm_strata_missing_study_design.py applies the reading rule to the committed tables.