Stratified incremental MSM#
This study validates three stratified targets of ordinary point-treatment TMLE with baseline strata,
strata=["V"]: incremental propensity-score interventions, marginal structural models (MSMs) with
a logit link or a continuous dose, and the cross-fitted natural-course mean with a missing
outcome. A stratum parameter is the marginal parameter of the law inside one of a fixed number of
baseline strata. The construction is the marginal result applied inside each cell of a finite
partition, which the natural-extension verdicts record as part
(i).
Each target solves one score block \(I(V=s) H_s / P_n(V=s)\) for each equation of its targeting. Each stratum curve is \(I(V=s) D_s / P_n(V=s)\). The incremental fit alternates outcome blocks and treatment-mechanism blocks. The linked MSM fit takes its stratum coefficients from a nested fluctuation of the blocks at each stratum’s own coefficients. Point-treatment TMLE states each construction.
What was tested#
setting |
declaration |
|---|---|
law L1 |
the baseline-strata law: \(V \in \{0, 1, 2\}\) with probabilities 0.5, 0.3 and 0.2, \(W \in \{0, 1, 2\}\) given \(V\), a binary treatment and a binary outcome. |
law L2 |
L1’s \(V\) and \(W\), a dose \(A \sim N(0.5 W - 0.4 V, 1)\), a continuous outcome for the identity link and a binary outcome for the logit link, both with an \(A V\) term |
L1 with a missing outcome |
L1, with the outcome observed with probability \(\operatorname{expit}(1.1 - 0.3 A + 0.2 W - 0.25 V)\) |
primary scenario |
|
fits |
in sample, with main-terms logistic regressions on \((A, W, V)\) and \((W, V)\), which are correct for L1. The natural-course fits are cross-fitted on ten folds |
Monte Carlo inference |
exact 99% intervals around every rate, and 99% intervals around every primary endpoint |
The comparator is R npcausal ipsi at commit 56a5ac1. npcausal has no stratified
estimator, so the runner calls ipsi once per stratum subset and multiplier, and once per
multiplier on every row for the marginal means. The two implementations differ in two declared
ways.
difference |
effect |
|---|---|
|
the paired difference scales like \(1/n_s\), so it is largest in the smallest stratum |
|
the marginal rows compare two correct but different regressions |
The MSM and natural-course targets have no stratified comparator. The property cells measure them against the exact truths.
Accuracy against known truth#
law |
estimand |
what was tested |
implementation |
bias (99% interval) |
coverage |
SE ratio |
result |
|---|---|---|---|---|---|---|---|
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged” |
|
-0.000095 to 0.000314 |
0.9469 |
0.9790 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged” |
R |
-0.000405 to 0.000021 |
0.9400 |
0.9529 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=0 |
|
-0.000276 to 0.000378 |
0.9494 |
0.9891 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=0 |
R |
-0.000614 to 0.000058 |
0.9437 |
0.9734 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=1 |
|
-0.000137 to 0.000573 |
0.9475 |
0.9686 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=1 |
R |
-0.000475 to 0.000266 |
0.9381 |
0.9439 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=2 |
|
-0.000182 to 0.000380 |
0.9469 |
0.9787 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=2 |
R |
-0.000385 to 0.000214 |
0.9375 |
0.9453 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged” |
|
-0.000348 to 0.000071 |
0.9481 |
0.9813 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged” |
R |
-0.000004 to 0.000433 |
0.9419 |
0.9551 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=0 |
|
-0.000374 to 0.000273 |
0.9487 |
0.9911 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=0 |
R |
-0.000111 to 0.000551 |
0.9456 |
0.9788 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=1 |
|
-0.000638 to 0.000125 |
0.9481 |
0.9676 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=1 |
R |
-0.000217 to 0.000576 |
0.9356 |
0.9472 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=2 |
|
-0.000519 to 0.000140 |
0.9425 |
0.9720 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=2 |
R |
-0.000113 to 0.000591 |
0.9350 |
0.9377 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged |
|
-0.000288 to 0.0011 |
0.9487 |
0.9870 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged |
R |
-0.000288 to 0.0011 |
0.9487 |
0.9870 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=0 |
|
-0.000797 to 0.0013 |
0.9450 |
0.9841 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=0 |
R |
-0.000797 to 0.0013 |
0.9456 |
0.9844 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=1 |
|
-0.000558 to 0.0018 |
0.9406 |
0.9712 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=1 |
R |
-0.000558 to 0.0018 |
0.9406 |
0.9717 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=2 |
|
-0.000641 to 0.0014 |
0.9544 |
1.0341 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=2 |
R |
-0.000641 to 0.0014 |
0.9544 |
1.0352 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5 |
|
-0.000223 to 0.0012 |
0.9500 |
0.9923 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5 |
R |
-0.000529 to 0.000953 |
0.9500 |
0.9887 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=0 |
|
-0.000793 to 0.0014 |
0.9444 |
0.9889 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=0 |
R |
-0.0011 to 0.0010 |
0.9437 |
0.9880 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=1 |
|
-0.000448 to 0.0022 |
0.9413 |
0.9777 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=1 |
R |
-0.000775 to 0.0018 |
0.9413 |
0.9772 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=2 |
|
-0.000664 to 0.0016 |
0.9513 |
1.0259 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=2 |
R |
-0.000852 to 0.0014 |
0.9494 |
1.0270 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two |
|
-0.000435 to 0.000964 |
0.9469 |
0.9808 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two |
R |
-0.000082 to 0.0013 |
0.9450 |
0.9822 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=0 |
|
-0.000890 to 0.0013 |
0.9506 |
0.9804 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=0 |
R |
-0.000621 to 0.0015 |
0.9487 |
0.9803 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=1 |
|
-0.000801 to 0.0016 |
0.9456 |
0.9615 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=1 |
R |
-0.000368 to 0.0020 |
0.9463 |
0.9622 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=2 |
|
-0.000773 to 0.0011 |
0.9513 |
1.0324 |
pass |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=2 |
R |
-0.000350 to 0.0016 |
0.9463 |
1.0316 |
pass |
Agreement with the canonical implementation#
Comparator-backed: a paired comparison with npcausal-stratified on the same samples. Each row reports estimate equivalence within the declared margin, the RMSE ratio and the coverage difference, and names its comparison kind.
law |
estimand |
what was compared |
paired difference |
share of margin used |
RMSE ratio bound |
coverage difference |
comparison kind |
calibration resolution |
result |
|---|---|---|---|---|---|---|---|---|---|
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged” |
0.000302 |
0.6214 |
0.9714 |
0.0069 |
different estimator |
0.0394 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=0 |
0.000329 |
0.4264 |
0.9835 |
0.0056 |
related target |
0.0358 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=1 |
0.000322 |
0.3818 |
0.9715 |
0.0094 |
related target |
0.0180 vs 0.0500 |
superior |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by 0.5” against “leave the observed treatment mechanism unchanged”, in stratum V=2 |
0.000184 |
0.2724 |
0.9574 |
0.0094 |
related target |
0.0376 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged” |
-0.000353 |
0.7097 |
0.9722 |
0.0062 |
different estimator |
0.0444 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=0 |
-0.000271 |
0.3558 |
0.9881 |
0.0031 |
related target |
0.0309 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=1 |
-0.000436 |
0.4810 |
0.9759 |
0.0125 |
related target |
0.0171 vs 0.0500 |
superior |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
difference in means under the incremental interventions “multiply the treatment odds by two” against “leave the observed treatment mechanism unchanged”, in stratum V=2 |
-0.000428 |
0.5397 |
0.9523 |
0.0075 |
related target |
0.0243 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged |
-5.990e-15 |
3.723e-12 |
1.0000 |
0 |
different estimator |
1.196e-13 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=0 |
-3.209e-17 |
1.334e-14 |
1.0000 |
-0.000625 |
related target |
0.000004 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=1 |
-2.741e-17 |
9.874e-15 |
1.0000 |
0 |
related target |
0.000016 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention leave the observed treatment mechanism unchanged, in stratum V=2 |
-3.006e-14 |
1.272e-11 |
1.0000 |
0 |
related target |
0.0020 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5 |
0.000302 |
0.1756 |
0.9986 |
0 |
different estimator |
0.0100 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=0 |
0.000329 |
0.1305 |
1.0009 |
0.000625 |
related target |
0.0046 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=1 |
0.000322 |
0.1062 |
1.0005 |
0 |
related target |
0.0038 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by 0.5, in stratum V=2 |
0.000184 |
0.0694 |
1.0010 |
0.0019 |
related target |
0.0054 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two |
-0.000353 |
0.2171 |
1.0020 |
0.0019 |
different estimator |
0.0033 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=0 |
-0.000271 |
0.1081 |
1.0009 |
0.0019 |
related target |
0.0029 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=1 |
-0.000436 |
0.1586 |
1.0008 |
-0.000625 |
related target |
0.0038 vs 0.0500 |
equivalent |
binary law with three unequal baseline strata, three incremental odds multipliers |
|
mean under the incremental intervention multiply the treatment odds by two, in stratum V=2 |
-0.000428 |
0.1916 |
0.9994 |
0.0050 |
related target |
0.0060 vs 0.0500 |
equivalent |
Repeated-sampling properties#
property |
cell |
role |
what was tested |
what must hold |
measured |
result |
|---|---|---|---|---|---|---|
|
|
positive |
dose coefficient of the identity-link dose MSM (1, a) over the grid, marginal: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0027 to 0.0069, margin 0.0209, SE ratio 0.9949 |
pass |
|
|
positive |
intercept coefficient of the identity-link dose MSM (1, a) over the grid, marginal: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0032 to 0.0019, margin 0.0111, SE ratio 1.1519 |
pass |
|
|
positive |
dose coefficient of the identity-link dose MSM (1, a) over the grid, in stratum V = 0: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0035 to 0.0066, margin 0.0219, SE ratio 1.3577 |
pass |
|
|
positive |
intercept coefficient of the identity-link dose MSM (1, a) over the grid, in stratum V = 0: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0033 to 0.0025, margin 0.0125, SE ratio 1.4455 |
pass |
|
|
positive |
dose coefficient of the identity-link dose MSM (1, a) over the grid, in stratum V = 1: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0017 to 0.0156, margin 0.0377, SE ratio 0.9778 |
pass |
|
|
positive |
intercept coefficient of the identity-link dose MSM (1, a) over the grid, in stratum V = 1: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0029 to 0.0060, margin 0.0193, SE ratio 1.1501 |
pass |
|
|
positive |
dose coefficient of the identity-link dose MSM (1, a) over the grid, in stratum V = 2: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0190 to 0.0140, margin 0.0714, SE ratio 0.6073 |
pass |
|
|
positive |
intercept coefficient of the identity-link dose MSM (1, a) over the grid, in stratum V = 2: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0103 to 0.0058, margin 0.0348, SE ratio 0.7541 |
pass |
|
|
positive |
dose coefficient of the logit-link dose MSM (1, a) over the grid, marginal: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0109 to 0.0040, margin 0.0323, SE ratio 1.3104 |
pass |
|
|
positive |
intercept coefficient of the logit-link dose MSM (1, a) over the grid, marginal: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0035 to 0.0047, margin 0.0178, SE ratio 1.3519 |
pass |
|
|
positive |
dose coefficient of the logit-link dose MSM (1, a) over the grid, in stratum V = 0: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0142 to 0.0058, margin 0.0435, SE ratio 1.4313 |
pass |
|
|
positive |
intercept coefficient of the logit-link dose MSM (1, a) over the grid, in stratum V = 0: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0052 to 0.0056, margin 0.0234, SE ratio 1.5417 |
pass |
|
|
positive |
dose coefficient of the logit-link dose MSM (1, a) over the grid, in stratum V = 1: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0202 to 0.0076, margin 0.0602, SE ratio 1.3009 |
pass |
|
|
positive |
intercept coefficient of the logit-link dose MSM (1, a) over the grid, in stratum V = 1: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0064 to 0.0086, margin 0.0325, SE ratio 1.3576 |
pass |
|
|
positive |
dose coefficient of the logit-link dose MSM (1, a) over the grid, in stratum V = 2: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0576 to -0.0143, margin 0.0939, SE ratio 1.0611 |
pass |
|
|
positive |
intercept coefficient of the logit-link dose MSM (1, a) over the grid, in stratum V = 2: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias 0.0033 to 0.0247, margin 0.0464, SE ratio 1.2582 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 0: both the outcome regression and the treatment mechanism are correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0040 to 0.0154, margin 0.0326, SE ratio 1.0058 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a) in stratum V = 0: both nuisances are misspecified |
bias interval must fall entirely outside the margin, with the reported standard error still on the scale of the empirical spread |
bias 0.1723 to 0.1922, margin 0.0334, SE ratio 0.9974 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 0: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0089 to 0.0106, margin 0.0328, SE ratio 0.9690 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 0: only the treatment mechanism is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0062 to 0.0136, margin 0.0333, SE ratio 1.0265 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 1: both the outcome regression and the treatment mechanism are correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0091 to 0.0215, margin 0.0515, SE ratio 1.0028 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a) in stratum V = 1: both nuisances are misspecified |
bias interval must fall entirely outside the margin, with the reported standard error still on the scale of the empirical spread |
bias 0.1743 to 0.2053, margin 0.0520, SE ratio 0.9782 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 1: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0095 to 0.0212, margin 0.0515, SE ratio 0.9373 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 1: only the treatment mechanism is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias -0.0110 to 0.0216, margin 0.0547, SE ratio 0.9703 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 2: both the outcome regression and the treatment mechanism are correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias 0.0066 to 0.0698, margin 0.1061, SE ratio 0.9559 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a) in stratum V = 2: both nuisances are misspecified |
bias interval must fall entirely outside the margin, with the reported standard error still on the scale of the empirical spread |
bias 0.1858 to 0.2450, margin 0.0994, SE ratio 0.9047 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 2: only the outcome regression is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias 0.0371 to 0.0985, margin 0.1031, SE ratio 0.8261 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a) in stratum V = 2: only the treatment mechanism is correctly specified |
bias interval inside the equivalence margin, with the reported standard error on the scale of the empirical spread |
bias 0.0309 to 0.0989, margin 0.1141, SE ratio 0.9176 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a, W), marginal: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9389 to 0.9639, SE ratio 0.9613 to 1.0427 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), marginal: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8131 to 0.8563, SE ratio 0.6819 to 0.7412 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), marginal: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8073 to 0.8511, SE ratio 0.6729 to 0.7287 |
pass |
|
|
positive |
intercept coefficient of the logit MSM (1, a, W), marginal: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9445 to 0.9683, SE ratio 0.9750 to 1.0572 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), marginal: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8089 to 0.8525, SE ratio 0.6854 to 0.7432 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), marginal: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8220 to 0.8643, SE ratio 0.6827 to 0.7399 |
pass |
|
|
positive |
baseline-covariate coefficient of the logit MSM (1, a, W), marginal: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9361 to 0.9617, SE ratio 0.9530 to 1.0295 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), marginal: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8073 to 0.8511, SE ratio 0.6821 to 0.7355 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), marginal: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8005 to 0.8449, SE ratio 0.6658 to 0.7212 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9300 to 0.9569, SE ratio 0.9644 to 1.0471 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8236 to 0.8657, SE ratio 0.6841 to 0.7426 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8136 to 0.8568, SE ratio 0.6752 to 0.7336 |
pass |
|
|
positive |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9394 to 0.9643, SE ratio 0.9689 to 1.0521 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8047 to 0.8487, SE ratio 0.6760 to 0.7329 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8194 to 0.8619, SE ratio 0.6790 to 0.7364 |
pass |
|
|
positive |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9322 to 0.9586, SE ratio 0.9534 to 1.0308 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8121 to 0.8553, SE ratio 0.6788 to 0.7360 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8037 to 0.8478, SE ratio 0.6675 to 0.7207 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9283 to 0.9555, SE ratio 0.9273 to 1.0105 |
fail |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8162 to 0.8591, SE ratio 0.6753 to 0.7359 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8037 to 0.8478, SE ratio 0.6498 to 0.7065 |
pass |
|
|
positive |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9311 to 0.9578, SE ratio 0.9564 to 1.0350 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8162 to 0.8591, SE ratio 0.7015 to 0.7588 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.7974 to 0.8421, SE ratio 0.6679 to 0.7242 |
pass |
|
|
positive |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9417 to 0.9661, SE ratio 0.9498 to 1.0300 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8210 to 0.8634, SE ratio 0.6847 to 0.7474 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8058 to 0.8497, SE ratio 0.6649 to 0.7203 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9289 to 0.9560, SE ratio 0.7818 to 0.9387 |
fail |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8037 to 0.8478, SE ratio 0.6035 to 0.6936 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W), in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8047 to 0.8487, SE ratio 0.5488 to 0.6571 |
pass |
|
|
positive |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9389 to 0.9639, SE ratio 0.9592 to 1.0452 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8115 to 0.8549, SE ratio 0.6764 to 0.7334 |
pass |
|
|
control |
intercept coefficient of the logit MSM (1, a, W), in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8126 to 0.8558, SE ratio 0.6714 to 0.7321 |
pass |
|
|
positive |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9541 to 0.9756, SE ratio 0.9623 to 1.0514 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8368 to 0.8775, SE ratio 0.7041 to 0.7630 |
pass |
|
|
control |
baseline-covariate coefficient of the logit MSM (1, a, W), in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8373 to 0.8779, SE ratio 0.6741 to 0.7351 |
pass |
|
|
positive |
cross-fitted natural-course mean with a missing outcome, marginal: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9389 to 0.9639, SE ratio 0.9734 to 1.0577 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, marginal: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8136 to 0.8568, SE ratio 0.6957 to 0.7510 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, marginal: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8162 to 0.8591, SE ratio 0.6819 to 0.7395 |
pass |
|
|
positive |
cross-fitted natural-course mean with a missing outcome, in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9355 to 0.9613, SE ratio 0.9672 to 1.0473 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8110 to 0.8544, SE ratio 0.6733 to 0.7317 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8141 to 0.8572, SE ratio 0.6781 to 0.7340 |
pass |
|
|
positive |
cross-fitted natural-course mean with a missing outcome, in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9350 to 0.9608, SE ratio 0.9608 to 1.0439 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8100 to 0.8534, SE ratio 0.6745 to 0.7343 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8152 to 0.8582, SE ratio 0.6716 to 0.7288 |
pass |
|
|
positive |
cross-fitted natural-course mean with a missing outcome, in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9294 to 0.9564, SE ratio 0.9752 to 1.0604 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8115 to 0.8549, SE ratio 0.6809 to 0.7391 |
pass |
|
|
control |
cross-fitted natural-course mean with a missing outcome, in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8094 to 0.8530, SE ratio 0.6826 to 0.7426 |
pass |
|
|
positive |
incremental contrast of multiplier 0.5 against 1 in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9222 to 0.9506, SE ratio 0.9342 to 1.0155 |
pass |
|
|
control |
incremental contrast of multiplier 0.5 against 1 in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8126 to 0.8558, SE ratio 0.6766 to 0.7363 |
pass |
|
|
control |
incremental contrast of multiplier 0.5 against 1 in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.7969 to 0.8416, SE ratio 0.6540 to 0.7123 |
pass |
|
|
positive |
incremental contrast of multiplier 2 against 1 in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9233 to 0.9515, SE ratio 0.9317 to 1.0145 |
pass |
|
|
control |
incremental contrast of multiplier 2 against 1 in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8126 to 0.8558, SE ratio 0.6698 to 0.7249 |
pass |
|
|
control |
incremental contrast of multiplier 2 against 1 in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.7916 to 0.8368, SE ratio 0.6517 to 0.7081 |
pass |
|
|
positive |
incremental mean at odds multiplier 1 in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9451 to 0.9687, SE ratio 0.9786 to 1.0607 |
pass |
|
|
control |
incremental mean at odds multiplier 1 in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8178 to 0.8605, SE ratio 0.6880 to 0.7479 |
pass |
|
|
control |
incremental mean at odds multiplier 1 in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8110 to 0.8544, SE ratio 0.6853 to 0.7413 |
pass |
|
|
positive |
incremental mean at odds multiplier 0.5 in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9411 to 0.9657, SE ratio 0.9699 to 1.0504 |
pass |
|
|
control |
incremental mean at odds multiplier 0.5 in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8136 to 0.8568, SE ratio 0.6763 to 0.7330 |
pass |
|
|
control |
incremental mean at odds multiplier 0.5 in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8089 to 0.8525, SE ratio 0.6789 to 0.7346 |
pass |
|
|
positive |
incremental mean at odds multiplier 2 in stratum V = 0: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9428 to 0.9670, SE ratio 0.9793 to 1.0618 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 0: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8241 to 0.8662, SE ratio 0.6955 to 0.7533 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 0: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8157 to 0.8586, SE ratio 0.6852 to 0.7441 |
pass |
|
|
positive |
incremental contrast of multiplier 0.5 against 1 in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9400 to 0.9648, SE ratio 0.9681 to 1.0462 |
pass |
|
|
control |
incremental contrast of multiplier 0.5 against 1 in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8110 to 0.8544, SE ratio 0.6885 to 0.7460 |
pass |
|
|
control |
incremental contrast of multiplier 0.5 against 1 in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8052 to 0.8492, SE ratio 0.6785 to 0.7334 |
pass |
|
|
positive |
incremental contrast of multiplier 2 against 1 in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9394 to 0.9643, SE ratio 0.9718 to 1.0513 |
pass |
|
|
control |
incremental contrast of multiplier 2 against 1 in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8178 to 0.8605, SE ratio 0.6865 to 0.7453 |
pass |
|
|
control |
incremental contrast of multiplier 2 against 1 in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8063 to 0.8501, SE ratio 0.6801 to 0.7361 |
pass |
|
|
positive |
incremental mean at odds multiplier 1 in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9389 to 0.9639, SE ratio 0.9570 to 1.0380 |
pass |
|
|
control |
incremental mean at odds multiplier 1 in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.7990 to 0.8435, SE ratio 0.6673 to 0.7239 |
pass |
|
|
control |
incremental mean at odds multiplier 1 in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8084 to 0.8520, SE ratio 0.6703 to 0.7262 |
pass |
|
|
positive |
incremental mean at odds multiplier 0.5 in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9350 to 0.9608, SE ratio 0.9653 to 1.0474 |
pass |
|
|
control |
incremental mean at odds multiplier 0.5 in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8262 to 0.8681, SE ratio 0.7051 to 0.7668 |
pass |
|
|
control |
incremental mean at odds multiplier 0.5 in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8094 to 0.8530, SE ratio 0.6760 to 0.7327 |
pass |
|
|
positive |
incremental mean at odds multiplier 2 in stratum V = 1: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9339 to 0.9600, SE ratio 0.9453 to 1.0272 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 1: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.7901 to 0.8354, SE ratio 0.6583 to 0.7127 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 1: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8105 to 0.8539, SE ratio 0.6622 to 0.7208 |
pass |
|
|
positive |
incremental contrast of multiplier 0.5 against 1 in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9344 to 0.9604, SE ratio 0.9546 to 1.0359 |
pass |
|
|
control |
incremental contrast of multiplier 0.5 against 1 in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8084 to 0.8520, SE ratio 0.6716 to 0.7303 |
pass |
|
|
control |
incremental contrast of multiplier 0.5 against 1 in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8031 to 0.8473, SE ratio 0.6670 to 0.7256 |
pass |
|
|
positive |
incremental contrast of multiplier 2 against 1 in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9305 to 0.9573, SE ratio 0.9518 to 1.0334 |
pass |
|
|
control |
incremental contrast of multiplier 2 against 1 in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8089 to 0.8525, SE ratio 0.6656 to 0.7232 |
pass |
|
|
control |
incremental contrast of multiplier 2 against 1 in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8026 to 0.8468, SE ratio 0.6652 to 0.7231 |
pass |
|
|
positive |
incremental mean at odds multiplier 1 in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9322 to 0.9586, SE ratio 0.9700 to 1.0550 |
pass |
|
|
control |
incremental mean at odds multiplier 1 in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8026 to 0.8468, SE ratio 0.6595 to 0.7196 |
pass |
|
|
control |
incremental mean at odds multiplier 1 in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8084 to 0.8520, SE ratio 0.6783 to 0.7381 |
pass |
|
|
positive |
incremental mean at odds multiplier 0.5 in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9311 to 0.9578, SE ratio 0.9665 to 1.0532 |
pass |
|
|
control |
incremental mean at odds multiplier 0.5 in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.7974 to 0.8421, SE ratio 0.6692 to 0.7266 |
pass |
|
|
control |
incremental mean at odds multiplier 0.5 in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8147 to 0.8577, SE ratio 0.6761 to 0.7367 |
pass |
|
|
positive |
incremental mean at odds multiplier 2 in stratum V = 2: both nuisances are correctly specified |
SE ratio and coverage intervals both inside their calibration bands |
coverage 0.9300 to 0.9569, SE ratio 0.9671 to 1.0518 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 2: a declared scale of independent noise is added to each estimate |
the SE-ratio interval must fall below the calibration band |
coverage 0.8247 to 0.8667, SE ratio 0.6905 to 0.7493 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 2: the reported standard errors are multiplied by a declared factor below one |
the SE-ratio interval must fall below the calibration band |
coverage 0.8094 to 0.8530, SE ratio 0.6781 to 0.7364 |
pass |
|
|
control |
the five marginal and fifteen stratum incremental parameters of one fit: the pointwise 95% intervals of the same fits, read jointly |
joint coverage upper endpoint must fall below the nominal rate |
joint coverage 0.6057 to 0.6616 |
pass |
|
|
positive |
the five marginal and fifteen stratum incremental parameters of one fit: the max-t multiplier band over every estimand the law reports, from the same-row centered influence curves |
joint coverage interval inside the calibration coverage band |
joint coverage 0.9261 to 0.9538 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 0: the same outcome regression in the unstratified incremental fit, its targeted mixture averaged inside the stratum |
bias interval must fall entirely outside the margin |
bias 0.0998 to 0.1017, margin 0.0031 |
pass |
|
|
positive |
incremental mean at odds multiplier 2 in stratum V = 0: the stratified fit with an outcome regression that omits the stratum, so the per-stratum score blocks do all the adjusting |
bias interval inside the equivalence margin |
bias -0.0011 to 0.0014, margin 0.0042 |
pass |
|
|
control |
incremental mean at odds multiplier 2 in stratum V = 2: the same outcome regression in the unstratified incremental fit, its targeted mixture averaged inside the stratum |
bias interval must fall entirely outside the margin |
bias -0.1583 to -0.1565, margin 0.0030 |
pass |
|
|
positive |
incremental mean at odds multiplier 2 in stratum V = 2: the stratified fit with an outcome regression that omits the stratum, so the per-stratum score blocks do all the adjusting |
bias interval inside the equivalence margin |
bias -0.0015 to 0.000807, margin 0.0039 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a, W) in stratum V = 0: the estimator fluctuates a misspecified outcome model, so targeting does all the adjusting |
bias interval inside the equivalence margin |
bias -0.0062 to 0.0141, margin 0.0342 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W) in stratum V = 0: the identical fit with every fluctuation step removed |
bias interval must fall entirely outside the margin |
bias -0.2498 to -0.2345, margin 0.0257 |
pass |
|
|
positive |
treatment coefficient of the logit MSM (1, a, W) in stratum V = 1: the estimator fluctuates a misspecified outcome model, so targeting does all the adjusting |
bias interval inside the equivalence margin |
bias -0.0021 to 0.0305, margin 0.0546 |
pass |
|
|
control |
treatment coefficient of the logit MSM (1, a, W) in stratum V = 1: the identical fit with every fluctuation step removed |
bias interval must fall entirely outside the margin |
bias -0.2498 to -0.2345, margin 0.0257 |
pass |
The property grid has six families.
family |
what it measures |
|---|---|
|
the fifteen stratum incremental parameters; the logit MSM \((1, a, W)\), marginal and in each stratum; and the cross-fitted natural-course mean, marginal and in each stratum. Each cell has a shrunken-SE control and a noise control |
|
the band over the twenty incremental parameters of one fit |
|
the stratified incremental fit against the unstratified fit’s mixture inside a stratum, with an outcome regression that omits \(V\) |
|
the stratified logit-MSM slope against the untargeted stratum projection of the same initial regression |
|
the logit MSM \((1, a)\) in each stratum under four nuisance configurations, and the dose MSM coefficients with a correct outcome regression |
The robustness family fits \((1, a)\), not \((1, a, W)\). Inside a stratum the logit model \((1, a, W)\) contains the outcome regression, so a fit with both nuisances wrong stays within 0.5 SD of the truth, and its control could not fail. The dose cells are robustness cells and not calibration cells. The package fits the dose density with bins, which is not the law’s density, so no result predicts a calibrated standard error.
Two positive cells are red, and both measure the logit-MSM slope:
interval_calibration/logit_v1_a__correctly_specified and logit_v2_a__correctly_specified.
Their SE ratios are 0.967 and 0.862, and the lower ends of their intervals are 0.927 and
0.782 against a floor of 0.93. Both cover nominally. Stratum 1 holds about 600 of the 2,000 rows,
and stratum 2 about 400.
The owner X8-logit-small-stratum in the roadmap records the
diagnosis. A declared diagnostic read stratum 2 only. It fitted the shipped unstratified logit MSM
on samples of stratum 2’s size from the law given \(V = 2\). That fit under-reports as well: 0.896,
against the stratified fit’s 0.867 on fresh draws. At four times the size the two read 0.997 and
1.006.
No diagnostic reads stratum 1, whose interval misses the floor by 0.003. The study declared
gated and moved to reporting before a re-run at the same seeds, which its declaration allows
for a finite-sample red with no defect.
Measured values and declared margins#
quantity |
value |
source |
|---|---|---|
|
1600 |
paired replications |
|
2000 |
observations per primary replication |
|
40 |
truth tests passing |
|
40 |
truth tests reported |
|
20 |
paired comparisons passing |
|
20 |
paired comparisons reported |
|
129 |
property cells passing |
|
131 |
property cells reported |
|
0.0634 |
largest primary standardized bias |
|
0.9350 |
lowest primary coverage |
|
0.7097 |
largest paired similarity-margin share |
|
0.9900 |
Monte Carlo confidence level |
|
0.0500 |
nominal test size |
|
0.9500 |
nominal interval coverage |
|
10000 |
bootstrap replications |
|
0.2500 |
necessity displacement threshold in standardized units |
|
0.2500 |
standardized-bias margin |
|
0.1000 |
union-model SE-ratio screen, lower limit |
|
10 |
union-model SE-ratio screen, upper limit |
|
0.9000 |
primary coverage floor |
|
0.9900 |
descriptive overcoverage threshold |
|
0.8000 |
primary SE-ratio lower screen |
|
1.2000 |
primary SE-ratio upper screen |
|
0.9300 |
calibration SE-ratio lower bound |
|
1.0700 |
calibration SE-ratio upper bound |
|
0.9200 |
calibration coverage lower bound |
|
0.9800 |
calibration coverage upper bound |
|
0.1000 |
type-I upper bound |
|
0.1500 |
paired similarity margin in pooled SDs |
|
1.1000 |
RMSE-ratio noninferiority bound |
|
-0.0250 |
coverage-difference noninferiority bound |
|
0.0500 |
calibration-excess noninferiority bound |
|
0.8000 |
minimum power lower bound |
|
-0.5000 |
expected root-n slope |
|
-0.6250 |
root-n slope lower bound |
|
-0.3750 |
root-n slope upper bound |
|
-0.2500 |
slower rate the interval must exclude |
|
0.7000 |
negative-control SE multiplier |
|
0.0050 |
exact L1 efficiency-bound standard error at primary n |
|
0.0053 |
exact L1 efficiency-bound standard error at primary n |
|
0.0043 |
exact L1 efficiency-bound standard error at primary n |
|
0.0031 |
exact L1 efficiency-bound standard error at primary n |
|
0.0050 |
exact L1 efficiency-bound standard error at primary n |
|
0.0058 |
exact L1 efficiency-bound standard error at primary n |
|
0.0050 |
exact L1 efficiency-bound standard error at primary n |
|
0.0032 |
exact L1 efficiency-bound standard error at primary n |
|
0.0158 |
exact L1 efficiency-bound standard error at primary n |
|
0.0180 |
exact L1 efficiency-bound standard error at primary n |
|
0.0163 |
exact L1 efficiency-bound standard error at primary n |
|
0.0106 |
exact L1 efficiency-bound standard error at primary n |
|
0.0166 |
exact L1 efficiency-bound standard error at primary n |
|
0.0198 |
exact L1 efficiency-bound standard error at primary n |
|
0.0182 |
exact L1 efficiency-bound standard error at primary n |
|
0.0113 |
exact L1 efficiency-bound standard error at primary n |
|
0.0164 |
exact L1 efficiency-bound standard error at primary n |
|
0.0176 |
exact L1 efficiency-bound standard error at primary n |
|
0.0154 |
exact L1 efficiency-bound standard error at primary n |
|
0.0106 |
exact L1 efficiency-bound standard error at primary n |
Limits#
The study covers two finite-support laws with one three-level stratum, and one continuous-dose law with the same strata.
The incremental and MSM fits are in sample. The cross-fitted forms of these targets are not measured here.
The dose fits use the package’s binned density on six bins. They publish no calibration cell and no band.
The MSM and natural-course targets have no stratified comparator, so only the truth measures them.
targeting_scheme="fold"andcv_evaluation=Truerefuse strata, so neither is measured.
Reproduction#
The fixture README, manifest, replications, paired decisions, and property results carry the protocol, provenance, and every published row.