Collaborative TMLE: an assignment predictor with an uncertain role

This notebook estimates the effect of a navigation offer on a successful transition home. One baseline variable predicts the offer strongly, and the study team cannot settle its role. The notebook keeps that variable in the adjustment set. It then compares two estimators of the same effect: an ordinary TMLE and outcome-adaptive collaborative TMLE (OAT C-TMLE). Each step shows its code, its output, and what the output tells you. Collaborative TMLE gives the algorithm.

The situation

A health system offers standard transition navigation when a discharge home is ordered. Nobody randomized the offer. Staff record three baseline variables at the order, before the offer.

variable	what it records	what the study team knows about its role
baseline_readiness	a standardized nurse rating of readiness to manage care at home	navigators offer more often to ready patients, and readiness raises success. It is a confounder
coordination_need	a standardized count of follow-up tasks to coordinate	it changes success. The team knows of no path to the offer
navigator_staffing	the standardized navigator staffing level on the discharge day	more staff means more offers. Low staffing can also mean a rushed discharge, so the team cannot rule out a path to success

The treatment transition_navigation is 1 for an offer and 0 for usual support. The outcome transition_success is 1 for a successful transition at 30 days. Step 4 defines success. The time order is the baseline variables, then the offer, then the outcome.

The data hold one observed outcome for each patient, under the arm that patient received. The analysis is about two counterfactual outcomes: \(Y^1\), success had the patient received the offer, and \(Y^0\), success under usual support. No patient shows both.

The diagram gives the assumed causal structure. A dotted arrow is one the team cannot confirm or exclude.

 baseline_readiness ──────────────────────┐
        │                                 │
        ▼                                 ▼
 transition_navigation ────────► transition_success ◄─── coordination_need
        ▲                                 ▲
        │                                 ┆
 navigator_staffing ┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┘

arrow or absence	what it asserts	what the analysis does with it
readiness to offer, and readiness to success	readiness is a common cause	the path offer ← readiness → success biases a crude comparison. Adjustment must block it
staffing to offer	staffing changes who receives the offer	it moves the propensity toward 0 and 1
staffing to success, dotted	unresolved	with the arrow, staffing is a confounder and the analysis must adjust for it. Without the arrow, staffing is an instrument, and adjustment is not needed. Adjustment is valid in both cases, so the adjustment set keeps staffing
coordination need to success	need changes success, and the offer changes how much	need is an outcome predictor and an effect modifier. It is not a confounder
offer to success	the effect under study	the estimand below
no unmeasured common cause of the offer and success	no unmeasured confounding	the data cannot test it. Step 10 measures how strong a violation would need to be

The diagram has no mediator, because the protocol analyzes the offer and not the completed contacts. Death before day 30 is a competing event. Step 4 gives the rule for it.

What you will learn

decision	the step that shows it
keep a variable of uncertain role in the adjustment set	this section
see what a strong assignment predictor does to an ordinary TMLE	Steps 6 and 8
configure OAT C-TMLE so that it reports an interval	Step 7
name the assumption that OAT C-TMLE adds	Step 7 and Method evidence
read the robustness value of the declared adjustment set	Step 10

The causal question

The program sponsor decides whether to offer navigation to every eligible discharge. The question is: in eligible adults with a discharge-home order, how much would the share of successful transitions change if every discharge received the offer, rather than usual support?

The estimand is the average treatment effect (ATE), \(E[Y^1] - E[Y^0]\), on the risk-difference scale. An estimand is the number the question asks for, written before any model is chosen.

Simpler quantities answer other questions. The unadjusted difference in success compares patients who differ in readiness before the offer. A logistic regression coefficient is a conditional odds ratio. In this law the effect of the offer on the risk-difference scale varies with need and with readiness. One model coefficient therefore does not give the population average that the decision needs.

The ATE is identified by the formula in Step 5 under the first four assumptions of the table. The interval also needs the fifth.

assumption	what it requires here	how it is supported
consistency	each patient’s observed outcome is the counterfactual outcome of the offer received	scientific judgment: one standardized offer and its records
no interference	one patient’s offer does not change another patient’s outcome	scientific judgment: reserved navigator capacity
independent rows	the patients are independent draws	the synthetic law draws staffing for each patient. Real staffing is shared by all discharges of a day. A real analysis would then need standard errors clustered by day, and a check that staffing does not change the delivered version of the offer
no unmeasured confounding	readiness, need, and staffing block every common cause of the offer and success	scientific judgment. The data cannot test it, and Step 10 gives its sensitivity
positivity	every kind of patient has some chance of each arm	partly checkable. Step 8 reads the fitted propensities, and that report describes the fitted model only

Why this method

TMLE fits an outcome regression Q and a treatment mechanism g. It then updates Q with a step weighted by \(1/g\). That weight is the clever covariate. Staffing makes the true g extreme. A correctly specified g that holds staffing gives large weights to a few patients. The weights add no bias protection for staffing if staffing has no path to success, and they add variance.

Dropping staffing from the adjustment set would avoid the weights. That is safe only if the dotted arrow is absent, and the team cannot confirm that. A dropped confounder biases the estimate. A kept instrument costs precision when no unmeasured confounder exists. With an unmeasured confounder, a kept instrument can also amplify the bias.

OAT C-TMLE resolves this conflict through the outcome model. For each arm \(a\), it fits g as the probability of arm \(a\) given the outcome prediction \(\bar Q_n(a, W)\), and not given \(W\) itself.

feature of OAT C-TMLE	the problem feature it answers
g reads only \(\bar Q_n(a, W)\)	if staffing does not predict success, \(\bar Q_n\) gives it a coefficient near zero, and g barely moves with it. If staffing does predict success, \(\bar Q_n\) carries it, and the fit adjusts for it through Q
one mechanism for each arm (oat_design="per_arm")	the outcome index differs between arms, because need changes the effect
ParametricOutcome(features="arm_interactions")	each arm gets its own coefficients, so the model can represent the effect modification by need
ParametricMechanism(degree=3)	a cubic in \(\operatorname{logit} \bar Q_n(a, W)\) for each arm’s assignment probability. The cubic need not be exact. With a correct Q, the interval still covers the ATE
fitted in sample, CrossFitting(enabled=False)	a registered study measures the interval of this in-sample parametric fit. Method evidence links it. A cross-fitted fit also reports an interval

The method adds one assumption. OAT C-TMLE fits the probability of each arm given \(\bar Q_n\), not the propensity score given \(W\). So in general only a correct Q gives a consistent estimate. An ordinary TMLE is doubly robust: a correct g alone also gives a consistent estimate. So the choice depends on which model the team trusts more. Here Q is a logistic model whose form matches the law, so the page compares the two estimators where both are valid.

Two other routes exist. The greedy and other selector strategies of C-TMLE report a point estimate and no interval. F18 records why. Inside a correct parametric outcome model, the plug-in g-computation estimate is at least as efficient as OAT C-TMLE, as the technical entry states. Both rest on a correct Q. Step 8 compares their spread and gives the reason this page reports OAT C-TMLE.

term	plain meaning	reference
nuisance	a model the estimate needs but the question does not ask about. Here, Q and g	point-treatment TMLE
targeting	a small update to Q, weighted by g, that removes first-order bias	targeting and bounds
influence curve	how much each row moves the estimate. Its variance gives the standard error	inference
positivity	every kind of patient has some chance of each arm	diagnostics
instrument	a variable that changes assignment and has no other path to the outcome	collaborative TMLE
outcome-adaptive mechanism	the assignment probability of an arm given the outcome prediction for that arm	the per-arm design

Step 1: set up

The setup imports the libraries and prints the installed cleverly version. Every fit below passes its learners, fold layer, and random seed explicitly, so a rerun reproduces the stored outputs.

import numpy as np
import pandas as pd
from numpy.polynomial.hermite_e import hermegauss
from scipy.special import expit
from sklearn.linear_model import LogisticRegression

import cleverly

pd.set_option("display.width", 130)
pd.set_option("display.max_columns", 20)
print(f"cleverly {cleverly.__version__}")

cleverly 0.1.3

What this output tells you. The stored outputs came from the version named above. A different version can print different numbers.

Step 2: the data

The data come from a synthetic law with a known answer. The function draw_discharges writes the structural equations out, so you can match each arrow of the diagram to a term. With \(W_1\) readiness, \(W_2\) need, and \(W_3\) staffing, all independent standard normal:

P(offer = 1 | W)          = expit(0.5 W1 + 1.6 W3)
P(success = 1 | offer, W) = expit( 0.3 + 0.8 W1 + 0.8 W2)   if offer = 1
                            expit(-0.3 + 0.8 W1 - 0.8 W2)   if offer = 0

This is the law binary_active of the registered C-TMLE study that Method evidence links, at the same sample size. Staffing has no term in the success equation, so in this law the dotted arrow is absent. The analysis does not use that fact.

The function arm_mean computes each true arm mean by quadrature. Within an arm, the linear index is normal, so a one-dimensional Gauss-Hermite rule gives the mean exactly to printed precision.

def draw_discharges(n, seed):
    """Draw n discharges from the structural equations of this page."""
    rng = np.random.default_rng(seed)
    readiness, need, staffing = rng.standard_normal((n, 3)).T
    offer_probability = expit(0.5 * readiness + 1.6 * staffing)
    offer = (rng.random(n) > 1.0 - offer_probability).astype(float)
    success_probability = np.where(
        offer == 1.0,
        expit(0.3 + 0.8 * readiness + 0.8 * need),
        expit(-0.3 + 0.8 * readiness - 0.8 * need),
    )
    success = (rng.random(n) < success_probability).astype(float)
    return pd.DataFrame(
        {
            "transition_success": success,
            "transition_navigation": offer,
            "baseline_readiness": readiness,
            "coordination_need": need,
            "navigator_staffing": staffing,
        }
    )


def arm_mean(intercept, slopes):
    """E[expit(intercept + slopes . W)] for standard-normal W, by Gauss-Hermite quadrature."""
    nodes, weights = hermegauss(80)
    scale = float(np.linalg.norm(slopes))
    return float(weights @ expit(intercept + scale * nodes) / weights.sum())


frame = draw_discharges(n=1_500, seed=44)
truth = {"ey0": arm_mean(-0.3, (0.8, -0.8)), "ey1": arm_mean(0.3, (0.8, 0.8))}
truth["ate"] = truth["ey1"] - truth["ey0"]
print(f"rows and columns: {frame.shape}")
print(list(frame.columns))
print(frame.head().round(3).to_string())
print()
print("known values of the synthetic law:")
for key in ("ey0", "ey1", "ate"):
    print(f"  {key}: {truth[key]:.4f}")

rows and columns: (1500, 5)
['transition_success', 'transition_navigation', 'baseline_readiness', 'coordination_need', 'navigator_staffing']
   transition_success  transition_navigation  baseline_readiness  coordination_need  navigator_staffing
0                 1.0                    1.0               1.446              0.102               0.327
1                 0.0                    1.0               1.136              0.824               0.579
2                 0.0                    0.0              -0.379             -1.019               0.376
3                 0.0                    1.0              -0.512              0.827               0.211
4                 1.0                    1.0              -0.745              0.268               0.709

known values of the synthetic law:
  ey0: 0.4407
  ey1: 0.5593
  ate: 0.1186

What this output tells you. The sample has 1,500 discharges and the five columns of the situation table. The outcome and the treatment are 0 or 1. The three baseline variables are standard normal draws.

The true success probability is 0.4407 under usual support and 0.5593 under the offer. The true ATE is 0.1186, about 12 percentage points. A real program has no truth. Every comparison against it below is a teaching device.

Step 3: association first

The code compares the two arms before any adjustment. It prints the success rate and the mean of each baseline variable by arm.

covariates = ["baseline_readiness", "coordination_need", "navigator_staffing"]
by_arm = frame.groupby("transition_navigation")[["transition_success", *covariates]].mean()
print(by_arm.round(3))
print()
print(f"share offered navigation: {frame['transition_navigation'].mean():.3f}")
unadjusted = by_arm.loc[1.0, "transition_success"] - by_arm.loc[0.0, "transition_success"]
print(f"unadjusted difference in success: {unadjusted:.4f}")
print(f"population ATE:                   {truth['ate']:.4f}")

                       transition_success  baseline_readiness  coordination_need  navigator_staffing
transition_navigation                                                                               
0.0                                 0.407              -0.171              0.007              -0.484
1.0                                 0.590               0.217             -0.022               0.502

share offered navigation: 0.510
unadjusted difference in success: 0.1827
population ATE:                   0.1186

What this output tells you. Offered patients succeed 0.1827 more often than the others. The true ATE is 0.1186. The arms differ before the offer. The mean readiness is 0.217 among offered patients and -0.171 among the others. Readiness raises success, so the crude difference mixes the effect of the offer with the effect of readiness.

Staffing also differs between the arms, 0.502 against -0.484. Its gap is the larger one, because staffing drives the offer more strongly than readiness does. The data cannot show whether that gap also reaches success. The mean need is close in both arms, as the diagram implies.

Step 4: write the protocol

A StudyProtocol records the scientific design before any model runs. Every result fitted from it carries its fingerprint. The page starts from navigation_protocol(), the protocol of the shared study design, and changes three fields with dataclasses.replace.

from dataclasses import replace

from cleverly.datasets import navigation_protocol

program = navigation_protocol()
protocol = replace(
    program,
    outcome="Successful transition: a 30-day transition score of at least 0.7 of the maximum",
    intercurrent_event_handling=(
        "Use the success indicator regardless of readmission",
        program.intercurrent_event_handling[1],
        "The protocol counts death before day 30 as an unsuccessful transition "
        "(composite strategy)",
    ),
    assumption_rationale=(
        "Readiness, coordination need, and discharge-day navigator staffing are recorded "
        "before the offer and cover the measured common causes of the offer and success",
        "Staffing may affect success through discharge quality, so the adjustment set keeps it",
        *program.assumption_rationale[1:],
    ),
)
print("\n".join(protocol.summary_lines()))

causal study protocol: schema 1; d6ac004f4e42dc00
target population: Adults with a discharge-home order at a participating hospital during the enrollment period
eligibility: ['Age 18 years or older', 'Discharge home ordered at a participating hospital']
time zero: Discharge-home order, after baseline measurement and before the navigation offer
treatment strategies: ['Offer standard transition navigation', 'Provide usual discharge support']
treatment versions: ['Bedside transition plan and two scheduled navigator contacts within 30 days', 'No access to the transition-navigation offer']
outcome: Successful transition: a 30-day transition score of at least 0.7 of the maximum
horizon: 30 days after discharge
intercurrent-event handling: ['Use the success indicator regardless of readmission', 'Analyze the offer regardless of completed contacts', 'The protocol counts death before day 30 as an unsuccessful transition (composite strategy)']
interference unit: Individual patient
assumption rationale: ['Readiness, coordination need, and discharge-day navigator staffing are recorded before the offer and cover the measured common causes of the offer and success', 'Staffing may affect success through discharge quality, so the adjustment set keeps it', 'The standardized offer and version records support consistency', 'Reserved navigator capacity and access controls support no interference']

What this output tells you. The first line gives the fingerprint d6ac004f4e42dc00. The table names the three fields this page changes.

protocol field	what this page changes
outcome	success is a 30-day transition score of at least 0.7 of the maximum. The program records the score itself
intercurrent-event handling	the readmission rule names the success indicator. Death before day 30 counts as an unsuccessful transition. That is the composite strategy, and it needs an outcome with a worst value, which a failed transition provides
assumption rationale	the three baseline variables, and the reason the set keeps staffing

The protocol has no field for the adjustment set or the method. The design in Step 5 holds the adjustment set, and each method in Steps 6 and 7 holds its configuration.

Step 5: design and identification

The design names the outcome, the treatment, and the three adjustment variables. Both estimators use this one design, so they target the same ATE under the same assumptions.

from cleverly import ATE, CausalStudy, PointTreatment

study = CausalStudy(
    frame,
    design=PointTreatment(
        outcome="transition_success",
        treatment="transition_navigation",
        adjustment=tuple(covariates),
    ),
    protocol=protocol,
)
effect = study.identify(ATE(reference=0))
print(effect.summary(protocol="fingerprint"))

average treatment effect, E[Y^a] - E[Y^reference]
identified by explicit-adjustment: E_W[E(transition_success | transition_navigation=a, W)] - E_W[E(transition_success | transition_navigation=0, W)] for a in [1]
adjustment/history: ['baseline_readiness', 'coordination_need', 'navigator_staffing']
required nuisances: ['outcome_regression', 'treatment_mechanism']
assumptions:
  - consistency: Y = Y^a when A = a
  - no interference: one unit's potential outcome does not depend on other units' treatment assignments
  - no unmeasured confounding: Y^a is independent of A given W
  - positivity: P(transition_navigation = a | W) > 0 almost surely for every supported treatment level a in [0, 1]
causal study protocol: schema 1; d6ac004f4e42dc00

What this output tells you. The formula averages the outcome regression over all patients, once with the offer and once without. It needs two nuisances, Q and g. The four assumptions are those of the table in “The causal question”. The last line prints the protocol fingerprint.

Step 6: ordinary TMLE

The ordinary TMLE uses the same outcome model as OAT C-TMLE in Step 7. Only g differs.

setting	value	why
outcome_learner	ParametricOutcome(features="arm_interactions")	an unpenalized logistic model of success with separate coefficients for each arm. It matches the form of the law
treatment_learner	LogisticRegression(C=np.inf, max_iter=1000)	an unpenalized logistic model of the offer on the three variables. It matches the form of the true g
CrossFitting(enabled=False)	in sample	both models have few coefficients, so in-sample fits meet the data-reuse condition that CV-TMLE names
Runtime(random_state=44, n_jobs=1)	fixed seed, one process	a rerun reproduces the outputs

The fit keeps the default bounds on g. The summary prints them.

from cleverly import CrossFitting, ModelSpec, Runtime, TMLEMethod
from cleverly.learners import ParametricOutcome

outcome_model = ParametricOutcome(features="arm_interactions")
in_sample = CrossFitting(enabled=False)
runtime = Runtime(random_state=44, n_jobs=1)

tmle_method = TMLEMethod(
    models=ModelSpec(
        outcome_learner=outcome_model,
        treatment_learner=LogisticRegression(C=np.inf, max_iter=1000),
    ),
    cross_fitting=in_sample,
    runtime=runtime,
)
tmle = effect.estimate(method=tmle_method)
print(tmle.summary(protocol="fingerprint"))

Targeted maximum likelihood estimation
======================================
n = 1500; covariates = 3; P(A=1) = 0.51
causal estimand: average treatment effect, E[Y^a] - E[Y^reference]
identification: explicit-adjustment; E_W[E(transition_success | transition_navigation=a, W)] - E_W[E(transition_success | transition_navigation=0, W)] for a in [1]
required nuisances: outcome_regression, treatment_mechanism
identification assumptions: consistency: Y = Y^a when A = a; no interference: one unit's potential outcome does not depend on other units' treatment assignments; no unmeasured confounding: Y^a is independent of A given W; positivity: P(transition_navigation = a | W) > 0 almost surely for every supported treatment level a in [0, 1]
causal study protocol: schema 1; d6ac004f4e42dc00
outcome family: binomial; fluctuation: logistic (iterative)
TMLE (in-sample nuisances): nuisances fitted in-sample (cross_fit=False)
propensity truncated to [0.01765, 0.9823]; ATT/ATC to [0.025, 0.975]
data f2d08ca0daecfd33 | folds e36af2092414a1d7 | cleverly 0.1.3 | 2026-10-16T01:17:01+00:00

estimand  psi      std_err  95% CI               p_value
--------  -------  -------  -------------------  -------
ate       0.16289  0.03466  [0.094954, 0.23082]  <1e-4  

What this output tells you. The ordinary TMLE estimates the ATE at 0.16289. Its standard error is 0.03466, and its 95% interval is [0.094954, 0.23082]. The interval contains the true 0.1186 on this draw.

The line propensity truncated to [0.01765, 0.9823] gives the default bounds on g at this sample size. Step 8 shows how many fitted propensities reach them.

Step 7: OAT C-TMLE

The collaborative fit keeps the outcome model of Step 6. It replaces the logistic g on the three variables with ParametricMechanism(degree=3), fitted for each arm on that arm’s outcome prediction.

strategy="oat" selects the outcome-adaptive mechanism. oat_design="per_arm" is the default, and the code writes it out. The interval needs the exact pair ParametricOutcome and ParametricMechanism. With another learner, the fit reports a point estimate and no interval.

from cleverly import CollaborativeTMLEMethod
from cleverly.learners import ParametricMechanism

collaborative_method = CollaborativeTMLEMethod(
    models=ModelSpec(
        outcome_learner=outcome_model,
        treatment_learner=ParametricMechanism(degree=3),
    ),
    cross_fitting=in_sample,
    runtime=runtime,
    strategy="oat",
    oat_design="per_arm",
)
collaborative = effect.estimate(method=collaborative_method)
print(collaborative.summary(protocol="fingerprint"))

Targeted maximum likelihood estimation
======================================
n = 1500; covariates = 3; P(A=1) = 0.51
causal estimand: average treatment effect, E[Y^a] - E[Y^reference]
identification: explicit-adjustment; E_W[E(transition_success | transition_navigation=a, W)] - E_W[E(transition_success | transition_navigation=0, W)] for a in [1]
required nuisances: outcome_regression, treatment_mechanism
identification assumptions: consistency: Y = Y^a when A = a; no interference: one unit's potential outcome does not depend on other units' treatment assignments; no unmeasured confounding: Y^a is independent of A given W; positivity: P(transition_navigation = a | W) > 0 almost surely for every supported treatment level a in [0, 1]
causal study protocol: schema 1; d6ac004f4e42dc00
outcome family: binomial; fluctuation: logistic (iterative)
TMLE (in-sample nuisances): nuisances fitted in-sample (cross_fit=False)
propensity truncated to [0.01765, 0.9823]; ATT/ATC to [0.025, 0.975]
C-TMLE outcome-adaptive fit used one Qbar feature per arm in 2 binary mechanisms. Its intervals read the stacked parametric curve of ParametricOutcome(features='arm_interactions') and ParametricMechanism(degree=3). Each interval is pointwise and covers the arm mean when the outcome model is correct, and its limit psi* otherwise. The g_bounds are constants of the fit, so with a wrong outcome model psi* can move with n while a bound binds. No simultaneous band is built (X58 part (b))
data f2d08ca0daecfd33 | folds e36af2092414a1d7 | cleverly 0.1.3 | 2026-10-16T01:17:01+00:00

estimand  psi     std_err  95% CI              p_value
--------  ------  -------  ------------------  -------
ate       0.1483  0.02826  [0.09292, 0.20368]  <1e-4  

What this output tells you. OAT C-TMLE estimates the ATE at 0.1483. Its standard error is 0.02826, and its 95% interval is [0.09292, 0.20368]. The interval contains the true 0.1186 on this draw.

The line that starts C-TMLE outcome-adaptive fit names the construction. The interval reads the stacked parametric curve. That curve includes the uncertainty from estimating the outcome prediction on which g is fitted. The line also states the condition: the interval covers the true arm mean when the outcome model is correct, and a different limit \(\psi^*\) otherwise.

The two estimates differ by about 0.015 on this draw. When Q is correct, both estimators converge to the same ATE whatever g is. A persistent large gap therefore signals a wrong Q.

Step 8 describes a review probe on this law, with both models correct. There, the absolute gap had a median of 0.0134 and a 95th percentile of 0.0381. So a gap of 0.015 is typical here. The probe gives no threshold for other laws.

Step 8: what staffing does to each assignment model

Both fits target the same ATE, with the same adjustment set and the same outcome model. The ordinary TMLE needs one correct nuisance model, Q or g. OAT C-TMLE needs a correct Q specifically. Staffing is the mechanism that separates them. It moves the full g toward 0 and 1. The outcome prediction gives it only a fitted coefficient near zero, so it barely moves the OAT g.

The code reads the support report of each fit. The report describes the fitted propensities and the weights \(1/g\) that targeting uses.

row	meaning
share of g below 0.1, above 0.9	the share of patients whose fitted propensity is near 0 or 1
truncated fraction	the share of patients whose fitted propensity reached the bounds of Step 6
treated ESS / n, control ESS / n	the effective sample size of each arm under the weights, as a share of the arm. A value near 1 means nearly equal weights
max |clever covariate|	the largest weight \(1/g\) in the targeting step

It then prints the two estimates side by side, and the ratio of their standard errors.

def weights_report(result):
    support = result.diagnostics.support()
    return {
        "share of g below 0.1": support.tail_mass[0.1]["below"],
        "share of g above 0.9": support.tail_mass[0.1]["above"],
        "truncated fraction": support.truncated["fraction"],
        "treated ESS / n": support.effective_sample_size["treated"]["ratio"],
        "control ESS / n": support.effective_sample_size["control"]["ratio"],
        "max |clever covariate|": support.clever_covariate_max["mean"],
    }


def estimate_report(result):
    point = result["ate"]
    low, high = point.ci
    return {
        "ATE": point.psi,
        "standard error": point.std_error,
        "95% CI lower": low,
        "95% CI upper": high,
    }


fits = {"TMLE": tmle, "OAT C-TMLE": collaborative}
print(pd.DataFrame({name: weights_report(fit) for name, fit in fits.items()}).round(4))
print()
estimates = pd.DataFrame({name: estimate_report(fit) for name, fit in fits.items()})
print(estimates.round(4))
se_ratio = collaborative["ate"].std_error / tmle["ate"].std_error
print(f"standard error ratio, OAT C-TMLE over TMLE: {se_ratio:.3f}")
print(f"population ATE: {truth['ate']:.4f}")

                           TMLE  OAT C-TMLE
share of g below 0.1     0.0680      0.0000
share of g above 0.9     0.0687      0.0000
truncated fraction       0.0100      0.0000
treated ESS / n          0.4496      0.9774
control ESS / n          0.6977      0.9974
max |clever covariate|  29.7790      3.5011

                  TMLE  OAT C-TMLE
ATE             0.1629      0.1483
standard error  0.0347      0.0283
95% CI lower    0.0950      0.0929
95% CI upper    0.2308      0.2037
standard error ratio, OAT C-TMLE over TMLE: 0.815
population ATE: 0.1186

What this output tells you. The ordinary TMLE fits g on staffing. Of its fitted propensities, 0.0680 lie below 0.1 and 0.0687 lie above 0.9. A share of 0.0100 reached the bounds. Its largest weight is 29.7790, and its effective-sample ratios are 0.4496 in the treated arm and 0.6977 in the control arm.

The OAT g has no fitted propensity outside 0.1 to 0.9. Its largest weight is 3.5011, and its effective-sample ratios are 0.9774 and 0.9974. The OAT g reads only the outcome prediction of each arm. That prediction moves with readiness and need, and staffing enters it only through a fitted coefficient near zero.

That small coefficient still matters for the interval. It is estimated, staffing predicts the offer, and so its sampling noise moves the OAT g. The stacked curve of Step 7 carries that term. The registered C-TMLE study has a control that leaves it out.

On this draw, the OAT standard error is 0.815 times the TMLE standard error. One draw does not show that this ratio holds in general. A review probe repeats the fits on 2,000 draws of this law, with the settings of this page. The probe is not a registered study. Its last column is the plug-in g-computation estimate of the same outcome model, with no targeting step.

over 2,000 probe draws	ordinary TMLE	OAT C-TMLE	g-computation
mean error of the estimate	+0.0006	+0.0006	+0.0006
empirical SD of the estimate	0.0358	0.0297	0.0297
mean standard error, over that SD	0.93	0.98	no interval
draws on which the interval contains 0.1186	1,857	1,881	no interval
mean share of g outside 0.1 to 0.9	0.193	0.0002	no g

The mean errors lie within twice their Monte Carlo standard error, about 0.0016. The OAT estimate varies less than the ordinary TMLE estimate. The OAT standard error was below the TMLE standard error on 1,963 of 2,000 draws.

The ordinary TMLE interval covered 1,857 of 2,000 times, below the nominal 95%. Its standard error averaged 0.93 of the spread. On this law and at this sample size, the influence-curve standard error of the ordinary TMLE understates its spread under extreme weights. The OAT interval covered 1,881 of 2,000 times. The registered C-TMLE study measures the OAT interval with more replications.

The g-computation estimate has the same spread as OAT C-TMLE. Both rest on a correct Q. The technical entry states that g-computation is at least as efficient inside a correct parametric outcome model. cleverly reports no interval for a point-treatment g-computation estimate. OAT C-TMLE gives a targeted estimate with the same spread and an interval that a registered study measures. That is the reason this page reports it, and not a precision gain over g-computation.

Step 9: diagnostics of the collaborative fit

The code runs the combined assessment of the OAT fit. It prints each check that ran or runs on request, and the nuisance report.

assessment = collaborative.assess()
ledger = assessment.to_frame()
ran = ledger[~ledger["status"].isin(["unavailable", "not_applicable"])]
print(ran[["surface", "check", "status"]].to_string(index=False))
print(f"needs attention: {tuple(item.name for item in assessment.attention)}")
print()
nuisance = assessment.report("nuisance_models")
print(f"treatment role: {nuisance.treatment_role}")
print(nuisance.summary())

    surface                 check    status
 validation       score_equations    passed
 validation               support completed
 validation       nuisance_models completed
diagnostics      truncation_curve  deferred
diagnostics                refute  deferred
sensitivity simulated_confounding  deferred
needs attention: ()

treatment role: collaborative_working_model
Nuisance model diagnostics
----------------------------------------
model          auc     brier   log_loss  r2  mse  cal_slope  cal_se  reg_slope
-------------  ------  ------  --------  --  ---  ---------  ------  ---------
propensity[0]  0.5209  0.2492  0.6915    -   -    1.0000     0.4862  -        
propensity[1]  0.5838  0.2445  0.6820    -   -    1.0000     0.1771  -        
outcome        0.7686  0.1958  0.5732    -   -    1.0000     0.0609  -        
Calibration slopes are in sample and carry no test. An unpenalized logistic model with an intercept has an in-sample slope of 1 by construction.

The propensity metrics describe the selected C-TMLE working mechanism.
They do not describe treatment given the complete adjustment set.
C-TMLE outcome-adaptive fit used one Qbar feature per arm in 2 binary mechanisms. Its intervals read the stacked parametric curve of ParametricOutcome(features='arm_interactions') and ParametricMechanism(degree=3). Each interval is pointwise and covers the arm mean when the outcome model is correct, and its limit psi* otherwise. The g_bounds are constants of the fit, so with a wrong outcome model psi* can move with n while a bound binds. No simultaneous band is built (X58 part (b))

VERDICT: C-TMLE working-model metrics are descriptive; inspect the selection and support reports.

What this output tells you. The score-equation check passed, and nothing needs attention. support and nuisance_models are completed: the calculation ran. completed is not a pass. The deferred rows run on request.

The treatment role is collaborative_working_model. The rows propensity[0] and propensity[1] describe the two per-arm mechanisms, with AUCs of 0.5209 and 0.5838. A low AUC is expected. Each mechanism describes the offer given the outcome prediction only, as the footer under the table says. It does not describe the offer given the complete adjustment set. The calibration slopes are in sample, so the report tests none, and the footer says so.

The outcome row describes Q, with an AUC of 0.7686 for success. No diagnostic here tests that Q is correct, and OAT C-TMLE needs that.

Step 10: sensitivity, what the fits cannot check

Sensitivity analysis asks how strong an unmeasured confounder would need to be to explain the result away. The omitted-variable bound needs the Riesz representer of the declared adjustment set. The ordinary TMLE estimates it, and the OAT fit does not, because its g conditions on the outcome prediction. So the code reads the bound on the ordinary TMLE fit. Point-treatment TMLE defines each quantity.

tmle_assessment = tmle.assess()
elements = tmle_assessment.report("elements")
robustness = tmle_assessment.report("robustness_value")
print(f"TMLE estimate:          {tmle['ate'].psi:.4f}")
print(f"sigma2:                 {elements.sigma2:.4f}")
print(f"nu2:                    {elements.nu2:.3f}")
print(f"robustness value:       {robustness['rv']:.3f}")
print(f"confidence-limit value: {robustness['rva']:.3f}")

TMLE estimate:          0.1629
sigma2:                 0.1957
nu2:                    8.042
robustness value:       0.122
confidence-limit value: 0.080

What this output tells you. The robustness value is 0.122. Take an unmeasured confounder at the worst alignment. If it explains a share of 0.122 of the residual variation of success and of the Riesz representer, it moves the TMLE estimate to zero. The value at the one-sided 95% limit is 0.080.

These values refer to the TMLE estimate of 0.1629, not to the OAT estimate of 0.1483. Both estimate the same ATE. The OAT estimate is closer to zero, so the same bias would reach zero sooner. Read the robustness value as optimistic for the OAT estimate.

The bound grows with \(\nu^2\), the second moment of the Riesz representer. Here the fit estimates \(\nu^2\) at 8.042. The representer of the ATE divides by g, so staffing raises \(\nu^2\).

A review probe computes the \(\nu^2\) of the law. It is about 10.14 for the full set and 9.14 with g clipped at the bounds of Step 6. Without staffing, it is 4.12. The fitted value lies below the law’s value, so the robustness value is optimistic here as well. A larger \(\nu^2\) lets a weaker confounder reach the null.

The review must decide whether an unrecorded factor could be that strong. Readiness is the measured benchmark. Point-treatment TMLE shows how to calibrate the bound against a measured covariate.

Method evidence

The fits on this page use one draw of a synthetic law. The registered studies in the table measure each method over repeated draws. Each study page gives its law, its nuisance models, and the result of each cell.

fit on this page	registered study	law of the study
Step 6, ordinary TMLE	Canonical point-treatment TMLE	another law
Step 7, OAT C-TMLE	Outcome-adaptive per-arm C-TMLE (parametric)	this page’s law, binary_active, at the same sample size

A review probe is a script under reviews/notebook-review/probes/. The step readings quote its results. A review probe is not a registered study.

The technical entry lists the evidence and the limits of the method.

Interpret the result

The question was how much the share of successful transitions would change if every eligible discharge received the offer, rather than usual support. OAT C-TMLE estimates an increase of 0.1483, about 15 percentage points, with the 95% interval [0.09292, 0.20368]. The interval is pointwise and describes the population of the synthetic law. The true ATE of 0.1186 lies inside it on this draw.

The reading rests on the registered study of Method evidence and on a correct outcome model. The assumptions of no unmeasured confounding, consistency, and no interference stay unverified. The robustness value of 0.122 in Step 10 refers to the TMLE estimate, and it is optimistic for this one.

conclusion	why the analysis does not support it
staffing has no effect on success	the method never tests the dotted arrow. It only makes g depend on what Q predicts
OAT C-TMLE is more precise than ordinary TMLE in general	the probe covers one law, one sample size, and correct models
OAT C-TMLE is more precise than g-computation	the probe shows the same spread, and the technical entry states that g-computation is at least as efficient inside a correct parametric outcome model
the interval holds with a flexible outcome learner	that fit reports no interval. F19 holds it
the estimate is causal for a real program	it needs the four assumptions of “The causal question”. The data test none of the first three, and positivity only in part

Where to go next

If the outcome model is the part you trust least, use the doubly robust point-treatment TMLE. With a strong assignment predictor, that route costs variance, and on this law it also cost coverage. If you expect one nuisance to be inconsistent whatever you choose, read the DR-TMLE case table. The technical entry lists every C-TMLE strategy and its inference status. The examples index lists every tutorial in the program.