{ "cells": [ { "cell_type": "markdown", "id": "title", "metadata": {}, "source": [ "# Point-treatment TMLE: did transition navigation improve the experience score?\n", "\n", "This tutorial answers one causal question from observational data. It writes the question as an\n", "estimand and fits a targeted maximum likelihood estimator (TMLE). It then checks the fit and says\n", "what the result supports.\n", "[Point-treatment TMLE](../technical-reference/point-treatment-tmle.md) gives the parameter, the\n", "influence curve, and the algorithm.\n", "\n", "## The situation\n", "\n", "A regional health plan offers a **standard transition-navigation protocol** to adults when a\n", "discharge home is ordered. The offer is a bedside plan and two scheduled navigator contacts within\n", "30 days. Discharge teams decided who received the offer. They used a recorded risk process, and\n", "nobody randomized the offer.\n", "\n", "Each row of the data is one discharge. The table gives the variables in time order, and separates\n", "the observed variables from the counterfactual ones.\n", "\n", "| variable | role | when it is fixed | observed? |\n", "| --- | --- | --- | --- |\n", "| `discharge_risk`, `prior_utilization`, `medication_burden`, `age` (together, W) | baseline covariates | before the offer | yes |\n", "| `transition_navigation` (A) | treatment: 1 for the offer, 0 for usual support | at the discharge-home order | yes |\n", "| `transition_score` (Y) | outcome: the score as a share of the maximum, so it lies between 0 and 1 | 30 days after discharge | yes |\n", "| $Y^1$ and $Y^0$ | the score the patient would have with the offer, and without it | 30 days after discharge | no. Each patient shows at most one of them, $Y = Y^A$ |\n", "\n", "The graph states the causal assumptions of the analysis.\n", "\n", "```text\n", " W ───────────────┐\n", " │ ▼\n", " └──────▶ A ────▶ Y\n", "\n", " W = discharge_risk, prior_utilization, medication_burden, age\n", " A = transition_navigation (the offer)\n", " Y = transition_score at day 30\n", " not drawn: an unrecorded cause U with U ──▶ A and U ──▶ Y\n", "```\n", "\n", "| arrow or absent arrow | what it asserts | in the synthetic law of this page |\n", "| --- | --- | --- |\n", "| W ──▶ A | the baseline profile changes the chance of an offer | all four covariates enter g, through a linear term, a square, an interaction, and an indicator |\n", "| W ──▶ Y | the baseline profile changes the score in either arm | all four covariates enter Q, through a sine, a square, an interaction, and an absolute value |\n", "| A ──▶ Y | the offer can change the score. The size of this effect is the target | the effect varies with the covariates |\n", "| no U | no unrecorded variable causes both the offer and the score | true by construction. In a real program it is an assumption |\n", "| no node after A | nothing measured after the offer enters the analysis | completed contacts and length of stay occur after the offer, so they are not adjusted for |\n", "\n", "All four covariates are confounders, because each opens a path A ◀── W ──▶ Y. The analysis must\n", "block these paths.\n", "\n", "There is no mediator, censoring variable, missing outcome, or death in this\n", "law. The outcome mean is the expit of a sum. On the score scale, every covariate therefore modifies\n", "the effect. `discharge_risk` and `prior_utilization` also change the effect on the logit scale. The\n", "graph cannot show whether a real U exists. Step 10 asks how strong it would need to be." ] }, { "cell_type": "markdown", "id": "plan", "metadata": {}, "source": [ "## The question\n", "\n", "The program sponsor must decide whether to offer navigation to every eligible discharge. The\n", "sponsor asks one question. How much would the mean 30-day transition score change if every\n", "eligible discharge received the offer, rather than usual support?\n", "\n", "| element | this question |\n", "| --- | --- |\n", "| target population | adults with a discharge-home order at a participating hospital |\n", "| intervention | the offer, against usual support, analyzed regardless of completed contacts |\n", "| counterfactual outcome | $Y^a$, the 30-day score under offer status $a$ |\n", "| estimand | the average treatment effect (ATE), $\\psi = E[Y^1] - E[Y^0]$ |\n", "\n", "The sponsor's decision applies to every eligible discharge, so the estimand is the ATE. The\n", "effect among the patients who received the offer (the ATT) answers a question about the past\n", "rollout. The effect among the others (the ATC) answers a question about an extension to them. The\n", "effect varies with `discharge_risk`, which also drives the offer, so the three differ.\n", "\n", "A regression coefficient on the offer is not the causal estimand of this question. It equals the ATE\n", "when the regression model is correct and has no effect modification. In general it does not.\n", "\n", "| identification assumption | what it means for this program | how it is assessed |\n", "| --- | --- | --- |\n", "| consistency | an offer always means the declared bedside plan and two scheduled contacts | scientific judgment, from the program's records of the offer |\n", "| no interference | one patient's offer does not change another patient's score | scientific judgment, from the capacity and access controls of the design |\n", "| no unmeasured confounding | the recorded baseline variables block every common cause of the offer and the score | scientific judgment. Step 10 measures how strong a violation would need to be |\n", "| positivity | each baseline profile has some chance of an offer and of usual support | partly checkable. Step 8 describes the fitted overlap |\n", "\n", "## Why this method\n", "\n", "Each choice below follows from a feature of the problem.\n", "\n", "| feature of the problem | method choice |\n", "| --- | --- |\n", "| one treatment decision at time zero, and one outcome at day 30 | point-treatment TMLE. No treatment or confounder varies over time, so longitudinal TMLE is not needed |\n", "| a static contrast, with a true propensity between 0.05 and 0.95 | the ATE of \"offer everyone\" against \"offer no one\". Positivity holds in this law, so a stochastic or modified policy is not needed |\n", "| the score is observed for every patient | no missing-outcome model and no censoring model |\n", "| both nuisance functions are nonlinear | flexible learners for Q and g. Step 7 shows what misspecified linear models do |\n", "| a flexible learner can overfit the rows it predicts | cross-fitting. [Method evidence](#method-evidence) links a registered study of this choice |\n", "| the score lies between 0 and 1 by its definition | `q_bounds=(0.0, 1.0)` and a logistic fluctuation, so the targeted predictions stay on the score's scale |\n", "| the sponsor needs an interval | the Wald interval from the targeted influence curve |\n", "\n", "## Terms\n", "\n", "| term | plain meaning | reference |\n", "| --- | --- | --- |\n", "| estimand | the number the question asks for, written before any model is chosen | [estimands](../user-guide/estimands.md) |\n", "| nuisance | a model the estimate needs but the question does not ask about. Here, the outcome regression Q and the treatment mechanism g | [point-treatment TMLE](../technical-reference/point-treatment-tmle.md) |\n", "| targeting | a small update to Q, weighted by g, that removes the first-order bias of the plug-in estimate | [targeting and bounds](../user-guide/methods-learners.md#targeting-and-bounds) |\n", "| influence curve | each row's contribution to the estimate's error. Its variance gives the standard error | [inference](../technical-reference/inference.md) |\n", "| cross-fitting | each row's nuisance prediction comes from models fit without that row | [CV-TMLE](../technical-reference/cv-tmle.md) |\n", "| positivity | every kind of patient has some chance of each arm, so both arms have data to compare | [diagnostics](../user-guide/results-assessment.md#diagnostics) |" ] }, { "cell_type": "markdown", "id": "setup-heading", "metadata": {}, "source": [ "## Step 1: set up\n", "\n", "Every fit below passes its learners, fold count, and random seed explicitly. A rerun therefore\n", "reproduces the stored outputs." ] }, { "cell_type": "code", "execution_count": 1, "id": "setup", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:32.259116Z", "iopub.status.busy": "2026-10-16T01:16:32.258814Z", "iopub.status.idle": "2026-10-16T01:16:33.795393Z", "shell.execute_reply": "2026-10-16T01:16:33.794857Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cleverly 0.1.3\n" ] } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "from sklearn.ensemble import HistGradientBoostingClassifier, HistGradientBoostingRegressor\n", "from sklearn.linear_model import LinearRegression, LogisticRegression\n", "\n", "import cleverly\n", "\n", "pd.set_option(\"display.width\", 130)\n", "pd.set_option(\"display.max_columns\", 20)\n", "print(\"cleverly\", cleverly.__version__)" ] }, { "cell_type": "markdown", "id": "setup-reading", "metadata": {}, "source": [ "**What this output tells you.** The stored outputs came from the version named above. Another\n", "version can print different numbers." ] }, { "cell_type": "markdown", "id": "data-heading", "metadata": {}, "source": [ "## Step 2: the data\n", "\n", "The data come from a synthetic law with a known answer, so later steps can compare each estimate\n", "with the truth. A real analysis has no truth. `navigation_data` draws the rows of the generator\n", "`make_nonlinear_bounded` and gives the columns the program's names." ] }, { "cell_type": "code", "execution_count": 2, "id": "data", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:33.796781Z", "iopub.status.busy": "2026-10-16T01:16:33.796417Z", "iopub.status.idle": "2026-10-16T01:16:34.884396Z", "shell.execute_reply": "2026-10-16T01:16:34.883862Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "rows and columns: (3000, 6)\n", "column names: ['transition_score', 'transition_navigation', 'discharge_risk', 'prior_utilization', 'medication_burden', 'age']\n", " transition_score transition_navigation discharge_risk prior_utilization medication_burden age\n", "0 0.748 0.0 0.359 1.511 -1.786 1.687\n", "1 0.554 1.0 -0.047 -0.800 -0.803 -1.083\n", "2 0.463 0.0 -0.224 0.834 0.584 0.638\n", "3 0.084 0.0 -1.695 -1.571 1.554 0.969\n", "4 0.948 1.0 2.183 1.210 -1.024 1.285\n", "\n", "known values of the synthetic law:\n", " ey1: 0.567\n", " ey0: 0.404\n", " ate: 0.163\n", " att: 0.177\n", " atc: 0.151\n" ] } ], "source": [ "from cleverly.datasets import navigation_data\n", "\n", "frame, truth = navigation_data(n=3_000, seed=21)\n", "print(\"rows and columns:\", frame.shape)\n", "print(\"column names:\", list(frame.columns))\n", "print(frame.head().round(3))\n", "print()\n", "print(\"known values of the synthetic law:\")\n", "for key in (\"ey1\", \"ey0\", \"ate\", \"att\", \"atc\"):\n", " print(f\" {key}: {truth[key]:.3f}\")" ] }, { "cell_type": "markdown", "id": "data-reading", "metadata": {}, "source": [ "**What this output tells you.** The columns are the ones the situation table names. Every score\n", "lies between 0 and 1. The covariates are independent standard-normal draws, so a negative `age`\n", "is below the average age. Real covariates would be correlated.\n", "\n", "The truth line `ate` is $E[Y^1] - E[Y^0] = 0.163$. The ATT is 0.177 and the ATC is 0.151, so the\n", "choice of estimand changes the answer." ] }, { "cell_type": "markdown", "id": "association-heading", "metadata": {}, "source": [ "## Step 3: association first\n", "\n", "The graph predicts that the unadjusted contrast is confounded. The recorded risk process offers\n", "navigation more often to patients with a higher `discharge_risk`, and that covariate also changes\n", "the score. The unadjusted difference $E[Y \\mid A = 1] - E[Y \\mid A = 0]$ equals the ATE only\n", "when no such path exists. The code compares the arms before any adjustment." ] }, { "cell_type": "code", "execution_count": 3, "id": "association", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:34.886008Z", "iopub.status.busy": "2026-10-16T01:16:34.885831Z", "iopub.status.idle": "2026-10-16T01:16:34.901427Z", "shell.execute_reply": "2026-10-16T01:16:34.900978Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " transition_score discharge_risk prior_utilization medication_burden age\n", "transition_navigation \n", "0.0 0.387 -0.219 0.004 -0.024 -0.011\n", "1.0 0.585 0.226 0.012 -0.047 0.053\n", "\n", "share offered navigation: 0.465\n", "unadjusted difference in mean score: 0.198\n", "population ATE: 0.163\n" ] } ], "source": [ "covariates = [\"discharge_risk\", \"prior_utilization\", \"medication_burden\", \"age\"]\n", "by_arm = frame.groupby(\"transition_navigation\")[[\"transition_score\", *covariates]].mean()\n", "print(by_arm.round(3))\n", "print()\n", "print(\"share offered navigation:\", round(float(frame[\"transition_navigation\"].mean()), 3))\n", "unadjusted = by_arm.loc[1.0, \"transition_score\"] - by_arm.loc[0.0, \"transition_score\"]\n", "print(f\"unadjusted difference in mean score: {unadjusted:.3f}\")\n", "print(f\"population ATE: {truth['ate']:.3f}\")" ] }, { "cell_type": "markdown", "id": "association-reading", "metadata": {}, "source": [ "**What this output tells you.** The offered patients score 0.198 higher on average, and the true\n", "ATE is 0.163. The arms differ before the offer. The mean `discharge_risk` is 0.226 among offered\n", "patients and -0.219 among the others.\n", "\n", "Two mechanisms of this law raise the unadjusted difference above the ATE. First, the offered\n", "patients would score higher than the others under usual support alone. Second, the effect grows\n", "with `discharge_risk`, so the offered patients gain more than the average patient. An estimator\n", "must compare patients with the same baseline profile. The next two steps state that comparison and\n", "the assumptions that make it possible." ] }, { "cell_type": "markdown", "id": "protocol-heading", "metadata": {}, "source": [ "## Step 4: write the protocol\n", "\n", "A `StudyProtocol` records the scientific design before any model runs. It uses target-trial\n", "terms: the population, the time zero, the strategies, the outcome, and the handling of later\n", "events. Every result fitted from it carries its fingerprint. `navigation_protocol()` in\n", "`cleverly.datasets` holds the program's record. A real analysis calls `StudyProtocol(...)` with the\n", "same ten fields." ] }, { "cell_type": "code", "execution_count": 4, "id": "protocol", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:34.902727Z", "iopub.status.busy": "2026-10-16T01:16:34.902471Z", "iopub.status.idle": "2026-10-16T01:16:34.905304Z", "shell.execute_reply": "2026-10-16T01:16:34.904854Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "causal study protocol: schema 1; 623fc2c240615d26\n", "target population: Adults with a discharge-home order at a participating hospital during the enrollment period\n", "eligibility: ['Age 18 years or older', 'Discharge home ordered at a participating hospital']\n", "time zero: Discharge-home order, after baseline measurement and before the navigation offer\n", "treatment strategies: ['Offer standard transition navigation', 'Provide usual discharge support']\n", "treatment versions: ['Bedside transition plan and two scheduled navigator contacts within 30 days', 'No access to the transition-navigation offer']\n", "outcome: Patient-reported transition score, as a share of the maximum score\n", "horizon: 30 days after discharge\n", "intercurrent-event handling: ['Use the transition score regardless of readmission', 'Analyze the offer regardless of completed contacts', 'The protocol scores death before day 30 as the worst transition score (composite strategy)']\n", "interference unit: Individual patient\n", "assumption rationale: ['The recorded baseline variables cover the measured common causes', 'The standardized offer and version records support consistency', 'Reserved navigator capacity and access controls support no interference']\n" ] } ], "source": [ "from cleverly.datasets import navigation_protocol\n", "\n", "protocol = navigation_protocol()\n", "print(\"\\n\".join(protocol.summary_lines()))" ] }, { "cell_type": "markdown", "id": "protocol-reading", "metadata": {}, "source": [ "**What this output tells you.** The first line gives the schema version and the fingerprint\n", "`623fc2c240615d26`. Three fields set the meaning of the answer.\n", "\n", "| protocol field | what it fixes for this question |\n", "| --- | --- |\n", "| target population and time zero | the effect is about adults with a discharge-home order at a participating hospital. Follow-up starts at the order, before the offer, so eligibility cannot depend on the offer |\n", "| treatment strategies and versions | the treatment is the **offer**. The protocol analyzes the offer regardless of completed contacts, so the estimand is the effect of offering navigation, not of receiving it |\n", "| outcome and intercurrent events | the score is a share of the maximum. A death before day 30 counts as the worst score. The synthetic law draws no deaths |\n", "\n", "The outcome field is the evidence for the score support that Step 6 declares. A fit cannot recover\n", "that support from the data. The protocol does not store the contrast. The typed estimand in the\n", "next step owns it." ] }, { "cell_type": "markdown", "id": "identify-heading", "metadata": {}, "source": [ "## Step 5: design and identification\n", "\n", "The design says which column plays which role. The estimand says which contrast the question\n", "needs. Identification writes the estimand as a quantity of the observed data, and lists the\n", "assumptions that make the two equal." ] }, { "cell_type": "code", "execution_count": 5, "id": "identify", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:34.906500Z", "iopub.status.busy": "2026-10-16T01:16:34.906263Z", "iopub.status.idle": "2026-10-16T01:16:34.913027Z", "shell.execute_reply": "2026-10-16T01:16:34.912583Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "average treatment effect, E[Y^a] - E[Y^reference]\n", "identified by explicit-adjustment: E_W[E(transition_score | transition_navigation=a, W)] - E_W[E(transition_score | transition_navigation=0, W)] for a in [1]\n", "adjustment/history: ['discharge_risk', 'prior_utilization', 'medication_burden', 'age']\n", "required nuisances: ['outcome_regression', 'treatment_mechanism']\n", "assumptions:\n", " - consistency: Y = Y^a when A = a\n", " - no interference: one unit's potential outcome does not depend on other units' treatment assignments\n", " - no unmeasured confounding: Y^a is independent of A given W\n", " - positivity: P(transition_navigation = a | W) > 0 almost surely for every supported treatment level a in [0, 1]\n", "causal study protocol: schema 1; 623fc2c240615d26\n" ] } ], "source": [ "from cleverly import ATE, CausalStudy, PointTreatment\n", "\n", "study = CausalStudy(\n", " frame,\n", " design=PointTreatment(\n", " outcome=\"transition_score\",\n", " treatment=\"transition_navigation\",\n", " adjustment=tuple(covariates),\n", " ),\n", " protocol=protocol,\n", ")\n", "effect = study.identify(ATE(reference=0))\n", "\n", "print(effect.summary(protocol=\"fingerprint\"))" ] }, { "cell_type": "markdown", "id": "identify-reading", "metadata": {}, "source": [ "**What this output tells you.** Under the four printed assumptions, the ATE equals the printed\n", "formula $E_W[Q(1, W) - Q(0, W)]$. Here $Q(a, W) = E(Y \\mid A = a, W)$. The formula needs two\n", "nuisances: Q, and the treatment mechanism $g(W) = P(A = 1 \\mid W)$. The question section states\n", "which assumptions need scientific judgment.\n", "\n", "The [shared study design](index.md#the-shared-study-design) states how the program supports\n", "consistency and no interference. An unrecorded discharge-team judgment that affects both the offer\n", "and recovery would violate no unmeasured confounding, and no estimator repairs that. A real\n", "protocol should also evaluate site, navigator team, calendar time, and language access as\n", "confounders." ] }, { "cell_type": "markdown", "id": "estimate-heading", "metadata": {}, "source": [ "## Step 6: estimate the ATE\n", "\n", "The method follows the choices of \"Why this method\". The configuration is written out in full.\n", "\n", "| setting | value | what it does |\n", "| --- | --- | --- |\n", "| `outcome_learner` | gradient boosting | fits Q |\n", "| `treatment_learner` | a shallow, regularized booster | fits g |\n", "| `CrossFitting(n_folds=5)` | five folds | predicts each row from models that did not see that row |\n", "| `Targeting(q_bounds=(0.0, 1.0))` | the declared score support | fixes the outcome scale from the protocol, not from the observed scores |\n", "| `Inference(alpha=0.05)` | 95% interval | sets the interval level |\n", "| `Runtime(random_state=21, n_jobs=1)` | fixed seed, one process | makes the fit reproducible |\n", "\n", "Do not use the default `HistGradientBoostingClassifier` as a treatment learner below 10,000 rows.\n", "Below that size, scikit-learn does not stop it early, and its fitted propensities are too extreme.\n", "The [rules for code cells](../development/example-notebooks.md#rules-for-code-cells) state this\n", "requirement, and Step 8 checks the calibration of the fitted propensities." ] }, { "cell_type": "code", "execution_count": 6, "id": "estimate", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:34.918219Z", "iopub.status.busy": "2026-10-16T01:16:34.917962Z", "iopub.status.idle": "2026-10-16T01:16:37.156754Z", "shell.execute_reply": "2026-10-16T01:16:37.156271Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Targeted maximum likelihood estimation\n", "======================================\n", "n = 3000; covariates = 4; P(A=1) = 0.4653\n", "causal estimand: average treatment effect, E[Y^a] - E[Y^reference]\n", "identification: explicit-adjustment; E_W[E(transition_score | transition_navigation=a, W)] - E_W[E(transition_score | transition_navigation=0, W)] for a in [1]\n", "required nuisances: outcome_regression, treatment_mechanism\n", "identification assumptions: consistency: Y = Y^a when A = a; no interference: one unit's potential outcome does not depend on other units' treatment assignments; no unmeasured confounding: Y^a is independent of A given W; positivity: P(transition_navigation = a | W) > 0 almost surely for every supported treatment level a in [0, 1]\n", "causal study protocol: schema 1; 623fc2c240615d26\n", "outcome family: gaussian; fluctuation: logistic (iterative)\n", "stacked CV-TMLE (Levy): nuisances cross-fitted over 5 folds; targeting: pooled\n", "propensity truncated to [0.0114, 0.9886]; ATT/ATC to [0.025, 0.975]\n", "data 67649085bf54f0fd | folds 01ad882979feb330 | cleverly 0.1.3 | 2026-10-16T01:16:37+00:00\n", "\n", "estimand psi std_err 95% CI p_value\n", "-------- ------- -------- ------------------ -------\n", "ate 0.16658 0.005482 [0.15584, 0.17733] <1e-4 \n", "\n", "estimate: 0.167\n", "standard error: 0.0055\n", "95% CI: (0.156, 0.177)\n", "population ATE: 0.163\n" ] } ], "source": [ "from cleverly import CrossFitting, Inference, ModelSpec, Runtime, Targeting, TMLEMethod\n", "\n", "\n", "def shallow_booster():\n", " \"\"\"A shallow, regularized boosted propensity learner.\"\"\"\n", " return HistGradientBoostingClassifier(\n", " max_depth=2,\n", " learning_rate=0.05,\n", " max_iter=200,\n", " l2_regularization=1.0,\n", " random_state=21,\n", " )\n", "\n", "\n", "flexible = TMLEMethod(\n", " models=ModelSpec(\n", " outcome_learner=HistGradientBoostingRegressor(random_state=21),\n", " treatment_learner=shallow_booster(),\n", " ),\n", " cross_fitting=CrossFitting(n_folds=5),\n", " targeting=Targeting(q_bounds=(0.0, 1.0)),\n", " inference=Inference(alpha=0.05),\n", " runtime=Runtime(random_state=21, n_jobs=1),\n", ")\n", "result = effect.estimate(method=flexible)\n", "\n", "estimate = result[\"ate\"]\n", "print(result.summary(protocol=\"fingerprint\"))\n", "print()\n", "print(f\"estimate: {estimate.psi:.3f}\")\n", "print(f\"standard error: {estimate.std_error:.4f}\")\n", "print(f\"95% CI: ({estimate.ci[0]:.3f}, {estimate.ci[1]:.3f})\")\n", "print(f\"population ATE: {truth['ate']:.3f}\")" ] }, { "cell_type": "markdown", "id": "estimate-reading", "metadata": {}, "source": [ "**What this output tells you.** The summary names the construction as stacked CV-TMLE. The\n", "nuisances are cross-fitted over 5 folds, and one targeting step runs on the pooled out-of-fold\n", "predictions.\n", "The propensity truncation bound is [0.0114, 0.9886].\n", "\n", "The estimate is 0.167 with a standard error of 0.0055. The 95% interval is (0.156, 0.177), and it\n", "contains the true ATE of 0.163 on this draw. One draw is not a coverage result. A review probe,\n", "`reviews/notebook-review/probes/point-treatment-tmle-final/sweep.py`, repeats this fit on 200\n", "draws of the same law. The interval contained the true ATE on 189 of them, and the mean standard\n", "error was 0.98 times the standard deviation of the estimates. The probe is not a registered study.\n", "\n", "The interval is asymptotically valid under positivity, the declared independence of the rows, and a rate\n", "condition on the nuisances. The rate condition asks that the errors of Q and g, multiplied\n", "together, shrink faster than $1/\\sqrt{n}$." ] }, { "cell_type": "markdown", "id": "failure-heading", "metadata": {}, "source": [ "## Step 7: what targeting can repair, and what it cannot\n", "\n", "Double robustness means that the TMLE point estimate stays consistent when *either* Q *or* g is\n", "consistent. The reason is the remainder of the estimator, which is a product of the error of Q and\n", "the error of g. The product shrinks when one factor shrinks.\n", "\n", "This step compares four estimators of the same ATE, with the same identification formula from\n", "Step 5. Name the mechanism before reading the numbers. The law's Q and g contain a sine, squares,\n", "interactions, and an absolute value. A main-terms linear or logistic regression cannot represent\n", "them, so each linear model is inconsistent here. The boosted learners can approximate them as n\n", "grows.\n", "\n", "| estimator | Q | g | expected behavior |\n", "| --- | --- | --- | --- |\n", "| g-computation plug-in | linear | none | inherits the bias of the linear Q |\n", "| TMLE | linear | booster | consistent through g. The g error shrinks, so the product remainder shrinks |\n", "| TMLE | booster | logistic | consistent through Q |\n", "| TMLE | linear | logistic | neither nuisance is consistent, so the remainder need not shrink |\n", "\n", "The plug-in averages $\\hat Q(1, W) - \\hat Q(0, W)$ over the rows. With a main-terms linear Q, that\n", "average equals the regression coefficient on the offer. The plug-in has no influence-curve\n", "interval, so the code prints none. Each TMLE fit copies the Step 6 method and changes only the\n", "learners." ] }, { "cell_type": "code", "execution_count": 7, "id": "failure-mode", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:37.159164Z", "iopub.status.busy": "2026-10-16T01:16:37.158980Z", "iopub.status.idle": "2026-10-16T01:16:39.615191Z", "shell.execute_reply": "2026-10-16T01:16:39.614179Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "plug-in, linear Q psi= 0.148 miss=0.015\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "linear Q, flexible g psi= 0.164 CI=(0.150, 0.179) miss=0.002\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "flexible Q, linear g psi= 0.167 CI=(0.157, 0.177) miss=0.005\n", "both linear psi= 0.144 CI=(0.130, 0.158) miss=0.019\n", "population ATE: 0.163\n" ] } ], "source": [ "from dataclasses import replace\n", "\n", "from cleverly.learners import thread_limit\n", "\n", "dr_points = {}\n", "\n", "features = frame[[\"transition_navigation\", *covariates]]\n", "with thread_limit():\n", " linear_q = LinearRegression().fit(features, frame[\"transition_score\"])\n", "offered = linear_q.predict(features.assign(transition_navigation=1.0))\n", "not_offered = linear_q.predict(features.assign(transition_navigation=0.0))\n", "plug_in = float(np.mean(offered - not_offered))\n", "print(\n", " f\"{'plug-in, linear Q':22s} psi={plug_in:6.3f} {'':20s}miss={abs(plug_in - truth['ate']):.3f}\"\n", ")\n", "\n", "\n", "def fit(outcome_learner, treatment_learner, label):\n", " method = replace(\n", " flexible,\n", " models=ModelSpec(outcome_learner=outcome_learner, treatment_learner=treatment_learner),\n", " )\n", " point = effect.estimate(method=method)[\"ate\"]\n", " dr_points[label] = point\n", " low, high = point.ci\n", " miss = abs(point.psi - truth[\"ate\"])\n", " print(f\"{label:22s} psi={point.psi:6.3f} CI=({low:.3f}, {high:.3f}) miss={miss:.3f}\")\n", "\n", "\n", "fit(LinearRegression(), shallow_booster(), \"linear Q, flexible g\")\n", "fit(\n", " HistGradientBoostingRegressor(random_state=21),\n", " LogisticRegression(max_iter=1000, random_state=21),\n", " \"flexible Q, linear g\",\n", ")\n", "fit(LinearRegression(), LogisticRegression(max_iter=1000, random_state=21), \"both linear\")\n", "print(f\"population ATE: {truth['ate']:.3f}\")" ] }, { "cell_type": "markdown", "id": "failure-reading", "metadata": {}, "source": [ "**What this output tells you.** The plug-in from the linear Q misses the true ATE by 0.015. TMLE\n", "with the same linear Q and the boosted g misses by 0.002, and its interval contains 0.163. TMLE\n", "with the boosted Q and the logistic g also contains it. With both models linear, TMLE misses by\n", "0.019, and its interval (0.130, 0.158) excludes the true value. Neither of its nuisances is\n", "consistent, so targeting cannot remove the product of their errors.\n", "\n", "One draw shows a pattern, not a property. Two review probes in\n", "`reviews/notebook-review/probes/point-treatment-tmle-final/` repeat these fits on 200 draws.\n", "`summary.log` labels the TMLE fits as \"Step 8\".\n", "\n", "| estimator | mean error over 200 draws | interval contains the truth | probe |\n", "| --- | --- | --- | --- |\n", "| plug-in, linear Q | -0.018 | no interval | `plugin.py` |\n", "| TMLE, linear Q and boosted g | -0.0022 | 195 of 200 | `sweep.py` |\n", "| TMLE, boosted Q and logistic g | -0.0006 | 190 of 200 | `sweep.py` |\n", "| TMLE, both linear | -0.0218 | 30 of 200 | `sweep.py` |\n", "\n", "The both-linear miss is therefore a bias, not sampling variation. Its mean error is larger than\n", "the plug-in's, so targeting with a wrong g can move a wrong Q further from the truth.\n", "\n", "| caution | why |\n", "| --- | --- |\n", "| a real analysis does not know which nuisance is consistent | the linear models here omit terms that only the synthetic law reveals |\n", "| one nuisance inconsistent | the point estimate can stay consistent, but the influence-curve interval need not be valid. [DR-TMLE](../technical-reference/dr-tmle/index.md) addresses this only in [the listed cases](../technical-reference/dr-tmle/theorem.md#the-conditioning-terms-are-linear-in-one-error) |\n", "| one law is not a general result | another law, another size, or other learners can change the pattern |\n", "\n", "This is why Step 6 uses flexible learners for both nuisances. Flexible learners make a consistent\n", "Q and g more likely. They do not guarantee it." ] }, { "cell_type": "markdown", "id": "assessment-heading", "metadata": {}, "source": [ "## Step 8: diagnostics, what the fit can check\n", "\n", "The assessment of the Step 6 fit holds its diagnostic reports. The code prints the rows that need\n", "attention and three reports. The support report describes overlap, the sample view of positivity.\n", "The nuisance report gives the calibration of the fitted propensities. The score check confirms\n", "that targeting solved its equation." ] }, { "cell_type": "code", "execution_count": 8, "id": "assessment", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:39.618018Z", "iopub.status.busy": "2026-10-16T01:16:39.617796Z", "iopub.status.idle": "2026-10-16T01:16:39.701676Z", "shell.execute_reply": "2026-10-16T01:16:39.695042Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "needs attention: ()\n", "\n", "Positivity / overlap diagnostics\n", "--------------------------------\n", "n = 3000; propensity truncated to [0.0114, 0.9886]\n", "\n", "g(W) quantile 0% 1% 5% 25% 50% 75% 95% 99% 100% \n", "------------- ------ ------ ------ ------ ------ ------ ------ ------ ------\n", "overall 0.0704 0.1314 0.2187 0.3703 0.4690 0.5589 0.7132 0.7840 0.8815\n", "treated 0.1087 0.1842 0.2800 0.4265 0.5040 0.5969 0.7305 0.7938 0.8815\n", "control 0.0704 0.1230 0.1949 0.3208 0.4311 0.5163 0.6723 0.7520 0.8799\n", "\n", "arm n effective n ESS / n top 1% weight top 5% weight\n", "------- ---- ----------- ------- ------------- -------------\n", "treated 1396 1212.5 0.869 0.034 0.112 \n", "control 1604 1447.7 0.903 0.028 0.101 \n", "\n", "threshold P(g < t) P(g > 1-t)\n", "--------- -------- ----------\n", "0.01 0.0000 0.0000 \n", "0.025 0.0000 0.0000 \n", "0.05 0.0000 0.0000 \n", "0.1 0.0027 0.0000 \n", "\n", "truncated: 0 unit(s) (0.00%); most extreme untruncated g(W) = 0.070385\n", "\n", "group equation target rows Kish-equivalent rows Kish / target Kish / total top 1% load top 5% load max |w h| zero load g bound clipped \n", "----- -------- ----------- -------------------- ------------- ------------ ----------- ----------- --------- --------- ---------------- ---------\n", "mean h1 3000/3000 1212.5 0.404 0.404 0.060 0.198 9.199 1604 [0.0114, 0.9886] 0 (0.00%)\n", "(absolute residual-multiplier load from the fitted equation; Kish values describe residual-multiplier concentration, not residual contributions or estimator information, influence, or precision; target rows are score-mask rows and include structural zeros)\n", "max |clever covariate| (mean): 9.199\n", "\n", "VERDICT: no truncation-driven fragility detected (0.0% of units clipped at the marginal treatment-mechanism bound [0.0114, 0.9886]). The narrowest arm has a Kish-equivalent weight count of 87% of its nominal rows. This is concentration, not estimator information, and no threshold is applied because none is derived. Absolute-load concentration is greatest for group 'mean', equation 'h1': 1212.5 Kish-equivalent rows out of 3000 score-mask rows (40%; 40% of 3000 total rows). This is a descriptive residual-multiplier concentration index. It excludes the residual and is not estimator effective sample size, information, actual influence, precision, or a positivity test.\n", "\n", "Nuisance model diagnostics\n", "----------------------------------------\n", "model auc brier log_loss r2 mse cal_slope cal_se reg_slope\n", "---------- ------ ------ -------- ------ ------ --------- ------ ---------\n", "propensity 0.6611 0.2291 0.6498 - - 0.9564 0.0662 - \n", "outcome - - - 0.6746 0.0176 - - 0.9546 \n", "Calibration slope: logistic recalibration of the out-of-fold predictions with one intercept per fold.\n", "A finding needs the 95.0% interval to lie above 0 and exclude 1, a Bonferroni level over 1 tested model(s).\n", "\n", "VERDICT: nuisance fits look reasonable.\n", "\n", "PASS: the targeting step solved the estimated efficient score equation.\n" ] } ], "source": [ "assessment = result.assess()\n", "print(\"needs attention:\", tuple(item.name for item in assessment.attention))\n", "\n", "support = assessment.report(\"support\")\n", "nuisance = assessment.report(\"nuisance_models\")\n", "scores = assessment.report(\"score_equations\")\n", "print()\n", "print(support.summary())\n", "print()\n", "print(nuisance.summary())\n", "print()\n", "print(scores.summary().splitlines()[-1])" ] }, { "cell_type": "markdown", "id": "assessment-reading", "metadata": {}, "source": [ "**What this output tells you.** No row needs attention.\n", "\n", "| output part | what it shows on this draw | what it cannot show |\n", "| --- | --- | --- |\n", "| positivity and overlap | no unit sits at the truncation bound. The smallest fitted propensity is 0.070385. The effective-sample-size ratio is 0.869 in the treated arm and 0.903 in the control arm | the true propensity. The report describes the fitted model |\n", "| nuisance model diagnostics | the out-of-fold calibration slope of the propensity is 0.9564, with a standard error of 0.0662. The verdict reads `nuisance fits look reasonable` | that the propensity model is correct |\n", "| score-equation check | targeting solved the estimated efficient score equation | that the estimate is unbiased |\n", "\n", "A calibration slope well below 1 means that the fitted propensities are more extreme than the\n", "observed offer rates. The default booster that Step 6 avoids produces that pattern below 10,000\n", "rows. A slope of 0.9564 with that standard error does not differ from 1.\n", "\n", "The fitted model can still miss part of the law. It puts a share of 0.0027 of the rows below 0.1,\n", "in the `P(g < t)` column. In the law, about 1.9% of patients have a true propensity below 0.1\n", "(review probe `reviews/notebook-review/probes/point-treatment-tmle-final/truth.py`). The report\n", "gives no positivity verdict, because no universal cutoff applies." ] }, { "cell_type": "markdown", "id": "truncation-heading", "metadata": {}, "source": [ "## Step 9: truncation, does the bound drive the estimate?\n", "\n", "Truncation clips each fitted propensity into a bound, so no row gets an extreme weight. A clip\n", "adds bias that the standard error does not include. The code retargets the estimate at a grid of\n", "bounds and at the fitted bound, without refitting the nuisances. The grid ends with two tight\n", "bounds, 0.1 and 0.2, which clip rows on this fit." ] }, { "cell_type": "code", "execution_count": 9, "id": "truncation-curve", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:39.703249Z", "iopub.status.busy": "2026-10-16T01:16:39.703088Z", "iopub.status.idle": "2026-10-16T01:16:39.731242Z", "shell.execute_reply": "2026-10-16T01:16:39.730760Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " bound psi std_err ci_lower ci_upper truncated_fraction\n", "0.0010 0.1666 0.0055 0.1558 0.1773 0.0000\n", "0.0050 0.1666 0.0055 0.1558 0.1773 0.0000\n", "0.0100 0.1666 0.0055 0.1558 0.1773 0.0000\n", "0.0114 0.1666 0.0055 0.1558 0.1773 0.0000\n", "0.0250 0.1666 0.0055 0.1558 0.1773 0.0000\n", "0.0500 0.1666 0.0055 0.1558 0.1773 0.0000\n", "0.1000 0.1666 0.0055 0.1558 0.1773 0.0027\n", "0.2000 0.1674 0.0053 0.1570 0.1778 0.0417\n" ] } ], "source": [ "curve = result.diagnostics.truncation_curve()\n", "columns = [\"bound\", \"psi\", \"std_err\", \"ci_lower\", \"ci_upper\", \"truncated_fraction\"]\n", "print(curve[columns].round(4).to_string(index=False))" ] }, { "cell_type": "markdown", "id": "truncation-reading", "metadata": {}, "source": [ "**What this output tells you.** The fitted bound is 0.0114, from the default rule\n", "$5 / (\\sqrt{n} \\log n)$ at $n = 3000$. From 0.0010 to 0.0500 the truncated fraction is 0.0000.\n", "The default bound therefore changes no row, and the estimate stays at 0.1666. At 0.2000 the truncated\n", "fraction is 0.0417, and the estimate moves to 0.1674. Over this grid, the choice of bound moves the reported\n", "estimate by less than one standard error. A flat curve does not verify positivity." ] }, { "cell_type": "markdown", "id": "sensitivity-heading", "metadata": {}, "source": [ "## Step 10: sensitivity, what the fit cannot check\n", "\n", "Diagnostics cannot see an unmeasured confounder.\n", "[Sensitivity analysis](../user-guide/results-assessment.md#sensitivity-analysis) asks how strong a\n", "confounder would need to be to explain the result away.\n", "\n", "| term | plain meaning |\n", "| --- | --- |\n", "| `cf_y` | the share of the remaining outcome variation that a hidden confounder explains |\n", "| `cf_d` | the share of the Riesz representer's second moment that only the hidden confounder adds. For the ATE, the representer is the inverse-propensity weight |\n", "| `rho` | how closely the confounder's two effects align. `rho=1` is the worst case |\n", "| robustness value | the equal `cf_y` and `cf_d` that moves the point estimate to zero at `rho=1` |\n", "| benchmark | a refit without one recorded covariate. It measures what that covariate adds to the outcome regression and to $\\nu^2$ |\n", "\n", "A benchmark makes the strengths concrete: \"a confounder as strong as `discharge_risk`\". The code\n", "benchmarks the strongest recorded confounder, `discharge_risk`, and a weaker one,\n", "`medication_burden`. It then bounds the bias at each benchmark's strengths, with the default\n", "`rho=1`.\n", "\n", "`benchmark()` reports gain statistics. Chernozhukov et al. (arXiv:2112.13398v6, Appendix E.5,\n", "Remark 8) use these gains as proxies for the strengths that the bound reads. The call\n", "`omitted_confounding(benchmark=record)` therefore passes them to the bound unchanged.\n" ] }, { "cell_type": "code", "execution_count": 10, "id": "sensitivity", "metadata": { "execution": { "iopub.execute_input": "2026-10-16T01:16:39.732493Z", "iopub.status.busy": "2026-10-16T01:16:39.732282Z", "iopub.status.idle": "2026-10-16T01:16:43.324401Z", "shell.execute_reply": "2026-10-16T01:16:43.323933Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "robustness value: 0.445\n", "robustness value at the 95% limit: 0.425\n", "nu^2, doubly robust estimate: 4.418\n", "\n", "as strong as discharge_risk:\n", " estimate without it 0.2198\n", " gain in outcome variation 1.3360; cf_y 1.0000\n", " gain in nu^2 0.0282; cf_d 0.0282\n", " bias <= 0.0474 at rho = 1\n", " bias-adjusted bounds [0.1191, 0.2140]\n", " one-sided 95% limits [0.1099, 0.2230]\n", "\n", "as strong as medication_burden:\n", " estimate without it 0.1699\n", " gain in outcome variation 0.1122; cf_y 0.1122\n", " gain in nu^2 0.0587; cf_d 0.0587\n", " bias <= 0.0233 at rho = 1\n", " bias-adjusted bounds [0.1433, 0.1899]\n", " one-sided 95% limits [0.1342, 0.1989]\n" ] } ], "source": [ "sensitivity = result.sensitivity\n", "robustness = sensitivity.robustness_value()\n", "strong = sensitivity.benchmark(covariates=(\"discharge_risk\",))\n", "benchmark = sensitivity.benchmark(covariates=(\"medication_burden\",))\n", "strong_bounds = sensitivity.omitted_confounding(benchmark=strong)\n", "bounds = sensitivity.omitted_confounding(benchmark=benchmark)\n", "print(f\"robustness value: {robustness['rv']:.3f}\")\n", "print(f\"robustness value at the 95% limit: {robustness['rva']:.3f}\")\n", "print(f\"nu^2, doubly robust estimate: {sensitivity.elements(estimand='ate').nu2:.3f}\")\n", "for name, record, bound in (\n", " (\"discharge_risk\", strong, strong_bounds),\n", " (\"medication_burden\", benchmark, bounds),\n", "):\n", " print()\n", " print(f\"as strong as {name}:\")\n", " print(f\" estimate without it {record.psi_short:.4f}\")\n", " print(f\" gain in outcome variation {record.gain_y:.4f}; cf_y {record.cf_y:.4f}\")\n", " print(f\" gain in nu^2 {record.gain_d:.4f}; cf_d {record.cf_d:.4f}\")\n", " print(f\" bias <= {bound.bias:.4f} at rho = {bound.rho:g}\")\n", " print(f\" bias-adjusted bounds [{bound.lower:.4f}, {bound.upper:.4f}]\")\n", " print(f\" one-sided 95% limits [{bound.ci_lower:.4f}, {bound.ci_upper:.4f}]\")" ] }, { "cell_type": "markdown", "id": "sensitivity-reading", "metadata": {}, "source": [ "**What this output tells you.** The robustness value is 0.445. A confounder with `cf_y` and `cf_d`\n", "both equal to 0.445 would move the point estimate to zero at `rho=1`. The value at the 95% limit\n", "is 0.425.\n", "\n", "| benchmark | estimate without it | `cf_y` | `cf_d` | bias at most | bias-adjusted bounds | one-sided 95% limits |\n", "| --- | --- | --- | --- | --- | --- | --- |\n", "| `discharge_risk` | 0.2198 | 1.0000 | 0.0282 | 0.0474 | [0.1191, 0.2140] | [0.1099, 0.2230] |\n", "| `medication_burden` | 0.1699 | 0.1122 | 0.0587 | 0.0233 | [0.1433, 0.1899] | [0.1342, 0.1989] |\n", "\n", "Without `discharge_risk`, the residual outcome variance rises by a gain of 1.3360. A share cannot\n", "exceed 1, so the benchmark clips `cf_y` to 1.0000. This strength means that the hidden confounder\n", "explains all of the remaining outcome variation. The bound is finite there, because the treatment\n", "strength `cf_d` stays at 0.0282.\n", "\n", "At the strengths of either benchmark and the worst alignment, both one-sided limits exclude zero\n", "on this draw.\n", "\n", "| limitation | effect on the reading |\n", "| --- | --- |\n", "| the bound reads the fitted $\\nu^2 = 4.418$, and the law's value is about 4.83 | the doubly robust estimate of $\\nu^2$ has expectation equal to the law's value minus the squared error of the fitted representer. It reads low whenever the fitted propensity has an error, so the bounds are slightly narrow. With the law's value, the bias bound would be about 5% larger, and the robustness value would be 0.431 (review probe `summarize.py`) |\n", "| a benchmark is one draw | over the 200 probe draws, the one-sided limits at the `medication_burden` strengths excluded zero on every draw (`summary.log`) |\n", "| a benchmark is an analogy | it assumes that a hidden confounder adds to the recorded covariates what the dropped covariate adds to the others |\n", "\n", "The analysis supports one statement. On this draw, a hidden confounder as strong as\n", "`discharge_risk`, at the worst alignment, does not move either one-sided limit to zero. The bound\n", "does not cover a confounder that changes the offer more than `discharge_risk` does. The\n", "program must judge whether an unrecorded team judgment could be that strong. The\n", "[omitted-variable bounds](../technical-reference/validation-methods.md#omitted-variable-bounds-robustness-value-benchmark-and-contours)\n", "section defines each quantity.\n" ] }, { "cell_type": "markdown", "id": "method-evidence", "metadata": {}, "source": [ "## Method evidence\n", "\n", "The fits on this page use one draw of a synthetic law. The registered study in the table\n", "measures the method over repeated draws. The study page gives its law, its nuisance models, and\n", "the result of each cell.\n", "\n", "| fit on this page | registered study | law of the study |\n", "| --- | --- | --- |\n", "| Step 6, CV-TMLE | [Stacked point-treatment CV-TMLE](../technical-reference/method-evidence/stacked-point-treatment-cv-tmle.md) | this page's law |\n", "\n", "No registered study uses the boosted learners or the five folds of this page.\n", "\n", "A review probe is a script under `reviews/notebook-review/probes/`. The step readings quote its\n", "results. A review probe is not a registered study.\n", "\n", "The [technical entry](../technical-reference/cv-tmle.md#validation-issues-special-to-this-method) lists the evidence and the limits of the method." ] }, { "cell_type": "markdown", "id": "interpret", "metadata": {}, "source": [ "## Step 11: answer the question\n", "\n", "The sponsor asked how much the mean 30-day transition score would change if every eligible\n", "discharge received the offer. The table gives the answer and its limits.\n", "\n", "| item | the answer on this page |\n", "| --- | --- |\n", "| estimate | the offer to every eligible discharge would raise the mean score by 0.167 of the maximum score, against usual support |\n", "| uncertainty | the 95% interval is (0.156, 0.177). It reflects sampling variation only |\n", "| population and intervention | adults with a discharge-home order at a participating hospital. The intervention is the offer, analyzed regardless of completed contacts |\n", "| supporting evidence | the diagnostics of Step 8, the flat truncation curve of Step 9, and the bounds of Step 10. The registered study supports the cross-fitting choice, and the review probe supports this configuration on this law |\n", "| unverified assumptions | consistency, no interference, and no unmeasured confounding. Positivity is only partly checked |\n", "\n", "The analysis does not support these conclusions:\n", "\n", "- the effect of completing the navigator contacts, because the protocol defines the offer as the treatment;\n", "- the effect among the patients who received the offer, which is the ATT;\n", "- the same effect at other hospitals or for other eligibility rules;\n", "- robustness to a hidden confounder that changes the offer more than `discharge_risk` does;\n", "- a general performance claim for the estimator, from one synthetic draw." ] }, { "cell_type": "markdown", "id": "where-next", "metadata": {}, "source": [ "## Where to go next\n", "\n", "This page treated discharges as independent rows. Read\n", "[CV-TMLE and cross-fitting](cross-fitting.ipynb) for the same question at network scale, where\n", "patients share navigator teams. If your concern is which baseline variables belong in the\n", "assignment model, read [collaborative TMLE](collaborative-tmle.ipynb).\n", "\n", "To report the ATT or the ATC instead, read [estimands](../user-guide/estimands.md). To keep a\n", "fit and replay its assessment later, use `result.save()` and `cleverly.load()`.\n", "\n", "The [examples index](index.md#the-program) lists every tutorial in the program." ] } ], "metadata": { "cleverly_execution": { "command": [ "python", "scripts/execute_notebook.py", "docs/examples/point-treatment-tmle.ipynb" ], "gated": { "code_source_sha256": "8757e4ce1de9ebe9f7e1d528919be11ebfce878292b52e83131152e8bfd1a310", "execution_payload_sha256": "b94c5abc28bfdeeeeac5d2621f1e70b9512f982f5ab190865f7f478180273520" }, "recorded": { "cleverly_commit": "4cea0d1ad32abd00580cd9f3a27d84a8c294db5f", "cleverly_version": "0.1.3", "cleverly_worktree_clean": true, "dependency_lock_sha256": "2e4f7ad3264ff9b35cf7588e5c93a0e20fc5c0852092ac1da365491543836682", "generator_files": [ "scripts/execute_notebook.py", "tests/notebooks.py" ], "generator_sha256": "f247771ab390b23e7e092d7eb35794e241c2ac84fc0526695df1cee1fc457a83", "package_source_sha256": "5fce085883c189351f5a0d142b216264208c189d01a46bb372a5e8fc68eed390" }, "schema_version": 5 }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.16" } }, "nbformat": 4, "nbformat_minor": 5 }