Sequential pricing: a three-month renewal price policy#
A price changes later engagement and the prices that the pricing engine sets next, and engagement steers those prices. This notebook estimates the effect of a price policy on month-3 renewal and revenue with longitudinal TMLE. The policy modifies the prices that the engine would set, and it stays inside the price band that the data support. Each step shows its code, its output, and what the output tells you. Modified treatment policies at a node gives the estimator.
The situation#
A subscription retailer sells a monthly plan on a three-month term. A subscriber who starts a term commits to three monthly payments, and no one can leave before the term ends. A pricing engine sets the price of each month.
Before each price, the engine scores whether the subscriber is engaged that month. The engine reads the usage, the engagement, and the gap between its last price and the last month’s centre. It draws a price inside a band whose limits rise with engagement. After the third price, the subscriber renews or lapses.
This page uses a synthetic law. The examples index states how it differs from the offers law of the retailer program.
element |
on this page |
|---|---|
data |
a wide frame of 10000 synthetic subscribers, one row each, with no missing value |
time order |
|
treatment |
the renewal price of each month, inside the band \([4 + 2e, 8 + 2e]\) for the month’s engagement \(e\). The centre of the band is \(6 + 2e\) |
outcome |
|
observed |
the usage, three engagement scores, the three prices that the engine set, and the renewal |
counterfactual |
the renewal and the revenue under a price policy that differs from the prices the engine set. The frame never shows them |
The causal graph#
The graph draws one month. The pattern of month \(t\) repeats for \(t = 1, 2, 3\).
node |
meaning |
|---|---|
\(W\) |
the usage score |
\(L_t\) |
the engagement score of month \(t\), recorded before the price |
\(A_t\) |
the renewal price of month \(t\) |
\(Y\) |
the renewal at the month-3 price |
\(R\) |
the month-3 revenue, \(Y A_3\) |
\(U\) |
an unrecorded cause |
W (usage) --> L_t, A_t, and Y for t = 1, 2, 3
One month t, with L_0 and A_0 at their starting values:
L_t --> A_t the band and its centre move with engagement
L_{t-1} --> A_t the engine reads the last gap from the last centre
A_{t-1} --> A_t the engine carries part of the last price gap
A_{t-1} --> L_t a price above the centre lowers the next engagement
L_{t-1} --> L_t engagement persists
L_t --> Y for t = 1, 2, 3
A_t --> Y for t = 1, 2, 3
Y --> R R = Y A_3
A_3 --> R R = Y A_3: the price a renewer pays
U (unrecorded cause) - - -> A_t and Y (assumed absent)
arrow or absent arrow |
what it asserts |
role |
where the law sets it |
|---|---|---|---|
|
usage raises engagement ( |
baseline confounder |
|
|
the centre and both band limits rise by |
confounder of the price. It also sets the support |
|
|
the engine adds |
engine persistence. A policy must carry it |
|
|
a price above the centre lowers engagement ( |
the treatment changes a later confounder |
|
|
the month-3 gap lowers the renewal ( |
the effect of the price |
|
|
month-3 engagement raises the renewal by |
a path from the confounder to the outcome |
|
|
revenue is the renewal times the month-3 price |
the outcome definition |
the generator, |
no |
the engine logs every input it reads, so no unrecorded cause moves both the price and the renewal |
encoded, and open. This is strong sequential randomization, and the engine log is the argument for it. It is scientific judgment |
none, an assumption |
no arrow between customers |
one customer’s price does not change another customer’s renewal |
encoded. This is no interference, and a field of the protocol |
none, an assumption |
Reading the graph. Two paths run from a price to the renewal through a later variable. One runs through the next engagement: \(A_t \to L_{t+1} \to Y\). The other runs through the engine’s next price: \(A_t \to A_{t+1} \to Y\). Engagement \(L_{t+1}\) also causes \(A_{t+1}\).
One outcome regression that holds the later engagement and the later prices at their recorded values blocks both paths. A regression that omits engagement leaves every price confounded. Step 8 shows both failures.
What you will learn#
after this notebook you can |
the step that shows it |
|---|---|
state a price policy as a modified treatment policy that stays inside the band |
the next section and Step 5 |
read from the graph why one regression fails and the sequential g-formula is needed |
the graph and Step 8 |
read the label |
Steps 6 and 7 and the method evidence |
read the support of a larger price move, and say why the effect saturates |
Steps 9 and 10 |
The question, the estimand, and the method#
The pricing lead asks one question. Suppose the firm had raised each subscriber’s monthly renewal price by 12% for three months. The raise applies wherever the raised price stays inside that month’s price band, and the engine’s price stays elsewhere. What share would renew in month 3? What would month-3 revenue be?
Both answers compare with the prices that the engine set. A second policy cuts each price by 15% wherever the cut price stays inside the band, and otherwise keeps the engine’s price.
part of the question |
on this page |
|---|---|
target population |
active monthly subscribers who start a three-month term at time zero, as the protocol of Step 4 states |
intervention |
policy \(d\), one map of the engine’s price for each of months 1 to 3. |
counterfactual outcome |
\(Y^{d}\) and \(R^{d}\), the renewal and the revenue if each month’s price had been the policy’s price, and the later engagement and the engine’s later prices had responded |
estimand |
\(\psi_d = E[Y^{d}]\), the renewal share under \(d\), and \(E[R^{d}]\), the mean revenue |
contrasts |
\(\psi_d - \psi_{\mathrm{natural}}\) for |
The policy keeps the engine’s price where the raised price would leave the band. A pure 12% scale assigns some subscribers a price above the ceiling. The fit refuses that plan before any learner, because that price lies outside the declared band.
A rule that raised the price and capped it at the ceiling would put a point mass at the ceiling. Its map is not invertible there, and the theory needs an invertible map (Díaz, Williams, Hoffman and Schenck 2023, Assumption 4). The plans here are piecewise smooth and invertible. The held share is the share of subscribers whose price a plan leaves unchanged.
A fixed price for every subscriber answers a different question. The fit refuses a static price on a continuous node, because no regular estimator of its mean exists. An elasticity regression of the renewal on the price does not answer the question either. Step 8 shows why.
assumption |
what it means here |
checkable from data, or scientific judgment |
|---|---|---|
strong sequential randomization |
at every month, each common cause of the price, the later engagement and prices, and the renewal is in the recorded history |
scientific judgment. The argument is the engine log, which records every input the engine reads. No data test exists |
positivity |
the price that a plan assigns lies in the support of the engine’s price given the history |
the declared band holds by design. The density ratios of Step 9 show the empirical part |
consistency |
a price is the quoted renewal price, unrounded, and a plan quotes the new price for one month |
scientific judgment |
no interference |
one subscriber’s price does not change another subscriber’s renewal |
scientific judgment. One account belongs to one subscriber |
Each method feature answers one feature of the problem.
feature of the problem |
feature of the method |
|---|---|
an earlier price changes engagement and the engine’s next price, and engagement confounds the next price |
sequential regression, which evaluates the g-formula with one regression for each month |
the price is continuous, and the policy moves it |
a modified treatment policy at a continuous node. The weight is a density ratio of the plan’s price law to the engine’s price law. |
the engine’s price law is unknown |
|
the nuisance models are main-terms or degree-2 parametric models |
no cross-fitting. The registered cells for this configuration fit in sample |
revenue is continuous |
|
the question compares policies with the engine’s prices |
|
Four estimators could answer the question. The table states what each one reads and when to use it. Modified treatment policies at a node and the interval under heavy cumulative density ratios give the conditions.
estimator |
what it reads |
when to use it |
|---|---|---|
LTMLE |
the outcome regressions and the density ratios, with a targeting step |
the default of this page. It is consistent when every outcome regression is correct, or when every density ratio is correct |
SDR |
the same, with no targeting step |
when the ratio or the outcome regression could be wrong at different months, because it needs only one of the two at each month. Its interval runs short of nominal under heavy cumulative density ratios, which the technical reference measures on another law |
g-computation |
the outcome regressions only |
when every outcome regression is correct. It adds no protection against a wrong regression |
IPW |
the density ratios only |
never for this plan. |
The table defines the terms this page uses most.
term |
plain meaning |
|---|---|
node |
one decision time. Here, the price of month 1, 2, or 3 |
history |
every column recorded before the price at a node |
natural price |
the price the engine sets. The code calls the engine’s own prices |
a rule that maps the price the engine would set to a new price. It reads the history |
|
price band and support |
the interval of prices the engine can set given the history, here \([4 + 2e, 8 + 2e]\). The analyst declares it |
held share |
the share of subscribers whose price a plan leaves unchanged because the moved price would leave the band |
density ratio |
the density of the plan’s price divided by the density of the engine’s price, at the engine’s price. It weights each subscriber |
effective n |
the sample size that equal weights would give with the same variance. It falls as the weights grow uneven |
a model the estimate needs but the question does not ask about. Here, the outcome, pseudo-outcome, and density-ratio models |
|
a small update to each node’s regression, weighted by the density ratios, that removes first-order bias |
|
how much each subscriber moves the estimate. Its variance gives the standard error |
Step 1: set up#
The setup imports the libraries and prints the installed cleverly version. Every fit below passes
its learners, its cross-fitting setting, and its random seed explicitly, so a rerun reproduces the
stored outputs.
import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
import cleverly
pd.set_option("display.width", 130)
pd.set_option("display.max_columns", 20)
print("cleverly", cleverly.__version__)
cleverly 0.1.4
What this output tells you. The stored outputs in this notebook came from the version named above. A different version can print different numbers.
Step 2: the data#
The generator make_sequential_pricing returns a wide frame with one row per subscriber. The code
prints the shape and the column names. It counts missing values, and it prints the true values of
the synthetic law.
The seed follows a fixed rule. The page uses 1000, the first seed of the range 1000 to 1199 that the review probe of Step 6 sweeps. The rule was set before the page ran with this seed. No draw was chosen for its results, so an interval on this page can miss its true value.
from cleverly.datasets import make_sequential_pricing
frame, truth = make_sequential_pricing(10_000, seed=1000)
print("rows and columns:", frame.shape)
print(list(frame.columns))
print(frame.head(6).round(3))
print()
print("missing values:", int(frame.isna().sum().sum()))
print("known values of the synthetic law:")
for plan in ("natural_prices", "price_up", "price_down"):
renewal = truth[f"ey_regimen[{plan}]"]
revenue = truth[f"ey_revenue[{plan}]"]
print(f" {plan:15s} mean renewal {renewal:.4f} mean revenue {revenue:.4f}")
for plan in ("price_up", "price_down"):
renewal = truth[f"ate_regimen[{plan} vs natural_prices]"]
revenue = truth[f"ate_revenue[{plan} vs natural_prices]"]
print(f" {plan} against natural_prices: renewal {renewal:+.4f} revenue {revenue:+.4f}")
rows and columns: (10000, 9)
['usage', 'engaged_1', 'price_1', 'engaged_2', 'price_2', 'engaged_3', 'price_3', 'renewed', 'revenue']
usage engaged_1 price_1 engaged_2 price_2 engaged_3 price_3 renewed revenue
0 0.053 0.0 6.586 1.0 9.119 1.0 7.696 1.0 7.696
1 0.260 0.0 4.130 0.0 6.248 1.0 6.872 1.0 6.872
2 -0.072 1.0 8.736 0.0 6.877 1.0 7.356 1.0 7.356
3 -0.817 1.0 7.768 1.0 8.664 1.0 8.672 1.0 8.672
4 0.071 1.0 7.596 0.0 4.978 0.0 7.147 0.0 0.000
5 -0.860 0.0 6.818 0.0 4.364 1.0 6.299 0.0 0.000
missing values: 0
known values of the synthetic law:
natural_prices mean renewal 0.7301 mean revenue 5.3765
price_up mean renewal 0.6298 mean revenue 5.0643
price_down mean renewal 0.8450 mean revenue 5.5019
price_up against natural_prices: renewal -0.1003 revenue -0.3122
price_down against natural_prices: renewal +0.1149 revenue +0.1254
What this output tells you. The frame has 10000 rows and 9 columns. It has no missing value, because these data have no censoring.
column |
role |
|---|---|
|
a baseline covariate, a standard-normal draw cut off at |
|
the engagement score of that month, recorded before the price |
|
the renewal price of that month. The generator does not round it |
|
1 if the subscriber renews at the month-3 price |
|
|
The true values come from pricing_truth(), which integrates the law by quadrature and does not
simulate it.
Under the engine’s own prices, the renewal share is 0.7301 and the mean revenue is
5.3765. Under price_up they are 0.6298 and 5.0643. Under price_down they are 0.8450 and 5.5019.
The true renewal contrast is -0.1003 for price_up and 0.1149 for price_down. The true revenue
contrast is -0.3122 for price_up and 0.1254 for price_down.
A real program has no truth. Every comparison against it below is a teaching device.
Step 3: association first#
A confounder is a variable that changes both the next price and the outcome. Engagement is one. It moves the band and the centre of the price, and it raises the renewal. A price also changes the next engagement. The code reads three arrows of the graph in the data. These tables show association only.
gap_1 = frame["price_1"] - (6 + 2 * frame["engaged_1"])
print("arrow A_1 -> L_2, share engaged in month 2, by the month-1 price against its centre:")
print(
frame.groupby(np.where(gap_1 > 0, "above the centre", "at or below"))["engaged_2"]
.mean()
.round(3)
.to_string()
)
print()
print("arrow L_t -> A_t, mean price by that month's engagement:")
by_engagement = {
f"month {t}": frame.groupby(f"engaged_{t}")[f"price_{t}"].mean() for t in (1, 2, 3)
}
print(pd.DataFrame(by_engagement).round(3).to_string())
print()
print("arrow L_3 -> Y, share renewed, by the month-3 engagement:")
print(frame.groupby("engaged_3")["renewed"].mean().round(3).to_string())
arrow A_1 -> L_2, share engaged in month 2, by the month-1 price against its centre:
above the centre 0.619
at or below 0.696
arrow L_t -> A_t, mean price by that month's engagement:
month 1 month 2 month 3
0.0 5.946 5.973 5.948
1.0 8.033 8.006 7.980
arrow L_3 -> Y, share renewed, by the month-3 engagement:
engaged_3
0.0 0.598
1.0 0.809
What this output tells you. Each table agrees with one arrow of the graph.
arrow |
what the table shows |
|---|---|
|
after a month-1 price above its centre, a share of 0.619 of the subscribers are engaged in month 2. After a price at or below the centre the share is 0.696 |
|
the mean month-1 price is 5.946 for subscribers who are not engaged and 8.033 for engaged subscribers. Months 2 and 3 show the same gap of about two |
|
the renewal share is 0.598 for subscribers who are not engaged in month 3 and 0.809 for engaged subscribers |
The graph makes engagement a confounder of every later price, and it shows that an earlier price changes engagement. One outcome regression cannot condition on engagement without blocking part of the earlier price effect. It cannot leave engagement out without confounding the later price.
Step 4: write the protocol#
A StudyProtocol records the scientific design before any model runs.
Point-treatment TMLE introduces its fields
and its fingerprint. The helper pricing_protocol() returns the record for this program. It names
the three strategies of the question, and it states that the data have no censoring.
from cleverly.datasets import pricing_protocol
protocol = pricing_protocol()
print("\n".join(protocol.summary_lines()))
causal study protocol: schema 1; 682ab02814f9837b
target population: Active monthly subscribers at the start of a three-month term
eligibility: ['Active subscription at the first monthly renewal price', 'A renewal price set by the pricing engine in each of months 1 to 3', 'One account per customer, so shared-household accounts are excluded', 'A three-month term that starts at time zero, with no exit before the month-3 renewal']
time zero: The first monthly renewal price, after the usage score and the month-1 engagement score and before the engine sets the price
treatment strategies: ["The pricing engine's own price in months 1 to 3", "Raise each monthly price by a declared factor where the raised price stays inside that month's price band, otherwise keep the engine's price. The question uses a 12% rise, and the support step also reads 6% and 25%", "Cut each monthly price by 15% where the cut price stays inside that month's price band, otherwise keep the engine's price"]
treatment versions: ['The renewal price the engine quotes, not rounded', "The declared factor on the engine's price, quoted for one month", "The 15% cut on the engine's price, quoted for one month"]
outcome: Renewal at the month-3 price, and month-3 revenue: the renewal times the month-3 price
horizon: Month 3, at the renewal after the third monthly price
intercurrent-event handling: ['None: the three-month term has no exit before the month-3 renewal', 'Every customer in the data has all three prices and the renewal, so the data have no censoring']
interference unit: Individual customer
assumption rationale: ['The pricing engine logs every input it reads, so usage and the monthly engagement score cover the common causes of the price, the later engagement, and the renewal. This is sequential randomization given the recorded history', "A price band that moves with engagement is declared for each month, so the policy keeps every price inside the support of the engine's price", 'A declared factor on the quoted price, for one month, supports consistency', 'One account per customer and no reference-price spillover support no interference']
What this output tells you. The first line gives the schema version and the fingerprint
682ab02814f9837b. The other lines repeat each field. Read them as a checklist.
protocol field |
the question it answers for this program |
|---|---|
target population and eligibility |
who the effect is about. Eligibility is a three-month term that starts at time zero, with no exit before the month-3 renewal. Time zero fixes the population, so no later event defines it |
time zero |
the first monthly renewal price, after the usage score and the month-1 engagement score and before the engine sets the price |
treatment strategies and versions |
the engine’s own prices, a price rise by a declared factor, and a 15% cut. The question uses a 12% rise, and Step 9 also reads 6% and 25%. Each plan quotes the new price for one month |
outcome and horizon |
the renewal at the month-3 price, and the month-3 revenue |
intercurrent-event handling |
none. The term has no exit before the month-3 renewal, so every subscriber has all three prices and the renewal. The data have no censoring |
interference unit |
the individual customer |
assumption rationale |
the engine log supports strong sequential randomization given the usage and the monthly engagement. The declared band supports positivity. One account per customer supports no interference |
Step 5: design and identification#
LongitudinalTreatment reads the wide frame. The argument treatment lists the price columns in
time order. The argument time_varying lists the columns recorded before each price, one list for
each month. The argument baseline lists usage. The argument continuous_treatment marks each
price as a continuous node and declares its support. PRICE_BANDS maps each price column to the
band \([4 + 2e, 8 + 2e]\) for the engagement \(e\) of that month.
A plan is a DynamicRegimen with one policy for each month. A policy is a ModifiedPolicy of
pieces. On each interval of the price, a piece applies one increasing map with a known inverse. The
declaration policy_kind="known" says that the policy is a fixed function, chosen before the fit.
The dictionary PRICE_PLANS holds the three plans. The typed RegimeContrast compares each plan
with natural_prices.
from cleverly import CausalStudy, LongitudinalTreatment, RegimeContrast
from cleverly.datasets import PRICE_BANDS, PRICE_PLANS
def pricing_design(outcome):
return LongitudinalTreatment(
outcome=outcome,
treatment=["price_1", "price_2", "price_3"],
baseline=["usage"],
time_varying=[["engaged_1"], ["engaged_2"], ["engaged_3"]],
continuous_treatment=PRICE_BANDS,
)
study = CausalStudy(frame, design=pricing_design("renewed"), protocol=protocol)
for month in (1, 2, 3):
print(f"history_names({month}):", study.data.history_names(month))
print()
print("plans:", list(PRICE_PLANS))
effect = study.identify(RegimeContrast(PRICE_PLANS, reference="natural_prices"))
print(effect.summary(protocol="fingerprint"))
history_names(1): ('usage', 'engaged_1')
history_names(2): ('usage', 'engaged_1', 'engaged_2')
history_names(3): ('usage', 'engaged_1', 'engaged_2', 'engaged_3')
plans: ['natural_prices', 'price_up', 'price_down']
contrast of each regime against the reference
identified by explicit-adjustment: sequential g-formula under the declared treatment regimen
history at price_1: ['usage', 'engaged_1']
history at price_2: ['usage', 'engaged_1', 'engaged_2', 'price_1']
history at price_3: ['usage', 'engaged_1', 'engaged_2', 'engaged_3', 'price_1', 'price_2']
required nuisances: ['sequential_outcome_regressions', 'treatment_and_censoring_mechanisms']
assumptions:
- consistency: each observed history equals the potential history under its realized regimen
- no interference: one unit's potential history does not depend on other units' regimens
- sequential exchangeability in the form the chosen interpretation names: strong sequential randomization (Díaz et al. 2023, Assumption 3) for the modified treatment policy effect, or standard sequential randomization (Assumption 2) for the LMTP-SI of their Theorem 1 (i)
- positivity: the dose a policy assigns, d(a_t, h_t), lies in the support of A_t given h_t wherever a_t does (Díaz et al. 2023, Assumption 1, which Williams & Díaz 2025 leave unchanged), and so does remaining under observation
- each policy is known and fixed before the fit, or its cap is a declared tuner with a declared target (X40 part (e)), and on a continuous node it is piecewise smooth invertible (Assumption 4); a policy reads the history and the unit's own treatment at its node, and its earlier treatment columns stand for the intervened values
interpretations, each with the assumption it adds:
- modified treatment policy effect: the mean outcome had each node's natural dose a_t been replaced by d_t(a_t, h_t), with the history the earlier replacements produce
needs strong sequential randomization: at every node, every common cause of the dose, the outcome, and the later covariates and doses is in the recorded history (Díaz et al. 2023, Assumption 3; the strong exchangeability of Williams & Díaz 2025, A_t independent of Y(a_t, d_{t+1}(A_{t+1}(a_t), H_{t+1}(a_t))) given H_t)
source: Díaz, Williams, Hoffman & Schenck (2023), Theorem 1 (ii) and Assumption 3
source: Williams & Díaz (2025)
- stochastic intervention (LMTP-SI): the mean outcome had each node's dose been drawn from the law of d_t(A_t, H_t) given the history
needs standard sequential randomization: at every node, every common cause of the dose and the next covariates, which at the last node are the outcome, is in the recorded history (Díaz et al. 2023, Assumption 2)
source: Díaz, Williams, Hoffman & Schenck (2023), Theorem 1 (i) and Assumption 2
causal study protocol: schema 1; 682ab02814f9837b
What this output tells you. The three history_names lines list the columns that the design
puts in the history of each month. Month 1 holds usage and engaged_1, the confounder of the
first price. Each later month adds that month’s engagement. The earlier prices join the history
through the policy, and the history at lines of the summary show them.
The history at price_3 line lists engaged_1, engaged_2, engaged_3, price_1, and price_2,
with usage. The summary names the estimand and its identification, the sequential g-formula under
the declared treatment regimen. The last line prints the fingerprint from Step 4.
The summary lists two interpretations of the policy. The first is the modified treatment policy effect: the mean outcome had each month’s natural price been replaced by the policy’s price, with the history that the earlier replacements produce. The question of this page asks for it. It needs strong sequential randomization. The second is a stochastic intervention that draws the price from the law of the policy’s price. It needs only standard sequential randomization, and it answers a different question.
The printed assumptions are those of the table in the plan. Strong sequential randomization is scientific judgment, and Step 11 says what could break it. Step 9 reads the empirical part of positivity.
Step 6: estimate the renewal contrasts with LTMLE#
The configuration is written out in full.
setting |
value |
what it does |
|---|---|---|
|
logistic regression |
fits the regression of the binary renewal at the last node |
|
linear regression |
fits the earlier regressions. Their targets are continuous, and the recursion clips their predictions to the unit interval |
|
degree-2 logistic classifier |
estimates each density ratio by classification. It takes the degree-2 terms of the history and the price, scales them, and fits a logistic regression with a large penalty constant and a tight tolerance. An unconverged fit leaves the largest ratios dependent on the platform. It has no indicator for the pieces of a plan |
|
|
the route that the registered study of this law uses. A learned density is the other route |
|
in-sample fit |
the models have few parameters, and the registered cells for this configuration fit in sample |
|
the default |
bounds the censoring and categorical treatment factors. This fit has neither, so it bounds nothing. Step 9 reads the weights directly |
|
fixed seed, one process |
makes the fit reproducible |
The code then compares each contrast with its true value. It prints the support label of each
estimate and the reason that the label gives.
from cleverly import CrossFitting, ModelSpec, Runtime, TMLEMethod
logistic = LogisticRegression(max_iter=1000, random_state=42)
ratio_model = make_pipeline(
PolynomialFeatures(2, include_bias=False),
StandardScaler(),
LogisticRegression(C=1e6, tol=1e-8, max_iter=100_000, random_state=42),
)
ltmle = TMLEMethod(
models=ModelSpec(
outcome_learner=logistic,
pseudo_learner=LinearRegression(),
treatment_learner=ratio_model,
ratio="classifier",
),
cross_fitting=CrossFitting(enabled=False),
runtime=Runtime(random_state=1000, n_jobs=1),
)
result = effect.estimate(method=ltmle)
print(result.summary(protocol="fingerprint"))
def against_truth(fit, truth_kind):
rows = []
for key, estimate in fit.estimates.items():
plan = key.removeprefix("ate_regimen[").removesuffix(" vs natural_prices]")
exact = truth[f"{truth_kind}[{plan} vs natural_prices]"]
low, high = estimate.ci
rows.append(
{
"contrast": plan,
"estimate": estimate.psi,
"std_error": estimate.std_error,
"ci_low": low,
"ci_high": high,
"truth": exact,
"covers": bool(low <= exact <= high),
"support": estimate.support.label,
}
)
return pd.DataFrame(rows).set_index("contrast")
print()
print(against_truth(result, "ate_regimen").round(4).to_string())
print()
for key, estimate in result.estimates.items():
print(f"{key}: {estimate.support.label}. {estimate.support.reason}")
Longitudinal TMLE (3 time points, n = 10000)
parameter estimate std. error 95% CI p-value
----------------------------------------- -------- ---------- ------------------ -------
ate_regimen[price_up vs natural_prices] -0.1025 0.0065 [-0.1152, -0.0899] <1e-4
ate_regimen[price_down vs natural_prices] 0.1050 0.0078 [0.0896, 0.1203] <1e-4
time points: 3
outcome family: binomial
regimens: natural_prices=(m:natural course/m:natural course/m:natural course), price_up=(m:price_up at 1/m:price_up at 2/m:price_up at 3), price_down=(m:price_down at 1/m:price_down at 2/m:price_down at 3)
a 'd' is a rule d_t(H_t) read off [W, L_1, ..., L_t]; its followers are a covariate-dependent set, so the counts below describe this sample
assigned arms, natural_prices: c51a991afd938ee0
assigned arms, price_up: ed3314cb33273f4d
assigned arms, price_down: 517e7cb67a19cdff
a 'q' is a known policy q_t(a | H_t) read off [W, L_1, ..., L_t] and the earlier treatments; each unit draws its arm there, and the plan's units are those whose observed arm the policy can draw
reference: natural_prices
cross-fitting: none -- nuisances fitted in sample
g_bounds: fixed [0.01, 1] on each cumulative treatment-and-censoring probability (package default; R ltmle-compatible heuristic -- inspect truncation diagnostics)
confidence level: 95%
random_state: 1000
causal estimand: contrast of each regime against the reference
identification: explicit-adjustment; sequential g-formula under the declared treatment regimen
required nuisances: sequential_outcome_regressions, treatment_and_censoring_mechanisms
identification assumptions: consistency: each observed history equals the potential history under its realized regimen; no interference: one unit's potential history does not depend on other units' regimens; sequential exchangeability in the form the chosen interpretation names: strong sequential randomization (Díaz et al. 2023, Assumption 3) for the modified treatment policy effect, or standard sequential randomization (Assumption 2) for the LMTP-SI of their Theorem 1 (i); positivity: the dose a policy assigns, d(a_t, h_t), lies in the support of A_t given h_t wherever a_t does (Díaz et al. 2023, Assumption 1, which Williams & Díaz 2025 leave unchanged), and so does remaining under observation; each policy is known and fixed before the fit, or its cap is a declared tuner with a declared target (X40 part (e)), and on a continuous node it is piecewise smooth invertible (Assumption 4); a policy reads the history and the unit's own treatment at its node, and its earlier treatment columns stand for the intervened values
interpretation modified treatment policy effect: the mean outcome had each node's natural dose a_t been replaced by d_t(a_t, h_t), with the history the earlier replacements produce; needs strong sequential randomization: at every node, every common cause of the dose, the outcome, and the later covariates and doses is in the recorded history (Díaz et al. 2023, Assumption 3; the strong exchangeability of Williams & Díaz 2025, A_t independent of Y(a_t, d_{t+1}(A_{t+1}(a_t), H_{t+1}(a_t))) given H_t)
interpretation stochastic intervention (LMTP-SI): the mean outcome had each node's dose been drawn from the law of d_t(A_t, H_t) given the history; needs standard sequential randomization: at every node, every common cause of the dose and the next covariates, which at the last node are the outcome, is in the recorded history (Díaz et al. 2023, Assumption 2)
causal study protocol: schema 1; 682ab02814f9837b
data 2355e4f63d35f973 | folds 23ddc51b088c5ff9 | cleverly 0.1.4 | 2026-10-16T19:15:51+00:00
natural_prices: 10000 of 10000 units followed it throughout; max weight 1.0, effective n 10000; max truncated share 0.0% at t=1
price_up: 10000 of 10000 units followed it throughout; max weight 13.7, effective n 3509; max truncated share 0.0% at t=1
price_down: 10000 of 10000 units followed it throughout; max weight 31.2, effective n 1554; max truncated share 0.0% at t=1
simultaneous 95% bands (multiplier bootstrap, critical value 2.221 vs 1.960 pointwise) [experimental]:
ate_regimen[price_up vs natural_prices] [-0.1169, -0.0882]
ate_regimen[price_down vs natural_prices] [0.0876, 0.1223]
estimate std_error ci_low ci_high truth covers support
contrast
price_up -0.1025 0.0065 -0.1152 -0.0899 -0.1003 True supported
price_down 0.1050 0.0078 0.0896 0.1203 0.1149 True supported
ate_regimen[price_up vs natural_prices]: supported. Row G39.ltmle.tau3 covers this configuration.
ate_regimen[price_down vs natural_prices]: supported. Row G39.ltmle.tau3 covers this configuration.
What this output tells you. The table reports two contrasts against natural_prices. The
price_up contrast is -0.1025, with a standard error of 0.0065 and a 95% interval of
(-0.1152, -0.0899). A 12% rise lowers the month-3 renewal share by about ten points. The interval
contains the true contrast of -0.1003.
The price_down contrast is 0.1050, with an interval of (0.0896, 0.1203). It contains the true
0.1149. Both intervals contain their true values on this draw. That is one draw, not a coverage
result.
The review probe reviews/notebook-review/probes/sequential-pricing/sweep.py repeats the fits of
this page on 200 draws, seeds 1000 to 1199. The two intervals covered on 186 and 187 of 200 draws. The Monte Carlo standard error of a count
near the nominal 190 is about 3 draws, so both counts lie about one standard error below it. The
bias of the estimates was -0.0002 and 0.0004, each with a Monte Carlo standard error of 0.0005. The
sweep raised SparseInfluenceWarning on 36 of 200 renewal fits. This draw raised none.
The support label of each estimate is supported, and the reason names the registered row.
The label means that a registered row covers this configuration: the engine, the number of nodes, the
splitting, the outcome scale, and the ratio route. It does not mean that a study measured this
learner, this plan, or these factors. The support
label makes no coverage claim. The
method-evidence section lists the gaps. The summary also prints simultaneous bands. They carry the
label experimental, and this page does not use them.
Step 7: estimate the revenue contrasts#
Revenue is renewed times the month-3 price. It is a continuous outcome with the declared support
\([0, 10]\). A fit of a modified treatment policy refuses a continuous outcome without q_bounds, so
the code declares Targeting(q_bounds=(0.0, 10.0)). The fit never clips an outcome. See the
limits of the continuous-policy
evidence.
The outcome learner is a linear regression. It is cheap, and it does not match the law, because revenue is zero for every subscriber who lapses and the price for every subscriber who renews. Everything else stays as in Step 6.
from cleverly import Targeting
revenue_method = TMLEMethod(
models=ModelSpec(
outcome_learner=LinearRegression(),
pseudo_learner=LinearRegression(),
treatment_learner=ratio_model,
ratio="classifier",
),
targeting=Targeting(q_bounds=(0.0, 10.0)),
cross_fitting=CrossFitting(enabled=False),
runtime=Runtime(random_state=1000, n_jobs=1),
)
revenue_study = CausalStudy(frame, design=pricing_design("revenue"), protocol=protocol)
revenue_result = revenue_study.identify(
RegimeContrast(PRICE_PLANS, reference="natural_prices")
).estimate(method=revenue_method)
print(against_truth(revenue_result, "ate_revenue").round(4).to_string())
print()
for key, estimate in revenue_result.estimates.items():
print(f"{key}: {estimate.support.label}. {estimate.support.reason}")
estimate std_error ci_low ci_high truth covers support
contrast
price_up -0.3129 0.0559 -0.4224 -0.2035 -0.3122 True supported
price_down 0.0743 0.0536 -0.0307 0.1792 0.1254 True supported
ate_regimen[price_up vs natural_prices]: supported. Row G39.ltmle.tau3_revenue covers this configuration.
ate_regimen[price_down vs natural_prices]: supported. Row G39.ltmle.tau3_revenue covers this configuration.
What this output tells you. The price_up revenue contrast is -0.3129, with an interval of
(-0.4224, -0.2035). The true contrast is -0.3122. A 12% rise loses revenue, because the subscribers
who lapse outweigh the higher price of those who stay.
The price_down revenue contrast is 0.0743, with an interval of (-0.0307, 0.1792). The true contrast
is 0.1254. The interval contains zero on this draw, so this fit does not tell whether the cut raises
revenue. In the review probe, the interval of the cut contained zero on 60 of 200 draws.
The linear outcome regression is wrong for revenue, and the probe shows a small bias. Over 200 draws
the estimates of the price_up and price_down contrasts had biases of 0.0099 and 0.0094, each
about two and a half Monte Carlo standard errors (0.0039 and 0.0038). The standard deviations of the estimates
are 0.0557 and 0.0537, so each bias is about a sixth of them.
The intervals covered on 184 and 191 of 200 draws. The count 184 lies between one and two standard
errors below the nominal 190, and 191 lies above it. The sweep raised SparseInfluenceWarning on 46
of 200 revenue fits. This draw raised none.
The probe revenue_learner.py refits 60 draws with a quadratic outcome learner. Both learners fit
the same draws, so their estimates are paired. For the price_up contrast the bias was 0.0028 with
the quadratic learner and 0.0132 with the linear learner, with Monte Carlo standard errors of
0.0075 and 0.0074. The standard error of a bias does not bound the standard error of a paired difference.
The quadratic learner moved the price_up estimate by 0.0104 on average, with a paired Monte Carlo
standard error of 0.0005. That is about the size of the bias over 200 draws. For price_down the
move was 0.0076, with a paired Monte Carlo standard error of 0.0010. The linear outcome learner
carries most of the price_up bias and about half of the price_down bias. With the quadratic
learner, 60 draws cannot tell the price_up bias from zero.
Both revenue fits are supported, and the reason names a registered row for this configuration. The
label does not cover the linear outcome learner or a revenue contrast. The method-evidence section
says so.
Step 8: the failure mode, one regression for a sequential question#
The question asks for a policy effect. The natural tool is one logistic regression of renewed on the usage, the three engagement scores, and the three prices,
with the prices replaced by the plan’s prices. This regression is correctly specified: the true
logit is linear in exactly these columns. It still answers a different question.
for the comparison |
on this page |
|---|---|
estimands |
LTMLE targets the policy effect: the mean renewal when the later engagement and the engine’s later prices respond to the plan’s prices. The regression with engagement targets the mean renewal under the plan’s prices, with engagement and the engine’s next natural prices held at the values they took under the engine’s own prices. That is a natural-direct-effect type of quantity. It is a different estimand. The regression without engagement targets an association |
assumptions added |
the regression adds that its model is correct, and it adds no sequential argument. A correct model does not repair the estimand |
mechanism |
a price changes the next engagement (\(A_t \to L_{t+1} \to Y\)) and the engine’s next price (\(A_t \to A_{t+1} \to Y\)). The regression holds both at their recorded values, which blocks both paths. Without engagement, \(L_t\) raises both the price and the renewal, so a high price looks harmless. Hernán and Robins (Chapter 20) call the first case treatment-confounder feedback |
evidence |
|
reading |
one draw shows no general property. The review probe repeats the comparison |
The two regressions are scikit-learn code and not cleverly fits, so the code runs them inside
with thread_limit():. The function planned_prices writes each plan’s rule. It multiplies the
engine’s price by the factor where the moved price stays inside the band, and it keeps the price
elsewhere.
from cleverly.learners import thread_limit
def planned_prices(plan, month):
price = frame[f"price_{month}"].to_numpy()
engaged = frame[f"engaged_{month}"].to_numpy()
if plan == "price_up":
return np.where(price < (8 + 2 * engaged) / 1.12, 1.12 * price, price)
return np.where(price < (4 + 2 * engaged) / 0.85, price, 0.85 * price)
def one_regression(columns, plan):
with thread_limit():
model = LogisticRegression(max_iter=2000, C=1e6).fit(frame[columns], frame["renewed"])
moved = frame[columns].copy()
for month in (1, 2, 3):
moved[f"price_{month}"] = planned_prices(plan, month)
return float(
np.mean(model.predict_proba(moved)[:, 1] - model.predict_proba(frame[columns])[:, 1])
)
prices = ["price_1", "price_2", "price_3"]
adjust_columns = ["usage", "engaged_1", "engaged_2", "engaged_3", *prices]
omit_columns = ["usage", *prices]
rows = []
for plan in ("price_up", "price_down"):
key = f"ate_regimen[{plan} vs natural_prices]"
low, high = result[key].ci
rows.append(
{
"plan": plan,
"truth": truth[key],
"ltmle": result[key].psi,
"ltmle_low": low,
"ltmle_high": high,
"adjust": one_regression(adjust_columns, plan),
"omit": one_regression(omit_columns, plan),
}
)
print(pd.DataFrame(rows).set_index("plan").round(4).to_string())
print()
print("exact effect when the plan's prices enter the renewal model and a path stays natural:")
paths = {
plan: {
"truth": truth[f"ate_regimen[{plan} vs natural_prices]"],
"engagement held": truth[f"ate_engagement_held[{plan}]"],
"engine held": truth[f"ate_engine_held[{plan}]"],
"both held": truth[f"ate_both_held[{plan}]"],
}
for plan in ("price_up", "price_down")
}
print(pd.DataFrame(paths).round(4).to_string())
truth ltmle ltmle_low ltmle_high adjust omit
plan
price_up -0.1003 -0.1025 -0.1152 -0.0899 -0.0796 -0.0202
price_down 0.1149 0.1050 0.0896 0.1203 0.0938 0.0268
exact effect when the plan's prices enter the renewal model and a path stays natural:
price_up price_down
truth -0.1003 0.1149
engagement held -0.0906 0.1050
engine held -0.0900 0.1081
both held -0.0804 0.0973
What this output tells you. The first table gives, for each plan, the true contrast, the LTMLE
estimate with its interval, and the two regressions. The adjust column is the regression with
engagement. The omit column is the regression without it.
For price_up, the truth is -0.1003, and LTMLE gives -0.1025. The adjust regression gives
-0.0796, which lies outside the LTMLE interval (-0.1152, -0.0899) on this draw. The omit
regression gives -0.0202.
The second table gives the exact limits. The adjust regression estimates the both held value,
-0.0804 for price_up, and not the truth. Each path supplies about half of the gap. Holding
engagement at its natural values gives -0.0906. Holding the engine’s next price gives -0.0900.
The regression is correctly specified, so the gap is structural and not a fitting error.
For price_down, the truth is 0.1149, and LTMLE gives 0.1050 with an interval of (0.0896, 0.1203).
The adjust regression gives 0.0938, and it lies inside that interval on this draw. Its exact limit
is 0.0973, which is further from the truth than the LTMLE estimate is. The omit regression gives
0.0268.
The omit regression targets an association. Engaged subscribers get higher prices and renew more
often, so a high price looks harmless and the estimate shrinks toward zero. Its limit is an
approximation from one draw of 400000 customers: -0.0203 for price_up and 0.0270 for
price_down. See the probe summarize.py.
The review probe repeats the comparison on 200 draws. The adjust regression was further from the
truth than LTMLE on 193 of 200 draws for price_up and on 185 for price_down. It lay outside the
LTMLE interval on 194 draws for price_up and on 127 for price_down. The omit regression lay
outside the LTMLE interval on every draw for both plans. The adjust estimate lay between the LTMLE
estimate and the omit estimate on 200 and 199 of 200 draws.
Report the LTMLE estimate. The regressions answer other questions, and a correct outcome model does not repair that.
Validation: what each check can and cannot show#
Steps 9 to 11 check the fit. None of them checks an identification assumption. Each reading names what its output does not show.
Step 9: support, a larger price move rests on fewer customers#
A larger factor changes the plan in two places. It pushes more natural prices past \(u_t / \text{factor}\), and the plan leaves those prices at the engine’s value. It also lifts the lowest price that the plan assigns to the factor times \(l_t\). The exact density ratio is zero for a subscriber whose engine price lies below that value, because the plan assigns no price there. The share of those subscribers grows with the factor.
In the held region, from \(u_t / \text{factor}\) to \(u_t\), the plan’s density adds the density of the raised prices to the density of the held prices. The exact ratio there is one plus the ratio from the raised prices. So the subscribers priced near the ceiling carry the largest ratios. A larger factor widens both regions, and the weight falls on fewer subscribers. The effect of the move also saturates.
The code fits a 6% rise, the 12% rise of the question, and a 25% rise against natural_prices in
one LTMLE fit. The function price_rise(factor) builds each plan. The code prints the zero-weight share and the held share at month 3, and the estimate for each plan. It then prints the support of month 3: the weights and the
effective sample size.
from cleverly.datasets import price_rise
rises = {
"natural_prices": PRICE_PLANS["natural_prices"],
"up_6": price_rise(1.06),
"price_up": PRICE_PLANS["price_up"],
"up_25": price_rise(1.25),
}
rise_fit = study.identify(RegimeContrast(rises, reference="natural_prices")).estimate(method=ltmle)
support = rise_fit.diagnostics.support().to_frame()
month3 = support[(support["time"] == 3) & (support["regimen"] != "natural_prices")]
columns = ["regimen", "time", "n_followed", "max_weight", "effective_n", "share_truncated"]
print(month3[columns].round(3).to_string(index=False))
print()
factors = {"up_6": 1.06, "price_up": 1.12, "up_25": 1.25}
ceiling = 8 + 2 * frame["engaged_3"]
lowest = 4 + 2 * frame["engaged_3"]
rows = []
for plan, factor in factors.items():
key = f"ate_regimen[{plan} vs natural_prices]"
low, high = rise_fit[key].ci
rows.append(
{
"plan": plan,
"zero_weight_share": float((frame["price_3"] < factor * lowest).mean()),
"held_share": float((frame["price_3"] >= ceiling / factor).mean()),
"estimate": rise_fit[key].psi,
"truth": truth[key],
"ci_width": high - low,
}
)
print(pd.DataFrame(rows).set_index("plan").round(4).to_string())
saturation = (
truth["ate_regimen[up_25 vs natural_prices]"] / truth["ate_regimen[up_6 vs natural_prices]"]
)
print(f"exact effect of up_25 over up_6: {saturation:.2f}; a linear response gives {25 / 6:.2f}")
print()
for key, estimate in rise_fit.estimates.items():
print(f"{key}: {estimate.support.label}")
regimen time n_followed max_weight effective_n share_truncated
up_6 3 10000 5.712 6629.661 0.0
price_up 3 10000 13.704 3508.672 0.0
up_25 3 10000 19.976 1977.153 0.0
zero_weight_share held_share estimate truth ci_width
plan
up_6 0.0559 0.0862 -0.0598 -0.0587 0.0129
price_up 0.1231 0.1916 -0.1025 -0.1003 0.0254
up_25 0.3044 0.4505 -0.1223 -0.1283 0.0393
exact effect of up_25 over up_6: 2.19; a linear response gives 4.17
ate_regimen[up_6 vs natural_prices]: supported
ate_regimen[price_up vs natural_prices]: supported
ate_regimen[up_25 vs natural_prices]: supported
What this output tells you. The first table lists the weights of month 3. The second table lists the held share and the estimate of each plan.
plan |
zero-weight share at month 3 |
held share at month 3 |
max weight |
effective n of 10000 |
width of the 95% interval |
|---|---|---|---|---|---|
|
0.0559 |
0.0862 |
5.712 |
6629.661 |
0.0129 |
|
0.1231 |
0.1916 |
13.704 |
3508.672 |
0.0254 |
|
0.3044 |
0.4505 |
19.976 |
1977.153 |
0.0393 |
A larger factor gives more subscribers an exact ratio of zero, and it leaves more prices at the engine’s value. The maximum weight rises, the effective sample size falls, and the interval widens. The estimates are -0.0598, -0.1025, and -0.1223, against the true -0.0587, -0.1003, and -0.1283. The exact effect saturates. The 25% rise moves the renewal 2.19 times as far as the 6% rise, and a linear response would move it 4.17 times as far.
The weights come from one model: the cumulative density ratio of the degree-2 classifier of Step 6. The support table describes that model. It cannot see an error in the ratio, and the classifier has no indicator for the pieces of a plan. The classifier returns a positive ratio at every price, so its weights are not the exact zeros above.
The column share_truncated is 0.0 in every row. On the classifier route, g_bounds reaches no
factor of a price node (the technical
entry). These
data have no censoring, so it bounds nothing here. A trim of the price ratio is open work: X33, part (d), ratio
trim.
A plan that assigned a price outside the declared band would stop the fit. The theory needs the assigned price inside the support of the engine’s price given the history (Assumption 1 of Díaz and co-authors, 2023). The held share is how the plans of this page stay inside the band.
All three estimates are supported. The label covers the configuration and not the factor. No
registered cell measures the 6% and 25% factors.
Step 10: diagnostics, what the fit can check#
result.assess() runs the checks that apply to the fit. The code prints the operations that ran. It
then prints the support report of the renewal fit for the three plans of the question. The report
shows the weights at each month. It ends with the nuisance models that the report scores, and the
role that it leaves out.
assessment = result.assess()
ledger = assessment.to_frame()
ran = ledger[~ledger["status"].isin(["unavailable", "not_applicable", "deferred"])]
print(ran[["surface", "check", "status"]].to_string(index=False))
support_report = assessment.report("support").to_frame()
columns = ["regimen", "time", "n_followed", "max_weight", "effective_n", "share_truncated"]
print()
print(support_report[columns].round(3).to_string(index=False))
kish_ratio = (support_report["effective_n"] / support_report["n_followed"]).min()
print(f"minimum effective-sample-size ratio: {kish_ratio:.1%}")
held_down = (planned_prices("price_down", 3) == frame["price_3"]).mean()
print(f"price_down, share of month-3 prices left at the engine's value: {held_down:.1%}")
print()
nuisance = assessment.report("nuisance_models")
outcome = nuisance.to_frame()
outcome = outcome[(outcome["role"] == "outcome") & (outcome["regimen"] == "price_down")]
print(
outcome[["role", "time", "n", "evaluation", "auc", "calibration_slope"]]
.round(3)
.to_string(index=False)
)
print(
"not scored:",
sorted({omission.reason for omission in nuisance.omissions if omission.role == "treatment"}),
)
surface check status
validation score_equations passed
validation support completed
validation nuisance_models completed
regimen time n_followed max_weight effective_n share_truncated
natural_prices 1 10000 1.000 10000.000 0.0
natural_prices 2 10000 1.000 10000.000 0.0
natural_prices 3 10000 1.000 10000.000 0.0
price_up 1 10000 3.352 6939.366 0.0
price_up 2 10000 7.352 5020.246 0.0
price_up 3 10000 13.704 3508.672 0.0
price_down 1 10000 4.646 5207.473 0.0
price_down 2 10000 13.093 2845.636 0.0
price_down 3 10000 31.240 1554.276 0.0
minimum effective-sample-size ratio: 15.5%
price_down, share of month-3 prices left at the engine's value: 19.7%
role time n evaluation auc calibration_slope
outcome 3 10000 in_sample 0.697 1.004
not scored: ['continuous node ratio estimated by classification, no density']
What this output tells you. The first table lists the operations that ran. The score-equation
check is passed. The support report and the nuisance report are completed. A completed result
means the calculation ran. It is not a pass.
The support table divides by the density ratios of the degree-2 classifier. It describes that model. It cannot see an error in the ratio.
plan |
max weight at month 3 |
effective n |
what it shows |
|---|---|---|---|
|
1.000 |
10000.000 |
equal weights |
|
13.704 |
3508.672 |
the weights are uneven, and the effective n is about a third of the sample |
|
31.240 |
1554.276 |
the heaviest weights of the three plans |
The price_down plan leaves unchanged the prices below \(l_t / 0.85\), because a 15% cut would take
them below \(l_t\). It cuts the rest. It leaves 19.7% of the month-3 prices at the engine’s value.
Its cumulative density ratio is a product of three monthly ratios. The maximum weight compounds from 4.646 at month 1 to 13.093 at month 2 and
31.240 at month 3. The minimum ratio of effective sample size to
followers is 15.5%, and it belongs to price_down at month 3. No weight is truncated.
The nuisance report scores the outcome regression. Its calibration slope is 1.004, and its auc is
0.697. The regression is fitted in sample, so a slope near 1 does not show that the model is correct.
The report leaves the density-ratio classifier unscored. Nothing in this report checks that
classifier.
In the review probe, the maximum weight of price_up at month 3 averaged 14.82, with a standard
deviation of 2.03. For price_down it averaged 31.76, with a standard deviation of 5.21. The
effective sample size over n averaged 0.361 for price_up and 0.154 for price_down.
Step 11: strong sequential randomization, what this page cannot probe#
The technical reference derives no omitted-variable bound for a regime, shift, or modified treatment policy contrast (X9). So this page runs no sensitivity analysis.
A covariate-drop benchmark cannot run. Dropping usage leaves no baseline covariate, and a
longitudinal fit refuses an empty baseline (X63, part
(c)). The covariate engaged_t defines
the price band and the pieces of each policy, so the benchmark cannot drop it.
The assumption at risk is strong sequential randomization. It fails when the engine reads an input that the log does not record. An example is a competitor’s price or a manual override by a pricing analyst. Such an input moves the price and the renewal together, and no output on this page would show it. The engine log is the argument for the assumption. It is not a test.
Method evidence#
The fits on this page use one draw of a synthetic law. The registered study in the table measures the method over repeated draws. The study page gives its law, its nuisance models, and the result of each cell.
fit on this page |
registered study |
law of the study |
|---|---|---|
Step 6, LTMLE of the renewal contrasts |
this page’s law |
|
Step 7, LTMLE of the revenue contrasts |
this page’s law |
|
Step 9, the factor ladder, for |
this page’s law |
The registered study leaves four parts of this page uncovered.
part of the page |
the gap |
|---|---|
the density ratio |
the registered cells read a learned ratio only for the |
the ratio classifier |
the study’s learned ratio is a screened quadratic classifier with indicators for the pieces of the plan. The degree-2 classifier of this page has none |
revenue |
one cell measures the |
the factors |
no cell measures the 6% and 25% factors |
A review probe is a script under reviews/notebook-review/probes/. The step readings quote its
results. A review probe is not a registered study. The probe of this page repeats its configuration
over 200 draws.
The technical entry lists the evidence and the limits of the method. The limits of the continuous-policy evidence state the limits that bear on the revenue outcome and the ratio learner.
What the analysis answers#
The question was what share of subscribers would renew in month 3, and what month-3 revenue would be, under one policy. The policy raises each monthly price by 12% where the raised price stays inside the band. It keeps the engine’s price elsewhere.
part of the answer |
what the page supports |
|---|---|
the estimate |
if the assumptions below hold, the 12% rise moves the month-3 renewal share by -0.1025 against the engine’s own prices, with a 95% interval of (-0.1152, -0.0899). It moves the mean month-3 revenue by -0.3129, with an interval of (-0.4224, -0.2035). The population is active monthly subscribers who start a three-month term. The intervention is the plan’s prices for three months, with the later engagement and the engine’s later prices responding |
the cut |
the 15% cut raises the renewal share by 0.1050, with an interval of (0.0896, 0.1203). Its revenue effect of 0.0743 has an interval of (-0.0307, 0.1792), which contains zero on this draw. The page does not resolve the sign of the revenue effect of the cut |
the held prices |
each plan leaves about a fifth of the prices at the engine’s value, because the moved price would leave the band. The estimate describes the plan with those prices held, and not a pure scale |
evidence behind the reading |
the exact truths, the single-regression comparison, the support report, the factor ladder, and the review probe. The registered study measures both plans with the exact ratio. It does not measure this learner or the 6% and 25% factors |
assumptions that stay unverified |
strong sequential randomization, consistency, and no interference. The sensitivity step runs no analysis |
conclusions the page does not support |
a general price elasticity, an optimal price, the effect of a price outside the declared band, or a claim about applied data. The law is synthetic and no applied data access exists. A statement that the method is unbiased in general. A revenue effect of the cut |
Where to go next#
The sequential offers tutorial fits categorical rules in the same retailer program. It adds censoring, a declared offer mechanism, and a paired comparison of two policies. The time-to-event tutorial handles disenrollment and a competing event. The longitudinal user guide shows how to read a panel and how to report the sequential baselines. Modified treatment policies at a node states the estimator. The examples index lists every tutorial in both programs.