cleverly.estimators.TMLEResult

class cleverly.estimators.TMLEResult(estimates: dict[str, ~cleverly.inference.influence.ParameterEstimate], repeats: tuple[~cleverly.estimators._nuisance.RepeatFit, ...], data: ~cleverly.data.causal_data.CausalData, config: ~cleverly.estimators.base.TMLEConfig, estimator: ~typing.Any = None, provenance: ~cleverly.provenance.Provenance | None = None, simultaneous: ~cleverly.inference.multiplier.SimultaneousBands | None = None, bootstrap: ~cleverly.inference.bootstrap.BootstrapResult | None = None, intermediate_value: float | None = None, extra: dict[str, ~typing.Any] = <factory>, identified_effect: ~typing.Any = None, method: ~typing.Any = None, parameter_keys: dict[str, ~typing.Any] = <factory>, fitted_method: str = 'unknown', solved_corrections: bool = False)

Store a fitted point-treatment TMLE result.

Behaves like a mapping from estimand name to ParameterEstimate, and carries the nuisance fits so that sensitivity and validation analyses can be run without refitting.

Parameters:

estimatesdict of str to ParameterEstimate

Estimates keyed by stable parameter alias.

repeatstuple of RepeatFit

One RepeatFit per draw of the cross-fitting split – a one-element tuple for an ordinary fit, R of them under repeats=R. This is where the nuisances and the fluctuations actually live; nuisance and fluctuations read through to the first entry, which is what keeps every analysis written against a single fit working unchanged. Anything that must account for all the draws – and every analysis that produces a number, as against one that describes a mechanism, must – iterates this.

dataCausalData

Validated study data and dataframe backend.

configTMLEConfig

Normalized implementation configuration.

estimatorAny or None

Fitted estimator that produced this result.

provenanceProvenance or None

Runtime, dependency, and data fingerprints.

simultaneousSimultaneousBands or None

Joint confidence bands for multi-parameter fits.

bootstrapBootstrapResult or None

Optional refit-bootstrap draws and failure counts.

intermediate_valuefloat or None

The level of the intermediate variable this fit targets, or None for an ordinary point-treatment fit. It is part of the estimand, not a setting: every estimate here is a controlled direct effect holding Z at this level, and the same data yields a different parameter at the other one. See cleverly.estimators.direct_effect.

extradict

Method-specific fitted artifacts.

identified_effectIdentifiedEffect or None

Causal question and identifying assumptions.

methodEstimationMethod or None

Typed public method configuration.

parameter_keysdict of str to ParameterKey

Structured identities for reported aliases.

fitted_methodstr

Method identity stamped by the estimator; unknown for unstamped artifacts.

solved_correctionsbool

Whether the fit participated in the guarded correction system.

Attributes:

assessment_cachedict

Saved diagnostic and sensitivity outputs. Not a constructor argument, so every constructed result owns its own cache. A result derived with dataclasses.replace() therefore starts empty and cannot serve the original result’s verdicts. Persistence restores the mapping without the constructor.

See also

cleverly.ParameterEstimate

One entry of estimates.

cleverly.assessment.DiagnosticsFacade

Reached as result.diagnostics.

cleverly.assessment.SensitivityFacade

Reached as result.sensitivity.

cleverly.longitudinal.LongitudinalResult

The same contract for a sequential fit.

Examples

>>> from sklearn.linear_model import LinearRegression, LogisticRegression
>>> from cleverly import ATE, CausalStudy, PointTreatment
>>> from cleverly.datasets import make_linear_ate
>>> frame, _ = make_linear_ate(n=200, seed=0)
>>> study = CausalStudy(
...     frame,
...     design=PointTreatment(
...         outcome="Y", treatment="A", adjustment=("W1", "W2", "W3", "W4")
...     ),
... )
>>> result = study.identify(ATE()).estimate(
...     outcome_learner=LinearRegression(),
...     treatment_learner=LogisticRegression(max_iter=1000),
...     cross_fit=False,
...     random_state=0,
... )

The result is a mapping from estimand alias to ParameterEstimate:

>>> sorted(result.estimates)
['ate']
>>> low, high = result["ate"].ci
>>> low < high
True

The interval bounds are ordered. The default validation battery reads the fitted artifacts without refitting:

>>> result["ate"].std_error > 0
True
>>> len(result.validate().items) > 0
True

Methods

TMLEResult.assess(*, include_refits: bool = False, include_retargets: bool = False, arguments: Mapping[str, Mapping[str, Any]] | None = None, random_state: int | None = None) → AssessmentReport

Run the applicable post-fit assessment battery.

Parameters:

include_refitsbool

Run requested operations that refit nuisance models.

include_retargetsbool

Include moderate retargets; cheap E-value retargets run by default.

argumentsmapping or None

Per-operation keyword arguments.

random_stateint or None

Common seed for stochastic refit operations.

Returns:

cleverly.AssessmentReport

Validation, diagnostics, and sensitivity in one report.

See also

TMLEResult.validate

Run the stored-artifact validation checks.

cleverly.assessment.DiagnosticsFacade.run_all

Run diagnostics alone.

Examples

>>> from sklearn.linear_model import LinearRegression, LogisticRegression
>>> from cleverly import ATE, CausalStudy, PointTreatment
>>> from cleverly.datasets import make_linear_ate
>>> frame, _ = make_linear_ate(n=80, seed=1)
>>> result = CausalStudy(
...     frame,
...     design=PointTreatment(
...         outcome="Y", treatment="A", adjustment=("W1", "W2", "W3", "W4")
...     ),
... ).identify(ATE()).estimate(
...     outcome_learner=LinearRegression(),
...     treatment_learner=LogisticRegression(max_iter=1000),
...     cross_fit=False,
...     random_state=0,
... )
>>> result.assess().validation.passed
True

TMLEResult.coefficients(scale: str = 'link') → Any

A working model’s coefficients, on the link scale or exponentiated.

scale="link" reports what the fit estimated: \(\hat\beta\), with a Wald interval on the scale the model is linear on. scale="ratio" reports \(e^{\hat\beta}\) with the interval exponentiated from that same scale, which is how the applied literature reports a log- or logit-link marginal structural model and the reason those links exist here.

What the exponential means depends on the link, and the two are not interchangeable. Under link="log" a coefficient exponentiates to a risk (or rate) ratio; under link="logit" to an odds ratio. Reporting one as the other is a real error rather than a wording preference, so the scale column of the returned frame names which it is, row by row. The intercept is a third thing again: \(e^{\beta_0}\) is a baseline mean (log) or a baseline odds (logit), not a ratio of anything, and its p-value tests \(\beta_0 = 0\) rather than any absence of effect. It is reported rather than dropped – it is a coefficient of the declared model like the others – and labelled "baseline".

Refused on an identity-link fit, where \(\beta\) is a risk difference and \(e^\beta\) is not a quantity; and on a fit with no working model at all, whose parameters are counterfactual means rather than coefficients.

Nothing is re-estimated: the interval and the p-value come from the influence curve the fit already reported, read on the scale scale names. This is a view, in the sense cleverly.longitudinal.LTMLEResult.curve() is one.

Parameters:

scale{“link”, “ratio”}

Coefficient scale to report.

Returns:

dataframe

One row per working-model coefficient in the input backend.

TMLEResult.contrast(function: Callable[[ndarray[tuple[Any, ...], dtype[float64]]], float], names: Sequence[str], *, name: str | None = None, scale: Literal['level', 'difference', 'ratio', 'fraction'] = 'difference', gradient: Callable[[ndarray[tuple[Any, ...], dtype[float64]]], ndarray[tuple[Any, ...], dtype[float64]]] | None = None, transform: Transform | None = None) → ParameterEstimate

A smooth function of several estimands, with correct inference.

Applies the delta method to the joint influence curve, so the correlation between the estimands is handled rather than ignored: \(D_\phi = \nabla\phi(\hat\psi)^\top D\). For example, result.contrast(lambda p: p[0] - p[1], ["ey1", "ey0"]) compares two means already stored on result.

Pass gradient when the function’s derivative is known in closed form. The default is a central difference, which is accurate to about 1e-10 relative – fine for reporting, but not for a test asserting agreement at 1e-12.

The result is an ordinary ParameterEstimate, so it carries its own influence curve and can itself be fed back into a contrast. It inherits the inference status of the estimates it reads, so a contrast of non-inferential estimates refuses an interval exactly as they do.

scale="ratio" reports a positive contrast with its interval on the log scale. transform= computes the interval and the test on the scale of a declared monotone map and maps the interval back with the inverse, as ci(contrast = list(f, f_inv, h, fh_grad)) in R drtmle 1.1.2 does.

Parameters:

functioncallable()

Smooth scalar function of the selected estimates.

namessequence of str

Aliases passed to function in order.

namestr or None

Alias for the derived estimate.

scalestr

Reported scale of the derived estimate. With transform= it must be "level" or "difference".

gradientcallable() or None

Analytic gradient. None uses central differences.

transformTransform or None

The map the interval and the test are computed on. None computes them on the reported scale, or on the log scale for scale="ratio".

Returns:

ParameterEstimate

Derived estimate with influence-curve inference.

TMLEResult.covariance(names: Sequence[str] | None = None) → ndarray[tuple[Any, ...], dtype[float64]]

Return joint covariance for selected estimates.

The estimands are not independent – they are functionals of one targeted distribution and share most of their influence curve – so a contrast built from two of them needs this rather than the two variances. Computed from the influence curves at the right independent unit, so a clustered fit gets the cluster-level covariance.

Each estimate declares its covariance_rule, and every selection size applies that one rule. The ordinary rule is the sample covariance of the curves. On a fold-evaluated CV-TMLE fit, the diagonal therefore differs from the stored cross-validated variance. The stacked cross-fitted natural-course mean declares the raw second moment, which is also its stored variance. A selection that mixes rules, or a raw second-moment selection on a clustered result, raises ValueError.

On a fit whose inference_status supplies no inference, the matrix is the plug-in covariance of the reported curves. It is a diagnostic and it licenses no interval, for the reason that status records.

Parameters:

namessequence of str or None

Aliases in output order. None selects all estimates.

Returns:

ndarray

Covariance matrix in the requested order.

TMLEResult.influence_frame() → Any

One column per estimand of per-observation influence-curve values.

On a fit whose inference_status supplies no inference, no result shows the column is the estimator’s influence curve, so this frame emits curves and not inference.

Returns:

dataframe

One column per estimand of per-observation influence-curve values.

TMLEResult.psi(name: str | None = None) → float

Return one point estimate.

Parameters:

namestr or None

Parameter alias. None requires a single-parameter result.

Returns:

float

Point estimate on its reported scale.

TMLEResult.ratio(numerator: str, denominator: str, *, kind: Literal['rr', 'or'] = 'rr', name: str | None = None) → ParameterEstimate

The risk ratio or odds ratio of two level estimates of this fit.

The interval is computed on the log scale and exponentiated, as lmtp_contrast(type = "rr") and type = "or" in R lmtp 1.5.4 compute it. The p-value tests a ratio of one. ParameterEstimate.wald_test() tests any positive null. The ratio inherits the inference status of its inputs.

Parameters:

numeratorstr

The name of the level in the numerator, such as "ey1".

denominatorstr

The name of the level in the denominator, such as "ey0".

kind{“rr”, “or”}, default=”rr”

A ratio of the levels, or a ratio of their odds. "or" needs a binary outcome.

namestr or None, default=None

The name of the derived estimate. None gives the registered name when the inputs are an arm and the reference arm, such as "rr" for ratio("ey[1]", "ey[0]") on a two-armed fit. Otherwise it gives "rr[<numerator> vs <denominator>]".

Returns:

ParameterEstimate

The ratio, on the "ratio" scale.

Raises:

CapabilityError

When kind="or" and the outcome is not binary.

ValueError

When an input is not a level, or a level is outside the domain of the log.

See also

cleverly.inference.ParameterEstimate.wald_test

Test the ratio at a null.

Examples

>>> from sklearn.linear_model import LogisticRegression
>>> from cleverly import CausalStudy, CounterfactualMean, PointTreatment
>>> from cleverly.datasets import make_binary_outcome
>>> frame, _ = make_binary_outcome(n=200, seed=0)
>>> study = CausalStudy(
...     frame,
...     design=PointTreatment(outcome="Y", treatment="A", adjustment=("W1", "W2")),
... )
>>> result = study.identify(CounterfactualMean()).estimate(
...     outcome_learner=LogisticRegression(max_iter=1000),
...     treatment_learner=LogisticRegression(max_iter=1000),
...     cross_fit=False,
...     random_state=0,
... )
>>> rr = result.ratio("ey[1]", "ey[0]")
>>> rr.name, rr.scale, rr.psi > 0
('rr', 'ratio', True)

TMLEResult.repeat_spread() → dict[str, float]

Split spread of the estimate across the cross-fitting draws, per estimand.

A diagnostic, and emphatically not a standard error. It measures how much the arbitrary fold assignment moved the answer: the R draws differ in nothing but the split, so this is the size of the fold noise a single fit carries silently.

Reported on the inference scale, which is the scale std_error is on: the original outcome scale for a level or a difference, and the log scale for a ratio. A ratio estimand therefore contributes \(\mathrm{sd}(\log\hat\psi_r)\). Read against std_error: a spread that is an appreciable fraction of the standard error means the split mattered, and one near zero means the nuisance fits were stable enough that repeating bought little. That is a way to read the number and not a threshold on it. No rule in this package assigns the ratio a status, and RepeatSpreadRow reports it without one. A spread taken on psi itself would not be comparable with the standard error of an rr or an or, and would be wrong by roughly the estimate’s own magnitude.

A draw whose psi is not finite, or whose ratio-scale psi is not positive, yields nan for that estimand rather than -inf or an exception. There is no spread on the inference scale to report there, and the caller needs the other estimands.

What it must not be used for is inference. It says nothing about the sampling variability of the estimand, so it is not an alternative to the reported standard error. The median variance rule already includes between-draw displacement.

Raises when there is only one draw, since the standard deviation of one number is not a diagnostic but an artefact.

Returns:

dict of str to float

Standard deviation across the cross-fitting draws, per estimand, on the inference scale of that estimand.

TMLEResult.save(path: Any) → Any

Write this result to a trusted joblib artifact; see cleverly.load().

Loading joblib data can execute arbitrary Python code. Only load files from a trusted source in a compatible Python and dependency environment. The artifact records cleverly.__version__. A load by a different version warns with VersionMismatchWarning and does not migrate the result.

Parameters:

pathpath-like

Destination file.

Returns:

Path

The destination path, as given.

TMLEResult.summary(*, protocol: Literal['full', 'fingerprint'] = 'full') → str

A printable report of the fit.

Parameters:

protocol{“full”, “fingerprint”}

"full" prints every field of the study protocol record. "fingerprint" prints only its schema and fingerprint, for a reader who has the record already.

Returns:

str

A printable report: the estimates, the settings, then the diagnostics.

TMLEResult.to_frame() → Any

Tidy results, one row per estimand, in the caller’s dataframe backend.

An ordinary fit emits std_err, ci_lower, ci_upper and p_value. A fit whose inference_status supplies no inference emits an inference column naming that status, and plugin_std_err, plugin_interval_lower and plugin_interval_upper in their place. Those three are a diagnostic and they are not a confidence interval. Exactly one of std_err and inference is present in any frame, so a consumer that has to tell the two apart has a total test. A fit that divided by a declared known treatment mechanism adds a treatment_mechanism column that reads "known".

Returns:

dataframe

One row per estimand, in the backend the data arrived in.

TMLEResult.validate() → ValidationReport

Run the inexpensive method-appropriate checks without refitting.

Returns:

ValidationReport

One item per check the fitted method declares, read off stored artifacts.

See also

cleverly.ValidationReport

The returned validation statuses.

cleverly.assessment.DiagnosticsFacade.run_all

Run the broader diagnostic set.

Examples

>>> from sklearn.linear_model import LinearRegression, LogisticRegression
>>> from cleverly import ATE, CausalStudy, PointTreatment
>>> from cleverly.datasets import make_linear_ate
>>> frame, _ = make_linear_ate(n=80, seed=1)
>>> study = CausalStudy(
...     frame,
...     design=PointTreatment(
...         outcome="Y", treatment="A", adjustment=("W1", "W2", "W3", "W4")
...     ),
... )
>>> result = study.identify(ATE()).estimate(
...     outcome_learner=LinearRegression(),
...     treatment_learner=LogisticRegression(max_iter=1000),
...     cross_fit=False,
...     random_state=0,
... )
>>> report = result.validate()
>>> len(report.items) > 0
True

Attributes

TMLEResult.assessment_family: ClassVar[str] = 'point'

Which family of assessment declarations applies to this result. A class constant rather than a property: the family is a property of the result type, and nothing a fit can do changes it. cleverly.assessment reads it to pick the capability table. The method identity is fitted_method, which is the only name for it, because two names for one value drift.

TMLEResult.ate

The average treatment effect, the most commonly wanted estimand.

TMLEResult.bootstrap: BootstrapResult | None = None

TMLEResult.ctmle_selection

The collaborative selection artifact a C-TMLE fit retained, if any.

The sibling of cv_targeting for the ctmle key, and typed for the same reason: a reader who reaches into extra gets Any and has to re-assert the type at every call site. None covers both an ordinary fit and a collaborative result whose artifact is absent, which the reports distinguish by fitted_method.

The two classes are imported inside the property because cleverly.estimators.ctmle imports this module, so the dependency only runs in the direction the package already has.

TMLEResult.cv_targeting

Fold-level detail for fold evaluation or fold-specific targeting.

Carries both the stacked and original fold-evaluated reports whichever one result[name] was configured to show, so the two can always be compared.

TMLEResult.diagnostics

Unified support, nuisance, score, and refutation diagnostics.

TMLEResult.estimate

The sole parameter estimate, with an explicit refusal for multi-parameter fits.

TMLEResult.estimator: Any = None

TMLEResult.fitted_method: str = 'unknown'

TMLEResult.fluctuations

The first draw’s solved fluctuations, keyed by target group.

Reads through to repeats for the reason nuisance does, and carries the same warning: on a repeated fit this is one draw’s epsilon and one draw’s targeted Qbar.

TMLEResult.identified_effect: Any = None

Fits made through CausalStudy record this effect. Direct estimator calls leave it unset.

TMLEResult.inference_status

The inference status every reported estimate declares.

One status per fit: the estimator stamps it on every estimate it produces, and inference_status() refuses a mix with ValueError. "influence_curve" when the fit reports no estimate, because such a fit refuses nothing.

TMLEResult.influence_curves

Return influence curves by parameter alias.

TMLEResult.intermediate_value: float | None = None

TMLEResult.method: Any = None

The normalized typed method configuration used by IdentifiedEffect.estimate.

TMLEResult.n

Return the number of observations.

TMLEResult.n_repeats

How many draws of the cross-fitting split this fit combined.

TMLEResult.nuisance

The first draw’s nuisance predictions.

The attribute every sensitivity analysis and validation diagnostic was written against, kept pointing at one object so that a fit with no repeats behaves exactly as it always has. On a repeated fit this is draw zero of R and nothing else: it is the right thing to describe a fitted mechanism with, and the wrong thing to compute a reported number from. Use repeats for the latter.

TMLEResult.nuisances

Every draw’s nuisance predictions, in fit order.

TMLEResult.provenance: Provenance | None = None

TMLEResult.replayability

Which post-fit operations this in-memory or loaded result can replay.

TMLEResult.score_verdict

This fit’s own answer to whether its interval is licensed.

The same object result.diagnostics.score_equations() returns, at the default tolerance, held here because summary() reports it and because a caller should not have to know which subsystem owns the question. Derived, never stored: it is recomputed from the fluctuations a result carries, so a fit reloaded from disk answers with its own records rather than with a flag written at fit time that nothing could check afterwards. Whole-result persistence retains those records directly instead of rebuilding a partial result graph. __getstate__() is what makes that true across a save, because the memo this property writes is otherwise part of the artifact.

Free: it reads cached arrays and refits nothing.

TMLEResult.sensitivity

Capability-aware sensitivity analyses for this fitted method.

TMLEResult.simultaneous: SimultaneousBands | None = None

TMLEResult.solved_corrections: bool = False

TMLEResult.split_plan

Return immutable assignments for every fitted repeat.

Bound to the rows that produced it, through the same data fingerprint provenance records. A fold label is a position, so a plan is only meaningful for the rows it was realised on: reuse it on reordered rows, or on rows a refutation replaced, and every label points at a different unit while the row count still agrees. validate() refuses that, and a caller who means to reuse the labels on other rows says so with unbound().

The plan carries each repeat’s FoldOrigin when random_partition() drew every repeat, and a fit accepts only such a plan. A stratified or in-sample fit returns a plan whose provenance is None: it records the split the fit ran, and no fit accepts it back.

TMLEResult.estimates: dict[str, ParameterEstimate]

TMLEResult.repeats: tuple[RepeatFit, ...]

TMLEResult.data: CausalData

TMLEResult.config: TMLEConfig

TMLEResult.extra: dict[str, Any]

TMLEResult.parameter_keys: dict[str, Any]

Alias-to-structured-key mapping. Routing must read this rather than parse aliases.

TMLEResult.assessment_cache: dict[str, Any]

Persistent assessment results, keyed by operation plus normalized arguments. Filling this mapping never changes the fitted parameter or its summary.

init=False because the key records the operation and its arguments and nothing about the result that answered them. dataclasses.replace used to pass the original’s mapping straight through, so a result derived by attach_bootstrap – which changes estimates, and therefore every sensitivity answer – served the original’s cached verdicts under a different method stamp. A constructed result now owns an empty cache, and joblib persistence restores the saved mapping with the instance state rather than through __init__.