cleverly.sensitivity.BenchmarkResult#
- class cleverly.sensitivity.BenchmarkResult(estimand: str, covariates: tuple[str, ...], cf_y: float, cf_d: float, rho: float, delta_psi: float, psi_long: float, psi_short: float, sigma2_long: float, sigma2_short: float, nu2_long: float, nu2_short: float, random_state: int | None = None)[source]#
Confounder strengths calibrated against covariates that were observed.
Interpretation:
cf_yandcf_dare the sensitivity parameters an unobserved confounder would need in order to be “as important as” the benchmark covariates, measured by how much dropping those covariates degrades the outcome regression and the Riesz representer.delta_psiis how much the estimate actually moved when they were dropped, andrhois the implied degree of adversity – a value well below 1 says the worst-caserho = 1is pessimistic for confounders like these.- Parameters:
- estimand
str Alias of the estimand benchmarked.
- covariates
tupleofstr The observed covariates the strength is calibrated against.
- cf_y
float Gain in explained outcome variation from adjusting for those covariates,
(R^2_long - R^2_short) / (1 - R^2_long), clipped to[0, 1]. The raw value isgain_y.- cf_d
float Gain in the Riesz representer’s second moment from adjusting for those covariates,
(nu2_long - nu2_short) / nu2_short, clipped to[0, 1]. The raw value isgain_d. Both gains are the benchmark convention of Chernozhukov et al. (arXiv:2112.13398v6, Appendix E.5, Remark 8) and of DoubleML. Remark 8 states that they serve as proxies for the shares thatomitted_variable_bounds()takes, so they enter it unchanged. Pass the whole record asbenchmark=.- rho
float The implied alignment of the two.
- delta_psi
float How far the estimate moved when they were dropped.
- psi_long
float Estimate with the benchmark covariates adjusted for.
- psi_short
float Estimate with them dropped.
- sigma2_long
float Residual outcome variance with them adjusted for.
- sigma2_short
float Residual outcome variance with them dropped.
- nu2_long
float Riesz second moment with them adjusted for.
- nu2_short
float Riesz second moment with them dropped.
- random_state
intorNone Seed this benchmark ran under. Pass it back to
benchmark()to obtain the benchmark again.Noneonly on a record built without a seed.
- estimand
- Attributes:
Methods#
Attributes#
- BenchmarkResult.gain_d#
The unclipped Riesz gain,
nu2_long / nu2_short - 1.A value of 1 or more gives
cf_d = 1, at which the bound is unbounded.
- BenchmarkResult.gain_y#
The unclipped outcome gain,
sigma2_short / sigma2_long - 1.A value above 1 says that a confounder as strong as the benchmark covariates would explain all of the remaining outcome variation.
cf_ythen holds the largest share, 1.
- BenchmarkResult.random_state: int | None = None#
Seed the short refit ran under, resolved rather than requested: an explicit seed, else the fit’s own, else one drawn here. A benchmark refits, so an estimator carrying no seed would otherwise give a different answer to the same question and cache the first one.