doubleml-pipeline

Measure what your marketing actually caused — not what merely happened alongside it.

doubleml-pipeline estimates the incremental sales and ROI of promotions, discounts, prices and media spend from the panel data you already have (date × region × product). It uses Double Machine Learning (DML) to remove the influence of confounders such as seasonality and holidays, and AutoML (FLAML) to pick the nuisance models for you. You decide what to measure; the pipeline runs model selection, cross-fitting, repetitions, confidence bounds, sensitivity checks and reporting.

Built by Google's APAC Marketing Effectiveness Taskforce. doubleml-pipeline refers to and builds upon the underlying DoubleML Python package published by doubleml.org.

• 📦 pip install doubleml-pipeline
• 🚀 Quick start — a dozen lines of code
• 📘 New to causal inference? Read Double Machine Learning in 10 minutes
• 📓 Example notebooks on realistic simulated marketing data — open them in Colab with one click
• 📚 Documentation — user guide, Saturation DML, advanced internals, API reference, FAQ

Important

Caveat on File Generation: For verification purposes, doubleml_pipeline currently writes CSV files and charts for every step of the process.

Table of contents

• Is it for me?
• What makes it different
• Installation
• Quick start
• What you need to decide
• How to read the results
• How it works
• Features
• Analyzing results with AI
• Documentation
• Community · License · Citation

---

Is it for me?

Use doubleml-pipeline when you want to answer questions such as:

• How much extra revenue did last quarter's discounts generate, and at what ROI?
• Does TV spend still pay off, or has it saturated?
• Where (regions, products) and when (months) does a promotion work best?
• How should the budget be reallocated across regions and periods?

What it needs: observational panel data — one row per period × region × product with the outcome, the treatment and the variables that drive both — and a treatment that actually varies across rows.

What it is not: a replacement for an A/B test when you can run one. It also cannot correct for drivers you did not measure; instead, its sensitivity analysis tells you how strong such an unmeasured confounder would have to be to overturn the result.

Why Double Machine Learning?

Estimates from observational data suffer from confounding: seasonality, holidays or competitor activity move both the treatment and the outcome, so a naive comparison credits the treatment with their effect. DML removes what the confounders can explain from both the outcome and the treatment with flexible ML models, and estimates the effect from what is left. Two guarantees make this trustworthy — Neyman orthogonality (small errors in the ML predictions do not bias the effect) and cross-fitting (every row is predicted by a model that never saw it) — and doubleml-pipeline adds the automation around it: model selection, repetitions, confidence bounds and diagnostics. The 10-minute introduction explains the idea without formulas.

---

What makes it different

Most DML libraries give you the estimator; you bring the learners, the tuning, the repetitions, the diagnostics and the reporting. doubleml-pipeline wraps all of those steps into one call and adds two capabilities built for marketing data: a saturation-aware treatment model and an agentic AI layer on top of the estimates.

An end-to-end, automated pipeline

run_full_pipeline takes your data frame and your choice of Y / T / X and runs every step that is usually hand-written notebook code:

Step	What the pipeline does	You set
Hyperparameter search	FLAML tunes every candidate learner once per treatment and side, within a time budget	ml_models_y, ml_models_t, time_budget
Candidate grid	every learner × fold setting is fitted in parallel and ranked by a combined error of the Y- and T-models	n_folds_list, n_jobs
Shortlist and repetitions	the best top_n candidates are re-estimated n_reps times into a weighted ensemble with an empirical 2.5–97.5% band	top_n, n_reps
Inference	2-way Cluster DML whose keys are chosen automatically from item, geo and a date block	geo_col, item_col, date_col
Business units	scaled inputs are descaled exactly, so incremental KPI and ROI come back in real-world units	CausalDataScaler
Diagnostics	ground-truth comparison on simulated data and OVB sensitivity analysis with robustness values and benchmark covariates	run_sensitivity, ground_truth_existence
Outputs	CSVs for every phase, charts, a self-contained HTML report, metadata.json and a runnable agent	output_dir

Details: How it works and Advanced configuration & internals.

Saturation DML (optional Phase 0)

Media spend has diminishing returns, and a linear treatment effect misprices the next dollar. Phase 0 estimates a response curve f(T; alpha) (tanh, exp, log or power) before the DML phases, decides with a one-standard-error test whether the curve beats a straight line, and hands the transformed treatment to Phases 1–4 — so the effect, the CATE and the ROI all account for saturation. On top of it sits a saturation-aware budget optimizer: a one-pass KKT allocation across regions, products and periods under a fixed budget or an ROI target, with six identification diagnostics that tell you when the data cannot pin the curve down. Enabling it is a one-key change (saturation_config={"saturation": {"family": "tanh"}}). Read more →

Agentic AI on top of the estimates

Every run ships with two ways to interrogate the results with Gemini, both built on the same CausalResultsReader tools that read the pipeline outputs and do the arithmetic deterministically — the LLM interprets DoubleML estimates, it never computes causal effects itself — and both carrying domain knowledge as Agent Skills (bundled: retail-analytics):

• In the browser, lightweight — the HTML report has an ✨ AI Assistant (Gemini) tab. It needs nothing but a Gemini API key: the report embeds the precomputed tool outputs (effects with intervals, ROI verdicts, breakdowns by region / product / period, diagnostics, budget optimization) and answers summary questions from them.
• In Python or Colab, full access — the generated ADK agent (adk web) and the notebook helper doubleml_pipeline.agent.Chat run all six tools live against the Phase 4 CSVs: any segment, cross-tabulation, date window or sort order, the profile of a single store or product, and several runs compared side by side.

Read more →

---

Installation

Before running the pipeline, ensure you have:

• Python 3.10 or later.

You can install the library directly from PyPI:

pip install doubleml-pipeline

Alternatively, to install from source:

git clone https://github.com/google/doubleml-pipeline.git
cd doubleml-pipeline
pip install .

---

Quick start

Basic example

Here is how to initialize and run the causal pipeline in a standard Python environment. The orchestrator is shown with its required parameters only — plus treatment_types, treatment_spend_cols and sales_col, which a non-spend treatment such as a discount rate needs for its ROI.

Tip

Ready-to-Use Simulation Datasets: Don't have a dataset on hand? We have prepared realistic marketing simulation datasets and end-to-end sample notebooks in the examples/ directory. Feel free to use them to explore the pipeline and test its features right away!

# Note: Module name uses underscores (_) instead of hyphens (-)
from doubleml_pipeline.preprocessing.scalers import CausalDataScaler
from doubleml_pipeline.workflow import CausalWorkflowOrchestrator
import pandas as pd

# 1. Load your dataset
df = pd.read_csv("path/to/your/marketing_data.csv")

# 2. Initialize the scaler with explicit scaling strategies per column
scaler = CausalDataScaler(
    standardize_cols=["holiday_index", "seasonality_factor"],
    population_standardize_cols=["competitor_spend"],
    population_median_normalize_cols=["sales_revenue", "tv_spend"],
    min_max_cols=["store_count"],
    median_normalize_cols=["discount_rate"],
    population_col="geo_population",
)

# 3. Initialize the orchestrator (required parameters only)
orchestrator = CausalWorkflowOrchestrator(
    y_col="sales_revenue",
    d_cols=["discount_rate"],
    x_cols_list=[
        ["holiday_index", "seasonality_factor", "competitor_spend"],
    ],
    ml_models_y=["lgbm", "xgboost"],
    ml_models_t=["lgbm", "xgboost"],
    n_folds_list=[5],
    output_dir="./causal_results",
    # A discount rate is a "percentage" treatment: declare its type, the
    # column holding its monetary cost (for ROI) and the sales column.
    treatment_types={"discount_rate": "percentage"},
    treatment_spend_cols={"discount_rate": "discount_cost"},
    sales_col="sales_revenue",
)

# 4. Execute the full 4-phase pipeline
#    (Grid Search -> Shortlist -> Ensemble -> Consolidation)
orchestrator.run_full_pipeline(
    df=df,
    scaler=scaler,
    top_n=3,
    n_reps=10,
)

The orchestrator will execute the models in parallel, select the best estimators, run repetitions to calculate robust confidence bounds, and write all dataframes, charts and the HTML report to the specified ./causal_results directory.

Saturation DML example

To model diminishing returns of the treatment (for example, deeper discounts adding less and less incremental sales), add a saturation_config with a saturation block to the same parameters. This enables Phase 0, which estimates the saturation curve f(T; alpha) of the treatment before the DML phases; see Saturation DML for every key and its default.

from doubleml_pipeline.workflow import CausalWorkflowOrchestrator

orchestrator = CausalWorkflowOrchestrator(
    y_col="sales_revenue",
    d_cols=["discount_rate"],
    x_cols_list=[
        ["holiday_index", "seasonality_factor", "competitor_spend"],
    ],
    ml_models_y=["lgbm", "xgboost"],
    ml_models_t=["lgbm", "xgboost"],
    n_folds_list=[5],
    output_dir="./causal_results_saturation",
    treatment_types={"discount_rate": "percentage"},
    treatment_spend_cols={"discount_rate": "discount_cost"},
    sales_col="sales_revenue",
    # "tanh" (default) | "exp" | "log" | "power"
    saturation_config={"saturation": {"family": "tanh"}},
)

Run it with the same run_full_pipeline(...) call as above. Two things to keep in mind:

• A saturated treatment must be left unscaled or use a divisive normalization — median_normalize_cols (as discount_rate above) or population_median_normalize_cols. standardize, population_standardize and min_max shift the treatment and are rejected by Phase 0 with a ValueError.
• The saturation-aware budget optimizer (optimize=True) exists for treatment_types = "spend" only. For a percentage treatment such as discount_rate, Phases 0–3 still use the saturation curve, but Phase 4 runs the simple optimizer: the orchestrator forces simple_optimization=True at construction and says so with a warning.

---

What you need to decide

The pipeline automates model selection, cross-fitting and reporting. Three choices remain yours, and they decide whether the estimate has a causal meaning:

Role	Parameter	Plain meaning	Example
Outcome Y	y_col	the KPI you want to move	sales_revenue
Treatment T	d_cols	the lever you control	discount_rate
Confounders X	x_cols_list	variables that drive both T and Y	holiday_index, seasonality_factor, competitor_spend

Your data should have one row per date × geo × item (geo and item are optional but recommended — they feed the cluster-robust inference and the breakdown charts):

date	geo	item	sales_revenue	discount_rate	discount_cost	holiday_index	seasonality_factor	competitor_spend	geo_population
2024-01-07	tokyo	shampoo	12,430	0.10	1,243	0	0.92	8,100	13,960,000
2024-01-07	osaka	shampoo	7,210	0.00	0	0	0.95	5,400	8,840,000
2024-01-14	tokyo	shampoo	14,980	0.15	2,247	1	1.08	8,300	13,960,000

Warning

Do not put variables that are caused by the treatment (store traffic after a promotion, search volume driven by a campaign) into x_cols_list: they absorb part of the effect you want to measure. Include the variables that influence both the treatment decision and the outcome, keep the list short, and check multicollinearity (VIF). The User Guide walks through this with a worked example.

Everything else — panel dimensions (date_col, geo_col, item_col), CATE covariates, sensitivity analysis, treatment types, optimization — has a default and is documented in the API Reference.

---

How to read the results

Every run writes to output_dir:

• The HTML report <treatment>_<outcome>[_<family>].html (for example discount_rate_sales_revenue.html; several treatments are joined with -, and _<family> is appended only when saturation is enabled) — one self-contained file with every chart, the summary tables and an AI assistant tab.
• CSV files for every phase, documented in the Output CSV Schema. The consolidated results live in step4_consolidated_results/.
• metadata.json with the run configuration and, by default, a runnable ADK agent under agents/.

Three numbers answer most questions:

1. Incremental KPI — how much of the outcome the treatment caused, with the empirical 2.5–97.5% band from the n_reps repetitions (step4_consolidated_results/graph/001_total/; per region and product in 002_geo/ and 003_item/).
2. ROI vs. threshold — incremental KPI ÷ spend, compared with opt_threshold_roi (default 1.0). Below the line, the money is better spent elsewhere.
3. Robustness value (RV) — how strong an unmeasured confounder would have to be to explain the effect away (graph/006_sensitivity/, with run_sensitivity=True). Compare it with the benchmark covariate: if the RV is smaller, treat the result with caution.

The screenshot at the top of this page shows the first of them: the 001_total tab with the incremental KPI, the ROI and the time series of the estimated effect against the known ground truth of a simulated dataset.

Then look at where and when the effect is largest — the CATE charts in graph/004_cate/ and, for percentage / impressions / price treatments, the ROI breakdowns under roi_output/<treatment>/ — and run the validation checks (ignorability, heterogeneity, positivity) described in the User Guide before acting on the numbers.

---

How it works

Phase 0 (optional)  Saturation: estimate the diminishing-returns curve f(T; alpha)
                    and hand fbar(T) to the next phases      -> docs/saturation.md
Phase 1             Grid search: fit the Y- and T-models for every candidate
                    learner x fold setting, in parallel
Phase 2             Shortlist: rank the candidates by combined error, keep top_n
Phase 3             Repetitions: re-estimate the shortlist n_reps times and build
                    a weighted ensemble with empirical 2.5-97.5% bounds
Phase 4             Consolidate & report: CATE, incremental KPI, ROI, sensitivity
                    analysis, budget optimization, charts, HTML report, agent

When the pipeline runs, it saves all intermediate evaluation metrics, candidate model repetitions, consolidated datasets, and visualization plots within the specified output_dir (e.g., ./causal_results/) as shown below:

The cross-fitting folds and the standard errors use a 2-way Cluster DML whose keys are chosen automatically from item_col, geo_col and a date block derived from date_col; the hyperparameters of the nuisance models are searched once per treatment and learner (hparam_policy="global"). Both are described in Advanced configuration & internals.

---

Features

• AutoML nuisance models — FLAML searches and tunes the learners you list ("lgbm", "xgboost", "rf", …) within time_budget seconds, once per treatment and learner.
• Model selection with repetitions — candidates are ranked by a combined error of the Y- and T-models, the best top_n are re-estimated n_reps times, and the effect is reported with empirical confidence bounds.
• Heterogeneous effects (CATE) — B-spline bases over the covariates you choose (covariates_for_cate_list) show how the effect varies by season, region or product.
• Sensitivity analysis (OVB) — robustness values and benchmark covariates (Cinelli & Hazlett 2020) quantify how strong an unmeasured confounder would have to be; details in Advanced configuration & internals.
• Saturation DML (optional) — Phase 0 estimates diminishing returns f(T; alpha) (tanh, exp, log or power) and the DML phases run on the transformed treatment. See Saturation DML.
• Budget optimization — a simple ROI-threshold optimizer or, for spend treatments with saturation, a one-pass KKT allocation under a fixed budget or an ROI target. See Saturation-aware optimization.
• Safe business-unit scaling — population- and median-based normalization with exact descaling, so KPI and ROI come back in real-world units.
• Priors for Bayesian MMM — fitted Normal / LogNormal / Beta parameters of the treatment contribution (%) and ROI, ready to use as prior knowledge (mind that CATE varies with conditions such as seasonality, and that MMM estimates an ATE while DML reports a CATE).
• Reports — CSVs (schema), publication-ready charts, a self-contained HTML report, and an ADK agent plus an in-browser Gemini chat to interrogate the results.

---

Analyzing results with AI (Browser Chat or ADK Agent)

There are two ways to explore the causal estimates with Gemini. Both use the same CausalResultsReader tools and the same bundled skills; they differ in what the tools can reach.

	In the browser (lightweight)	In Python or Colab (full access)
How	✨ AI Assistant (Gemini) tab of the HTML report	adk web on the generated agent, or Chat / ask in a notebook
Needs	the HTML file and a Gemini API key	Python with doubleml-pipeline (and google-adk) installed
Data the model sees	the outputs of the tools, precomputed at report time with their default arguments (effects with intervals, ROI vs. threshold, breakdowns by region / product / period and CATE covariate — up to 50 rows each, model diagnostics, budget optimization)	the tools run live against the Phase 4 CSVs with any arguments
Good for	executive summaries, "which treatment pays back", "where and when does it work"	drill-downs: arbitrary segments, cross-tabulations, date windows, sort orders, the profile of one store or product (get_segment_profile), comparisons across several runs

1. Directly in the HTML report (no server required): Open <output_dir>/<treatment>_<outcome>[_<family>].html in your browser, select the ✨ AI Assistant (Gemini) tab, and enter a Gemini API Key to chat with the precomputed CausalResultsReader outputs and the bundled retail-analytics skill. The tab shows the exact JSON the model receives (Inspect Precomputed CausalResultsReader Snapshot).
2. Local ADK Agent (generate_agent=True): By default, the pipeline also writes a runnable ADK agent to <output_dir>/agents/causal_ml_insight_agent/. The agent answers business questions such as "Which promotion pays back, where, and when?" from the pipeline outputs. Its tools read the results and do the arithmetic deterministically, so the LLM interprets the DoubleML estimates instead of computing causal effects itself. The domain knowledge comes from Agent Skills bundled in skills/, starting with retail-analytics. Coding agents can use these skills directly too.

cp causal_results/agents/causal_ml_insight_agent/.env.example \
   causal_results/agents/causal_ml_insight_agent/.env  # Set your project.
adk web causal_results/agents

3. In a notebook (Colab or Jupyter): the same agent, without a server. Chat keeps the conversation and renders an interactive widget with chat.ui(); ask answers a single question.

from doubleml_pipeline.agent import Chat

chat = Chat("./causal_results", api_key="YOUR_GEMINI_API_KEY")
chat.ui()  # or: await chat.ask("Which promotion pays back, where, and when?")

To regenerate the agent with your own skills or model, or to add one to the results of an earlier run:

from doubleml_pipeline.agent import generate_agent

generate_agent("./causal_results", skills_dir="./my_skills")

If you run one orchestrator per channel, one agent can cover all of the runs. It compares the channels side by side but never adds their effects up, since each run is estimated separately:

from doubleml_pipeline.agent import generate_agent

generate_agent(
    ["results/Spend_TV", "results/Spend_Digital", "results/Spend_Flyer"],
    output_dir="results",
)
# adk web results/agents

---

Documentation

Document	Read it when
Double Machine Learning in 10 minutes	you are new to causal inference or want the vocabulary (Y / T / X, CATE, cross-fitting) mapped to the parameters
User Guide	you are preparing data, choosing variables and normalization, setting up CATE and the ML options, or validating a result — a chapter per step of the workflow, with a worked case study and a checklist
Saturation DML	your treatment has diminishing returns (media spend) or you want the saturation-aware budget optimizer
Advanced configuration & internals	treatment types and ROI charts, optimization modes, Cluster DML keys, hyperparameter policy, parallelism, sensitivity formulas
FAQ	something errors, warns, or takes longer than expected
API Reference	you need every parameter, default and return value
Output CSV Schema	you are reading the CSV outputs programmatically
Slides	you want the original slide deck of the User Guide (one image per slide) for a presentation
Example notebooks: IN_skincare , JP_retailer	you want an end-to-end run — loading realistic simulated panel data, engineering lagged features, scaling, running the pipeline and plotting the results (a single discount-rate treatment in IN_skincare; multi-treatment, lagged flyer spend and a custom CATE formula in JP_retailer)

---

Community

We welcome contributions!

• Reporting Issues: If you encounter bugs or want to request features, please open an issue in the GitHub repository issue tracker.
• Submitting Changes: Please see CONTRIBUTING.md for our guidelines on submitting pull requests.

License

Apache License 2.0 — see LICENSE.

Citation

If you use doubleml-pipeline in your work, please cite the repository (GitHub offers the same reference under Cite this repository, generated from CITATION.cff):

@software{doubleml_pipeline,
  title   = {doubleml-pipeline: automated causal inference with Double Machine
             Learning and AutoML},
  author  = {{Google LLC}},
  year    = {2026},
  version = {0.2.2},
  url     = {https://github.com/google/doubleml-pipeline}
}

---

Acknowledgments & Dataset Credits

We gratefully acknowledge our partners for their collaboration on the simulation datasets included under examples/:

• examples/IN_skincare/: Simulation data and example materials provided in collaboration with Analytic Edge.
• examples/JP_retailer/: Simulation data designed and provided by Hakuhodo Inc., informed by real-world Japanese marketing dynamics.

---

References

1. Debiased Machine Learning method Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, James Robins. Double/debiased machine learning for treatment and structural parameters. arXiv:1608.00060

2. Hyperparameter Tuning for Causal Inference with Double Machine Learning: A Simulation Study Martin Spindler, et al. arXiv:2402.04674

3. Debiased Machine Learning of Conditional Average Treatment Effects and Other Causal Functions Vira Semenova, Victor Chernozhukov. arXiv:1702.06240

4. Multiway Cluster Robust Double/Debiased Machine Learning Harold D. Chiang, Kengo Kato, Yukitoshi Matsushita, Takuya Ishihara. arXiv:1909.03489