Skip to content

docs: migrate narrative RST off fit-time aggregate= (sweep PR-A) - #763

Merged
igerber merged 5 commits into
mainfrom
docs/aggregate-postfit-rst
Aug 11, 2026
Merged

docs: migrate narrative RST off fit-time aggregate= (sweep PR-A)#763
igerber merged 5 commits into
mainfrom
docs/aggregate-postfit-rst

Conversation

@igerber

@igerber igerber commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • Migrates the narrative-RST half of the fit-time aggregate= teaching sweep (the 3.9-deprecated M-020 family) onto post-fit results.aggregate(type=...): choosing_estimator.rst drops its second full refit for results.aggregate('event_study'); python_comparison.rst now mirrors the differences package's post-fit idiom it is paired against; r_comparison.rst's aggregation block maps 1:1 onto R's three aggte() calls (simple / event_study / group; the fit-time 'all' has no post-fit counterpart).
  • Rewrites troubleshooting.rst's event-study-plot remedy coherently: a base_period="universal" fit whose aggregate('event_study') container is inspected for its actual is_reference row(s) and plotted directly - no hard-coded reference_period=-1, because under the varying default every pre-period point is an estimated effect with no common anchor, and on gapped grids even the universal base can anchor at an event time other than -1 (the REGISTRY positional-base contract).
  • The troubleshooting bootstrap passage deliberately KEEPS fit-time aggregate='event_study' with a comment naming it the documented until-4.0 exception: post-fit recompute levels raise on a bootstrapped fit (staggered_results.py fail-closed gate), 'simple' relays. This matches the AI guides and docs/migration-4.0.md.
  • The two rewritten blocks previously died on allowlisted NameErrors before their teaching lines executed (r_comparison:block2, troubleshooting:block8); both are now self-contained and removed from _CONTEXT_DEPENDENT_SNIPPETS, so tests/test_doc_snippets.py actually executes them. Block counts per file are unchanged, so the surviving positional allowlist keys still point at their intended blocks.
  • CHANGELOG gains the first post-3.9 [Unreleased] section with this entry. A follow-up PR (sweep PR-B) migrates the tutorial notebooks and lands after this one.

Methodology references (required if estimator / math changes)

  • Method name(s): N/A - documentation-only; no estimator or math changes. The troubleshooting rewrite follows the REGISTRY's varying-base common-reference warning and positional universal-base contract (verified by execution: the varying-base container carries no is_reference row, so renormalizing a plot around e=-1 would shift every point).
  • Paper / source link(s): N/A
  • Any intentional deviations from the source (and why): None

Validation

  • Tests added/updated: tests/test_doc_snippets.py - removed r_comparison:block2 and troubleshooting:block8 from _CONTEXT_DEPENDENT_SNIPPETS (the rewritten blocks are self-contained and now execute under the harness). Full tests/test_doc_snippets.py + tests/test_docs_ia.py pass locally (138 passed, 5 skipped), including the r_comparison argument-mapping-table gate.
  • Backtest / simulation / notebook evidence (if applicable): N/A (notebooks are sweep PR-B)

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Migrates the narrative-docs half of the fit-time aggregate= teaching sweep
(TODO row, M-020 family) onto post-fit results.aggregate(type=):
choosing_estimator.rst drops its second full refit, python_comparison.rst
now mirrors the differences package's post-fit idiom, r_comparison.rst maps
1:1 onto R's three aggte() calls, and troubleshooting.rst's event-study-plot
remedy plots the aggregate('event_study') container with an explicit
reference_period. The troubleshooting bootstrap passage deliberately keeps
fit-time aggregate= with a comment naming it the documented until-4.0
exception (post-fit recompute levels raise on a bootstrapped fit; 'simple'
relays). The two rewritten blocks are now self-contained and removed from
_CONTEXT_DEPENDENT_SNIPPETS so the snippet harness actually executes them.
…ew R1)

Verified by execution: the varying-base default's event-study container has
no reference row (is_reference all False; e=-1 is an estimated pre-treatment
effect), so plot_event_study(es, reference_period=-1) would renormalize
around an estimate - the REGISTRY varying-base common-reference warning.
The universal-base fit marks e=-1 as the true reference, keeping the
section's specify-the-reference-explicitly lesson coherent.
…eview R2)

On gapped period grids the universal-base positional reference can sit at an
event time other than -1 (the reference_event_times multi-anchor case the
common-reference guards exist for), so the example now inspects the
container's is_reference row(s) and plots without a manual override.
@github-actions

Copy link
Copy Markdown

Overall assessment: ✅ Looks good

Executive summary

  • No P0/P1 findings.
  • Prior hard-coded e=-1 concern is resolved.
  • Aggregation, bootstrap, and positional-base guidance matches the Methodology Registry.
  • One minor copy-paste documentation issue remains.
  • Full tests could not run because pytest/numpy are unavailable; changed snippets passed static parsing.

Methodology

No findings. CallawaySantAnna behavior matches the registry’s universal/varying-base and bootstrap contracts.

Code Quality

No findings.

Performance

No findings.

Maintainability

No findings.

Tech Debt

  • Severity: P3 — informational
  • Impact: TODO.md still lists the migrated narrative sites as pending, overstating the remaining sweep.
  • Concrete fix: Prune completed RST sites while retaining the outstanding API/tutorial work.

Security

No findings.

Documentation/Tests

  • Severity: P2
  • Impact: troubleshooting.rst imports only plot_event_study but constructs CallawaySantAnna; copying the rewritten block into a fresh session raises NameError. The snippet test passes because its namespace preloads every public package symbol.
  • Concrete fix: Use from diff_diff import CallawaySantAnna, plot_event_study.

…lock (CI review)

The block constructs CallawaySantAnna but only imported plot_event_study -
a copy-paste into a fresh session would NameError; the snippet harness
masked it by preloading every public name.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 4498f15917ee588470631be186f64aec4afec0eb


Overall assessment: ✅ Looks good

Executive summary

  • No P0/P1 findings.
  • The prior missing import is resolved.
  • Aggregation, bootstrap, and positional-base guidance matches the registry and implementation.
  • Changed snippets pass static parsing; runtime tests could not run because pytest/numpy are unavailable.
  • One minor stale tracker entry remains.

Methodology

No findings. This documentation-only change correctly describes CallawaySantAnna aggregation and universal-base behavior.

Code Quality

No findings.

Performance

No findings.

Maintainability

No findings.

Tech Debt

  • Severity: P3 — informational
  • Impact: TODO.md:L81 still names the now-migrated narrative RST sites as outstanding, obscuring the remaining API-page/tutorial work.
  • Concrete fix: Remove the completed RST sites from that row while retaining the outstanding migration scope.

Security

No findings.

Documentation/Tests

No findings. The prior CallawaySantAnna import issue is fixed at docs/troubleshooting.rst:L238-L256.

… P3)

The narrative-RST half this PR completes no longer reads as outstanding;
the row now carries only the notebook half and the blocked API pages.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: eab34b30ec580571df9bf12d2fda09997a1c6878


Overall assessment: ✅ Looks good

Executive summary

  • No P0/P1 findings.
  • The prior stale TODO.md scope is corrected.
  • Aggregation, bootstrap, and positional universal-base guidance matches the registry and implementation.
  • Changed snippets parse successfully; runtime tests could not run because pytest/numpy are unavailable.

Methodology

No findings. The documentation matches CallawaySantAnna aggregation and base-period contracts.

Code Quality

No findings.

Performance

No findings.

Maintainability

No findings.

Tech Debt

No findings. TODO.md:L81 now clearly separates completed RST work from remaining notebook/API work.

Security

No findings.

Documentation/Tests

No findings. The newly unskipped r_comparison:block2 and troubleshooting:block8 are self-contained under the snippet harness and parse successfully.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Aug 11, 2026
@igerber
igerber merged commit 905fe8c into main Aug 11, 2026
14 of 15 checks passed
@igerber
igerber deleted the docs/aggregate-postfit-rst branch August 11, 2026 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant