Fixed-fit inference core: shared influence object, compressed exact CL2, exact Bell-McCaffrey df, accessor alignment - #124
Merged
Conversation
…default Ports inference-core/refund.patch (pinned at 5c895253, bundle id fixed-fit-core-2026-09-09) onto f8a8fa6. The patch's structural changes are kept (shared pffr_influence() core, compressed exact CL2, exact G x G Gram moment df, coef.pffr(sandwich = NULL) inheriting the fit, crit = "z" default, fit-time cluster =, unsupported families erroring); upstream semantics win wherever they are newer. Conflict resolutions -------------------- * R/pffr-core.R, gam_sandwich_cluster_cl2() body (the only textual conflict). HEAD had the dense per-cluster loop, extended by f8a8fa6 with the P-LB5 hat-invariant monitors; the patch replaces the whole body with a call into the influence core. Took the patch's body and re-implemented the upstream detection on top of it: the covariance now carries max_obs_leverage, min_block_eig_rel and hat_invariant_violation, and the verbatim upstream "NOT trustworthy" warning is emitted here (once per covariance call). * R/pffr-influence.R (copied file, modified). Ported the P-LB5 monitors into pffr_influence_core() so they also cover the compressed exact path: h_ii is read off H_gg = U Hsmall U' (no dense hat block), and the smallest residual-block eigenvalue is recorded relative to the block's own scale, including the unit eigenvalues of the orthogonal complement when the block is rank deficient. Both are new columns of core$diagnostics. pffr_influence_vcov() sets the attributes and calls pffr_hat_invariant_violation() with upstream's gating: the block-eigenvalue invariant only for the exact block, the per-cluster hat eigenvalue only for the shortcut, h_ii on every path. Verified on the upstream hardening fixture: the degenerate Poisson fit trips h_ii = 8.18e84 and a relative block eigenvalue of -2.1e-5 on the exact path, and both upstream P-LB5 tests still pass. * R/pffr-core.R, gam_sandwich_cluster_cl2() roxygen @returns. Kept upstream's text (documents min_block_eig_rel, max_obs_leverage and hat_invariant_violation) plus the patch's @PARAM influence. The merged man/gam_sandwich_cluster_cl2.Rd matches. * R/pffr-core.R, resolve_cl2_adjustment(). Kept upstream entirely: the 5e9 cost cap and its measurement rationale. The patch does not touch it, and the only remaining "5e8" mentions are upstream's own measured-timing sentence ("~0.03 s extra at proxy 5e8") and the upstream test comment, both consistent with the 5e9 code. * R/pffr.R, sandwich metadata. Upstream (9a4e3c8) wires cl2_adjustment into the pffr_build_metadata() call; the patch set ret$cl2_adjustment and a fresh ret$Vsandwich_cache afterwards, which is now redundant. Dropped both redundant assignments so the metadata field is set exactly once, and kept only the patch's new ret$cluster. The fit-time grouping and adjustment are inherited by coef()/predict()/plot() through $pffr as before. * DESCRIPTION. Kept PffrInferenceCore: fixed-fit-core-2026-09-09 and updated PffrBaseCommit from 5c89525... to f8a8fa6..., the actual base of this port. * NEWS.md, man/pffr.Rd, R/pffr.R prose and the test files merged cleanly; upstream's helper-exactcl2.R and test-pffr-exactcl2.R are untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The patch replaced satterthwaite_df_kernel()'s diagonal-only moment match
df = (sum_g ||q_g||^2)^2 / sum_g ||q_g||^4, clamped to [1, G], with the full
residualized Gram Gamma = diag(||q_g||^2) - T' C T. That is the intended
default, but historical Satterthwaite results were scored with the diagonal
formula, so re-scoring comparisons need it from the same q_g.
* pffr_influence_df() gains df_gram = c("full", "diagonal"); "diagonal" uses
Gamma = diag(||q_g||^2), which makes tr(Gamma)^2 / tr(Gamma^2) exactly the
deleted expression. The zero-variance NA rule and the [1, G] clamp are the
shared ones, and coincide with the deleted kernel's (for a diagonal Gram
the guard tr > 100 eps sum(q2) reduces to sum(q2) > 0).
* Threaded through pffr_df_context() (stored in the context),
pffr_df_from_context() (with an optional override) and coef.pffr(), all
defaulting to "full". Documented as a historical shortcut for re-scoring
comparisons only.
* Known-answer tests in test-pffr-inference-core.R: "diagonal" matches a
direct re-implementation of the deleted formula on the dense G = 7 fixture
(all three adjustments) and on the unequal-cluster OLS fixture to 1e-10;
"full" is bit-identical to the previous default; balanced intercept-only
OLS with G singleton clusters gives exactly G for "diagonal" and G - 1 for
"full".
* DESCRIPTION: PffrInferenceCore stays fixed-fit-core-2026-09-09 (asserted by
inference-core/run-tests.R); new PffrInferenceCoreRevision:
2026-09-17-integrated distinguishes this integrated build from the pinned
patch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ported test-pffr-inference-core.R checks the working-model moment df against clubSandwich::coef_test(..., vcov = "CR2") behind skip_if_not_installed(), so R CMD check reported it under "unstated dependencies in 'tests'". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Item 1 (warning gating regression): gam_sandwich_cluster_cl2() no longer warns whenever any block was floored. Upstream's gating is restored -- only cl2_adjustment = "shortcut" with n_capped_clusters > 0 warns, with upstream's text including the sandwich = "cluster" recommendation -- and a covariance call now emits at most ONE warning: when a hat invariant is violated only the P-LB5 warning is raised. Item 2 (df_gram = "diagonal" semantics): documented in the roxygen of pffr_influence_df(), pffr_df_context() and coef.pffr()'s df_gram, and in NEWS, that "diagonal" drops the off-diagonal residualization but reuses the covariance's own q_g, so it reproduces the historical (pre-2026-09) df exactly only together with cl2_adjustment = "shortcut". The deleted kernel is checked in verbatim as tests/testthat/helper-historical-df.R (historical_satterthwaite_df) and asserted against both at core level and through coef() on a fitted pffr fixture; the adjustment-dependent diagonal_df_reference test is retitled to say what it actually verifies. Item 3 (df_gram end to end): workflow tests that "full" is the default, that "diagonal" changes df and interval widths, and that df_gram survives coef.pffr() -> pffr_df_context() -> pffr_df_from_context(). Item 4 (NA interval limits): tests for the undefined-df branch, plus plot.pffr()/print(summary()) tolerance; the parametric-coefficient branch of coef.pffr() now also returns NA limits with a warning instead of silently substituting the Gaussian quantile, matching compute_pointwise_ci() and the documentation. Documented in NEWS. Item 5 (max_block_kappa): back to upstream's |max eig| / |min eig| (differs from max|eig| / min|eig| exactly when an eigenvalue is negative), and a rank-0 block now reports min_block_eig_rel = 1 (its residual block is the identity). Item 6 (lower-bound monitors): pffr_influence_core() records min_obs_leverage and min_hat_eig, pffr_hat_invariant_violation() flags either below -tol, both are exposed as covariance attributes and in sandwich_info, and a synthetic indefinite-bread test proves the check fires where the upper-bound monitors stay silent. Item 7 (cache key): pffr_influence() gains dof_correction / edf_type arguments (inheriting the fit by default), threaded from coef.pffr() through pffr_df_context(), and both enter the cache key, so an explicit override can no longer read back a stale expected_sampling_variance. Item 8 (docs/NEWS): NEWS records the crit = "z" default, NA limits, fit-time errors for custom-score families under sandwich = "auto", the cluster = seq_len(n) escape hatch (also in ?coef.pffr), and the newly populated sandwich_info slots; the df formula is restored to satterthwaite_df_kernel()'s roxygen and pffr_influence() gets a real @returns. Validation: testthat::test_local(filter = "pffr") 0 failed / 1114 passed (was 1033); focused installed suite 61 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…I tests Item 1 (hat-invariant monitors are CL2-only): NEWS and the sandwich_info comment in gam_sandwich_store() claimed the monitors are populated on "every cluster-robust path" and that CR1 is "equally affected". CR1 goes through gam_sandwich_cluster(), which builds no per-cluster leverage geometry at all, so it has nothing to monitor and leaves the slots NA. Both now say cl2 only, while keeping the true part -- CR1 shares the same bread, so switching to it removes the diagnostic rather than the problem. NEWS's "at most one warning per cluster-robust covariance call" is now "at most one leverage-related warning per CL2 covariance call", noting that option-validation warnings (an ignored dof_correction, say) are separate; gam_sandwich_cluster_cl2()'s @returns says the same. Item 2 (one undefined-df warning per coef() call): the smooth-term block (via compute_pointwise_ci(), once per term) and the parametric block each warned independently, so a fit with several affected blocks produced several identical warnings. Both now report through pffr_warn_undefined_df(); coef.pffr() opens a collection window with pffr_begin_undefined_df() and closes it via on.exit(), emitting one warning however many blocks are affected. Outside a window -- direct internal calls, as in the existing unit tests -- the warning still fires immediately. Nested windows keep the outermost in charge. New test drives coef() with pffr_df_from_context() mocked to return one non-finite df per block and asserts exactly one warning plus NA limits in both blocks. Item 3 (public-API tests): the cap/invariant/monitor behaviour was asserted only through refund:::pffr_vcov() and refund:::gam_sandwich_cluster_cl2(). test-pffr-inference-workflow.R now covers it the way a user meets it: pffr(sandwich = "cl2", cl2_adjustment = "shortcut") on the influential fixture warns once at fit time with the cap text while the exact path does not, and coef(crit = "satterthwaite") afterwards is silent (the covariance is computed once, at fit time); a benign cl2 fit's sandwich_info carries finite max_obs_leverage / min_obs_leverage / min_hat_eig with hat_invariant_violation NULL; the P-LB5 fixture through pffr() yields exactly one invariant warning, which coef() does not re-raise. Item 4 (wrong test comment): test-pffr-inference-core.R claimed the max_block_kappa ratio "must come out NEGATIVE" on an indefinite block. It cannot: the kernel is abs(max(values)) / abs(min(values)), so the assertion at hand is 1.5 / 0.5 = 3. Reworded to what is actually checked -- the ratio is built from the extreme eigenvalues by value, each taken in absolute value, and the signed min_block_eig is what records the indefiniteness. Item 5 (cache key): dof_correction / edf_type scale $correction only on the CR1 path (the type == "cluster" guard), so keying them for cl2 made an edf-fitted object queried as cl2 cache a second, byte-identical influence object per combination. They are now keyed as "none" for cl2 and unchanged for cluster. Item 6: PffrBaseCommit tracks the merged base 07e09ff. Validation: testthat::test_local(filter = "pffr") 0 failed / 1142 passed (was 1122); focused installed suite 61 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QSoB6DyAfeyS9PytuzQs3x
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the fixed-fit inference core (research patch
fixed-fit-core-2026-09-09)onto
pffr-refactor, on top of the merged rank guard. Exact and shortcut CL2and the working-model moment df now share one compressed per-cluster influence
object instead of three partly redundant code paths.
What this adds
per-cluster influence object (
pffr_influence()). Cached geometry is sharedwith the df calculation; sampling columns, the B₂ allowance and the
finite-cluster factor remain separate.
pffr(cluster = subject)captures one grouping label per curve, stores it,and propagates it through coefficient, prediction and plotting covariance.
Custom grouping currently requires dense responses; missing labels error.
cluster = NULLmeans "inherit"; passcluster = seq_len(<n curves>)toforce by-curve clusters.
coef.pffr(sandwich = NULL)inherits the fit-time covariance, and pointwisecritical values now default to
crit = "z"(previouslycrit = "auto",which switched to the Satterthwaite reference at
G < 150). Passcrit = "auto"or"satterthwaite"to opt in.an observation-level HC covariance — at fit time under
sandwich = "auto"as well as in the accessors. Existing approximate-score families keep their
approximation warnings.
intervals return
NAlimits with a warning rather than silentlysubstituting the Gaussian quantile.
summary()/print.summary.pffr()ignore non-finite df;
plot.pffr()draws SE bands and is unaffected.eigenvalues, condition numbers and floor counts are exposed in the core.
Saturated blocks warn;
sandwich_infocarries the formerly path-specificslots (
max_leverageon the exact path,min_block_eig/max_block_kappaon the shortcut path).pffr_influence_df()/coef.pffr()gaindf_gram = c("full", "diagonal").No automatic thresholds are relaxed. The
G <= 150, max D_g <= 500CL2selection rule and the exact/shortcut rule remain computational heuristics and
provide no calibration guarantee.
P-LB5 hat-invariant monitors on the compressed path
Upstream's P-LB5 monitors (
max_obs_leverage,min_block_eig_rel,hat_invariant_violation) were re-implemented insidepffr_influence_core()/pffr_influence_vcov()rather than left on the olddense path, so they fire on the compressed exact path too. The check also
covers the lower bounds (
h_ii >= 0,eigen(H_gg) >= 0), so an indefinitepenalized bread is detected instead of slipping past the upper-bound monitors;
a synthetic indefinite-bread test proves it fires where the upper-bound
monitors stay silent. At most one leverage-related warning is emitted per CL2
covariance call: the invariant warning when an invariant is broken, otherwise
the shortcut leverage-cap warning (the exact path's
(1 - leverage_cap)^2residual-eigenvalue floor is a routine safeguard and stays silent, as
upstream). These monitors are CL2-only —
sandwich = "cluster"(CR1) forms noper-cluster leverage geometry, so it has nothing to monitor and leaves the
slots
NA.Caveat:
df_gram = "diagonal"df_gram = "diagonal"drops the off-diagonal residualization but reuses thesame
q_gas the requested covariance. The historical (pre-2026-09)Satterthwaite df always used the shortcut leverage weight
(I - H_gg)^{-1/2},so
"diagonal"reproduces those historical numbers exactly only incombination with
cl2_adjustment = "shortcut". The deleted historical kernelis checked in verbatim as
tests/testthat/helper-historical-df.Rand assertedagainst, both at core level and through
coef()on a fittedpffrfixture.Validation
testthat::test_local(filter = "pffr"): 0 failed, 0 errors, 1142 passed(1 skipped:
clubSandwichnot installed; 1 known gaulsscoefbootgradient warning).
R CMD check(remote = FALSE, no manual): 0 errors, 3 warnings, 2 notes —all pre-existing (Rd macros in
compute_scat_scores.Rd, lost braces inpffr_jackknife_core.Rd, undeclarednumDeriv/withrtest imports,.github, aqtglobal). Vignette build and re-build clean.Review
Two adversarial review rounds with three external models. Round 1 drove the
fixes recorded in the branch's NEWS entry (warning gating restored to
upstream's,
df_gramsemantics documented and pinned by tests,NAintervallimits in the parametric branch too,
max_block_kappaback to upstream'sdefinition, lower-bound monitors,
dof_correction/edf_typein the influencecache key). Round 2 found no regressions and produced this branch's last
commit: the hat-invariant documentation corrected to say CL2-only, the
undefined-df warning unified so one
coef()call warns at most once howevermany blocks are affected, public-API tests for the cap/invariant/monitor
behaviour through
pffr()andcoef(), and a cache key that no longer storesa duplicate influence object when an edf-fitted model is queried as CL2.
🤖 Generated with Claude Code
https://claude.ai/code/session_01QSoB6DyAfeyS9PytuzQs3x