Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ print(problem.names, sampler.results.logz[-1])
- [`docs/recipes.md`](docs/recipes.md): every supported use case with its
spelling and expected behaviour. Each recipe is a test under
`test/recipes/`.
- [`examples/`](examples/): nine notebooks, the tutorials for the recipes,
- [`examples/`](examples/): the notebooks, the tutorials for the recipes,
from a line to an error-model comparison on real α + ⁴⁴Ca data and a
hierarchical calibration.
- [`docs/design.md`](docs/design.md): the maintainer's description of the
Expand Down
1 change: 1 addition & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ Covariance terms
rxmc.terms.proportional_error
rxmc.terms.systematic
rxmc.terms.kernel
rxmc.terms.kernel_param_names
rxmc.terms.ones
rxmc.terms.ym
rxmc.terms.averaging
Expand Down
10 changes: 8 additions & 2 deletions docs/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ proportional_error(parameter, averaging=False, log=True, on=None)
systematic(parameter, basis, log=True, basis_params=(), on=None, coords=None)
kernel(kernel, coords=None, amplitude=None, amplitude_params=(), jitter=1e-10,
prefix="discrepancy", params=None, on=None) -> KernelTerm
kernel_param_names(kernel) -> list[str] # the keys of a params= mapping
# bases and amplitudes: ones, ym, averaging, x_basis(scale), exp_growth(scale, base=ones),
# constant_amplitude, exp_growth_amplitude(scale)
```
Expand All @@ -253,7 +254,11 @@ reads `c.ym`, never `c.y`. `kernel` derives one `Parameter` per free
hyperparameter element in sklearn's log-theta space, bounded by the log of
the kernel's bounds so it compiles with a uniform prior there; `params=`
passes the objects instead, which is also how hyperparameters are shared
between per-comparison kernels. Two kernel terms with derived names and
between per-comparison kernels. `params=` is a mapping keyed by
`kernel_param_names(kernel)` (sklearn's `theta` order is alphabetical by
attribute name); a positional list is accepted only for a kernel with one
free hyperparameter, because with two or more a misread order would swap
priors silently. Two kernel terms with derived names and
one prefix fail compile on the duplicate name.

Hierarchy is sharing plus `meta`: a hyperparameter shared by several
Expand Down Expand Up @@ -652,7 +657,7 @@ Two tiers, chosen per assertion.
Three index tests keep the documents honest: `test_recipes_index.py`
(one file per `## NN.` heading, each quoting its recipe; the recipe-18
legend equal to the tests' legend), `test_notebooks_index.py` (each
notebook cites the recipes of the design's table, all nine exist), and
notebook cites the recipes of the design's table, and every one exists), and
`test_regression.py` (the 0.x pins). `test/helpers.py` holds the dense
references and `STUDY_LEGEND`, the labelled error-model forms of recipe
18 built against hand-written matrices.
Expand All @@ -670,6 +675,7 @@ a time unless noted.
| `sharing_error_models` | 5 | dynesty | two experiments with opposite normalisation defects: sharing a parameter, a mode per dataset, one mode spanning both, and the assembled covariance seen directly; one panel per error model | 102 s |
| `normalization_and_covariance_structure` | 3, 4, 6, 27 | dynesty | six treatments of four experiments' normalisations on a cubic with a free, non-zero constant term, one of them badly mis-quoted and alone in its range; Peelle's puzzle in the two-point case it was found in; a gallery of covariance structures from `matrix(theta)` | 1003 s |
| `gp_discrepancy` | 7 | emcee, dynesty | mean-zero discrepancies with amplitudes growing in x: four rungs on a toy line, three on n+⁴⁰Ca missing its surface absorption; the three predictive objects (model plus discrepancy, plus experimental, and the conditioned regression it does not use), with the equations | 483 s |
| `spectral_mixture_discrepancy` | 7, 33, 40 | MAP, emcee | a defect with two scales on a line, a three-component spectral-mixture discrepancy written as a user sklearn kernel and keyed `params=`; periodogram-seeded multistart MAP; each component backed out with `gp_predictive_draws(conditioned=True)`; the identifiability of θ, with the L² projection θ* as the only recoverable target; Plumlee's orthogonal kernel as a second user kernel; a chain against each Laplace approximation, which agrees for Plumlee and is several times too narrow and misplaced for Kennedy–O'Hagan | 930 s |
| `robust_likelihoods` | 9, 12, 39 | emcee | Student-t versus Gaussian on three gross outliers; the iterative rejection loop, including the round that over-rejects and recovers; tempering a correctly specified 200-point line, with the prior on the corner plot and the predictive coverage | 80 s |
| `error_scale_and_usu` | 34 | emcee | a global scale on the reported errors under both likelihoods; a USU offset on the technique we suspect | 90 s |
| `local_optical_model_calibration` | 10, 14, 15, 16, 17, 18, 19, 21, 40 | dynesty | EXFOR O1199007, p + ⁴⁰Ca at 35 MeV, which quotes no systematics, read from a plain dict: the unit contract; log-space residuals with the jitr α+Ca error model (reported statistics plus inferred point-to-point and common relative errors, as custom terms) against linear residuals with the reported errors alone, compared by evidence with the Jacobian, by predictives for the potential alone (on a grid) and with every error term (at the measured angles), and by three coverage curves each; the singular guard | 929 s |
Expand Down
1 change: 1 addition & 0 deletions docs/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Beyond the Gaussian
:maxdepth: 1

examples/gp_discrepancy.ipynb
examples/spectral_mixture_discrepancy.ipynb
examples/robust_likelihoods.ipynb
examples/error_scale_and_usu.ipynb

Expand Down
3 changes: 2 additions & 1 deletion docs/groundup_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,8 @@ kernel(kernel, coords=None, amplitude=None, amplitude_params=(), jitter=1e-10,
# KernelTerm(Term) adds kernel, n_kernel, amplitude, jitter so predictive.gp_predictive_draws
# can condition the discrepancy from the term alone. A derived hyperparameter is
# bounded by the log of the kernel's bounds (a uniform prior in log-theta).
# params=: the hyperparameter Parameter objects, one per free element in kernel.theta order;
# params=: the hyperparameter Parameter objects, keyed by name (kernel_param_names; positional
# only for a single free hyperparameter; superseded detail: see design.md);
# None derives fresh ones named f"{prefix}_{name}". Pass the same objects to share
# hyperparameters between per-block kernels. Two kernel terms with derived
# names and the same prefix fail compile on the duplicate name; the error says
Expand Down
6 changes: 6 additions & 0 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -638,6 +638,12 @@ Expected behaviour:
- Without `params=[ell]`, each `kernel` call would derive its own length
scale, one per dataset. That is also a legitimate model; it is just not
this one.
- `params=[ell]` is positional because Matern here has one free
hyperparameter. A kernel with several takes a mapping keyed by
`T.kernel_param_names(k)`, e.g. for `ExpSineSquared`
`params={"length_scale": ell, "periodicity": per}`;
a positional list for it is refused, since sklearn orders hyperparameters
alphabetically and a misread order would swap priors silently.
- The comparisons stay uncorrelated, so the constraint is on the structured
path.

Expand Down
2 changes: 1 addition & 1 deletion examples/normalization_and_covariance_structure.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@
"id": "240f15bb-1321-402c-915b-ef269093abfd",
"metadata": {},
"source": [
"### Generate synthetic data"
"## Generate synthetic data"
]
},
{
Expand Down
1,023 changes: 1,023 additions & 0 deletions examples/spectral_mixture_discrepancy.ipynb

Large diffs are not rendered by default.

169 changes: 124 additions & 45 deletions src/rxmc/covariance.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,19 @@
``B`` must be positive definite. A block covered only by modes is singular in
``B`` even when ``Sigma`` is not; when every term is constant this is caught at
construction and reported with the block's label.

A covariance that is not finite at a ``theta`` (a term returning inf or NaN,
or values whose squares, sums or low-rank products overflow) is zero density
there: :meth:`distance` returns a non-finite ``d2`` or ``logdet`` and the
compiled constraint reads it as ``log_likelihood == -inf``. The likelihood
path never scans terms for this: the factorisations run unchecked and a
non-finite input always yields a non-finite ``logdet``. A constant term is
checked once, at construction, and a non-finite contribution raises
``ValueError`` naming it. :meth:`matrix` checks too and raises, because a
predictive has no "zero density" to return. One gap remains: constant terms
whose contributions are each finite but whose sum overflows, in a block a
parametric term also touches, are zero density everywhere rather than an
error at construction.
"""

from __future__ import annotations
Expand All @@ -49,9 +62,16 @@ class _SingularCovariance(ValueError):
"""


def chol_logdet(Sigma):
"""Lower Cholesky factor and log-determinant of a positive-definite matrix."""
L = sla.cholesky(np.asarray(Sigma, dtype=float), lower=True)
def chol_logdet(Sigma, check_finite=True):
"""Lower Cholesky factor and log-determinant of a positive-definite matrix.

``check_finite=False`` skips scipy's scan: a non-finite ``Sigma`` then gives
a non-finite factor and log-determinant (or ``LinAlgError``) instead of
``ValueError``.
"""
L = sla.cholesky(
np.asarray(Sigma, dtype=float), lower=True, check_finite=check_finite
)
return L, 2.0 * float(np.sum(np.log(np.diag(L))))


Expand Down Expand Up @@ -156,12 +176,14 @@ def __init__(self, entries, x, y, offsets, active, *, meta=None, labels=None):
self.constant_entries = [e for e in self.entries if e.term.is_constant]
self.parametric_entries = [e for e in self.entries if not e.term.is_constant]
self.is_constant = not self.parametric_entries
self._D0, self._U0, self._M0, self._X0 = self._pieces(
self.constant_entries, None, None
)
with np.errstate(over="ignore", invalid="ignore"):
self._D0, self._U0, self._M0, self._X0 = self._pieces(
self.constant_entries, None, None, check=True
)
self._cache = None
if self.is_constant:
self._factor(self._D0, self._U0, self._M0, self._X0)
with np.errstate(over="ignore", invalid="ignore"):
self._factor(self._D0, self._U0, self._M0, self._X0)
elif not self.dense:
# modes never enter B, so a block no parametric diag or matrix term
# touches has a constant B: a singular one fails now, by label, rather
Expand All @@ -170,8 +192,12 @@ def __init__(self, entries, x, y, offsets, active, *, meta=None, labels=None):

# -- assembly -------------------------------------------------------------

def _pieces(self, entries, ym, theta):
"""``(D, U_columns, M_blocks, cross_blocks)`` from a set of entries."""
def _pieces(self, entries, ym, theta, check=False):
"""``(D, U_columns, M_blocks, cross_blocks)`` from a set of entries.

``check`` raises ``ValueError`` naming the first term whose contribution
is not finite on its active rows; the likelihood path leaves it off.
"""
n = self.n_active
D = np.zeros(n)
U = []
Expand All @@ -190,14 +216,20 @@ def _pieces(self, entries, ym, theta):
labels=e.labels,
)
pos, keep = e.pos[e.keep], e.keep
if t.kind == "matrix":
part = v[np.ix_(keep, keep)]
else:
part = v[keep] ** 2 if t.kind == "diag" else v[keep]
if check and not np.all(np.isfinite(part)):
self._nonfinite(t, theta is None)
if t.kind == "diag":
D[pos] += v[keep] ** 2
D[pos] += part
elif t.kind == "mode":
col = np.zeros(n)
col[pos] = v[keep]
col[pos] = part
U.append(col)
else:
sub = v[np.ix_(keep, keep)]
sub = part
if e.block is None:
X.append((pos, sub))
else:
Expand All @@ -211,11 +243,11 @@ def _pieces(self, entries, ym, theta):
M[b][np.ix_(local, local)] += sub
return D, U, M, X

def _assemble(self, ym, theta):
def _assemble(self, ym, theta, check=False):
"""Constant pieces plus the parametric ones at ``theta``."""
if self.is_constant:
return self._D0, self._U0, self._M0, self._X0
D, U, M, X = self._pieces(self.parametric_entries, ym, theta)
D, U, M, X = self._pieces(self.parametric_entries, ym, theta, check)
D = D + self._D0
U = list(self._U0) + U
M = [
Expand All @@ -237,12 +269,17 @@ def _dense_matrix(self, D, U, M, X):
# -- factorisation ---------------------------------------------------------

def _factor(self, D, U, M, X):
"""Factor the covariance; cached when constant."""
"""Factor the covariance; cached when constant.

Unchecked: a non-finite input gives a non-finite ``logdet`` (or a
``LinAlgError``, depending on where LAPACK meets it), both of which the
compiled constraint reads as zero density.
"""
if self.is_constant and self._cache is not None:
return self._cache
try:
if self.dense:
L, logdet = chol_logdet(self._dense_matrix(D, U, M, X))
L, logdet = chol_logdet(self._dense_matrix(D, U, M, X), False)
factors = ("dense", L, logdet)
else:
blocks = []
Expand All @@ -255,11 +292,14 @@ def _factor(self, D, U, M, X):
B = np.diag(D[pos])
if M[b] is not None:
B = B + M[b]
L, ld = chol_logdet(B)
L, ld = chol_logdet(B, False)
logdet += ld
W = (
sla.solve_triangular(
L, np.column_stack([c[pos] for c in U]), lower=True
L,
np.column_stack([c[pos] for c in U]),
lower=True,
check_finite=False,
)
if U
else np.zeros((pos.size, 0))
Expand All @@ -268,7 +308,7 @@ def _factor(self, D, U, M, X):
Ws.append(W)
if U:
W = np.vstack(Ws) if Ws else np.zeros((0, len(U)))
Ls, lds = chol_logdet(np.eye(len(U)) + W.T @ W)
Ls, lds = chol_logdet(np.eye(len(U)) + W.T @ W, False)
logdet += lds
else:
Ls = None
Expand All @@ -277,9 +317,33 @@ def _factor(self, D, U, M, X):
error = ValueError if self.is_constant else _SingularCovariance
raise error(self._singular_message(D)) from err
if self.is_constant:
if not np.isfinite(logdet):
raise ValueError(
"the constraint's constant covariance is not finite: its "
"terms are finite one by one but overflow when combined "
"(e.g. a mode of magnitude ~1e200)"
)
self._cache = factors
return factors

def _nonfinite(self, term, constant: bool):
"""Raise ``ValueError``: ``term`` contributed inf or NaN on its active rows."""
msg = f"{term!r} contributes a non-finite value on its active rows"
if constant:
raise ValueError(
msg + "; a constant term is evaluated once, so this is an error "
"in its declaration (an array with NaN, values whose squares "
"overflow, or a fn that divides by zero on the data)"
)
raise ValueError(msg + " at this theta")

def has_nan(self, ym, theta) -> bool:
"""Whether the assembled covariance at ``theta`` contains NaN (diagnostic)."""
with np.errstate(over="ignore", invalid="ignore"):
D, U, M, X = self._assemble(ym, np.asarray(theta, dtype=float))
arrays = [D, *U, *(m for m in M if m is not None), *(x for _, x in X)]
return any(np.isnan(a).any() for a in arrays)

def _constant_blocks(self) -> list[int]:
"""The blocks whose ``B`` no parametric diag or matrix entry touches."""
varying = set()
Expand Down Expand Up @@ -328,40 +392,55 @@ def distance(self, ym, theta=()):
r"""``(d2, logdet)`` of the residual ``y - ym`` on the active rows.

``ym`` is the full-length stacked prediction; ``theta`` the flat
parameter vector the entries' gathers index into.
parameter vector the entries' gathers index into. Either value is
non-finite where the covariance (or the residual) overflows: zero
density, read as such by the compiled constraint.
"""
theta = np.asarray(theta, dtype=float)
r = (self.y - np.asarray(ym, dtype=float))[self.active]
kind, *rest = self._factor(*self._assemble(ym, theta))
if kind == "dense":
L, logdet = rest
z = sla.solve_triangular(L, r, lower=True)
return float(z @ z), float(logdet)
blocks, Ls, logdet = rest
d2 = 0.0
w = None
for blk in blocks:
if blk is None:
continue
pos, L, W = blk
z = sla.solve_triangular(L, r[pos], lower=True)
d2 += float(z @ z)
w = W.T @ z if w is None else w + W.T @ z
if Ls is not None and w is not None:
s = sla.solve_triangular(Ls, w, lower=True)
d2 -= float(s @ s)
return d2, float(logdet)

def matrix(self, ym, theta=(), entries=None):
"""The dense covariance on the active rows, for display and tests.
with np.errstate(over="ignore", invalid="ignore"):
kind, *rest = self._factor(*self._assemble(ym, theta))
if kind == "dense":
L, logdet = rest
z = sla.solve_triangular(L, r, lower=True, check_finite=False)
return float(z @ z), float(logdet)
blocks, Ls, logdet = rest
d2 = 0.0
w = None
for blk in blocks:
if blk is None:
continue
pos, L, W = blk
z = sla.solve_triangular(L, r[pos], lower=True, check_finite=False)
d2 += float(z @ z)
w = W.T @ z if w is None else w + W.T @ z
if Ls is not None and w is not None:
s = sla.solve_triangular(Ls, w, lower=True, check_finite=False)
d2 -= float(s @ s)
return d2, float(logdet)

def matrix(self, ym, theta=(), entries=None, check=True):
"""The dense covariance on the active rows, for display and predictives.

``entries`` restricts the sum to a subset of :attr:`entries` (the
objects themselves, compared by identity): the covariance of *part* of
the error model, as a predictive draw of the model plus its discrepancy
but not the experimental errors needs. Default: every entry. A subset
is assembled directly, without the constant-piece cache.

Raises ``ValueError`` if the matrix is not finite, naming the term when
one term alone is responsible; ``check=False`` returns it as is.
"""
theta = np.asarray(theta, dtype=float)
if entries is None:
return self._dense_matrix(*self._assemble(ym, theta))
return self._dense_matrix(*self._pieces(entries, ym, theta))
with np.errstate(over="ignore", invalid="ignore"):
if entries is None:
pieces = self._assemble(ym, theta, check=check)
else:
pieces = self._pieces(entries, ym, theta, check=check)
Sigma = self._dense_matrix(*pieces)
if check and not np.all(np.isfinite(Sigma)):
raise ValueError(
"the covariance is not finite at this theta: every term is "
"finite on its own, but their sum or a low-rank product overflows"
)
return Sigma
Loading
Loading