Repository navigation
Conversation
Remove the 0.x tests, notebooks, in-package samplers, drivers and the declaration classes. The eight modules the rewrite harvests stay in place for one commit so that git mv can carry their history into the new layout. The 0.x package remains at v0.1.0 and on legacy/0.x.
git mv only, no edits: likelihood_model -> likelihood, covariance -> terms, observation_from_measurement -> units, model_comparison -> diagnostics, the two reaction observation modules -> reactions/. Their bodies are rewritten in milestones 1, 5 and 6.
Minimal package init and smoke test; pytest fast/slow tiers (slow marker, deselected by default); py312 lint targets; pandas and scikit-learn out of the runtime requirements and into the examples extra with dill; docs reduced to the design and recipes pages; a README for the branch; CI with format, tests and docs jobs on main and rewrite plus a scheduled slow job; a trusted-publishing workflow on version tags.
…prior Frozen dataclass with eq=False, so sharing is by object; bounds validated and coerced; prior, unit and latex carried for Problem and plotting.
Drop the contextual flag, the context keyword and per_observation_scaling; scale() uses the new Parameter signature. Transform, composition, log/exp and the finite-difference derivative are unchanged.
The measurement adapters return inside from_measurement in milestone 5.
Gaussian, StudentT(nu=None) with a bounded default parameter, Chi2 and the log_likelihood helper; the Cholesky helper moves to covariance in milestone 3. Test helpers gain a dense reference assembly used by the term tests now and the structured covariance tests later.
Term is a frozen dataclass: fn, params, kind, on, coords, constant. No bind, no support indices, no caches; value() evaluates on a given support and compile resolves on= to rows. TermContext gains meta(key) for per-point dataset metadata. Factories keep their bodies with support= renamed on= and the _term suffix dropped; kernel() accepts params= to share hyperparameters between comparisons. Tests port TermKinds, Coords, Factories, Kernel and the alpha+Ca study forms against a hand-built dense reference, plus the term-level halves of recipes 19 and 27.
x4i3 converts every EXFOR cross section to barns while parsing and exfor_tools labels the result from a three-entry table, so measurement unit labels come from a small fixed vocabulary. parse_unit(label) maps that vocabulary (plus the obvious spellings) to a factor into the internal unit and a quantity kind, and rejects anything else loudly. The 0.x pint path could not parse the real labels (barns/ster, b/Sr); it only worked for hand-built measurements.
Frozen, identity-hashed; x is opaque beyond its first dimension; y_err validated; norm_err and offset_err stored as a float or an (n,) array; meta copied. No comparison transform, mask or solver state.
Model(fn, params) binds to a grid by closing over it; subclasses override bind() to build solvers from meta. Composition happens on the bound predictors: | applies a mean transform after the prediction, + and * are x-dependent corrections, parameters concatenated left to right. polynomial(order) is the generic example.
Comparison binds a model to a dataset in a parameter-free space, with the delta-method errors and the per-point log-Jacobian as constants; reported_terms() turns the dataset's offset and normalisation systematics into rank-one modes, propagated at the data and at the prediction respectively. Constraint holds the comparisons, terms, likelihood, weight and per-comparison masks; masked, masked_where and complement share every object with the original. Every term's on= is resolved eagerly and an array term's shape is checked against its support.
The study labels (L0, E0, L1, ...) meant nothing to a reader who had not seen the study. test/helpers.py now carries STUDY_LEGEND, one line per label, and study_form(label, x, y, ym) builds the terms, values and the hand-built dense reference; the term tests parametrise over it and the structured covariance tests will too. Recipe 18 and the design document point at the legend.
Sigma = diag(D) + blockdiag(M_b) + U U^T over the active rows of a constraint: diag terms into D, mode terms as columns of U (a mode across comparisons is one column with entries in several blocks), matrix terms into their block, and a matrix term that crosses blocks forces the dense path. Per-block Cholesky plus a rank-r correction gives (d2, logdet); constant pieces are evaluated once and a fully constant covariance is factored once, with a singular block reported by label at construction. Verified against the hand-built dense reference on every study form, across blocks, with masks, and on the dense fallback.
Recipe 18 now carries the full legend table; test/helpers.py keeps an identical copy and says so, and the design document cites the recipe. A docs reader never has to open the test suite.
ParameterIndex assigns each distinct Parameter a slot in first-seen order (predictors, terms, likelihood, constraint by constraint) and rejects duplicate names with a hint. The prior is assembled so every slot is covered exactly once: a marginal truncated to its bounds, the uniform on finite bounds, or one joint block; a frozen multivariate normal gets a whitening unit-cube map, other joints supply their own. Compiled constraints stack the comparisons, check finiteness on the active points, stack per-point metadata for terms, resolve every term's rows and gather, and build the structured covariance. Problem exposes log_prior, log_likelihood (weighted sum), log_posterior (prior first), prior_transform, sample_prior, predict, chi2, columns and the black-box-bayes aliases, and pickles with dill. Tests cover the index, the prior rules, compile errors, masked views, and emcee, dynesty and dill drivers.
One test file per recipe under test/recipes, docstring quoting the recipe's intent, assertions its expected-behaviour bullets that need no converged chain; a linear-Gaussian oracle supplies closed-form posteriors for the linear recipes. offset() and normalization() now take the nuisance parameter as their first positional argument, like every other nuisance factory and like the recipes spell them; magnitude= stays a keyword.
…heck Recipe tests for external samplers, custom terms, preprocessing, the singular-covariance error, shared hyperparameters, a GP across energies and the hyperprior block. Two library changes they exposed: a joint prior block may introduce a parameter no model or term uses (its hyperparameter), which now gets a slot after the constraints' parameters; and the block factor B is checked at construction whenever every diag and matrix term is constant, since modes never enter B, so a block covered only by modes fails by label before any likelihood call. Term is exported from the package.
SafeBayes prefix differences equal the Gaussian conditional; KDUQ deltas are linear fractions (log=False, also in the recipe); Peelle's puzzle is pinned by the two-point closed form and by the ordering data-built < live prediction-built < t0 across replicates, with the recipe text corrected to say the live mode keeps a log-determinant pull that the t0 refit removes; cut posteriors by closure and by module weights; an emulator term seeing the model's values; MAP + Laplace against the oracle; error scale and USU modes; energy-dependent parameters; Legendre modes; and the eight-schools model in its three spellings.
GLS with Sigma = sigma^2 I + s^2 y y^T collapses by Sherman-Morrison to t = ybar / (1 + (s/sigma)^2 sum (y_i - ybar)^2), exact per dataset and sampler-free; the test asserts it, that identical points give no bias, and that replicate means lie between the leading-order expectation 1/(1 + (n-1) s^2) and the truth. The recipe text now quotes that form instead of the 1/(1 + n s^2) factor it had copied unverified.
ElasticXS and IsobaricAnalogPN build their jitr workspace from the dataset's kinematics in bind() and return a Predictor closing over it; the elastic basis is cached per model and energy so a plotting grid or a held-out view reuses it, and the cache is dropped on pickling. The harvested set_up_solver functions take one grid; rutherford() is the closed form of the kinematics and momentum_transfer() a free function. No compound-correction hook. Real-solve tests with lmax=10 pin finiteness, units, the cache, composition, solver-setting forwarding, the Lane-term IAS, and a dill round trip of a reaction Problem.
Reads the exfor_tools fields, converts angles to radians and cross sections to b/sr through the fixed label table, converts dXS/dA to and from dXS/dRuth with the closed-form Rutherford cross section (a per-angle norm), scales the dimensionful errors with the data and leaves the fractional normalisation error alone, and fills meta with the kinematics a reaction model needs. Every failure names what is wrong.
Reported systematics become modes only when asked for; an EXFOR measurement lands in b/sr and radians with the Rutherford conversion as a per-angle norm; one elastic model serves the data grid and a plotting grid from one cached basis; a cross section and an analysing power from one measurement share an angle-calibration mode across comparisons. Recipe 37 now notes that a spanning basis sees the gathered stack and must split a finite difference at the seam by per-point metadata.
A term on several comparisons sees their rows gathered into one stack, so a basis that differentiates or smooths along the grid would straddle the seam. TermContext now carries segments (the slice of the support belonging to each spanned comparison, in constraint order), labels, and split(a); the covariance computes them once per entry from the block offsets. Outside a problem a term has one segment. Recipe 37 takes its angle-calibration slope per segment, and the design and recipe docs describe the new fields.
A kernel term closed over its sklearn kernel and nothing on the Term exposed it, but the predictive band promised by recipe 7 must condition the discrepancy from the term alone. KernelTerm is a Term subclass carrying the kernel, the count of free hyperparameter elements (so the log-theta and amplitude columns follow from params), the amplitude and the jitter; the covariance treats it as any matrix term.
… score predictive_draws and heldout_log_predictive take a compiled Problem and chain rows in problem.names order; coverage, sharpness, the log posterior predictive and the evidence bookkeeping are lifted from 0.x. A held-out problem built from fit.complement() scores the marginal of its rows, which is wrong when a term spans the split (a GP over several experiments); given=fit_problem computes the Gaussian conditional under the full covariance instead, and equals the marginal when nothing spans. split_samples and the module-level log_jacobian are gone: columns come from problem.columns and the Jacobian from problem.log_jacobian().
total_predictive_band(problem, term, predictor, x_pred, samples) locates the kernel term's constraint and covariance entry, takes its training rows, comparison space, kernel and amplitude columns from the problem, and conditions the discrepancy with the rest of the declared covariance as the regression noise, so a chain with nuisance columns needs no column arithmetic. The amplitude is evaluated at the prediction grid through a TermContext; physical=True maps draws back through the space's inverse. GP conditioning against sklearn is lifted from 0.x.
The two long-NUTS references were accepted under the classic R-hat; rerun with the same seeds under the rank-normalised one they pass (1.001 and 1.001, min bulk ESS 2475 and 3920) with means and widths identical to before, so no row scored against them changes.
…98 s -> 518 s) The cross-section ladder's posterior has one mode, so pocomc runs at the benchmark's single-mode setting, 1024/512, rather than the 4096/2048 that separated modes need. Gate against the committed run at 4096/2048 and other seeds: log Z -467.18 / 96.73 / 103.88 against -467.21 / 96.78 / 103.91, Delta log Z 563.9 and 7.1 with the same verdicts, all twelve medians within 0.07 posterior sd, envelope fraction 0.56 against 0.58. The caveat now says the two runs agree, and that two runs of one algorithm cannot reveal a shared bias. Intervals are 68 %, as in the sibling notebooks; the corner plots are density-normalised; two plot labels use mathtext.
…(3845 s -> 1342 s) - The log-space error model is median one: point_to_point = noise(log sigma_A) and common = offset(log sigma_B), log-space standard deviations with log-uniform priors on [0.05, 2], in place of the jitr mean-one moment map written as custom terms. The derivation says why median one and gives E[y/ym] correctly; sigma_A = 0.197 [0.176, 0.222], a factor of about 1.22 per angle; log Z 198.9, still about 730 above the linear model. - The NUTS cross-check, unconverged by design, is gone; the benchmark's numbers say why a gradient sampler started at the MAP is no check here. - One log-space run (n_seeds = 1, raise it for the spread); its island split is 15 %, as the benchmark's reference. - The linear model, with one narrow mode, runs pocomc at 1024/512. Gate against the committed run: log Z -526.69, Vv 45.4 [45.1, 45.6] at 68 % (the old 90 % interval scaled), chi2/N 35.2, coverage 0.06 / 0.26. - predictive_draws(physical=True) replaces the hand-written exp; 68 % intervals; coverage on the 0.05 grid; the predictives are 'model only', '+ statistical', '+ point-to-point' and 'full predictive'; the duplicate coverage table goes. Recipes 10, 11, 14, 15, 16, 17, 18, 21, 40.
The check against the earlier run at 4096/2048 shows the resolution is enough; both runs used seeds 10, 11 and 12, as 3ab75f6's message also wrongly said otherwise. Text only.
…53 s -> 1765 s) - The error model is median one, T.noise(log sigma_A) + T.offset(log sigma_B) with log-uniform priors on [0.05, 2], replacing the jitr moment map; sigma_A, sigma_B and the GP amplitude A are all log-space standard deviations and are compared directly. Their posterior correlations are printed: log sigma_A and log A are only weakly correlated (-0.10, -0.14), so the data do tell the uncorrelated error and the discrepancy apart. - The growth power p is uniform on [0, 16]; its posterior, 7.5 [4.4, 11.5], no longer presses against the bound. The verdict holds: Delta log Z = 6.96 +/- 0.05, power of q favoured. - One pocomc run per model (n_seeds = 1). A scratch run with n_seeds = 3 (outside the repo; its first runs reproduce these exactly) gave per-run log Z within 0.09 and 0.04, Delta log Z 6.95 +/- 0.05, and family weights 0.45/0.45/0.45 and 0.73/0.75/0.73; the prose quotes it, from this model and prior. - 68 % bands, coverage on the 0.05 grid, 'model only' and 'full predictive', the Rutherford-ratio label, 'power of $q$' in plot labels, jnp.sin in power_of_q, and the coverage title naming its 73 points. Recipe 19 is no longer cited (no custom Term remains).
- gp_discrepancy: the growing-amplitude toy fit had landed at R-hat 1.010 (not converged by the notebook's own rule) while the prose said every fit passed; the toy emcee runs are now 40 000 steps and all converge (R-hat at most 1.005, runs of at least 174 tau). The half-widths at x = 0.2 and 5, the toy envelope fractions (81 % / 68 %) and the toy intervals are read from this run; a sentence says the single-mode pocomc setting can bias GP hyperparameters on the benchmark. 588 s. - local_optical: the Rutherford-ratio label follows the shared convention; the benchmark's omp problem is called its version of this problem, with an earlier parametrisation of the two errors, not 'this very problem' (Phase 4 makes them identical); the island-weight sentence cites the benchmark's 4096/2048 runs, not stale two-run numbers; the counting-error shift is at most 0.15 %; 'model only' throughout. 1324 s. - alpha_ca: alpha_0 for the starting parameters (theta is the angle); the three-run numbers are labelled as an uncommitted n_seeds = 3 run, with LaTeX minus signs; 'a factor of 1.6 to 1.7'. 1760 s. - docs/benchmarks.md describes omp as using the earlier parametrisation.
- local_optical: the benchmark's omp problem differs in the parametrisation *and the priors* of the two errors, so its posterior and evidence differ a little (log Z -6.63 against -6.71); said so here and in design.md. Recipe 11 is dropped from its list: the notebook only points to it, it does not hold data out. - alpha_ca: the three-run spread (0.09 for the constant amplitude) is about three times pocomc's quoted 0.03, so the quoted errors understate it; the verdict still does not depend on the seed. Text only; no rerun.
- omp is now exactly local_optical's log-space problem (median-one noise + offset errors on log-uniform log-space sds), and two problems join it: alpha_gp, exactly alpha_ca's power-of-q GP problem, and n40ca_gp, exactly gp_discrepancy's GP rung (via n40ca_ladder(), which builds all three rungs). Each was checked against the notebook's own cells: identical log posterior at five prior draws (to 4e-11 for omp). - Their references pool three pocomc runs at 8192/4096 and are corroborated by one dynesty run (2000 live points); a disagreement is recorded (corroborated: false) rather than refused. - Parallel-safe output: --references gives each reference job its own file; merge_results.py merges per-job files into the committed results, refusing two inputs with the same entry. - sweep_pocomc.py sweeps pocomc's resolution on a problem (saving each run, resumable, mode share where there are two modes); gp_seed_check.py reruns gp's ladder at its settings with three other seed sets. - benchmarks/hpc/: a SLURM job script, site settings, submit_all.sh (31 jobs) and the runbook. Tests for the problems and the merge; the drivers end to end in the slow tier. docs/benchmarks.md lists the problems and notes the pending rows.
The references, sweeps and seed check ran their pieces in sequence inside one job; now every independent run is an array task (72 in 10 array jobs): a reference's three pocomc seeds and its dynesty corroboration, each sampler, each (setting, seed) of a sweep, each (seed set, rung) of the gp check. benchmarks/hpc/task.py maps $SLURM_ARRAY_TASK_ID to its run and writes results/hpc/<kind>/<problem>/<index>.json; assemble.py pools the reference parts (reference.pool_parts, split out of long_pocomc, which is unchanged in behaviour) and joins the sweep and seed-check pieces, skipping any problem with a missing piece. submit_all.sh sizes each array from task.py --count. Tests cover the index mapping and assembly.
benchmarks/README.md maps the directory: the problems, the density and sampler benchmarks, the references, sweeps, seed check, merging and the cluster runner. The hard-coded submit_all.sh gives way to hpc/submit.py, which submits any subset as array jobs: --problems (default every benchmark problem), --kinds ref/run/sweep/seed, and within a kind --samplers, --settings, --seeds or --seed-sets. A selection keeps each task's array index, so a result lands in the same file whichever subset produced it, and a failed task is rerun by selecting it again. Walltimes come from a per-(kind, problem) table, overridable with --time. task.py now handles every problem: a reference that is not pooled (quadrature, a long NUTS run) is one task, and assemble.py takes it whole. A test pins the selection.
The coverage curves of the six notebooks that draw them now use np.linspace(0.05, 0.95, 19), not np.linspace(0, 1, 21), whose levels 0 and 1 coverage_curve does not take (level 1 is the draws' min to max, so it plotted an under-coverage that is not there). robust and normalization look the levels they quote up by value. local_optical keeps each model's pocomc setting next to its problem; gp_discrepancy gives the doubled emcee run only to the growing-amplitude toy fit (the other three converge at 20 000 steps, R-hat at most 1.004). All seven reran; every printed number is unchanged (the quoted coverages are computed at explicit levels, with the same seeds). design.md has the runtimes of the uncontended runs (linear 15 s, gp 582 s, local_optical 1332 s, alpha_ca 1763 s); robust, normalization and hierarchical were timed while other work shared the machine, and keep their earlier runtimes, their computation being unchanged.
- References: one dynesty run could not have corroborated a pocomc pool (single runs were 0.12 to 0.37 sd off on omp), so three dynesty runs (rslice, 1000 live points) are pooled the way the pocomc runs are, as three more array tasks, and the pools must agree on log Z within twice the combined error, on the mode share within 0.03 and on the moments within 0.2 sd; the checks are recorded and named, not enforced. The mode share (problems.mode_share, moved from sweep_pocomc) is computed for every reference run, dynesty included (run_dynesty gains the summary hook). - assemble.py no longer aborts on one problem's failed pool: it keeps it, marked accepted: false, with its runs. Pooled references keep each run's seed; whole references record their wall time; references carry their parameter names, and a report refuses to score a run against a reference of other parameters. - merge_results.py: the docstring's command is the right one; anything that is not a reference or sampler file (a sweep) is refused; a problem whose parameters changed drops the rows no input replaces. - gp_seed_check.py records log Z as the notebook prints it (with the Jacobian). env.sh gives each task its own PyTensor compile cache and drops two thread settings nothing reads. submit.py validates problem names and asks for --problems or --all.
…_FLAGS intra_op_parallelism_threads is not an XLA flag; JAX 0.12 starts with it but ignores it. --xla_cpu_multi_thread_eigen=false and OMP_NUM_THREADS remain.
The end-to-end driver test ran pocomc on the GP problems (about 19 ms per likelihood call), which took most of an hour. The sweep is now tested end to end on linear with short runs (28 s), resume included; the seed check's summaries, the mode share and the merge are tested directly. run_pocomc passes n_total and n_evidence, if given, to pocomc's run (its defaults, 4096 each, otherwise), so short runs are possible.
What the cluster tasks compute: - gp_seed_check's weighted quantiles centre each draw's weight on its value (equal weights give the midpoint median, 2.5 of 1..4, not 2.0). - job.sbatch runs each task through srun --cpu-bind=cores: XLA does not read OMP_NUM_THREADS, and binding keeps its threads on the task's cores when tasks share a node. env.sh's comment no longer claims otherwise. - A seed task builds only its own rung of the ladder, and n40ca_gp only the GP rung (n40ca_ladder(rungs=...)), not all three. What scores them, later: - bench_samplers.usable_reference refuses a reference that does not name its parameters or names others, and flags one not accepted or not corroborated (also marked in the tables); sweep_pocomc's report uses it. The committed references now carry their names; the committed omp reference names the earlier log_s/log_b, so it can no longer score the redefined omp. The committed tables are unchanged. - merge_results drops a redefined problem's stale reference along with its rows, unless an input brings a new one. - sweep_pocomc and gp_seed_check create their output's directory; seed tasks refuse any problem but gp. A dead line in a test is gone.
omp now defines local_optical's exact problem, so the earlier problem's reference and sampler rows describe nothing the benchmark runs; they are removed (git keeps them) and come back from the cluster refresh. The results table is regenerated without them; the references table states the new population-reference method for omp, alpha_gp and n40ca_gp; the correct/fastest row and the note above Results say omp is pending, and that the omp numbers the text still quotes are of the earlier version.
gradient_based_sampling: - the hierarchy's MLE search is L-BFGS-B, not Nelder-Mead; - emcee is "cheaper and agrees with NUTS" on the GP, not "as good": agreement between two samplers is not a check, and on the benchmark's exact gp neither gets the widths within the tolerance; - the benchmark's NUTS had 31 divergences, not "a similar number" to this run's 85; - the Laplace widths on b are standard deviations the tails dominate; - "about twelve seconds"; "wrong means or widths", not "biased"; - the recipes link. normalization_and_covariance_structure: - ignoring the systematics is the most overconfident answer, not the most wrong (Peelle's is further off); - treatment 6 is close to the truth, its 68 % interval missing a0 = 1; - leaving a dataset out is recipe 30, not 28; - the start ball is wider than most posterior widths, and the modes reference is itself NUTS; - the covariance cells fill theta by name through problem.columns; - rerun (figures unchanged). The rest: - hierarchical_calibration: over-coverage from a nominal 0.3 up; the sentence on energies between datasets rewritten. - robust_likelihoods: KDUQ cited at first use; a note that its scaled term needs model_only=True on a grid. - linear_calibration: Bayes' rule over log sigma. - Recipe lists: robust adds 17, 33, 40 and normalization 33, 40, 41, in cell 0, design.md and the index test. - design.md: runtimes re-timed on a quiet machine (robust 64 s, normalization 318 s, hierarchical 294 s); emcee unmixed in tau too.
job.sbatch and env.sh no longer carry a site's account, partition or venv. submit.py takes --account, --partition, --constraint, --venv (exported to env.sh as RXMC_VENV) and --sbatch for any other option, and reads them from an @file kept outside the repository.
Three dynesty runs put 0.55, 0.69 and 0.80 of alpha_gp's mass in the deeper family, too scattered to test the pocomc pool's 0.740 against the 0.03 bar; six more seeds (3003-3008) take task indices 6-11 of its ref job.
84 tasks on amd20 nodes at bf6e086/12db2ca: references, the samplers, the pocomc sweeps and gp_discrepancy's seed check (results/hpc/), merged into results/samplers.json and references.json. All three references are accepted; omp's and n40ca_gp's are corroborated by dynesty. alpha_gp's is not: nine dynesty runs pool to a deeper-family share of 0.787 +/- 0.035 against pocomc's 0.740, a gap within dynesty's own scatter, and log Z and the moments agree; it is kept, marked corroborated: false, and the docs say why. docs/benchmarks.md: the new rows, references and correct/fastest entries; the discussion's omp numbers are still the earlier version's.
…esults The omp text quoted the earlier parametrisation; it now reads from the cluster refresh, and covers alpha_gp and n40ca_gp too. The sweep table is the new one for all three problems (4096/2048 correct on every seed but one, omp's seed 100); the discussion's bullets on separated modes, the ensembles, the evidence, PyMC against blackjax and the cost per call carry the new numbers. A new section sets gp_seed_check's three seed sets against the gp_discrepancy notebook: its log Z, Delta log Z and volume medians hold to a small fraction of their intervals.
proportional_error and normalization are fractions of the prediction
(or of the data and prediction averaged), which is a relative error only
in linear space. On a comparison with space=log the term sees log(ym),
so the "fraction" scaled with log(sigma) in whatever units the data
carry. A Constraint now refuses them there (Term(relative=True)), with
a message naming the log-space equivalents: in log space a multiplicative
error is additive, so an uncorrelated relative error is noise() and a
normalisation is offset(). reported_terms() already maps reported
uncertainties by the delta method and is unaffected.
This is a change of public behaviour: code that combined either term
with a log-space comparison now fails at Constraint construction.
The guard found two such uses besides gp_discrepancy's (fixed with its
rerun): recipe 18's L2y ("a normalisation mode, sys * ym") in log space
is L2n's offset mode, so L2y is gone from the legend and recipe 18 and
the design example use L2n; a test_problem test now builds its log-space
mode with offset. design.md's terms section gains the linear/log table.
On the three optical-model problems the reference is pooled pocomc,
checked by pooled dynesty, and the two differ by up to 0.07 (omp), 0.16
(alpha_gp) and 0.15 (n40ca_gp) sd in their moments, which is coarser
than the 0.1 tolerance. That gap is now the reference's
resolution r (bench_samplers.resolution): an error more than r above the
tolerance is wrong (✗), one within r of it either way is unresolved (?),
and a run is correct unless shown wrong. Exact and NUTS references keep
r = 0, so the other problems' verdicts are unchanged. The sweep report
uses the same verdict; the uncorroborated-reference warning goes to
stderr, out of the Markdown.
docs/benchmarks.md, from the three reviews and the regenerated tables:
- every n40ca_gp sampler but emcee is within resolution; HMC is fastest,
pocomc fastest with log Z; omp and alpha_gp's pocomc are ?;
- 4096/2048 is wrong on one seed of nine (omp's 100), every lower
setting on at least one seed per problem but alpha_gp's 2048/1024;
- the dynesty/pocomc settings rule stated as it is (every setting swept
is correct on elastic, none on both funnels; rslice 250 on gp alone),
the sweep's data noted as not in results/;
- the scoring, references and alpha_gp-corroboration text says what r
is and why alpha_gp's verdicts are ?; pooled log Z error explained;
- elastic's vmapped row is 3.0 ms, not 3.9; the PyMC and per-call
bullets merged and shortened; the funnel bullet shortened;
- history wording removed ("an earlier run", "with three runs the gap
was"); prose math in LaTeX.
The gp seed-check section follows with the gp_discrepancy rerun.
Narrative that referred to earlier runs, versions or process now states the result: - local_optical: omp is this notebook's problem, not "its version ... which differs in the parametrisation and priors"; the gradient-sampler numbers are the refreshed ones (HMC and PyMC 0.30 to 0.32 off at R-hat 1.005 to 1.007; blackjax NUTS the one flagged, 1.69); the island share 13 % in the reference, 14 to 16 % at this setting, 23 to 85 % at a quarter of it; the linear model's setting justified by elastic and by its evidence, 726 below. - alpha_ca: the n_seeds = 3 run that was never committed is replaced by the benchmark's three runs of the same problem (log Z -84.50 to -84.55; 71 to 74 % in the deeper family, dynesty's check 79 %); the reference's self-agreement is stated; "a few minutes" is half an hour. - design.md: the stale omp parenthetical, "a three-run result quoted", "the 0.x quartic", "the 0.x pins", "was removed", "JAX rxmc", "as it is"; the performance summary covers alpha_gp and n40ca_gp, and the rule for the notebooks' samplers is stated as applied; the release section states the release path, not the milestone history; the hierarchical "in detail" paragraph, a restatement of the notebook, is gone; the gradient row's 50-tau claim corrected. - docs/groundup_design.md, the plan the rewrite was built from, moves out of the repository with the other plans; its pointers are gone. - README: pocomc by preference for optical models (not nested sampling), pocomc among the samplers, one link to the documentation. - recipes.md: the "rewrite must support" sentence, "pre-0.1", "the authors have confirmed", section numbers (§6); recipe 12's coverage claim qualified. index.rst points to installation for 0.x. - Notebooks: "the 0.x normalisation example", KDUQ's tempering as the authors' private communication, black-box-bayes's name, robust's scaled term on comp.y_err as recipe 34 has it. - Prose math in design.md, recipes.md and README in LaTeX.
The cross-section ladder's "proportional error" rung put proportional_error on a log-space comparison, so its error scaled with log(sigma in b/sr): about 0.26 gamma forward and 5.5 gamma backward, in whatever units the data carry. In log space a relative uncorrelated error is noise(); the rung (and the benchmark's copy of the ladder) now uses it. pocomc now runs the ladder at 4096/2048: on n40ca_gp, this notebook's GP rung, the sweep finds 1024/512 wrong on two seeds of three (0.29 and 0.40 sd, beyond the reference's resolution), while every 4096/2048 seed is within it. Rerun (1256 s): Delta log Z 558.3 (proportional over bare) and 12.8 (GP over proportional, was 7.1); the proportional error is 6 sd out in V_v, 4 in R_v and 10 in W_v, the GP 2.7 in V_v and 8 to 9.6 elsewhere; the envelope holds the defect over 58 % of the grid. The toy half is unchanged. The seed check reruns at the same settings: log Z within 0.06, Delta log Z 12.8 in every set, medians within 0.16 of a half-width, and the GP rung within 0.12 sd of n40ca_gp's reference. Its nine cluster pieces, of the old rung, are removed; the JSON is the local run.
- docs/conf.py enables MyST's dollarmath and amsmath. Without them no page but the API reference had a math node, so MathJax never loaded and every $...$ in the docs and the notebooks showed as literal text. - benchmarks: verdict() calls a NaN error wrong, not unresolved; resolution()'s docstring says which references have r = 0. - The guard's message names the refused space and says every non-identity space is refused; its test covers on=None across mixed spaces and on= a dataset. - Statements about pocomc at 4096/2048 say what the benchmark shows (not shown wrong; wrong on one omp seed of three) in recipes.md, local_optical and gp_discrepancy; design.md's rule for the notebooks' samplers asks for evidence that agrees with the reference. - docs/benchmarks.md: the gp seed-check section from the rerun; dynesty's share range 0.55 to 0.90; pocomc's ESS on gp about 4500; the density baseline named as rxmc 0.x (v0.1.0); prose uncertainties in LaTeX. - Pointers to the removed groundup_design.md in terms.py and pyproject.
Rerunning for the figure titles ("vermillion", for Wong's colour, not
"orange") showed that the GP's NUTS at target acceptance 0.95 is not
reliably divergence-free: this run gave 17 divergences and R-hat 1.010
where the committed one gave none, and three seeds gave 4 to 17. The
run is deterministic within an environment (the same seed twice gives
the same chains), so the committed run was a favourable one. At 0.99
three seeds gave no divergences; with 2000 draws per chain the R-hat
is 1.004 to 1.007.
The notebook now runs the GP's NUTS at 0.99 with 2000 draws: 904
effective draws in 60 s, R-hat 1.004, no divergences; emcee is about six
times faster (96 against 15 per second); the 0.8 run gives 41
divergences. The text and the design row follow (runtime 366 s).
…ces; dynesty's settings swept
- reference.py: alpha_gp and n40ca_gp pool fifty dynesty runs (seeds
3000-3049), not nine and three: single runs scatter about 0.1 in
alpha_gp's mode share against a 0.03 bar, and the pools' moment gaps (0.16,
0.15 sd) set those references' resolution. Existing task indices keep
their runs (new ones are 12-52 and 6-52).
- sweep_pocomc.py sweeps dynesty's settings as well as pocomc's: on gp,
modes and elastic (PRIOR_PROBLEMS) seven settings, dynesty rwalk 200/500,
rslice 250/500 and pocomc 512/256 to 2048/1024, so the docs' settings
table can be regenerated from committed data. Settings are named by
label ("pocomc 1024/512", "dynesty rslice 250"); runs saved before keep
reading as pocomc. --grid prints several problems' sweeps side by side.
- hpc: sweep tasks come from settings_for(problem); submit.py selects sweep
settings by label and any kind by --indices; ref walltimes for alpha_gp
and n40ca_gp follow their cluster runs (2 h, 1 h).
- Tests for the labels, the grid, --indices and a dynesty sweep end to end.
Claude-Session: https://claude.ai/code/session_012XSD4kQVCWxa2N29LhXjnu
…p's data committed - alpha_gp: fifty dynesty runs pool to a deeper-family share of 0.756 (standard error 0.015) against pocomc's 0.740, within the 0.03 bar; the moment gap falls from 0.16 to 0.05, so the reference is corroborated and its ? marks go. n40ca_gp's fifty move its gap to 0.07 / 0.12. - The dynesty/pocomc settings table on gp, modes and elastic is regenerated from cluster runs (sweep_pocomc.py --grid, seeds 100-102). Every setting is correct on elastic; none on gp or modes on more than one seed, and the earlier local result for rslice 250 on gp does not reproduce. - docs: the pooled log Z error (half the runs' range) grows with the number of runs; against the standard error, dynesty is about three errors below pocomc on n40ca_gp. Claude-Session: https://claude.ai/code/session_012XSD4kQVCWxa2N29LhXjnu
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Significant re-write, superseding #43 and #42