Skip to content
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,7 @@ out/
.classpath
.DS_Store
/bin/
.project
.project

# JVM crash logs from long benchmark runs
hs_err_pid*.log
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,10 @@ private static List<ModelSpec> modelSpecs(double alpha) {
specs.add(new ModelSpec(String.format("NNIRegCCD[CO_OCCURRING,b=%.3f]", beta),
tr -> new NNIRegCCD(tr, 0.0, PairingMode.CO_OCCURRING, alpha, beta)));
}
// full-support KRegCCD (reserve depth 2); compare the three tail modes to
// show the normalisation effect on mean logP: NONE (super-normalised),
// BOUND (upper bound, sub-normalised), SAMPLED (Knuth, near-exact)
// full-support KRegCCD (reserve depth 2); compare the three tail modes to show the
// normalisation effect on mean logP. All three are sub-normalised by Theta(mu^2) (see
// KRegCCD.TailMode); they differ only in the O(mu^3) truncation term: NONE omits it,
// BOUND over-corrects it, SAMPLED estimates it (landing within noise of NONE).
for (double mu : new double[]{0.01, 0.05, 0.1, 0.2}) {
specs.add(new ModelSpec(String.format("KRegCCD[rd=2,none,mu=%.2f]", mu),
tr -> new KRegCCD(tr, 0.0, mu, alpha, 2, KRegCCD.TailMode.NONE)));
Expand Down
1,157 changes: 1,157 additions & 0 deletions src/main/java/ccd/model/CRegCCD.java

Large diffs are not rendered by default.

31 changes: 22 additions & 9 deletions src/main/java/ccd/model/KRegCCD.java
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,11 @@
* {@code eps} is solved from only the cheap low orders {@code j = 1..k} (boundaries
* {@code 3..k+2}; {@code N_1} is {@code O(m^2)}, {@code N_2} is {@code O(m^3)} in
* the number {@code m} of observed subclades), while regions of any depth are still
* scored with that {@code eps}. The omitted deep orders contribute {@code O(mu^2/m)}
* to the reserve — a rounding error — so this barely affects probabilities while
* keeping the model tractable and full-support. Reserve depth {@code k = 2} (the
* scored with that {@code eps}. Since {@code eps ~ mu / N_1}, the omitted deep orders contribute
* {@code O(mu^(k+1))} to the reserve — {@code O(mu^3)} at the default {@code k = 2}, a rounding
* error — so this barely affects probabilities while keeping the model tractable and full-support.
* (Measured by exhaustive enumeration on 6- and 7-taxon sets: the truncation shifts total
* probability mass with a log-log slope in {@code mu} of 2.00 / 2.97 / 3.96 at {@code k = 1/2/3}.) Reserve depth {@code k = 2} (the
* default) is recommended; an op-budget (scaled to the largest clade's N_1, overridable via
* {@code -Dkreg.enumOps})
* bounds even the low-order enumeration on pathological clades, and a clade with
Expand Down Expand Up @@ -112,13 +114,24 @@ private static final class BudgetExceeded extends RuntimeException {
* How to correct for the omitted reserve tail (orders &gt; reserve depth) when
* discounting red splits by {@code 1 - mu - tail(C)}:
* <ul>
* <li>{@link #NONE}: no correction ({@code tail = 0}); slightly
* super-normalised (inflates held-out scores by the tail).</li>
* <li>{@link #BOUND}: geometric upper bound on the tail; provably
* sub-normalised (never inflates).</li>
* <li>{@link #SAMPLED}: Knuth estimate of the actual tail; near-exactly
* normalised (up to Monte-Carlo noise).</li>
* <li>{@link #NONE}: no correction ({@code tail = 0}). The truncation alone inflates by
* {@code O(mu^(k+1))}, but that is dominated by the maximality deficit below, so the
* model is net <em>sub</em>-normalised.</li>
* <li>{@link #BOUND}: geometric upper bound on the tail; sub-normalised (never inflates).</li>
* <li>{@link #SAMPLED}: Knuth estimate of the actual tail. This removes the
* {@code O(mu^(k+1))} truncation, but NOT the maximality deficit, so it is
* <em>not</em> exactly normalised — it lands within Monte-Carlo noise of {@code NONE}.</li>
* </ul>
*
* <p><b>All three modes are sub-normalised by {@code Theta(mu^2)}</b>, and no tail correction
* removes it: a blue region is only scored at its <em>maximal</em> top, so each boundary part
* that is itself a reserving clade contributes {@code (1 - mu - tail)} rather than 1 — the
* {@code O(mu)}-per-reserving-boundary-part gap noted on {@link SamplingFidelity}. Summing
* {@link #getProbabilityOfTree} over every tree on 7 taxa (2 training trees, {@code k = 2})
* gives total mass {@code -4.7e-3 / -6.3e-3 / -4.7e-3} from 1 for NONE/BOUND/SAMPLED at
* {@code mu = 0.05}, and {@code -2.6e-6 / -2.7e-6 / -2.7e-6} at {@code mu = 0.001};
* the deficit scales as {@code mu^2} and persists at reserve depths where the truncation
* has fully converged.
*/
public enum TailMode {NONE, BOUND, SAMPLED}

Expand Down
Loading