From 8fcc914d91f4c8dd2f827788ccdb9b5a6c49f71e Mon Sep 17 00:00:00 2001 From: seabbs-bot Date: Wed, 30 Sep 2026 18:19:26 +0100 Subject: [PATCH] docs: bring the composable recurrences page up to API v0.1 Redraw the package map with ComposableRecurrences as a foundation that ComposableTuringIDModels, ComposedDistributions and CensoredDistributions plan to build on, marked as planned. Drop the Reactant, EpiAwareADTools and distribution-stack boxes. Move the example, modifiers and status to the v0.1 API and what is merged on main. Co-authored-by: Sam Abbott --- approaches/composable-recurrences.qmd | 187 ++++++++++++-------------- styles.css | 6 +- 2 files changed, 88 insertions(+), 105 deletions(-) diff --git a/approaches/composable-recurrences.qmd b/approaches/composable-recurrences.qmd index 20bbe39..f12cbbb 100644 --- a/approaches/composable-recurrences.qmd +++ b/approaches/composable-recurrences.qmd @@ -76,12 +76,16 @@ page-layout: full ```{.julia .code-overflow-scroll} using ComposableRecurrences +using ComposableRecurrences: Depletion, Floor, Add, with_state # Generation interval, lag 1 first: weights on y[t-1], y[t-2], y[t-3] g = [0.6, 0.3, 0.1] # Two strata mixed by a contact matrix K = [0.9 0.1; 0.2 0.8] -infections = Recurrence(g; coupling = K) +# Modifiers are structs in a tuple, applied in order after each step +N = PerStratum([1_000.0, 500.0]) +infections = Recurrence(g; coupling = K, + modifiers = (Depletion(N, Floor()), Add(0.5))) # Reporting delay, lag 0 first: P(delay = 0, 1, 2, 3) reports = Convolution([0.0, 0.5, 0.3, 0.2]) @@ -91,10 +95,9 @@ R = fill(1.1, 2, 10) y = infections(R; history = ones(2, 3)) cases = reports(y) -# Stop, keep the state, and resume for a forecast -y1, state = infections(R[:, 1:5]; history = ones(2, 3), - return_state = true) -y2 = infections(R[:, 6:end]; history = state) +# Keep the state after day 5, then resume for a forecast +y1, st = with_state(infections, R; history = ones(2, 3), stop = 5) +y2 = infections(R; state = st) y ≈ hcat(y1, y2) ``` @@ -103,7 +106,7 @@ y ≈ hcat(y1, y2) ::: -The example runs on the main branch of [ComposableRecurrences.jl](https://github.com/EpiAware/ComposableRecurrences.jl). +The example uses API v0.1 of [ComposableRecurrences.jl](https://github.com/EpiAware/ComposableRecurrences.jl), which is landing on the main branch now. The package is not registered yet, so install it from the repository to try it. ## The approach @@ -114,6 +117,7 @@ This approach builds all of them from two operators rather than one bespoke loop A `Recurrence` feeds its output back into its own history. At each step it convolves the last `L` values with its kernel, mixes strata through a coupling, scales by a gain, adds an input, and passes the result through a tuple of modifiers. +An operator is built from structs, and those structs compose. A renewal process is a `Recurrence` with the generation interval as kernel and `R_t` as gain. A random walk is a `Recurrence([1.0])` with the noise as the added input. @@ -130,142 +134,125 @@ Every kernel is written lag first, the way a generation interval or AR coefficie A `Recurrence` kernel starts at lag 1 and a `Convolution` kernel at lag 0, so nothing is ever reversed by hand. Operators are built once, outside the model, and called with the per-day inputs that are usually sampled. Chaining is function composition, as in `reports(infections(R; history))`. +Every time-indexed input is read at absolute time, so `with_state` returns the output with its state and a later call resumes from `state`. ### Strata, time variation and coupling -Wrappers name the extra axis of a coefficient. -`PerStratum` gives each stratum its own kernel, `TimeVarying` lets a kernel or coupling change by day, and `Pairwise` gives each pair of strata its own lag kernel. -The coupling mixes strata before the gain, as a contact or mobility matrix does. +Wrappers name the extra axis of a coefficient or parameter. +`PerStratum` gives each stratum its own value, and `TimeVarying` lets a kernel, coupling or parameter change by day. +`TimeVarying(P, Primary())` indexes a delay by the day of the primary event rather than the day it is observed. +`Pairwise` is a kernel that gives each pair of strata its own lags, so it replaces the coupling. +Otherwise the coupling mixes strata before the gain, as a contact or mobility matrix does. Couplings can be dense, sparse or diagonal matrices, and any real element type is accepted, so `Float32` and dual numbers pass through. ### Modifiers -Modifiers are applied in order after each step and carry their own state. -They handle what the linear core cannot, such as susceptible depletion or moving cases between strata. -A modifier is a plain type with an `apply!` method, so a modelling package can add its own. +Modifiers are structs composed in a tuple. +They are applied in order after each step and carry their own state. +They handle what the linear core cannot. +`Depletion(N)` depletes susceptibles by a hazard, and `Depletion(N, Floor())` by a floored fraction. +`Add(b)` adds an input at that point in the tuple, such as imports after depletion. +`Redistribute` moves cases between strata and `Clamp` bounds them. +Variants such as `Floor()` are structs too. +A new modifier or variant is a struct with a `forward` method for its step, so a modelling package can add its own. ### Plug-in adjoints Reverse-mode AD is slow when each step copies its lag window into a new state. The operators own a history buffer instead, which avoids that copy. -Native adjoints will plug in per operator, modifier or coupling by adding a method, with no AD declarations from the user. +Native adjoints will plug in per operator, modifier or coupling by adding a `pullback!` method, with no AD declarations from the user. `NoAdjoint(op)` turns an adjoint off, so its gain can be timed and the adjoint dropped if a backend catches up. A [benchmark script](https://github.com/EpiAware/ComposableRecurrences.jl/blob/main/benchmark/gradients.jl) in the repository times plain AD gradients of the operators under Mooncake and Enzyme. ### How it fits with the other approaches -[Composable Turing models](composable-turing-models.qmd) will build on these operators. +The operators are a foundation for the other approaches. +[Composable Turing models](composable-turing-models.qmd) will build its models on them. Its renewal, delay and latent process steps become calls to `Recurrence` and `Convolution`, with the priors and observation models staying in the Turing layer. -The [composed distributions](composed-distributions.qmd) stack supplies the delay pmfs that become `Convolution` kernels. -The operators have no dependency on either, so they can also be used in other models. +The [composed distributions](composed-distributions.qmd) stack will use them inside its packages. +ComposedDistributions can run a renewal across a distribution tree, taking each step's kernel from the tree ([#82](https://github.com/EpiAware/ComposedDistributions.jl/issues/82)). +CensoredDistributions may build on them in the same way. +The operators depend on none of these packages, so they can also be used in other models. ## Package map -Solid arrows are data or code that will flow once the package is adopted. -The dashed grey arrow is a test-time link. -The dashed amber arrow is an optional future backend, not a dependency. +Arrows point from a package to the packages that use it. +The dashed grey arrow is a test and documentation dependency, and it is the only link in place today. +The dashed teal arrows are planned and not yet implemented. ```{=html} ``` -The package has no runtime dependency on any other EpiAware package. -Today ComposableTuringIDModels runs its own renewal, delay and latent process loops. +None of the three packages depends on ComposableRecurrences yet. +Today each runs its own renewal and delay loops. Moving them onto one set of operators gives one forward kernel and one adjoint to maintain and test. ## Status This approach is under development. -The core operators are merged on the main branch of ComposableRecurrences.jl. -These are `Recurrence`, `Convolution`, the `TimeVarying`, `PerStratum` and `Pairwise` wrappers, and the coupling and modifier interfaces. +The core operators and the built-in modifiers are merged on the main branch of ComposableRecurrences.jl. +These are `Recurrence`, `Convolution`, the `TimeVarying`, `PerStratum` and `Pairwise` wrappers, the coupling and modifier interfaces, and `Depletion`, `Imports`, `Redistribute` and `Clamp`. The test suite differentiates them with ForwardDiff, ReverseDiff, Mooncake and Enzyme through plain AD. -Use-case tests against copies of the ComposableTuringIDModels steps they will replace are written but not yet switched on. +Use-case tests check them against copies of the ComposableTuringIDModels and other model steps they will replace. -The built-in modifiers (`Depletion`, `Imports`, `Redistribute`, `Clamp`) and the native Mooncake and Enzyme adjoints are in progress. +The API v0.1 consolidation is in progress. +It makes every option a struct, renames `Imports` to `Add`, makes `Pairwise` a kernel, and replaces `return_state` with `with_state`. +The native Mooncake and Enzyme adjoints are also in progress. A first registered release follows them, and then the port of ComposableTuringIDModels onto the operators. -An FFT method for long fixed kernels and a Reactant backend are later extensions. +An FFT method for long fixed kernels is a later extension. ## Where next @@ -273,7 +260,7 @@ An FFT method for long fixed kernels and a Reactant backend are later extensions tracks development, and its documentation will follow the first release. - Building a full model with priors and observations? See the [composable Turing models approach](composable-turing-models.qmd). -- Need the delay distributions that feed a `Convolution`? See the +- Composing delay distributions? See the [composed distributions approach](composed-distributions.qmd). :::::: diff --git a/styles.css b/styles.css index 831820f..431b07c 100644 --- a/styles.css +++ b/styles.css @@ -330,17 +330,13 @@ h1, h2, h3 { letter-spacing: -0.02em; } .cr-map .cr-band-t { fill-opacity: 0.6; font-size: 10.5px; letter-spacing: 0.08em; } -.cr-map .cr-node.cr-opt { - stroke: var(--epi-accent); stroke-dasharray: 5 4; -} .cr-map .cr-e { stroke: var(--epi-primary); stroke-width: 1.5; fill: none; } .cr-map .cr-e.cr-dash { stroke: currentColor; stroke-opacity: 0.5; stroke-dasharray: 5 4; } -.cr-map .cr-e.cr-warn { stroke: var(--epi-accent); stroke-dasharray: 5 4; } +.cr-map .cr-e.cr-plan { stroke-dasharray: 6 4; } .cr-map .cr-ah { fill: var(--epi-primary); } .cr-map .cr-ah.cr-muted { fill: currentColor; fill-opacity: 0.5; } -.cr-map .cr-ah.cr-warn { fill: var(--epi-accent); } /* Value propositions + foundations grid ----------------------------------- */ .feature-grid {