GeiserX · Nov 13, 2026 · Nov 13, 2026 · Nov 13, 2026 · Nov 13, 2026 · Nov 13, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,12 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n
 
 ## Unreleased
 
+- The three builders of the committed asset tables now run in the Rust `og` binary: `og dev build cn-intervals` (the interval rows of `bin/cn_loci.tsv`), `og dev build callability` (`assets/callability_genes.tsv`) and `og dev build sv-rule-regions` (`assets/sv_rule_regions.tsv`). They replace `scripts/ci/build-cn-intervals.py`, which is gone (the script is now `scripts/ci/build-cn-intervals.sh`), and the Python inside `scripts/ci/build-callability.sh` and `scripts/ci/build-sv-rule-regions.sh`. The scripts still download the pinned files, and the three tables are the same, byte for byte, as the committed ones. With the downloads in the cache, the callability table takes 2.5 s and 27 MB instead of 5.5 s and 39 MB, and the SV rule table 4.9 s and 30 MB instead of 10.5 s and 217 MB. What changes on purpose:
+  - The three scripts share one download function, `scripts/ci/lib/fetch-pinned.sh`. A file the cache folder already holds with the right sha256 is read where it is, not copied. A 403 or 404 fails at once, including after an earlier retry. Other failures, stalled transfers included, get at most six retries, 30 s apart, within a five-minute retry window. `build-cn-intervals.py` tried three times, 30 s apart.
+  - The marker line of `bin/cn_loci.tsv` names `scripts/ci/build-cn-intervals.sh`. No row changes.
+  - `build-cn-intervals.sh --check` counts the lines only in the file and the lines only in the rebuild. The Python counted only the lines the rebuild lacks, so a catalog with an added line said "0 line(s) differ".
+  - The SV rule builder takes each gene's span from the first GENCODE record of its HGNC id and skips the `_PAR_Y` copies, as the callability builder does. The Python kept the last record, so a pseudogene in a pseudoautosomal region would have taken its chrY span. No gene of the pinned files is one, so the table does not change.
+
 - The container helpers check (`scripts/ci/check-container-helpers.sh`, Guard job `helper-binaries`) now derives its table with the Rust `og` binary, `og dev helpers`. It replaces the Python inside the script, which is gone; the script keeps the docker probe, the orad download and its self-test. On this tree the table and the rule 1 output are the same, byte for byte. The static run takes 0.01 s and 6 MB where the Python took 0.2 s and 15 MB. What changes:
   - A helper called in a process's `shell:` block is checked too. The Python read only `script:` blocks, so such a call went unchecked. No module has one today.
   - A helper called in a process's `exec:` block always fails the check: Nextflow runs that block on the host, where no image applies.

diff --git a/bin/cn_loci.tsv b/bin/cn_loci.tsv
@@ -66,7 +66,7 @@ HP	gene	chr16	HP_UNIT1:72056529-72057466:HP exons 3 and 4, the first copy of the
 LCE3BC	interval	chr1	LCE3BC_DEL:152600233-152614098:LCE3C to LCE3B, inside the LCE3C_LCE3B deletion	LCE3_FLANK_L:152520000-152560000:flank on the LCE3D side, outside the deletion;LCE3_FLANK_R:152640000-152680000:flank on the LCE3A side, outside the deletion	.	sum	flanks	2	0	.,-Q 1	.	.	8	1.5	round:0.3	.	0=LCE3B and LCE3C deleted on both chromosomes (LCE3C_LCE3B-del on both), a common genotype that goes with a higher psoriasis risk|1=LCE3B and LCE3C deleted on one chromosome (LCE3C_LCE3B-del on one)|2=no LCE3B/LCE3C deletion seen|more={n} copies of the LCE3B/LCE3C stretch: more than two, a duplication	bedcov	cn-depth	none	19169253
 CCL3L1	gene	chr17	CCL3L3:36194868-36196758:CCL3L3, the one copy of CCL3L1/CCL3L3 on the reference (ENST00000619989)	CCL3L_FLANK_L:36112000-36148000:single-copy sequence between the CCL3 and CCL3L3 duplications;CCL3L_FLANK_R:36460000-36500000:single-copy sequence past the duplications	.	sum	flanks	2	0	.,-Q 1	.	.	8	1.5	round:0.3	.	0=no copy of CCL3L1 or CCL3L3|1=one copy of CCL3L1/CCL3L3 in all|more={n} copies of CCL3L1/CCL3L3 in all: the count differs widely between people, and no reading of it is given	bedcov	cn-depth	none	15637236
 CYP21A2	gene	chr6	CYP21A2_BODY:32038950-32041644:CYP21A2 from the end of the C4A model to its last base (NM_000500.9);CYP21A1P_BODY:32006220-32008909:the same stretch of CYP21A1P (NR_040090.1)	RCCX_FLANK_L:31945000-31975000:single-copy sequence before the RCCX duplication;RCCX_FLANK_R:32070000-32100000:single-copy sequence in TNXB past the duplication	PSV_In2G:32039080-32039081:32006352 AC>G c.293-13A/C>G (In2G, intron 2);PSV_I172N:32039425-32039426:32006689 T>A c.518T>A (p.I172N, exon 4);PSV_V237E:32039809-32039810:32007073 T>A c.713T>A (p.V237E, exon 6 cluster);PSV_V281L:32040109-32040110:32007373 G>T c.844G>T (p.V281L, exon 7);PSV_Q318X:32040420-32040421:32007685 C>T c.955C>T (p.Q318X, exon 8);PSV_R356W:32040534-32040535:32007799 C>T c.1069C>T (p.R356W, exon 8)	sum	flanks	2	0	.,-Q 1	0.02	.	8	1.5	psv:0.4	.	none=no sign of the 30 kb chimeric deletion or of a pseudogene conversion at the six sites read; short reads see neither a change elsewhere in CYP21A2 nor two copies on one chromosome and none on the other|deletion=one RCCX module fewer and one CYP21A2 copy fewer at the differing sites: the pattern of the 30 kb CYP21A1P/CYP21A2 chimeric deletion, a common cause of carrier status for 21-hydroxylase deficiency (congenital adrenal hyperplasia); unconfirmed: a clinical test (MLPA or long-range PCR) confirms it|conversion=fewer CYP21A2 copies carry the gene's own base at {sites} than at the other sites: a possible conversion of one CYP21A2 copy to the pseudogene sequence there, a carrier pattern for 21-hydroxylase deficiency (congenital adrenal hyperplasia) when it sits on one chromosome; which chromosome carries it is not known; unconfirmed: a clinical test confirms it|pseudogene_deletion=one RCCX module fewer with CYP21A2 kept at every site read: a CYP21A1P (pseudogene) deletion, common and not a carrier finding	bedcov	cn-depth	paraphase:rccx	34285390,27228465
-# --- interval rows: written by scripts/ci/build-cn-intervals.py; do not edit by hand ---
+# --- interval rows: written by scripts/ci/build-cn-intervals.sh; do not edit by hand ---
 1q21.1_TAR_syndrome_proximal_BP2-BP3	interval	chr1	1q21.1_TAR_syndrome_proximal_BP2-BP3_t1:145686999-145809382:1q21.1 recurrent (TAR syndrome) region (proximal, BP2-BP3) (includes RBM8A);1q21.1_TAR_syndrome_proximal_BP2-BP3_t2:145823828-145923649;1q21.1_TAR_syndrome_proximal_BP2-BP3_t3:145927476-146048495	1q21.1_TAR_syndrome_proximal_BP2-BP3_L:143034587-143134587:flank left of the region;1q21.1_TAR_syndrome_proximal_BP2-BP3_R:149926380-150026380:flank right of the region	.	pooled	flanks	2	0	.	.	.	8	1.5	round:0.3	.	.	bedcov	cn-depth	cnvpytor:cnvs	.
 1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4	interval	chr1	1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_t1:145687000-145809382:1q21.1 recurrent (TAR syndrome) region (proximal and distal, BP1-BP4) (includes RBM8A and GJA5);1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_t2:145823828-145923649;1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_t3:145927476-146048496;1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_t4:147105903-147473859;1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_t5:147473859-147841815;1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_t6:147842996-147917509	1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_L:143034587-143134587:flank left of the region;1q21.1_TAR_syndrome_proximal_and_distal_BP1-BP4_R:149926380-150026380:flank right of the region	.	pooled	flanks	2	0	.	.	.	8	1.5	round:0.3	.	.	bedcov	cn-depth	cnvpytor:cnvs	.
 1q21.1_distal_BP3-BP4	interval	chr1	1q21.1_distal_BP3-BP4_t1:147105904-147473860:1q21.1 recurrent region (distal, BP3-BP4) (includes GJA5);1q21.1_distal_BP3-BP4_t2:147473860-147841815;1q21.1_distal_BP3-BP4_t3:147842996-147917509	1q21.1_distal_BP3-BP4_L:143034587-143134587:flank left of the region;1q21.1_distal_BP3-BP4_R:149926380-150026380:flank right of the region	.	pooled	flanks	2	0	.	.	.	8	1.5	round:0.3	.	.	bedcov	cn-depth	cnvpytor:cnvs	.

diff --git a/bin/copy_number.py b/bin/copy_number.py
@@ -31,7 +31,7 @@
            "call", "interval", "readings", "input", "owner", "cross_check", "pmids"]
 KINDS = ("gene", "interval", "organelle")
 # GRCh38 names of the primary chromosomes (the ClinGen BED spells chrX 'chrx':
-# scripts/ci/build-cn-intervals.py normalises it, and a row with any other
+# og dev build cn-intervals normalises it, and a row with any other
 # spelling is refused here).
 CHROMS = {f"chr{c}" for c in list(range(1, 23)) + ["X", "Y", "M"]}
 INPUTS = ("bedcov", "cnvpytor-level")

diff --git a/docs/cn-loci.md b/docs/cn-loci.md
@@ -89,7 +89,7 @@ A site is low when its estimate is more than 0.6 copies under the larger of the
 
 ### The interval rows
 
-`scripts/ci/build-cn-intervals.py` writes these rows, everything in `bin/cn_loci.tsv` after its marker line; do not edit them by hand. Step 35 (and the PARASCOPY module) runs `og cn bed --kind interval`, one `samtools bedcov` over every segment of every row, then `og cn intervals`, which writes one line per row to `<sample>_cn_intervals.tsv`: locus, name, chrom, start, end, status, copies, expected, change (`deletion`, `duplication`, `no change`, or `.` when not called), estimate and reason. The step log lists the rows with a change. The rows give a total copy count only, never breakpoints.
+`scripts/ci/build-cn-intervals.sh` writes these rows with `og dev build cn-intervals`, everything in `bin/cn_loci.tsv` after its marker line; do not edit them by hand. Step 35 (and the PARASCOPY module) runs `og cn bed --kind interval`, one `samtools bedcov` over every segment of every row, then `og cn intervals`, which writes one line per row to `<sample>_cn_intervals.tsv`: locus, name, chrom, start, end, status, copies, expected, change (`deletion`, `duplication`, `no change`, or `.` when not called), estimate and reason. The step log lists the rows with a change. The rows give a total copy count only, never breakpoints.
 
 - **The ClinGen recurrent regions**: every region row of ClinGen Dosage Sensitivity's recurrent CNV BED, version 2.1, GRCh38 (CC0, pinned by `CLINGEN_RECURRENT_CNV_URL` and its sha256 in `versions.env`). The file's breakpoint and LCR rows are the segmental duplications that bound the regions. They are cut out of the targets, as are GIAB v3.6 segmental duplications, assembly gaps, satellites and the regions of every pinned Parascopy model (CHRNA7 in 15q13.3 stays with Parascopy). Targets are cut to pieces of at most 500 kb, and their depth is pooled. Controls are two 100 kb windows, one on each side of the cluster of ClinGen rows the region belongs to, clear of all of that and of the targets of the hand rows below (so no row's control sits on another row's target); a side with no clear window within 3 Mb (a chromosome end, as next to 22q11.2 and 15q11.2) is left out. The BED spells its chrX rows `chrx`; the script normalises them, and the engine refuses any contig name that is not a GRCh38 chromosome. The classification of a CNV found here follows the ACMG and ClinGen technical standards (Riggs 2020, PMID 31690835); a call from this step is a lead for a clinical test, not a classification.
 - **Two more ClinGen curations**: the 1p36 terminal region (ISCA-37434, haploinsufficiency score 3) and the Xq28 region that holds MECP2 (ISCA-46304, triplosensitivity score 3: the MECP2 duplication), from ClinGen's region curation list, read on 2026-11-04.
@@ -100,7 +100,7 @@ A row on chrX or chrY is read against the sample's own count there: its controls
 
 The couple screen (step 39) reads the table with `couple_report.py screen --cn-intervals ID=<id>_cn_intervals.tsv`: each carrier allele is present, absent or not assessable for each partner, and a male partner's AZF deletions are listed the same way (present means the deletion is there). With `--cnvpytor ID=<id>_cnvs.vcf.gz` (step 18), a CNVpytor call that disagrees with the depth call is printed beside it. Step 39 and the Nextflow couple screen pass both when the partner has them. Both reports show the table in step 35's section: the counts by status and each row with a deletion or a duplication.
 
-To change a row, change the script (its hand rows or a pinned download) and run `scripts/ci/build-cn-intervals.py`; `--check` fails when the catalog is not what the downloads give.
+To change a row, change its rule or hand row in `rust/crates/og-dev/src/build/cn_intervals.rs`, or a pinned download in `scripts/ci/build-cn-intervals.sh`, rebuild og (`scripts/setup.sh --og`) and run `scripts/ci/build-cn-intervals.sh`; `--check` fails when the catalog is not what the downloads give.
 
 ## One owner per locus
 
@@ -118,7 +118,7 @@ pypgx owns the genes it calls from the BAM, the `bam` rows of `assets/pgx_genes.
 
 1. Check that no pinned caller owns the gene: Parascopy (`assets/parascopy_models_v1.7.txt`), pypgx, Paraphase.
 2. Choose single-copy controls near the gene, outside any common deletion or duplication there and outside segmental duplications.
-3. Add the row to `bin/cn_loci.tsv`, above its marker line, and cite the method in `pmids`. An interval row (everything after the marker) is never added by hand: add it to `scripts/ci/build-cn-intervals.py` and rerun the script.
+3. Add the row to `bin/cn_loci.tsv`, above its marker line, and cite the method in `pmids`. An interval row (everything after the marker) is never added by hand: add it to the hand rows of `rust/crates/og-dev/src/build/cn_intervals.rs` and rerun `scripts/ci/build-cn-intervals.sh`.
 4. Run it from a step: `og cn bed --locus NAME`, `samtools bedcov` with the row's passes, then `og cn call --locus NAME --bedcov ... --table ...`.
 5. Add made-up bedcov tables for the row to `tests/test_copy_number.py`, at every count it can call and at each guard.