Update nf-core modules and local module tools

.pre-commit-config.yaml

@@ -41,6 +41,9 @@ repos:
     hooks:
       - id: nextflow-lint
         files: '\.nf$|nextflow\.config$'
+        # nf-core component tests are nf-core/modules' own; some include modules this pipeline does not install
+        # (subworkflows/nf-core/deepvariant/tests includes deepvariant/rundeepvariant)
+        exclude: "^(modules|subworkflows)/nf-core/.*/tests/"
         args: ["-output", "json"]
   - repo: local
     hooks:

CHANGELOG.md

@@ -66,6 +66,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - [#182](https://github.com/IntGenomicsLab/lrsomatic/pull/182) - Added `--vcf` to the default `vep_args` so VEP writes VCF output rather than its default tab-delimited format (@AmberVerhasselt).
 - [#183](https://github.com/IntGenomicsLab/lrsomatic/pull/183) - Corrected the `github` URL for Laurens Lambrechts in `manifest.contributors`, which was a copy of @MariosEft97's, to @laulambr. RO-Crate metadata updated to match (@ljwharbers).
 
+### `Dependencies`
+
+- [#212](https://github.com/IntGenomicsLab/lrsomatic/pull/212) - Updated all nf-core modules and the `bam_stats_samtools` subworkflow to the current nf-core/modules master with `nf-core modules update` (the `utils_*` subworkflows stay on the template's versions, because their master needs nf-schema 3.0.0), and bumped the tools further where nf-core lags the release and a single-tool biocontainer could be used (recorded as module patches). The callers follow the new module interfaces: `.tbi`/`.csi`/`.bai` outputs are now one `.index` channel, `WGET` takes the file suffix in its input tuple, and most modules report versions through the `versions` topic. The variant callers (Clair3, ClairS, DeepVariant, DeepSomatic) are updated in a separate PR (@ljwharbers).
+
+  | Tool            | Old                    | New    | Notes                                                                                                                                                                                                                                                                                    |
+  | --------------- | ---------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+  | `bcftools`      | 1.20 / 1.22            | 1.24   | All nf-core and local bcftools modules, `VCFSPLIT`, `VEPPLUGIN_EVE`/`VEPPLUGIN_REVEL`. `BCFTOOLS_NORM` now runs with `--check-ref w`: 1.24 aborts when a REF differs from the FASTA, and Clair3 writes `N` where GRCh38 has IUPAC codes (e.g. `B` at chr3:16902883), which 1.22 accepted |
+  | `samtools`      | 1.21 / 1.22.1 / 1.23.1 | 1.24   | Including `METAEXTRACT`; `minimap2/align` stays on nf-core's samtools 1.23.1                                                                                                                                                                                                             |
+  | `minimap2`      | 2.29                   | 2.30   | 2.31 has no minimap2 + samtools image yet                                                                                                                                                                                                                                                |
+  | `mosdepth`      | 0.3.11                 | 0.3.14 |                                                                                                                                                                                                                                                                                          |
+  | `nanoplot`      | 1.46.1                 | 1.48.0 |                                                                                                                                                                                                                                                                                          |
+  | `ensembl-vep`   | 115.2                  | 116.1  | 116.2's biocontainer lacks `Math::CDF`; the default `--vep_cache_version` (113) is unchanged                                                                                                                                                                                             |
+  | `severus`       | 1.6                    | 1.7    |                                                                                                                                                                                                                                                                                          |
+  | `cramino`       | 1.3.0                  | 2.0.0  | **N50, N75, median and mean read length now use the full read length** (primary alignments, clipped bases included), so the report's N50 goes up; the old aligned values are new `... aligned` rows. `% from total alignments` is renamed `% from total reads`                           |
+  | `fibertools-rs` | 0.8.1                  | 0.13.1 | **Fiber-seq BAMs now carry nucleosomes, MSPs and FIREs in the MolecularAnnotation tags `Ma` (`nuc`, `msp` and `fire` intervals) and `Aq` (their qualities)** instead of `ns`/`nl`/`as`/`al`/`aq`; downstream tools reading the old tags need `ft convert-tags` or an update              |
+  | `wakhan`        | 0.4.3                  | 0.4.4  | 0.5.0 changes the output directory layout and has no biocontainer yet                                                                                                                                                                                                                    |
+
+- [#213](https://github.com/IntGenomicsLab/lrsomatic/pull/213) - Updated the small variant callers. **Germline and somatic calls change** with the new models (@ljwharbers).
+
+  | Tool          | Old   | New    | Notes                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+  | ------------- | ----- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+  | `clair3`      | 1.2.0 | 2.0.3  | Clair3 v2 runs on PyTorch and **cannot read the v1 TensorFlow models** the pipeline downloaded from ONT's CDN. Models bundled with the image (HKU's and ONT's recent R10.4.1 SUP/HAC ones, `hifi_revio`) are now used from the image with no download; other models are fetched as PyTorch directories from HKU (ONT's Rerio catalogue for `r9`/`r10` names) by the new `CLAIR3_MODEL` step, which replaces `WGET`/`UNTAR` and, unlike `WGET`, checks the TLS certificate (Clair3 unpickles the files). The GPU image is `hkubal/clair3:v2.0.2_gpu`, the newest published |
+  | `clairs`      | 0.4.4 | 0.5.1  | Bundles Clair3 2.0.3 for its germline step                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+  | `deepvariant` | 1.9.0 | 1.10.0 | nf-core update; `make_examples` now takes its flags from the model (`--checkpoint`): the ONT model is 199 wide with the new fuzzy channel, and both models call easy sites with the small model                                                                                                                                                                                                                                                                                                                                                                           |
+  | `deepsomatic` | 1.7.0 | 1.10.0 | `make_examples_somatic` now takes its flags from the model (`--checkpoint`) instead of a hard-coded channel list and pileup width; the paired PacBio model is 147 wide with 9 channels. The small model is kept off explicitly (`--nocall_small_model_examples`), because the module does not pass its calls on                                                                                                                                                                                                                                                           |
+
+- [#214](https://github.com/IntGenomicsLab/lrsomatic/pull/214) - Updated Wakhan from 0.4.4 to 0.5.0, run from `docker.io/ljwharbers/wakhan:0.5.0-histfix-1afd44d` (`oras://docker.io/ljwharbers/wakhan-sif:0.5.0-histfix-1afd44d` for Singularity/Apptainer) because no biocontainer was built for it. The image carries upstream commit 1afd44d4, which fixes the `IndexError` in `update_coverage_hist` that killed WAKHAN on some samples (KolmogorovLab/Wakhan#64). **The `wakhan/` output layout changes**: each solution is now `solution_{ploidy}_{purity}_{confidence}/` with `integer_profile.{bed,vcf,html,pdf}` (one boundary-aligned table for both haplotypes, replacing the two `*_HP_{1,2}.bed` files), `subclonal_profile.*`, `genes/genes_copynumber_states.*` and a `HiScanner_plots_data.zip` for the Wakhan web viewer; ranked solutions are `solution_rank_{n}`. Wakhan 0.5.0 no longer writes the per-chromosome `variation_plots`, `genome_copynumbers_details` plots or per-solution `loh_regions*.bed`. `LRSOMATICREPORT` now runs lrsomatic_report v1.6.1 (`ghcr.io/ljwharbers/lrsomatic-report:1.6.1`), which finds the copy-number plots in both layouts; v1.6.0 silently dropped them for 0.5.0 output. The WAKHAN stub now writes the outputs the module declares, and the version comes from `wakhan --version` (@ljwharbers).
+
 ## v1.1.0 - [2026-04-28]
 
 ### `Added`

conf/modules.config

@@ -122,20 +122,21 @@ process {
     withName: '.*:BCFTOOLS_NORM' {
         ext.prefix = { "${meta.id}.${meta.caller}_norm" }
         // 'consensus' splits multi-allelics so BCFTOOLS_ISEC can match them per ALT; BCFTOOLS_NORM_REJOIN rejoins them.
-        ext.args = { meta.split ? "-m -any -Oz" : "-Oz" }
+        // Warn, not abort, on REF mismatches: Clair3 writes N where GRCh38 has IUPAC codes, which bcftools 1.24 rejects
+        ext.args = { meta.split ? "-m -any -Oz --check-ref w" : "-Oz --check-ref w" }
         publishDir = [
             enabled: false
         ]
     }
     // Rejoin split multi-allelics ('consensus' only); LongPhase keys variants by position and cannot take two records at one POS.
     withName: '.*:GERMLINE_CONSENSUS:BCFTOOLS_NORM_REJOIN' {
         ext.prefix = { "${meta.id}_germline_sorted" }
-        ext.args   = { '-m +any --output-type z --write-index=tbi' }
+        ext.args   = { '-m +any --check-ref w --output-type z --write-index=tbi' }
         publishDir = [ enabled: false ]
     }
     withName: '.*:SOMATIC_CONSENSUS:BCFTOOLS_NORM_REJOIN' {
         ext.prefix = { "${meta.id}_somatic_sorted" }
-        ext.args   = { '-m +any --output-type z --write-index=tbi' }
+        ext.args   = { '-m +any --check-ref w --output-type z --write-index=tbi' }
         publishDir = [ enabled: false ]
     }
 
@@ -179,12 +180,12 @@ process {
     // Split multi-allelics in both inputs so the CHROM,POS,REF,ALT verdict match works per ALT.
     withName: '.*:DS_VERDICT_SPLIT_DS' {
         ext.prefix = { "${meta.id}.deepsomatic_split" }
-        ext.args   = { '-m -any --output-type z --write-index=tbi' }
+        ext.args   = { '-m -any --check-ref w --output-type z --write-index=tbi' }
         publishDir = [ enabled: false ]
     }
     withName: '.*:DS_VERDICT_SPLIT_DV' {
         ext.prefix = { "${meta.id}.deepvariant_split" }
-        ext.args   = { '-m -any --output-type z --write-index=tbi' }
+        ext.args   = { '-m -any --check-ref w --output-type z --write-index=tbi' }
         publishDir = [ enabled: false ]
     }
 
@@ -226,7 +227,7 @@ process {
     // Rejoin split multi-allelics; LongPhase keys variants by position and cannot take two records at one POS.
     withName: '.*:DS_GERMLINE_REJOIN' {
         ext.prefix = { "${meta.id}.deepvariant_germline" }
-        ext.args   = { '-m +any --output-type z --write-index=tbi' }
+        ext.args   = { '-m +any --check-ref w --output-type z --write-index=tbi' }
         publishDir = [ enabled: false ]
     }
 
@@ -239,12 +240,10 @@ process {
         ]
     }
 
+    // DeepVariant 1.10 checkpoints carry their make_examples flags in model.example_info.json, read via
+    // --checkpoint: the ONT model is 199 wide with the new fuzzy channel, and both enable the small model
     withName: '.*DEEPVARIANT_MAKEEXAMPLES' {
-        ext.args = {
-            meta.platform == 'pb'
-                ? '--channel_list "BASE_CHANNELS,haplotype,base_6ma" --alt_aligned_pileup "diff_channels" --pileup_image_width "147"'
-                : '--channel_list "BASE_CHANNELS,haplotype" --alt_aligned_pileup "diff_channels" --pileup_image_width "99"'
-        }
+        ext.args = { "--checkpoint '/opt/models/${meta.platform == 'pb' ? 'pacbio' : 'ont_r104'}'" }
         publishDir = [
             enabled: false
         ]
@@ -272,12 +271,12 @@ process {
         ]
     }
 
+    // DeepSomatic 1.10 checkpoints carry their make_examples flags (channels, pileup width, thresholds) in
+    // model.example_info.json, which make_examples_somatic reads when given --checkpoint; the paired PacBio
+    // model is 147 wide and 9 channels, so hard-coded flags would no longer match call_variants.
+    // The small model stays off: the module does not collect its calls, so a checkpoint enabling it would drop them
     withName: '.*DEEPSOMATIC_MAKEEXAMPLES' {
-        ext.args = {
-            meta.paired_data
-                ? '--channel_list "BASE_CHANNELS,haplotype" --alt_aligned_pileup "diff_channels" --pileup_image_width "99"'
-                : '--channel_list "BASE_CHANNELS,haplotype,allele_frequency" --alt_aligned_pileup "diff_channels" --pileup_image_width "99"'
-        }
+        ext.args = { "--checkpoint '/opt/models/deepsomatic/${meta.platform == 'pb' ? 'pacbio' : 'ont'}${meta.paired_data ? '' : '_tumor_only'}' --nocall_small_model_examples" }
         publishDir = [
             enabled: false
         ]
@@ -299,15 +298,7 @@ process {
     }
 
     withName: '.*DEEPSOMATIC_CALLVARIANTS' {
-        ext.args = {
-            meta.platform == 'pb'
-                ? (meta.paired_data
-                    ? "--checkpoint '/opt/models/deepsomatic/pacbio'"
-                    : "--checkpoint '/opt/models/deepsomatic/pacbio_tumor_only'" )
-                : (meta.paired_data
-                    ? "--checkpoint '/opt/models/deepsomatic/ont'"
-                    : "--checkpoint '/opt/models/deepsomatic/ont_tumor_only'")
-        }
+        ext.args = { "--checkpoint '/opt/models/deepsomatic/${meta.platform == 'pb' ? 'pacbio' : 'ont'}${meta.paired_data ? '' : '_tumor_only'}'" }
         publishDir = [
             enabled : false
         ]
@@ -739,25 +730,13 @@ process {
         ]
     }
 
-    withName : '.*:WGET' {
-        ext.args = {
-                    [
-                        "--no-check-certificate"
-                    ].join(' ').trim() }
-        ext.suffix = { "tar.gz" }
-        publishDir = [
-            enabled: false
-        ]
-    }
-
-    withName : '.*:UNTAR' {
+    withName : '.*:CLAIR3_MODEL' {
         publishDir = [
             enabled: false
         ]
     }
 
     withName : '.*:WGET_(REVEL|EVE)' {
-        ext.suffix = { "zip" }
         publishDir = [
             enabled: false
         ]