Skip to content

Adding leafcutter2 module - #1429

Open
jaempawi wants to merge 6 commits into
StatFunGen:mainfrom
jaempawi:leafcutter2-module
Open

jaempawi wants to merge 6 commits into
StatFunGen:mainfrom
jaempawi:leafcutter2-module

Conversation

@jaempawi

@jaempawi jaempawi commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Add LeafCutter2 splicing calling module

Adds splicing_calling_leafcutter2.ipynb alongside the existing splicing_calling module, so LeafCutter1 can eventually be retired. LeafCutter2 adds a functional label to every junction — PR (productive), UP (unproductive / NMD target), NE (ambiguous), IN (intergenic) — appended as a fifth colon field on the intron ID, e.g. chr10:134786:179993:clu_1_+:PR (Dai et al., 2025).

The existing module is untouched; this one sits beside it.

Workflows

Workflow Purpose
leafcutter2 regtools junction extraction, then clustering, classification and quantification
leafcutter2_preprocessing the first half only, stopping at the list of .junc files
leafcutter2_cluster entry point for junctions extracted elsewhere
star2junc converts STAR SJ.out.tab to the BED6 junction format LeafCutter2 reads

leafcutter2 keeps a default of 1 since its junctions always come from regtools.

Guards

Each one tested to fire:

  • chromosome naming disagreement between junctions and GTF, checked before clustering
  • junction lists mixing BED6 and BED12, which one offset cannot suit
  • empty result after filtering, rather than a later opaque ValueError
  • unrecognised strand values, before they reach regtools as literals
  • colliding BAM basenames, which would silently overwrite a sample's junctions

The [global] block now derives BAM paths, strandness and sample IDs from one filtered row collection, so a blank or NA path drops that sample without shifting the next one's strand. This bug is inherited verbatim from splicing_calling.ipynb, which still has it — worth a follow-up there. It is not silent corruption: SoS fails loudly with Length of provided properties (3) does not match number of groups (2).

GTF, FASTA, .fai and the junction files named in --juncfiles are now declared dependencies. --output-prefix was added so two workflows can share one --cwd, and so a manifest named cohort.v1.sample_list.txt does not collapse to cohort.

Dependency: LeafCutter2 >= 2.0

star2junc calls leafcutter2-star2junc, which exists only in 2.x.

The only conda build of leafcutter2 in any of our channels is 0.0.1 (dnachun/noarch, 2026-01-27) — conda-forge and bioconda have none. It packages the pre-2.0 repository scripts with no __init__.py and no .dist-info, and its classification step fails with:

AttributeError: module 'leafcutter2.Transcript_tools' has no attribute 'ClassifySpliceJunction'

@dnachun is building 2.0.1 for the channel. Once it lands:

  1. add leafcutter2 to StatFunGen/pixi-setup envs/full_packages.txt, beside regtools
  2. the skipped tests below go live, and the remaining workflows can be verified on SCC

To reproduce the verified run before then — note LeafCutter2 is not on PyPI, so pip install leafcutter2 fails:

python -m venv ~/lc2-venv
~/lc2-venv/bin/pip install git+https://github.com/leafcutter2/leafcutter2.git   # 2.0.1
export PATH=~/lc2-venv/bin:$PATH

Tests

tests/notebooks/molecular_phenotypes/calling/test_splicing_calling_leafcutter2.py, currently 2 passed, 1 skipped:

  • test_cluster_requires_explicit_offset — needs no binary and no reference data, so it runs on every CI runner today
  • test_star2junc — skips unless leafcutter2-star2junc is on PATH, which doubles as the >= 2.0 probe
  • test_cluster_and_classify — additionally gated on XQTL_REFERENCE_DATA, since the GTF and FASTA are not in the repository; asserts the five-field ID, the PR/UP/NE/IN label, that cluster_ratios.gz cells carry /, and that junction_counts.gz does not

The guards mean CI is green now and the coverage becomes real the moment the conda package lands, without a red X in the meantime.

Files

  • code/SoS/molecular_phenotypes/calling/splicing_calling_leafcutter2.ipynb
  • tests/notebooks/molecular_phenotypes/calling/test_splicing_calling_leafcutter2.py
  • tests/fixtures/splicing_calling/leafcutter/protocol_example.sample_list.txt
  • tests/fixtures/splicing_calling/leafcutter/protocol_example.bam_list.txt
  • tests/fixtures/splicing_calling/leafcutter/protocol_example.junction_files.txt
  • tests/fixtures/splicing_calling/leafcutter/SAMPLE_001.SJ.out.tab
  • tests/fixtures/splicing_calling/leafcutter/SAMPLE_002.SJ.out.tab

The two SJ.out.tab fixtures are new, derived from the shipped .junc files, so the star2junc MWE runs off the repository as written (~1 s). All four MWEs use the tests/fixtures data with protocol_example.* naming and SAMPLE_001/SAMPLE_002.

Anak Empawi and others added 3 commits September 25, 2026 11:10
TWAS weights released on Synapse use the legacy list layout, which pecotmr
0.8.x does not read. This adds a protocol-level conversion layer, as agreed,
rather than a backwards-compatibility loader in pecotmr.

- twas_weights_conversion.R: convert legacy weight RDS (univariate and
  multicontext) into one pecotmr TwasWeights per gene. Weights and CV metrics
  are copied unchanged; carries traitPos (gene TSS) for cTWAS block placement;
  writes the xqtl_meta row and the context -> data type rows.
- twas_ctwas.ipynb: new optional twas_weights_conversion workflow (skip it for
  weights already in TwasWeights format), with Input/Output/MWE docs and the
  Overview updated; MWE commands now use the converted example tables.
- ctwas_1: --gwas_study now selects the GWAS (cTWAS takes one per LD block) and
  the run fails early if several remain; new --ctwas_method (weights with several
  methods and no ensemble); pass the chromosome's genotype prefix as --ld-sketch
  so ctwasPipeline sees a real LD file rather than a "<chrom-meta>" placeholder.
- ctwas_assemble.R: drop study elements and LD blocks with no GWAS variants
  (they carry no genome build and broke combineGwasSumStats).

Tested: conversion is exact for eQTL, pQTL and sQTL weights (weights compared
element by element); TWAS runs on converted weights for 8 studies in the APOE
region; cTWAS assemble passes. cTWAS parameter estimation and fine-mapping are
not yet tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Csueazw8VDi9a3mhLvgjQg
- fine_mapping_conversion.R: convert legacy FunGen-xQTL QTL fine-mapping
  (*.univariate_bvsr.rds) into pecotmr QtlFineMappingResult, one per study and
  gene, with the same study/context labels as twas_weights_conversion.
- twas_ctwas.ipynb: new optional fine_mapping_conversion step; twas passes
  --fine-mapping-result when --fine_mapping_dir has a file for the gene, plus
  --mr_method and --mr_pip_cutoff; Inf is passed to R as Inf; docs note that
  --rsq_cutoff 0 --rsq_pval_cutoff inf keeps every method.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Csueazw8VDi9a3mhLvgjQg
@jaempawi
jaempawi force-pushed the leafcutter2-module branch from 15edca7 to 1d55835 Compare October 1, 2026 23:54
- regtools_junctions.sh: optional --min-mapq (default 0, off). regtools has no
  mapping-quality filter, so multimapping reads add junction support at every
  location they map to; --min-mapq 255 keeps STAR's uniquely mapped reads only.
- splicing_calling_leafcutter2.ipynb: --min-mapq on the junction step, and a new
  differential_splicing workflow. It splits the LeafCutter1-format counts by
  chromosome, runs leafcutter_ds.R on each part as its own job, then joins the
  parts and recomputes the Benjamini-Hochberg FDR over all clusters, so results
  match a single unsplit run.
- leafcutter_ds_split.py / leafcutter_ds_merge.py: split and join helpers; the
  split checks the groups file against the count table before any job runs.
- leafcutter_ds.R: LeafCutter's script, restricted to samples present in both
  the counts and the groups file.
- Docs: Overview, Input, Output, example, help text and troubleshooting updated;
  NE and IN label descriptions corrected.
- Tests for the split, the FDR recomputation and --min-mapq validation.

Checked on a 200-brain cohort: --min-mapq 255 reproduces junctions extracted
with samtools view -q 255 exactly, and differential_splicing on chr21/22 gives
identical p-values to an unsplit leafcutter_ds.R run.
leafcutter2_2 and leafcutter2_cluster carried the same inline block: the
mixed-format and chromosome-naming checks, the leafcutter2 call, and the
empty-result check. It now lives once in leafcutter2_cluster.sh, which both
steps call, matching the other steps of the module.

Outputs on the repository fixtures are byte-identical before and after
(junction counts, cluster ratios, LeafCutter1 tables, classifications).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant