diff --git a/README.md b/README.md index 10c540b..c41bd39 100755 --- a/README.md +++ b/README.md @@ -200,6 +200,29 @@ $ python3 SigProfilerMatrixGenerator cnv_matrix_generator BATTENBERG ./SigProfilerMatrixGenerator/references/CNV/example_input/Battenberg_test.tsv BATTENBERG-TEST ./SigProfilerMatrixGenerator/references/CNV/example_output/ ``` +## Reproducing earlier analyses + +The corrected reference archives use the usual genome names (for example, +`GRCh38`). The previously distributed archives remain available under matching +`_Legacy` names (for example, `GRCh38_Legacy`). To rerun an earlier analysis, +install the Legacy reference and pass that same name to matrix generation: + +```bash +SigProfilerMatrixGenerator install GRCh38_Legacy +SigProfilerMatrixGenerator matrix_generator previous_project GRCh38_Legacy /path/to/original_inputs +``` + +Replace `GRCh38` with the genome used in the original analysis. Legacy archives +are available for `GRCh38`, `GRCh37`, `mm9`, `mm10`, `mm39`, `rn6`, `rn7`, `dog`, +and `c_elegans`. An older reference already installed as `GRCh38` is not +automatically renamed to `GRCh38_Legacy`; install the Legacy identity explicitly. +Use the corrected name for new analyses. For an exact reproduction, also retain +the original input data, software versions, and analysis settings: the Legacy +archive alone cannot reproduce differences caused by software changes. + +See the [Legacy reference guide](docs/Legacy-References.md) for Python and +offline installation instructions. + **SUPPORTED GENOMES** This tool currently supports the following genomes: diff --git a/docs/Currently-Supported-Genomes.md b/docs/Currently-Supported-Genomes.md index 15d6a65..10464d8 100644 --- a/docs/Currently-Supported-Genomes.md +++ b/docs/Currently-Supported-Genomes.md @@ -11,6 +11,9 @@ These are the currently supported genomes: +For analyses made with the formerly distributed reference archives, see the +[Legacy reference guide](Legacy-References.md) for installation and usage. + - [GRCh38.p12 [GRCh38]][6]
*GRCh38.p12 [GRCh38] (Genome Reference Consortium Human Reference 38), INSDC Assembly GCA_000001405.27, Dec 2013. Released July 2014. Last updated January 2018.
This genome was downloaded from ENSEMBL database version 93.38.* @@ -26,7 +29,7 @@ new analyses. `GRCh37_Legacy` retains the historical labels for reproducibility. `mm39` uses corrected transcription-strand data for new analyses. The previously distributed reference is available as `mm39_Legacy` for reproducing historical results; existing local `mm39` installations must be reinstalled -after upgrading. See [the mm39 reference record](../reference_data/mm39/README.md). +after upgrading. See [the mm39 reference record](https://github.com/SigProfilerSuite/SigProfilerMatrixGenerator/blob/master/reference_data/mm39/README.md). - [GRCm38.p6 [mm10]][9]
*GRCm38.p6 [mm10] (Genome Reference Consortium Mouse Reference 38), INDSDC Assembly GCA_000001635.8, Jan 2012. Released July 2012. Last updated March 2018.
This genome was downloaded from ENSEMBL database release 94 (previously documented as "93.38"; see `reference_data/mm10/README.md`). Use `mm10` for corrected transcription-strand annotation; use `mm10_Legacy` only to reproduce historical results.* diff --git a/docs/Installation-Python.md b/docs/Installation-Python.md index add3014..ed1dc03 100644 --- a/docs/Installation-Python.md +++ b/docs/Installation-Python.md @@ -161,6 +161,10 @@ $ python3
This example installs the custom human 37 assembly reference files but you can install any of the available [genome assemblies][30]. The installation will use bash commands as default. +To reproduce an analysis that used a previously distributed reference archive, +install and select its `_Legacy` identity instead. See the +[Legacy reference guide](Legacy-References.md) for supported names and examples. + If the server has firewall in place, **wget** will not work. The `genInstall.install` command has an additional `rsync` parameter that must be set to **True** which acts as a wget equivalent. ``` $ python3 diff --git a/docs/Legacy-References.md b/docs/Legacy-References.md new file mode 100644 index 0000000..a358b0b --- /dev/null +++ b/docs/Legacy-References.md @@ -0,0 +1,79 @@ +# Legacy Reference Archives + +Corrected transcription-strand references are distributed under the standard +genome names. The archives distributed before those corrections are retained +under `_Legacy` names so earlier analyses can be reproduced. Use the corrected +name for new analyses; select a Legacy name only when comparing with results +generated using the former reference. + +| Genome | Corrected reference/archive | Historical reference/archive | +| --- | --- | --- | +| GRCh38 | `GRCh38` / `GRCh38.tar.gz` | `GRCh38_Legacy` / `GRCh38_Legacy.tar.gz` | +| GRCh37 | `GRCh37` / `GRCh37.tar.gz` | `GRCh37_Legacy` / `GRCh37_Legacy.tar.gz` | +| mm9 | `mm9` / `mm9.tar.gz` | `mm9_Legacy` / `mm9_Legacy.tar.gz` | +| mm10 | `mm10` / `mm10.tar.gz` | `mm10_Legacy` / `mm10_Legacy.tar.gz` | +| mm39 | `mm39` / `mm39.tar.gz` | `mm39_Legacy` / `mm39_Legacy.tar.gz` | +| rn6 | `rn6` / `rn6.tar.gz` | `rn6_Legacy` / `rn6_Legacy.tar.gz` | +| rn7 | `rn7` / `rn7.tar.gz` | `rn7_Legacy` / `rn7_Legacy.tar.gz` | +| dog | `dog` / `dog.tar.gz` | `dog_Legacy` / `dog_Legacy.tar.gz` | +| c_elegans | `c_elegans` / `c_elegans.tar.gz` | `c_elegans_Legacy` / `c_elegans_Legacy.tar.gz` | + +## Reproduce a Previous Analysis + +Use a SigProfilerMatrixGenerator release that recognizes the `_Legacy` genome +names. Install the Legacy reference, then select that same name when generating +matrices. For example, if the previous analysis used the former GRCh38 archive: + +```bash +SigProfilerMatrixGenerator install GRCh38_Legacy +SigProfilerMatrixGenerator matrix_generator previous_project GRCh38_Legacy /path/to/original_inputs +``` + +The equivalent Python calls are: + +```python +from SigProfilerMatrixGenerator import install as genInstall +from SigProfilerMatrixGenerator.scripts import SigProfilerMatrixGeneratorFunc as matGen + +genInstall.install("GRCh38_Legacy") +matGen.SigProfilerMatrixGeneratorFunc( + "previous_project", "GRCh38_Legacy", "/path/to/original_inputs" +) +``` + +Substitute the appropriate genome name from the table. An older reference +installed under the standard name is **not** automatically renamed to its +`_Legacy` identity after a software upgrade. Install and select the Legacy +identity explicitly. If you also need the corrected reference for new analyses, +install it under the standard name; the two identities can coexist. + +## Install from a Downloaded Archive + +The archives are available at +`ftp://alexandrovlab-ftp.ucsd.edu/pub/tools/SigProfilerMatrixGenerator/`. +If automatic downloading is unavailable, download the exact archive filename +from the table into a local directory, then pass **the directory**, not the +archive path, to the CLI: + +```bash +SigProfilerMatrixGenerator install GRCh38_Legacy --local_genome /path/to/archive-directory +``` + +For this example, the directory must contain `GRCh38_Legacy.tar.gz`. The CLI +checks the installed chromosome files against the registered reference +checksums. Do not rename a corrected archive to a Legacy filename or vice versa. + +## Reproducibility Limits + +The Legacy archive preserves the previously distributed reference data, but +does not restore an older version of the tool. For the closest possible +reproduction, keep the original input files, SigProfilerMatrixGenerator and +dependency versions, command arguments, and any BED/exome files. Differences +caused by code or settings may remain even when the Legacy archive is used. +Record the reference identity (`GRCh38` versus `GRCh38_Legacy`) with each run. + +The separate `GRCh37_havana`, `GRCh38_havana`, and `mm10_havana` references +are not substitutes for these Legacy archives. See +[Reference Correctness](TSB-Reference-Correctness.md) for the scope of the +corrections and [Supported Genomes](Currently-Supported-Genomes.md) for genome +descriptions. diff --git a/docs/TSB-Reference-Correctness.md b/docs/TSB-Reference-Correctness.md index 8977015..e5bfacf 100644 --- a/docs/TSB-Reference-Correctness.md +++ b/docs/TSB-Reference-Correctness.md @@ -129,6 +129,8 @@ Verification does not delete or replace files. An installation created by an earlier release under a corrected default name will fail verification after upgrading; reinstall the corrected identity, or install the corresponding `*_Legacy` identity to reproduce an earlier analysis. +See the [Legacy reference guide](Legacy-References.md) for installation and +matrix-generation examples. ## Tests and Release Limits diff --git a/mkdocs.yml b/mkdocs.yml index b187615..3b1c283 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -66,6 +66,7 @@ nav: - Python: Installation-Python.md - R: Installation-R.md - Supported Genomes: Currently-Supported-Genomes.md + - Legacy References: Legacy-References.md - Reference Correctness: TSB-Reference-Correctness.md - Usage: - SBS, ID, DBS Input: Using-the-Tool-SBS-ID-DBS-Input.md