computeMatrix: decode gzipped BED lines in sortMatrix (fixes #1423) - #1451
Closed
cindykrafft wants to merge 1 commit into
Closed
computeMatrix: decode gzipped BED lines in sortMatrix (fixes #1423)#1451cindykrafft wants to merge 1 commit into
cindykrafft wants to merge 1 commit into
Conversation
With the default --sortRegions keep, computeMatrix re-reads the regions
file in computeMatrixOperations.sortMatrix to restore the input order.
openPossiblyCompressed opens a gzipped file in binary mode, and while
getNext decodes the header lines, loadBED iterated the remaining lines
as bytes, so every gzipped BED crashed with
TypeError: startswith first arg must be bytes or a tuple of bytes, not str
loadGTF already decoded its lines; do the same in loadBED. Tests for
computeMatrix and computeMatrixOperations sort on a gzipped copy of the
existing test BEDs, which fail before this change.
Fixes deeptools#1423
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TaHntBDKuZJpMAAMenkC44
Member
|
Thanks for the work, this was adressed in #1452 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Welcome to deepTools GitHub repository! Please check the following regarding
your pull request :
Fixes #1423.
This is the
deeptoolsintervals/parse.py#L15pattern you pointed at in the issue thread: the sameif not isinstance(line, str): line = line.decode('ascii')guard thatgetNext()uses there and thatloadGTF()already has a few lines further down in this file, applied toloadBED(). Opening it because the issue is still open since March and unassigned, but obviously please close this if this is not needed.Cause. With the default
--sortRegions keep,computeMatrixre-reads the regions file incomputeMatrixOperations.sortMatrix()to restore the input order.deeptoolsintervals.openPossiblyCompressed()opens a gzipped file in binary mode;getNext()decodes the header/first line, butloadBED()then iterates the remaining lines of the handle asbytes, soline.startswith("#")raisesTypeError: startswith first arg must be bytes or a tuple of bytes, not str. The matrix itself is computed fine (the deeptoolsintervals parser handles compressed files); the crash happens right before the matrix is written, so everycomputeMatrixrun on a.bed.gzfails unless--sortRegions no/ascend/descendis given.computeMatrixOperations sort -R regions.bed.gzfails the same way.loadGTF()in the same file already decodes its lines.Fix. Decode non-
strlines inloadBED()exactly asloadGTF()does (two lines indeeptools/computeMatrixOperations.py). No change for plain-text BED files.Tests.
test_heatmapper.py::test_computeMatrix_gzipped_bedrunscomputeMatrix reference-pointon a gzipped copy of the existingtest2.bedand checks the matrix equalsmaster.mat;test_computeMatrixOperations.py::testsortGzippedBEDrunscomputeMatrixOperations sorton a gzipped copy ofcomputeMatrixOperations.bedand checks the same md5 as the existingtestsort. Both fail onmasterwith theTypeErrorabove and pass with the fix.Run.
pytest deeptools/test/test_heatmapper.py deeptools/test/test_computeMatrixOperations.py: 21 passed + 2 failed (the new tests) before, 23 passed after; full suite 100 passed with the fix (the two remaining failures,test_plotCoverage_defaultandtest_tools, fail identically on unmodifiedmasterin this environment).flake8with the CI options is clean on the changed files. ACHANGES.txtbullet is included under anunreleasedheading.Found while working through open issues in Mytochondria, a volunteer project that verifies fixes for the software behind published results (methods and harnesses: https://github.com/cindykrafft/mytochondria/tree/main/audits/deeptools)
Generated by Claude Code