Skip to content

Reimplement lazy importing using importlib - #459

Merged
lohedges merged 1 commit into
develfrom
fix_biosimspace_545
Aug 5, 2026
Merged

Reimplement lazy importing using importlib#459
lohedges merged 1 commit into
develfrom
fix_biosimspace_545

Conversation

@lohedges

@lohedges lohedges commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This PR switches to using our own internal, native Python lazy import system. This fixes several issues with the third party package that we were using:

  • Fixed spurious isinstance()/pickling failures across process boundaries. This was the original production bug that started this work: lazily-loaded modules could end up as two distinct class objects for the same module path when independently loaded in separate worker processes (joblib/multiprocessing), breaking type checks on objects passed between them.
  • Fixed the same duplicate-object bug for direct submodule imports reached before the parent package had been touched (e.g. via import pkg.sub.leaf, or pickle resolving a class's module).
  • Silent corruption when third-party code introspects sys.modules. Encountered via openff/nagl: PyTorch's custom_op registration calls inspect.getmodule(), which checks hasattr(module, 'file') on every entry in sys.modules — under lazy_import, that forced every lazily-loaded module in the process to load all at once, mid-import of unrelated code, with no error raised. Not a crash, but a silently wrong result (an empty/corrupted return value, NAGL charge assignment silently disabled) that's much harder to catch than a clean failure.

This also removes a GPLv3 dependency from our stack.

Written and debugged with help from Claude.

  • I confirm that I have merged the latest version of devel into this branch before issuing this pull request (e.g. by running git pull origin devel): [y]
  • I confirm that I have added a test for any new functionality in this pull request: [y]
  • I confirm that I have added documentation (e.g. a new tutorial page or detailed guide) for any new functionality in this pull request: [y]
  • I confirm that I have added a changelog entry to the changelog (we will add a link to this PR as part of the review): [y]
  • I confirm that I have permission to release this code under the GPL3 license: [y]

@lohedges lohedges added bug Something isn't working enhancement New feature or request labels Aug 4, 2026
@lohedges
lohedges force-pushed the fix_biosimspace_545 branch from 69354be to 9a420d9 Compare August 5, 2026 08:38
@lohedges
lohedges force-pushed the fix_biosimspace_545 branch from 9a420d9 to b216c00 Compare August 5, 2026 09:11
@lohedges
lohedges force-pushed the fix_biosimspace_545 branch from b216c00 to ee5db44 Compare August 5, 2026 09:25
@lohedges

lohedges commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

CI failures are caused by actions flakiness and intermittent I/O and connection glitches.

@lohedges
lohedges merged commit 19e4934 into devel Aug 5, 2026
2 of 5 checks passed
@lohedges
lohedges deleted the fix_biosimspace_545 branch August 5, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant