Skip to content

Update dependencies.py to allow for compiling with -j flag - #382

Merged
max-models merged 10 commits into
develfrom
parallel-compile
Sep 29, 2026
Merged

max-models merged 10 commits into
develfrom
parallel-compile

Conversation

@max-models

@max-models max-models commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Enable parallel kernel compilation with struphy compile -j N, preserving dependency order and keeping serial compilation as the default.

  • Add -j/--jobs to the CLI and jobs to Compiler and Compiler.compile().

  • Replace import-based dependency discovery with static AST parsing, avoiding kernel execution and temporary source copies.

  • Enable parallel compilation in CI, defaulting to the available CPU count capped at 8, with a configurable override.

  • Add tests for dependency detection, dependency graph consistency, and job-count propagation through the CLI and Python API.

@max-models
max-models marked this pull request as ready for review September 21, 2026 08:43
@max-models
max-models requested a review from spossann September 21, 2026 08:43
@max-models
max-models marked this pull request as draft September 27, 2026 12:25
@max-models
max-models marked this pull request as ready for review September 29, 2026 11:22

@spossann spossann left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got a 2x speed up when using struphy compile -j 2 - nice! Should we make -j 2 the default?

@max-models
max-models merged commit f733e58 into devel Sep 29, 2026
30 checks passed
@max-models
max-models deleted the parallel-compile branch September 29, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants