Skip to content

refactor: separate model loading and execution contexts - #33

Open
zhangzhenghao wants to merge 1 commit into
mainfrom
refactor/models
Open

zhangzhenghao wants to merge 1 commit into
mainfrom
refactor/models

Conversation

@zhangzhenghao

@zhangzhenghao zhangzhenghao commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Refactor all six model architectures following llama.cpp's separation of architecture loading and architecture-specific execution (reference: ggml-org/llama.cpp at caa8191df159f764fb0160e3d2fc6be008948aee).

  • Keep validated configuration, tensor contracts, capability queries, and embedded backbone ownership in each model specialization.
  • Move inference helpers and TensorOps into nested architecture-specific contexts behind a shared ModelContext contract and build_arch_context factory.
  • Construct each context once after parameter/tensor validation and retain backend selection and uploaded-weight caches across calls.
  • Preserve public APIs, GGUF contracts, errors, numerical operation order, embedded CLaMP3/MERT and MuQ-MuLan/MuQ reuse, and existing feature-test access.

This is an execution-organization refactor of the existing eager TensorOps runtime, not a conversion to a persistent whole-model GGML graph.

Validation

  • Independent code review approved exact tree 4204cd39b45a066869fbebfafbc39b4b1d502fd1; no blocking findings.
  • Static and shared Release builds succeeded with GGML_NATIVE=OFF.
  • Unmodified main baseline default CTest: 26/26 passed.
  • Refactored shared default CTest: 26/26 passed.
  • Refactored static CTest with all six architecture oracle options enabled: 42/44 passed, including CLAP, CLaMP3, MERT, MERT2, MuQ, bundled frontend, and quantized encoder parity.
  • Both optional MuQ-MuLan failures were reproduced on unmodified main (519a84e3b6dafe1534ecf93693f96855712adcfb): muq_mulan_model_parity expects an obsolete --audio-features CLI error string; its early exit prevents F16 fixture generation, causing muq_mulan_oracle_pipeline to fail on the missing fixture.
  • An external supplemental harness relaxed only that obsolete error-wording assertion while preserving command rejection and all numerical assertions. MuQ-MuLan F32/F16 native/CLI/resampling comparisons then passed, and the unchanged downstream oracle pipeline passed with both fixtures present. Repository tests were not modified; this is supplemental evidence, not a claim that the unmodified full CTest suite is green.
  • Independent comparison confirmed pre-existing model method bodies are unchanged after context-scope normalization. git diff --check passed.

Remote CI

Verified on head 1e6f4d1284ede148e5c2442b2c649426023a4b99: 8/9 checks passed.

  • Python CPU wheel/build/install/binding tests: all four platforms passed (Linux x64/ARM64, macOS, Windows x64), run 40396825113.
  • Native CI: Linux x64/ARM64, macOS, and Windows x64 passed, run 40396825231.
  • Windows ARM64 fails while generating audio fixtures: the installed external ffmpeg-N-127043-g5a54fcf75e-winarm64-gpl/bin/ffmpeg.exe crashes converting mono.wav to mono.flac with exit status 3221225477. This is not a model-source compiler/test failure.
  • Confirmed the identical FFmpeg binary, command, and exit status on the current main baseline 519a84e3b6dafe1534ecf93693f96855712adcfb, CI run 40102240289 (job 120896596496). Workflow, fixture generator, and root CMake files are unchanged by this PR. The Windows ARM64 failure is an existing dependency issue, not introduced by this refactor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants