Skip to content

fix: support llama.cpp runtimes across desktop platforms - #4

Open
FangzhiLi wants to merge 4 commits into
wmd3i:mainfrom
FangzhiLi:fix/multiplatform-llama-runtime
Open

fix: support llama.cpp runtimes across desktop platforms#4
FangzhiLi wants to merge 4 commits into
wmd3i:mainfrom
FangzhiLi:fix/multiplatform-llama-runtime

Conversation

@FangzhiLi

Copy link
Copy Markdown
Collaborator

Summary

  • add versioned llama.cpp b9990 runtimes for macOS ARM64, macOS Intel x64, and Windows x64
  • resolve the repo runtime from the current operating system and CPU architecture in Web development mode
  • stage and validate only the matching runtime and packaged backend for each Electron target
  • document the supported platforms, build commands, and local Qwen GGUF download behavior

Why

The local Demo flow already used llama-server, but the repo lookup and Electron packaging were tied to a single macOS runtime path. macOS Intel and Windows development could not reliably select their platform runtime, and Intel packaging was not reproducible from the current repo layout.

User impact

Users on macOS ARM64, macOS Intel x64, and Windows x64 can use the bundled platform runtime without installing Ollama or llama.cpp separately. The Qwen GGUF model remains an on-demand download and is not committed to the repository.

Validation

  • PYTHONPATH=backend python3 -m unittest backend.tests.test_llama_runtime -v — 4 passed
  • cd frontend && npm test — 131 passed
  • runtime preparation/architecture validation passed for macos-arm64, macos-x64, and windows-x64
  • git diff --check upstream/main...HEAD — passed
  • macOS ARM64 and x64 llama-server --version both report b9990 (259ae1df8)

Follow-up validation

  • macOS Intel and Windows Electron/Web Demo flows still need testing on matching physical or CI hosts.
  • Full frontend typecheck/build is currently blocked by an existing upstream/main reference to the missing frontend/src/components/icons/DbIcon module; this PR does not modify that unrelated UI code.

@FangzhiLi
FangzhiLi marked this pull request as ready for review July 22, 2026 21:58
@FangzhiLi
FangzhiLi requested a review from hanxu12 July 22, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant