Skip to content

feat: replace dummy CLI and server with native inference - #32

Merged
zhenghaoz merged 2 commits into
mainfrom
feat/real-embedding-serve
Oct 24, 2026
Merged

zhenghaoz merged 2 commits into
mainfrom
feat/real-embedding-serve

Conversation

@zhangzhenghao

Copy link
Copy Markdown
Collaborator

Summary

  • Replace hard-coded dummy CLI embeddings with native GGUF model inference for text and audio; preserve prompt/file batching, output formats and true cosine similarity.
  • Load a real model once for clap serve, expose real text embeddings and projection dimensions, validate requests and serialize inference on the shared model handle.
  • Remove the standalone clap-model example executable, migrate its callers to clap embedding, and update documentation.
  • Add deterministic stdlib-generated GGUF fixtures with independent numerical oracles so default CLI/HTTP tests exercise actual inference without downloads or PyTorch.

Verification

  • Rebased onto current main.
  • Static Release build and CTest: 26/26 passed.
  • Shared Release build and CTest: 26/26 passed.
  • Default CLI/HTTP integration tests verify numerical outputs, different inputs, batching, dimensions, concurrent calls and failure handling. Two optional model-fixture CLI cases are skipped in default CTest.
  • Prior implementation validation also exercised optional model-fixture CLI cases and staged installation; fresh latest-main validation above covers both complete default suites.
  • CUDA/ROCm inference is not claimed; these interfaces use the existing native CPU/optional Vulkan model runtime.

@zhenghaoz
zhenghaoz merged commit 519a84e into main Oct 24, 2026
8 of 9 checks passed
@zhenghaoz
zhenghaoz deleted the feat/real-embedding-serve branch October 24, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants