Repository navigation
Conversation
llama.cpp is pinned at f05c8b278, a commit whose vendored ggml is the same 0.26.0 as zee's GGML_BASE, and `make llama-lib` builds only libllama with -DLLAMA_USE_SYSTEM_GGML=ON against the ggml parakeet installs: one ggml in the process, as with whisper. The build stops if the submodule moves off LLAMA_BASE. internal/llama loads a GGUF chat model on Metal and completes an already-formatted prompt greedily until an end-of-generation token. Checked with Gemma 4 E2B (Q4_K_M). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
post_transcribe.model "gemma-4-e2b" runs the rewrite in-process through internal/llama instead of Groq. The model is registered in localmodel under a new llama engine (kept out of the STT menus and benchmarks), loaded when a recording starts and kept resident. The local prompt names the transcript's language: SessionResult.Lang carries the forced language or local Whisper's detection, and main falls back to the configured language. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
post_transcribe.provider is "groq" (the default) or "local", like the top-level provider/model pair. Each provider has its own default model, so "provider": "local" alone runs Gemma 4 E2B. A model that does not fit the provider, or an unknown provider, fails with a clear error instead of a request to Groq. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
info.json's post_transcribe block and the post_transcribe log line now name the provider (groq or local) beside the model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zee links libllama.a since the local post-transcribe backend, but test-integration and the GoReleaser hook built only whisper-lib, so the link failed (CI's integration job, and the next release build). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SessionResult.Lang is now set by cloud sessions too: the forced language, else what the provider detected — Groq Whisper's verbose_json name, Deepgram's multi-mode languages, Mistral's language, ElevenLabs' language_code — normalized to ISO 639-1 (golang.org/x/text/language for "tur"/"tr", a name lookup for "Turkish"). OpenAI's gpt-4o-transcribe reports none. main.go passes result.Lang as is; its fallback is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Checked live: voxtral-mini-latest returns language null without a
language parameter; ElevenLabs returns ISO 639-3 ("tur", "eng").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Same quality as Q4_K_M on the private eval (50 cases x 2 profiles: 20 vs 22 hand-checked failures, 0 flips each, same latency) at 0.57 GB less to download. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each profile in post_transcribe.profiles carries its whole system prompt
(required), a tray description and an overlay colour (optional). Prompts
are Go text/template over Vars ({{.Language}}, {{.Hints}}); an unknown
variable fails the rewrite with the list of variables, and a test keeps
docs/reference.md listing every one. zee seeds the defaults from the
embedded defaults.json the first time and folds the earlier prompts and
colors maps in once; from then on config.json is the only copy. The
defaults render exactly the prompt the eval measured (guarded by a test).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Turn off parakeet-server (an example HTTP server) and ggml's OpenMP and ccache lookups, neither found nor used; the libraries zee links are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Load returned nil on a parse error, so startup's post_transcribe normalize saved the defaults over the user's file. Load now returns the error, still leaving the defaults live, and startup does not save then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
omitempty dropped an empty profiles map on save, so a user who deleted every profile got the defaults back on the next start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An earlier prompt that replaced a default kept the default's
description, a custom one got none, and a "{{" in it broke the
template. The instruction is now the description, and its "{{" is
quoted so it stays text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the Intel slice and Linux, provider local reported the model as not downloaded, though placing it would not help. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generation that hit max_tokens, or dropped a piece that did not fit, returned the partial text as success, and zee pasted it. Both are now errors, so the raw transcript is pasted. The reply also gets the room the prompt leaves in the context when that is less than max_tokens, instead of refusing a prompt longer than n_ctx - max_tokens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The KV-cache claim was not checked, and the <turn|> check could not fail: special tokens render empty. Greedy replies must now match, and a reply cut off before end-of-generation fails Generate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Gemma model stayed resident after a switch to Groq or to None. A tray pick and Reload Config now call posttranscribe.Release, which frees it unless the config still runs it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ange 806db2b kept startup from saving the defaults over a corrupt config.json, but the first later setting change (tray pick, hotkey, tag colour) still did. Load now marks the settings as standing in for an unparsed file, and Update does not save until a Load or Reload succeeds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3febc8a quoted "{{" only, so an instruction ending in "{" joined the
template's "{{if .Hints}}" into "{{{if" and the profile did not parse.
Each "{" is now quoted on its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itself Release took localMu, which loadLocal holds for the whole load, so picking None in the tray during a load froze the menu bar. It now unloads in the background. A recording keeps the config it started with, so its rewrite could reload the model after the user moved off it; finishTranscription now calls Release after the rewrite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sumerc
force-pushed
the
local-post-transcribe
branch
from
October 26, 2026 21:49
00e5faa to
fd79b72
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — local post-transcribe rewrite, no cloud call.
Plan
f05c8b278(its ggml is the same 0.26.0 as zee'sGGML_BASE);make llama-libbuilds only libllama with-DLLAMA_USE_SYSTEM_GGML=ON→ one ggml in the process, as with whisper/parakeet.internal/llama: cgo wrapper — load a GGUF chat model on Metal, greedy completion until end-of-generation. Checked with Gemma 4 E2B Q4_K_M.posttranscribe: local backend (Gemma 4 chat format, thinking off) next to Groq, selected bypost_transcribe.provider: "local"(groqstays the default; each provider has its own default model, and a model that does not fit the provider is an error); loaded at recording start, kept resident. The language line ("The transcript is in .") comes fromSessionResult.Lang(forced language or local Whisper's detection), else the configured language.localmodel: Gemma 4 E2B registered (enginellama, checksum, opt-in); kept out of the STT menus andbench-local.models-v4release and bumplocalmodel.Version/install.sh'sMODELS_TAG. Until then, placegemma-4-E2B-it-Q4_K_M.ggufin the models folder by hand.Open
Why Gemma 4 E2B + language line
On the private eval (2026-10-22, 166 transcripts × 3 profiles, M5 Pro): ~0.27 s per rewrite (vs ~0.49 s for Groq gpt-oss-120b), ~3.1 GB; the language line stops Turkish → English flips (65 → 0). Known weak spot: it sometimes translates English words inside Turkish into Turkish.
🤖 Generated with Claude Code