Conversation
Engine & Model → Apple Intelligence gains two switches:
- Fall Back to Open Source Model (on by default, today's behavior): when
Apple Intelligence rejects the text's language, the router retries with
the selected Open Source model. Off returns no suggestion
("appleLanguageUnsupported") and never loads the local model.
- Keep Fallback Model Loaded (off by default): keeps the local runtime
started while Apple Intelligence is selected, so the first fallback
suggestion does not wait for a multi-GB model to load.
The row names the model the fallback uses and warns when no Open Source
model is selected. The fallback always uses the selected Open Source model
because the local runtime holds one model at a time. AppDelegate now
re-evaluates the runtime when either switch changes, as it does for an
engine change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 10 minutes. View limit detailsLimit details: You’ve used all 4 included reviews currently available. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (10)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| return SuggestionResult( | ||
| generation: request.generation, rawText: "", text: "", latency: 0, | ||
| suppressionReason: "appleLanguageUnsupported" | ||
| ) |
There was a problem hiding this comment.
When Apple Intelligence rejects a language and fallback is off, this return skips the router's quality accounting. The coordinator also skips counting it because the result has a suppression reason. As a result, these requests are missing from both the generated count and the appleLanguageUnsupported suppression count.
| return SuggestionResult( | |
| generation: request.generation, rawText: "", text: "", latency: 0, | |
| suppressionReason: "appleLanguageUnsupported" | |
| ) | |
| let result = SuggestionResult( | |
| generation: request.generation, rawText: "", text: "", latency: 0, | |
| suppressionReason: "appleLanguageUnsupported" | |
| ) | |
| recordQualityOutcome(result) | |
| return result |
| Toggle(isOn: Binding( | ||
| get: { suggestionSettings.keepsFallbackModelLoaded }, | ||
| set: { suggestionSettings.setKeepsFallbackModelLoaded($0) } | ||
| )) { | ||
| SettingsRowLabel( | ||
| title: "Keep Fallback Model Loaded", | ||
| description: "Load the fallback model in advance so its first suggestion doesn't wait for it " + | ||
| "to load. Uses the model's memory (several GB) while Apple Intelligence is selected.", | ||
| systemImage: "memorychip" | ||
| ) | ||
| } | ||
| .disabled(!suggestionSettings.isAppleLanguageFallbackEnabled) |
There was a problem hiding this comment.
“Keep Fallback Model Loaded” has no search item or row anchor. Its search terms, including “keep loaded” and “preload,” belong to the fallback switch's item, so searching for the new setting highlights the other switch instead. Give the keep-loaded switch its own indexed item and anchor so search can take users directly to it.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…s in a linked folder - The Open Source section is hidden while Apple Intelligence is the engine, so the fallback model could not be chosen anywhere visible. The Apple Intelligence section now has a Fallback Model picker (the shared Open Source selection, since the runtime holds one model). The warning now reports "no downloaded models" instead of reading the stopped runtime's selection, which is nil while it is unloaded. - Model discovery resolves a symlinked models folder before walking it: FileManager.enumerator does not descend into a symlinked root, so a folder linked to an external drive or another location listed nothing and the app cleared the saved model selection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| .font(.caption) | ||
| .foregroundStyle(.orange) | ||
| } else { | ||
| Picker(selection: selectedModelBinding) { |
There was a problem hiding this comment.
When “Keep Fallback Model Loaded” is off, changing this picker still calls runtimeModel.selectModel, which prepares and loads the selected GGUF. The model can then remain in memory while Apple Intelligence is selected, despite the user choosing not to keep it loaded.
This branch has not been deployed
1 participant
Summary
When Apple Intelligence rejects a language (for example Macedonian),
SuggestionEngineRoutersilently retries with the Open Source model. Users had no way to see or control that. This adds two switches under Engine & Model → Apple Intelligence:suppressionReason: "appleLanguageUnsupported"and never touches the local model, so unsupported languages get no suggestion and no multi-GB model load.AppDelegate.startRuntimeIfPreferredEngineRequiresItkeeps the local runtime started under Apple Intelligence. Without it, the first fallback suggestion waits for the GGUF to load on demand (LlamaRuntimeManager.preparedRuntime()).The rows name the model the fallback uses and warn when no Open Source model is selected. The fallback always uses the selected Open Source model, because the local runtime holds one model at a time and a separate fallback model would mean swapping weights mid-typing. AppDelegate now re-evaluates the runtime when either switch changes, as it already did for engine changes.
Persisted as
cotabbyAppleLanguageFallbackEnabled(defaulttrue) andcotabbyKeepFallbackModelLoaded(defaultfalse) in the engine settings domain, and registered inSettingsIndexso search finds them.Validation
New router tests:
The existing fallback tests still pass unchanged.
Not run by hand yet in a signed build;
swiftlint --strictnot run.Risk / rollout notes
🤖 Generated with Claude Code
The PR is not yet safe to merge because changing the fallback model can retain several GB of model memory despite keep-loaded being off.
Summary
The PR adds persisted Apple Intelligence language-fallback and keep-loaded controls, exposes the shared Open Source model as the fallback choice, updates runtime startup, and supports discovery through a symlinked models folder.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Apple Intelligence rejects language] --> B{Fallback enabled?} B -- No --> C[Empty suppressed result] B -- Yes --> D[Selected Open Source model] D --> E[Generate fallback suggestion] F[Keep loaded enabled] --> G[Prepare local runtime in advance]Reviews (2) · Last reviewed commit: "Pick the fallback model in the Apple Int..."