Repository navigation
Test the Grok ACP fallback against a fake CLI and fix two bugs - #969
Merged
Merged
Conversation
added 4 commits
October 21, 2026 15:52
The ACP fallback spawned plain grok from PATH. The native installer puts it under ~/.grok/bin, which is often not on PATH, so the fallback reported NotInstalled. It now uses the same resolver as the login flow. A message with a method is a server request or notification, never a reply. Matching on id alone let a server request that reused the pending id end the wait with no result.
examples/fake_grok_acp stands in for grok agent stdio. The end-to-end tests spawn it through the production spawn path and cover the handshake, billing, the upstream credits and legacy shapes, noise and garbled frames, JSON-RPC errors, auth errors, early exit, partial frames, oversized lines, and timeouts. Each test checks that the child was started with no console window and is not running afterwards. Provider-level tests cover the missing and expired login fallback and the isolated-account guard. The CLI program is injectable for tests only.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 32 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds end-to-end tests for the Grok ACP billing fallback (#958) against a scripted fake
grok agent stdio. The fallback had only in-memory pipe tests and a single missing-binary check, and it had never been run through a real child process. This PR runs it through one. It also fixes two bugs the fake exposed. Branchtest/grok-acp-fake-server, based on origin/main546b49e3d.rust/examples/fake_grok_acp.rs): a cargo example, so there is no new dependency and no node or python in CI. It speaks newline-delimited JSON-RPC 2.0 on stdio and takes a scenario argument. It writes its pid and whether it has a console window to a pid file before the handshake.rust/src/providers/grok/acp_process_tests.rs): they go through the productionspawn_child, so the stdio, framing, handshake, billing request, parsing, kill and reap all run for real. Scenarios: credits answer in the upstreamconfigshape withsubscription_tier; the legacymonthlyLimitshape; noise (non-JSON, notifications, reply to another id, server request reusing the pending id); truncated frames;-32601; auth errors on initialize and on billing; hang on initialize; hang on billing; early exit; half a frame then EOF; a 300 KiB line.CREATE_NO_WINDOWreports one, which shows the probe can detect an inherited console.fetch_cli_acp_without_login. A failing child keeps the original login error. An isolated account never starts the child.CliBillingRunner::Process(#[cfg(test)]) takes a program path, arguments and budgets. The production runner is unchanged apart from the resolver fix below.Bugs fixed
grokfrom PATH. The production runner spawnedCommand::new("grok"). The native installer puts the CLI under~/.grok/bin/grok.exe, which is often not on PATH. The fallback then reportedNotInstalledeven with the CLI installed. It now usesaccounts::login::executable(), the resolver the login flow already uses: the native path first, thenwhich. Found by reading the code, not reproduced with a real install.reply_formatched onidalone. A message with amethodis a request or notification, never a reply. A server request that reused the pending id ended the wait withParse("Grok CLI reply had no result"). Reproduced: thenoisetest fails without the fix. Upstream Swift has the same id-only match, so this goes beyond upstream parity.Windows differences and not ported
grokCLI was run. Per the user's choice, the tests use the scripted fake only.detect_cli_version(providers/grok/mod.rs,Command::new("grok") --version) still uses PATH only. It is outside the ACP fallback, so it is unchanged here. Follow-up candidate.Commands run
cargo fmt --all -- --check: cleancargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings: cleancargo clippy --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml --all-targets -- -D warnings: cleancargo test --manifest-path rust/Cargo.toml --lib providers::grok: 174 passed, 0 failed (156 existing, 18 new; the probe-control test is Windows-only)cargo test --manifest-path rust/Cargo.toml: 5212 passed, 0 failed, 2 ignored (run before the final test-only edits; the grok filter was rerun after them)cargo test --manifest-path apps/desktop-tauri/src-tauri/Cargo.toml: 756 passed, 0 failedprocess_testsbinary was started withStart-Process -WindowStyle Hidden, which gives the test process a console. The inherited-console control ran (not skipped) and passed, and all 17 process tests in that build passed (the liveness test was added after this run and passed in the later grok run). This is the only run where the hidden-spawn assertion is meaningful, because a shell without a console makes the control skip.noisetest without themethodfilter fails withParse("Grok CLI reply had no result").Note: the fake is a cargo example. Run the full
cargo test, orcargo build --manifest-path rust/Cargo.toml --example fake_grok_acpfirst. A--libfilter alone does not build it, and the process tests then fail with a message that says so.No UI proof: no frontend file changed. Backend only.
Coordinator follow-ups
CREATE_NO_WINDOW. Without it, a test runner that has no console would give the child a visible console window.detect_cli_versionnow uses the same CLI resolver as login and ACP (~/.grok/bin/grok.exefirst, then PATH) instead of a baregrok.cargo test --lib providers::grok174 passed; clippy-D warningsonrust/clean.