Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -596,6 +596,8 @@ Findings that change the backlog:
- **V4b Auto-mode disclosure review (2026-10-03, `1729220`).** Adversarial review confirmed that Auto mode also approves configured MCP tool calls, not only shell commands. The footer tooltip, command-permissions explainer, and Code task system prompt now state that MCP tools run without approval unless an exact project server/tool deny rule blocks them, and warn that they may affect external services. Regression checks cover the tooltip and prompt wording. Windows CI [#37148187560](https://github.com/jay23606/codev/actions/runs/37148187560) passed 936 portable tests (2 opt-in skips), 35 Avalonia headless UI tests on Windows/Linux/macOS, 8 Windows UI tests, and native credential-vault smoke checks. Desktop preview [#37148187564](https://github.com/jay23606/codev/actions/runs/37148187564) passed app/CLI build, launch, and packaging on all four RIDs; publishing was skipped for the PR. This review found no additional policy-routing bypass, but it does not satisfy Q3's full manual security review.
- **V4c. Real-use log (first pass, 2026-10-09).** The first Codev dogfood reports have been read and triaged in [`docs/real-use-log-release-2.md`](docs/real-use-log-release-2.md). They map to B1 (Auto prompts), B15 (local-model cold load), P2/P6 (saved hosted-key reconnect), collapsed activity/readability, sidebar ordering, and the existing Windows hidden-process follow-up. The [PR #62 Windows package preview #37991522656](https://github.com/jay23606/codev/actions/runs/37991522656) ran the reported deletion-plus-Git-status command in Auto without a prompt, then verified an exact Deny after switching back to Auto; PR #65 records the evidence. Confirm the behavior again after the public Release 2 install; native manual items remain open.

- **V4d. Release 2 cross-platform MCP preview coverage (2026-10-09, packaged run pending).** The existing Linux and macOS packaged-app scripts now configure a loopback Streamable HTTP MCP server and the deterministic mock-model MCP turn. They assert tool discovery, the expected echo call, the bounded untrusted result path, and a completed Auto turn; cleanup tracks the mock server process. Git Bash syntax validation, embedded Python compilation (32 blocks), Node syntax checks, and `git diff --check` pass. The scheduled October 11 four-RID preview is the first runtime check of these additions. Until that run passes, this is coverage added, not a verified cross-platform runtime result; native macOS/Linux interaction remains separately open.

**Done when:** V4a stays green, the checklist is committed and has been run at least once, and the first real-use log has been read and its items triaged.

#### V5. Measure whether Codev's own features help on real, large projects
Expand Down
2 changes: 1 addition & 1 deletion docs/release-plan-2026-10.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ The Release 1 ledger leaves 65 manual workflows open, including native macOS/Lin

**Release 2 checkpoint (verified 2026-10-09 at main `80c911e`; product code through PR #77, documentation through PR #80).** Subsequent product changes include the exact-Deny Auto regression, persisted sidebar reorder, profile and hosted-key fixes, and a clarification in the Auto/MCP tooltips when saved MCP permissions cannot be read. PR #77's Windows/Linux/macOS CI [#38003412218](https://github.com/jay23606/codev/actions/runs/38003412218) and post-merge CI [#38003676335](https://github.com/jay23606/codev/actions/runs/38003676335) passed. PR #79 documented the Auto/profile/MCP denial-composition review and focused test evidence in `DESIGN.md`; its Windows/Linux/macOS CI [#38004723785](https://github.com/jay23606/codev/actions/runs/38004723785) and post-merge CI [#38004977342](https://github.com/jay23606/codev/actions/runs/38004977342) passed. PR #80 refreshed this checkpoint; its main-branch Windows/Linux/macOS CI [#38005530614](https://github.com/jay23606/codev/actions/runs/38005530614) passed. PR #74 added the eight-archive prepublish gate; PR #76 changed the release preview workflow to run packaged interaction checks only on manual dispatch or version-tag pushes. Their PR CI [#38001121514](https://github.com/jay23606/codev/actions/runs/38001121514) [#38002345137](https://github.com/jay23606/codev/actions/runs/38002345137) and post-merge CI [#38001372222](https://github.com/jay23606/codev/actions/runs/38001372222) [#38002651235](https://github.com/jay23606/codev/actions/runs/38002651235) passed. The last exact-head packaged preview remains PR #62 [#37991522656](https://github.com/jay23606/codev/actions/runs/37991522656); run one new four-target preview by manual dispatch against the frozen October 11 candidate. Do not run packaged previews for ordinary PRs or local desktop smokes during routine work. Clean-machine installation, physical accessibility, and native macOS/Linux interaction remain explicit limitations. Keep the October 11 freeze and October 13 Release 2 target below.

Target Release 2 for **Tuesday, October 13, 2026** (Day 5 after the October 8 publication; acceptable window: Day 3–5). On October 9–10 (Days 1–2), collect install reports, triage reproducible issues, and fix release blockers. On October 11 (Day 3), freeze the candidate and run exact-head Windows/Linux/macOS CI plus all four packaged desktop/CLI checks by manually dispatching the desktop-preview workflow once against the frozen main commit. The workflow runs only on manual dispatch and version-tag pushes; pull requests and ordinary merges use the Windows/Linux/macOS CI workflow without packaged interaction smokes. Use a local desktop smoke only if a release-critical interaction remains uncovered by hosted checks and focused tests; routine local runs stay disabled. Publish on October 12–13 only if exact-head checks pass, update the release notes for verified changes and remaining limitations, and confirm all eight README latest-download links. If a blocker moves the candidate, move Release 2 to Day 3–5 after the verified fix.
Target Release 2 for **Tuesday, October 13, 2026** (Day 5 after the October 8 publication; acceptable window: Day 3–5). On October 9–10 (Days 1–2), collect install reports, triage reproducible issues, and fix release blockers. On October 11 (Day 3), freeze the candidate and run exact-head Windows/Linux/macOS CI plus all four packaged desktop/CLI checks by manually dispatching the desktop-preview workflow once against the frozen main commit. The four-RID preview will also run the new Linux/macOS Streamable HTTP MCP Auto scenarios, which were added to the existing packaged checks without changing the preview cadence. The workflow runs only on manual dispatch and version-tag pushes; pull requests and ordinary merges use the Windows/Linux/macOS CI workflow without packaged interaction smokes. Use a local desktop smoke only if a release-critical interaction remains uncovered by hosted checks and focused tests; routine local runs stay disabled. Publish on October 12–13 only if exact-head checks pass, update the release notes for verified changes and remaining limitations, and confirm all eight README latest-download links. If a blocker moves the candidate, move Release 2 to Day 3–5 after the verified fix.

**Earlier Release 2 candidate evidence (October 9, code head `e729b85`).** The four-target preview [#37885221048](https://github.com/jay23606/codev/actions/runs/37885221048) passed before the later Release 2 fixes. Subsequent PR #35 removed packaged previews from regular pushes to `main`; PR #76 later removed them from pull requests as well. This evidence is historical and does not validate current main `7cb7d2b`; the exact current candidate still needs the one manual four-target preview scheduled above. The prior main CI's intermittent Linux headless review-test failure was fixed layout-synchronously in PR #34. Treat Release 2 scope as stabilization and recheck [`docs/release-notes.md`](release-notes.md) against the exact release commit before tagging.

Expand Down
85 changes: 84 additions & 1 deletion scripts/smoke-packaged-linux-app.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export CODEV_DATA_ROOT="$smoke_root/data"
log_path="$smoke_root/app.log"
app_pid=''
mock_pid=''
mcp_http_pid=''
cleanup() {
if [[ -n "$app_pid" ]] && kill -0 "$app_pid" 2>/dev/null; then
kill "$app_pid" 2>/dev/null || true
Expand All @@ -26,6 +27,7 @@ cleanup() {
fi
if [[ -n "$app_pid" ]]; then wait "$app_pid" 2>/dev/null || true; fi
if [[ -n "$mock_pid" ]]; then kill "$mock_pid" 2>/dev/null || true; wait "$mock_pid" 2>/dev/null || true; fi
if [[ -n "$mcp_http_pid" ]]; then kill "$mcp_http_pid" 2>/dev/null || true; wait "$mcp_http_pid" 2>/dev/null || true; fi
rm -rf -- "$smoke_root"
}
trap cleanup EXIT
Expand All @@ -38,7 +40,7 @@ assert_search_focus() {

mock_port_path="$smoke_root/mock-ollama.port"
mock_request_log="$smoke_root/mock-ollama-requests.jsonl"
node ./scripts/mock-ollama-server.js --activity-summary --port-file "$mock_port_path" --request-log "$mock_request_log" >"$smoke_root/mock-ollama.log" 2>&1 &
node ./scripts/mock-ollama-server.js --activity-summary --mcp-http --port-file "$mock_port_path" --request-log "$mock_request_log" >"$smoke_root/mock-ollama.log" 2>&1 &
mock_pid=$!
for _ in {1..200}; do
if [[ -s "$mock_port_path" ]]; then break; fi
Expand All @@ -54,8 +56,40 @@ for _ in {1..200}; do
done
if [[ ! -s "$mock_port_path" ]]; then cat "$smoke_root/mock-ollama.log" >&2; echo 'Mock Ollama did not report its loopback port within 20 seconds.' >&2; exit 1; fi
mock_port="$(cat "$mock_port_path")"
mcp_http_port_path="$smoke_root/mock-mcp-http.port"
mcp_http_call_log="$smoke_root/mock-mcp-http-calls.jsonl"
node ./scripts/mock-mcp-http-server.js --port-file "$mcp_http_port_path" --call-log "$mcp_http_call_log" >"$smoke_root/mock-mcp-http.log" 2>&1 &
mcp_http_pid=$!
for _ in {1..200}; do
if [[ -s "$mcp_http_port_path" ]]; then break; fi
mcp_http_state="$(ps -p "$mcp_http_pid" -o stat= 2>/dev/null || true)"
if [[ -z "$mcp_http_state" || "$mcp_http_state" == Z* ]]; then
if wait "$mcp_http_pid"; then mcp_http_status=0; else mcp_http_status=$?; fi
mcp_http_pid=''
cat "$smoke_root/mock-mcp-http.log" >&2
echo "Mock MCP HTTP server exited before startup completed (exit $mcp_http_status)." >&2
exit 1
fi
sleep 0.1
done
if [[ ! -s "$mcp_http_port_path" ]]; then cat "$smoke_root/mock-mcp-http.log" >&2; echo 'Mock MCP HTTP server did not report its loopback port within 20 seconds.' >&2; exit 1; fi
mcp_http_port="$(cat "$mcp_http_port_path")"
mkdir -p "$CODEV_DATA_ROOT/Codev"
printf '{"Theme":"dark","OllamaEndpoint":"http://127.0.0.1:%s/","DefaultProjectCommandPermissionMode":"Auto"}\n' "$mock_port" >"$CODEV_DATA_ROOT/Codev/avalonia-settings.json"
python3 - "$CODEV_DATA_ROOT/Codev/mcp-servers.json" "$mcp_http_port" <<'PY'
import json
import sys

with open(sys.argv[1], "w", encoding="utf-8") as target:
json.dump([{
"id": "smoke-http",
"name": "Packaged smoke HTTP MCP",
"transport": "Http",
"enabled": True,
"url": f"http://127.0.0.1:{sys.argv[2]}/mcp",
"oauthEnabled": False,
}], target)
PY
"$app_path" >"$log_path" 2>&1 &
app_pid=$!

Expand Down Expand Up @@ -422,6 +456,55 @@ PY

echo 'Linux packaged Code task read, searched, created a file, and verified a command in one Auto turn.'

# Exercise configured Streamable HTTP MCP discovery and execution in Auto.
mcp_prompt='Run the packaged MCP Streamable HTTP smoke.'
before_mcp_count="$(jq --arg id "$conversation_id" '[.[] | select(.Id == $id) | .Messages[]] | length' "$conversations_path")"
xdotool windowfocus --sync "$window_id"
xdotool key --clearmodifiers ctrl+l
xdotool type --clearmodifiers --delay 1 "$mcp_prompt"
xdotool key --clearmodifiers Return

for _ in {1..300}; do
if jq -e --arg id "$conversation_id" --argjson before "$before_mcp_count" --arg prompt "$mcp_prompt" \
'.[] | select(.Id == $id) | .Messages as $messages |
($messages | length) >= ($before + 2) and
$messages[-2].Content == $prompt and
$messages[-1].Role == "assistant" and
$messages[-1].Content == "Packaged MCP Streamable HTTP call passed."' \
"$conversations_path" >/dev/null 2>&1; then break; fi
if ! kill -0 "$app_pid" 2>/dev/null; then cat "$log_path" >&2; echo 'Packaged Avalonia app exited during the Linux MCP HTTP smoke.' >&2; exit 1; fi
sleep 0.1
done

python3 - "$conversations_path" "$conversation_id" "$mock_request_log" "$mcp_http_call_log" <<'PY'
import json
import sys

prompt = "Run the packaged MCP Streamable HTTP smoke."
with open(sys.argv[1], encoding="utf-8") as source:
conversations = json.load(source)
conversation = next((item for item in conversations if item.get("Id") == sys.argv[2]), None)
messages = conversation.get("Messages", []) if conversation else []
if not any(message.get("Role") == "user" and message.get("Content") == prompt for message in messages):
raise SystemExit("The packaged Linux MCP prompt was not persisted.")
if not messages or messages[-1].get("Role") != "assistant" or messages[-1].get("Content") != "Packaged MCP Streamable HTTP call passed.":
raise SystemExit(f"The packaged Linux MCP call did not finish: {messages[-4:]!r}")
with open(sys.argv[3], encoding="utf-8") as source:
requests = [json.loads(line) for line in source if line.strip()]
turn = [request for request in requests if request.get("last_user_message") == prompt]
if len(turn) != 2 or turn[0].get("last_role") != "user" or \
len([name for name in turn[0].get("tool_names", []) if name.startswith("mcp_smoke-http_echo_")]) != 1 or \
turn[1].get("last_role") != "tool" or not turn[1].get("last_tool_name", "").startswith("mcp_smoke-http_echo_") or \
turn[1].get("keep_alive") != "30m":
raise SystemExit(f"The packaged Linux MCP discovery/result sequence was unexpected: {turn!r}")
with open(sys.argv[4], encoding="utf-8") as source:
calls = [json.loads(line) for line in source if line.strip()]
if calls != [{"name": "echo", "message": "packaged HTTP marker"}]:
raise SystemExit(f"The packaged Linux HTTP MCP fixture received an unexpected call: {calls!r}")
PY

echo 'Linux packaged Code task discovered and called the Streamable HTTP MCP echo tool in Auto without approval.'

# Verify the platform primary-modifier shortcut creates and activates a fresh chat.
previous_conversation_id="$(jq -r '.' "$active_path")"
xdotool windowfocus --sync "$window_id"
Expand Down
Loading
Loading