Skip to content

Commit 306aba4

Browse files
claudeclaude
committed
audit-cli-tests: docker/composer/bun/cargo test suites; T60-T64; C13-C24 reconciled
Assisted-by: Claude Code:claude-opus-5-5
1 parent 0a40df2 commit 306aba4

5 files changed

Lines changed: 138 additions & 46 deletions

File tree

‎coverage/files.tsv‎

Lines changed: 25 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -117,31 +117,31 @@ crates/socket-patch-cli/tests/common/pty_io.rs 100 audit-cli-tests cbbbdb9 2026-
117117
crates/socket-patch-cli/tests/common/rollback_json.rs 207 audit-cli-tests cbbbdb9 2026-11-01 - delete events, warning_codes, events_with_mode (= mode_events) and assert_rollback_envelope copies of envelope.rs; the last drifted (skipped == count vs <=, 3 statuses vs 10, no error/dryRun checks); re-export envelope readers; 6 single-caller helpers
118118
crates/socket-patch-cli/tests/common/update_fixture.rs 616 audit-cli-tests cbbbdb9 2026-11-01 - delete dead raw_asset/delay_asset; sha256_file dups common::sha256_hex, real_binary dups oracle::binary; sums_builder test computes expected with same Sha256 (format-only)
119119
crates/socket-patch-cli/tests/common/yarn_classic_vex.rs 520 audit-cli-tests cbbbdb9 2026-11-01 - REQUIRED_ENV/yarn_e2e_required exact copy of yarn_berry_common L88-L92 (T50); hosted 'reverted' cell asserts only code != 0 L494-L500 - assert errorCode; Hosted->redirected mapping 3×; 1.22.22 default spelled twice; drop #1671/#1734 history
120-
crates/socket-patch-cli/tests/composer_e2e_common/mod.rs 394 audit-cli-tests - -
121-
crates/socket-patch-cli/tests/contract_gradle_codes.rs 176 audit-cli-tests - -
122-
crates/socket-patch-cli/tests/contract_paid_required.rs 59 audit-cli-tests - -
123-
crates/socket-patch-cli/tests/crawl_fd_limit_e2e.rs 300 audit-cli-tests - -
124-
crates/socket-patch-cli/tests/docker_e2e_cargo.rs 481 audit-cli-tests - -
125-
crates/socket-patch-cli/tests/docker_e2e_composer.rs 498 audit-cli-tests - -
126-
crates/socket-patch-cli/tests/docker_e2e_deno.rs 608 audit-cli-tests - -
127-
crates/socket-patch-cli/tests/docker_e2e_gem.rs 524 audit-cli-tests - -
128-
crates/socket-patch-cli/tests/docker_e2e_golang.rs 450 audit-cli-tests - -
129-
crates/socket-patch-cli/tests/docker_e2e_maven.rs 519 audit-cli-tests - -
130-
crates/socket-patch-cli/tests/docker_e2e_npm.rs 1107 audit-cli-tests - -
131-
crates/socket-patch-cli/tests/docker_e2e_nuget.rs 624 audit-cli-tests - -
132-
crates/socket-patch-cli/tests/docker_e2e_pypi.rs 928 audit-cli-tests - -
133-
crates/socket-patch-cli/tests/docker_e2e_sbt.rs 631 audit-cli-tests - -
134-
crates/socket-patch-cli/tests/docker_e2e_vendor_composer.rs 530 audit-cli-tests - -
135-
crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs 843 audit-cli-tests - -
136-
crates/socket-patch-cli/tests/docker_e2e_vendor_maven.rs 558 audit-cli-tests - -
137-
crates/socket-patch-cli/tests/docker_e2e_vendor_nuget.rs 515 audit-cli-tests - -
138-
crates/socket-patch-cli/tests/docker_e2e_vendor_pypi_pm.rs 963 audit-cli-tests - -
139-
crates/socket-patch-cli/tests/docker_vendor_common/mod.rs 322 audit-cli-tests - -
140-
crates/socket-patch-cli/tests/e2e_bun_lockb.rs 2068 audit-cli-tests - -
141-
crates/socket-patch-cli/tests/e2e_cargo.rs 761 audit-cli-tests - -
142-
crates/socket-patch-cli/tests/e2e_composer.rs 312 audit-cli-tests - -
143-
crates/socket-patch-cli/tests/e2e_composer_vendor_dir_hints.rs 148 audit-cli-tests - -
144-
crates/socket-patch-cli/tests/e2e_composer_version_identity.rs 391 audit-cli-tests - -
120+
crates/socket-patch-cli/tests/composer_e2e_common/mod.rs 394 audit-cli-tests cbbbdb9 2026-11-02 - make fixture_composer_json, TRANSPORT_ATTEMPTS private; move copy_dir_recursive to common (T64; docker_e2e_vendor_composer includes the module only for it); delete e2e_vendor_composer_build `fn composer` forwarder; required() truthiness differs (T50); ~10 comment lines
121+
crates/socket-patch-cli/tests/contract_gradle_codes.rs 176 audit-cli-tests cbbbdb9 2026-11-02 - clean; merge with contract_paid_required into one contract binary (C31)
122+
crates/socket-patch-cli/tests/contract_paid_required.rs 59 audit-cli-tests cbbbdb9 2026-11-02 - delete get_emits_the_paid_required_envelope_status L43-59 (greps source for Status::PaidRequired; behaviour pinned by get_failure_paths/get_invariants); drop bare #982 + history comment; fold into a contract binary
123+
crates/socket-patch-cli/tests/crawl_fd_limit_e2e.rs 300 audit-cli-tests cbbbdb9 2026-11-02 - scan() L151-191 hand-scrubs SOCKET_* and keeps ambient SOCKET_NO_CONFIG, no SOCKET_NO_UPDATE_CHECK: use hermetic::command (12-fd macOS cell has 1 fd slack); hoist ample-JSON parse (L220-225, L262-267); ~12 history comment lines
124+
crates/socket-patch-cli/tests/docker_e2e_cargo.rs 481 audit-cli-tests cbbbdb9 2026-11-02 - use T60 harness (cov_docker_args, skip_if_no_image ungated, assert_vex_agent_attested, GIT_SHA256_FN, inline docker run L402-416); zero beforeHash+--force (T62); '"applied": 1,' grep L239-270 field-order dependent; host asserts L423, L438-441 recheck script gates; ~45 comment lines
125+
crates/socket-patch-cli/tests/docker_e2e_composer.rs 498 audit-cli-tests cbbbdb9 2026-11-02 - use T60 harness (5 copied helpers); plain_sha256 L63-67 → common::sha256_hex; global test L481-498 lacks API-hit check and VEX leg; verify_snippet greps JSON; zero beforeHash+--force (T62); ~35 comment lines (L69-93 restates bash, bare #424 ×2)
126+
crates/socket-patch-cli/tests/docker_e2e_deno.rs 608 audit-cli-tests cbbbdb9 2026-11-02 - skip gate L497-524 reads SOCKET_PATCH_REQUIRE_DOCKER, set nowhere (T60); deno_jsr_script_pins_scan_to_the_mock_api L475-488 asserts its own script string; host asserts L556-570, L598-603 recheck script gates; api_url_for_container copies npm; /patches/blob mock likely unhit; ~50 comment lines
127+
crates/socket-patch-cli/tests/docker_e2e_gem.rs 524 audit-cli-tests cbbbdb9 2026-11-02 - plain_sha256 → common::sha256_hex; '"applied": 1' grep L155 matches 10 (executed); mock check L462-473 path-only + unwrap_or_default; verify_snippet fn → const; global test lacks VEX leg; history comments L92-95, L289-291, L360-362, #424; T60 helpers
128+
crates/socket-patch-cli/tests/docker_e2e_golang.rs 450 audit-cli-tests cbbbdb9 2026-11-02 - zero beforeHash L98 + --force (T62); drop redundant !received.is_empty() L413-420 and hash_after assert L402-409; 5 summary grep -E → envelope parse; delete 'old check would miss' L212-214, L422-424; GIT_SHA256_FN → git_blob_sha; T60 helpers; ~45 comment lines
129+
crates/socket-patch-cli/tests/docker_e2e_maven.rs 519 audit-cli-tests cbbbdb9 2026-11-02 - zero beforeHash + --force (T62); '"applied": 1,' depends on field order; redundant hash_after assert L458-465; format!("{}", r.method)=="POST" L489, L515; GIT_SHA256_FN copy; T60 helpers; ~45 comment lines
130+
crates/socket-patch-cli/tests/docker_e2e_npm.rs 1107 audit-cli-tests cbbbdb9 2026-11-02 - delete npm_test_infrastructure_smoke (asserts own wiremock); delete dead workspace_root, `let _ =`, `const _: Option<Duration>`, unused imports, unexecuted run.sh write L578-582; delete SOCKET_PATCH_TEST_HOST host mode (4 tests return silently); berry git_sha256 L868 → prelude git_blob_sha; pin rollback failure; T60/T62; ~70 comment lines
131+
crates/socket-patch-cli/tests/docker_e2e_nuget.rs 624 audit-cli-tests cbbbdb9 2026-11-02 - plain_sha256 → common::sha256_hex; fold local_script/global_script (~120 duplicated lines); zero beforeHash + --force (T62); summary greps → assert_summary/envelope; mock check path-only + unwrap_or_default; T60 helpers; ~40 comment lines
132+
crates/socket-patch-cli/tests/docker_e2e_pypi.rs 928 audit-cli-tests cbbbdb9 2026-11-02 - sha256_hex → common; drop run_container _api_url; one bash fn for scan/sync/apply/verify ×3; gate SYNC_RC (L263, L381, L510); uv_venv CoW check vacuous when nlink==1 (L555-591); rename uv_tool ..._full_apply_chain (never applies) L910-928; self-test L733-750 asserts own script; ~45 comment lines; T60/T62
133+
crates/socket-patch-cli/tests/docker_e2e_sbt.rs 631 audit-cli-tests cbbbdb9 2026-11-02 - hex_of L83 → common sha helpers; stop calling sbt_jdk("1.13.0") for the env override L503; reuse sbt_uses_ivy in sbt_jdk L271-274; cache jars() download; delete zip readback L108-116; version_selection L613-626 asserts own scripts, docker-cfg only; useCoursier-false test L593 passes with 0 cells; inline required/sbt_versions/mill_versions
134+
crates/socket-patch-cli/tests/docker_e2e_vendor_composer.rs 530 audit-cli-tests cbbbdb9 2026-11-02 - assert_vex_attested_from_host L394-422 (contains "(vendored)") → vex_e2e_common::assert_attested(Marker::Vendored), already used at L467 (T63); stale L194 'no product auto-detect'; psr_ver() for 3 reads; drop STAGE3 '"type": "path"' grep L343 (host oracle covers); move render() to docker_vendor_common
135+
crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs 843 audit-cli-tests cbbbdb9 2026-11-02 - build STAGE2/3 from one template, delete STAGE2_CK/STAGE3_CK near-copies (L220-252 vs L617-646); inline render; merge pair oracles with checksums flag; replace Ruby TCPServer API stub L302-328 with host PatchApi; assert_attested (T63); stale doc L14-15 (--offline), L19-20
136+
crates/socket-patch-cli/tests/docker_e2e_vendor_maven.rs 558 audit-cli-tests cbbbdb9 2026-11-02 - TAMPER probe L255-274 only logs on Maven >=3.9.2 yet echoes VERIFIED (likely vacuous in CI, unexecuted); fold with_purl_env into render; assert_attested (T63); copy_tree → common (T64); stale STAGE2 'see module doc' L208-210; 'mirrors' + bare #973 L54-64
137+
crates/socket-patch-cli/tests/docker_e2e_vendor_nuget.rs 515 audit-cli-tests cbbbdb9 2026-11-02 - `omitted` closure L406-419 → vex_e2e_common::assert_omitted; assert_vex_attested_from_host → assert_attested (T63); copy_tree → common (T64); drop 'mirrors composer capstone' comments; ~10 comment lines
138+
crates/socket-patch-cli/tests/docker_e2e_vendor_pypi_pm.rs 963 audit-cli-tests cbbbdb9 2026-11-02 - pass __GHSA__/__CVE__ through STAGE1_VENDOR_COMMON, delete eval wrappers L159-160, L363-364 and Rust replace L911-919; move poetry/pdm/pipenv VEX legs host-side on vex_e2e_common (pdm L437-440 never checks status); RED probes L202-205, L403-406, L579-582 should grep the reason; stale --offline doc L14-15; ~25 comment lines (T63)
139+
crates/socket-patch-cli/tests/docker_vendor_common/mod.rs 322 audit-cli-tests cbbbdb9 2026-11-02 - delete dead run_in_image L203-205 (hidden by allow(dead_code)); make cov_docker_args private; json_assert_fns L313-321 greps JSON in-container, assert on host via envelope.rs; Maven-only m2 case in run_with_fixture L162-167 → caller env; pass SOCKET_TELEMETRY_DISABLED once via -e (18 stage exports); see T60
140+
crates/socket-patch-cli/tests/e2e_bun_lockb.rs 2068 audit-cli-tests cbbbdb9 2026-11-02 - drop dead _target params of frozen/mock_api + L1348 match; inline frozen/installed_target; split production_scoped_rollback (hidden behind env in native_binary_alias_and_transitive L1328-1336); --offline stripped (T61); cli_code bypasses prepare_command (L101-120, inline L1109-1116); bundled skip L1165-1175 derived from CLI output; envelope::all_codes for .to_string().contains ×6
141+
crates/socket-patch-cli/tests/e2e_cargo.rs 761 audit-cli-tests cbbbdb9 2026-11-02 - duplicate operands !contains("No packages found") L204, L269; vex checked by contains("not_affected") L692-698, no exit check L716-723; serde_json::from_str ×4 → envelope::output_envelope; manifest_pruned L313-321 → rollback_json::manifest_removed; fake_registry_crate vs cargo_lock_source::stage_registry_copy (drifted); locks via cargo_e2e_matrix::write_lock; ~30 comment lines
142+
crates/socket-patch-cli/tests/e2e_composer.rs 312 audit-cli-tests cbbbdb9 2026-11-02 - start_proxy/assert_proxy_served_scans/run copied from e2e_cargo/maven/nuget (T64); delete !contains("No packages found") L202-205, L260-263; third test string-matches batch body L305-311, parse it; drop essay comments L166-177, L189-196, L236-239, L251-254 + banners; #463 history L267
143+
crates/socket-patch-cli/tests/e2e_composer_vendor_dir_hints.rs 148 audit-cli-tests cbbbdb9 2026-11-02 - use hermetic::scrub_extra(Extra::Composer) not env_remove(COMPOSER_VENDOR_DIR) L99 (COMPOSER_HOME leaks); include common/hermetic.rs only (24 of 26 tests are common selftests, executed); fix vendor() doc L96 (3-tuple, --offline stripped, T61); share psr/log LOCK/ORIGINAL/PATCHED with version_identity; drop #658 narration
144+
crates/socket-patch-cli/tests/e2e_composer_version_identity.rs 391 audit-cli-tests cbbbdb9 2026-11-02 - run_json/read_json → envelope::output_envelope; warnings parse L368-371 → envelope::warning_codes; share psr/log fixture with dir_hints; ~4 comment lines
145145
crates/socket-patch-cli/tests/e2e_embedded_vex.rs 1085 audit-cli-tests - -
146146
crates/socket-patch-cli/tests/e2e_gem.rs 642 audit-cli-tests - -
147147
crates/socket-patch-cli/tests/e2e_golang.rs 418 audit-cli-tests - -

‎doc/05-tests.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
## Part 5
22

3-
_Last checked against main @ cbbbdb9 on 2026-11-02 by audit-core._
3+
_Last checked against main @ cbbbdb9 on 2026-11-02 by audit-cli-tests._
44

55
Tests:Fake tests (asserting on their own mocks, recomputing expected values with the code under test, unable to fail, claiming more than they check), mock overuse where a tempdir or a real install is cheap, sentence-matching assertions, duplicate tests, and real coverage gaps. Per-directory tallies and the fixes that landed.
66

@@ -101,4 +101,9 @@ _Last checked against main @ cbbbdb9 on 2026-11-02 by audit-core._
101101
- Can't fail: `save_and_clear_covers_every_bound_global_env_var` iterates the list it checks; `hermetic::HOSTILE_SEEDS` seed and remove the same list {{T59}}; the redirect test's "forbidden" mock in `cli_scan_silent` is outranked by the prebuilt mount (executed).
102102
- Duplicates: ~85 per-subcommand `GlobalArgs` flag tests beside `cli_global_args` {{T57}}; `cli_parse_remove`'s 5 binary tests copy `remove/remove_invariants`; 7 exit-code-only `run()` tests in `cli_parse_list`.
103103
- Scaffolding: dead `stderr_chatter` filter ×11 and `has_command`/`sha256_hex` copies {{T55}}; 21 `common` selftests re-run in ~104 binaries {{T58}}.
104+
- **Tally for `tests/docker_*`, `composer_e2e_common`, `contract_*`, `crawl_fd_limit_e2e`, `e2e_bun_lockb`, `e2e_cargo`, `e2e_composer*` (25 files, 14.7K lines, audit-cli-tests 2026-11-02 05:00Z): about 45 fake or weak tests.**
105+
- Prove less than they claim: 8 of 9 docker agent chains serve a zero `beforeHash` and `apply --force`, so only the force path runs {{T62}}; 49 `vendor`/`repair --offline` calls in 16 suites run online because `prepare_command` strips the flag {{T61}}; `binary_shared_bundled_record_hosted_pin_is_managed` skips on the CLI output it tests; pypi `uv_tool…_full_apply_chain` never applies; the vendor_maven TAMPER probe only logs on Maven ≥ 3.9.2.
106+
- Can't fail: `npm_test_infrastructure_smoke` asserts its own wiremock; deno/pypi/sbt script tests assert their own generated strings; `contract_paid_required` greps the source for a variant; e2e_cargo's `!a && !a` duplicate operand.
107+
- Weak oracles: gem's `"applied": 1` grep matches 10 (executed) {{T60}}; `assert_vex_attested_from_host` ×4 checks only `(vendored)` beside `assert_attested` {{T63}}.
108+
- Scaffolding: docker harness copied ×8-11 with 7 ungated image skips {{T60}}; `copy_tree` ~20+, scan-proxy ×4, `hosted_pin_count` ×8 {{T64}}.
104109
- **Coverage gaps:** a lockfile-only calm skip and a genuine miss serialize identically (`skipped`/`package_not_installed`), and an apply no-matching-variant miss emits no event, so the JSON tests can't check either (executed; C13); no offline round-trip test of the cargo matrix's `Cargo.lock` codec.

0 commit comments

Comments
 (0)