Skip to content

Commit 0d4e7cf

Browse files
committed
Merge remote-tracking branch 'origin/main' into agent/fix-ci-get-envelope-asserts
# Conflicts: # scripts/ci-test-durations.json
2 parents 576b8e9 + 75e49ae commit 0d4e7cf

31 files changed

Lines changed: 4954 additions & 4013 deletions

‎crates/socket-patch-cli/CLI_CONTRACT.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1442,7 +1442,8 @@ command's integration tests check its output.
14421442
| `vendor_ledger_missing` | `failed` (artifact-level: `uuid` + `details.{ecosystem,path}`, no purl) | repair (v5.0) and `vendor --check`: a lockfile references `.socket/vendor/<eco>/<uuid>/` but the vendor ledger has no entry for it; repair no longer rebuilds ledger entries from lockfiles. Recovery: restore `.socket/vendor/state.json` from version control and re-run `repair`, or `git checkout -- <lockfile>` and re-vendor. |
14431443
| `vendor_wiring_unknown_revert_blocked` | `skipped` (beside the `failed`/`revert_failed` event) | vendor --revert: the ledger entry was reconstructed by a pre-v5 `repair` without wiring records and the live lockfile still resolves through the artifact — the revert refuses (fail-closed) instead of deleting a tarball the lock points at. Recovery: `socket-patch repair`, then restore the pre-vendor lock (or re-lock without the override) and re-run the revert. repair: an npm ledger entry whose `flavor` this release does not know (written by a newer socket-patch) is skipped, never health-checked or rebuilt, and the artifact, wiring and ledger stay as found (a lone `skipped` event; the run's exit is unaffected). Recovery: upgrade socket-patch. |
14441444
| `stale_install` | `skipped` | vex (in-run `scan --mode hosted --vex`): a hosted stale-install probe found positively unpatched installed bytes, so the purl is omitted even under `--vex-no-verify` (see the gem / Python stale-install guards). |
1445-
| `dependency_not_installed` | `skipped` | vex: a hosted npm-family copy carries every patched file, but the patch changes its `package.json` and a dependency it declares (other than an optional one) does not resolve from the copy (no `node_modules/<name>` in it or above its real path), for example because the lockfile never locked it (a yarn classic pin written before the served-manifest check, #1462). Omitted until the dependency is installed. |
1445+
| `dependency_not_installed` | `skipped` | vex: a hosted npm-family copy carries every patched file, but the patch changes its `package.json` and a dependency it declares (other than an optional one) does not resolve from the copy (no `node_modules/<name>` in it or above its real path), for example because the lockfile never locked it (a yarn classic pin written before the served-manifest check, #1462), or the nearest copy that resolves (an installed optional one included) is outside the range the patched `package.json` declares, unless the project forces that dependency with npm `overrides`, `pnpm.overrides` or yarn `resolutions` (#1538). Lock-only (no installed copy): a yarn classic pin whose block's `dependencies:` / `optionalDependencies:` sub-maps are not exactly the ones the patched `package.json` declares, each locked (#1538). Omitted until the dependency is installed or locked. |
1446+
| `dependency_manifest_unverifiable` | `skipped` | vex (lock-only): a yarn classic hosted pin whose patch changes `package.json`, and the patched `package.json` could be read neither from `.socket/blobs` nor from the patch API (`--offline`, a fetch failure), so the dependencies the lock installs with it cannot be checked (#1538). |
14461447
| `record_unavailable` | `skipped` | vex (manifest-less): a lockfile-wired patch has no local record (manifest, this run's hosted records or a pre-v5 redirect ledger, vendor ledger) and none could be fetched — `--offline`, transport error, 404, or a refused (paid) patch. Omitted, never attested from the `socket-patch.vendor.json` marker. |
14471448
| `record_mismatch` | `skipped` | vex (manifest-less): the record found for a wired patch names another package or another patch uuid than the wiring. |
14481449
| `vendor_unwired` | `skipped` | vex: a vendor-ledger entry whose committed artifact no lockfile/config wires any more (reverted lock, leftover ledger or artifact). Applies under `--no-verify` too. |

‎crates/socket-patch-cli/src/commands/vex.rs‎

Lines changed: 146 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,14 @@
1616
//! * `--json` (requires `--output`): VEX JSON to file, envelope JSON
1717
//! to stdout. This is the CI integration shape.
1818
19-
use std::collections::HashMap;
19+
use std::collections::{HashMap, HashSet};
2020
use std::path::{Path, PathBuf};
2121

2222
use clap::Args;
2323
use socket_patch_core::api::client::ApiClient;
2424
use socket_patch_core::manifest::schema::{PatchManifest, PatchRecord};
2525
use socket_patch_core::telemetry::{track_vex_failed, track_vex_generated, TelemetryAuth};
26-
use socket_patch_core::vex::verify::DEPENDENCY_NOT_INSTALLED;
26+
use socket_patch_core::vex::verify::{npm_manifest_after_hash, DEPENDENCY_NOT_INSTALLED};
2727
use socket_patch_core::vex::{
2828
build_document, detect_product, BuildOptions, Document, FailedPatch, VendorContext,
2929
VerifyOutcome,
@@ -541,6 +541,118 @@ async fn hosted_gem_mirror_captured(
541541
captured
542542
}
543543

544+
/// Lock-only verdicts of hosted yarn classic pins (#1538): `purls` the
545+
/// plan may attest from the lock alone, mapped to the omission tag of each
546+
/// whose patch changes the package's `package.json` while the pinned
547+
/// block's dependency sub-maps are not exactly the ones that manifest
548+
/// declares, each locked. Yarn 1 installs what the lock names, so such a
549+
/// pin (one an earlier release wrote, #1462) installs the patched package
550+
/// without its patched dependencies: `dependency_not_installed`. A patched
551+
/// `package.json` that can be read neither from `.socket/blobs` nor from
552+
/// the patch API is `dependency_manifest_unverifiable`.
553+
async fn yarn_classic_lock_dependency_gaps(
554+
common: &GlobalArgs,
555+
params: &VexBuildParams,
556+
plan: &Plan,
557+
purls: &[&str],
558+
) -> HashMap<String, &'static str> {
559+
let mut gaps = HashMap::new();
560+
for purl in purls {
561+
let blocks: Vec<_> = plan
562+
.hosted
563+
.get(*purl)
564+
.into_iter()
565+
.flat_map(|wiring| &wiring.refs)
566+
.filter_map(|r| r.yarn_classic_block.as_ref())
567+
.collect();
568+
let Some(after_hash) = plan
569+
.view
570+
.patches
571+
.get(*purl)
572+
.and_then(npm_manifest_after_hash)
573+
else {
574+
continue;
575+
};
576+
if blocks.is_empty() {
577+
continue;
578+
}
579+
let pkg = patched_blob(common, params, after_hash)
580+
.await
581+
.and_then(|bytes| serde_json::from_slice::<serde_json::Value>(&bytes).ok())
582+
.filter(serde_json::Value::is_object);
583+
let gap = match pkg {
584+
None => DEPENDENCY_MANIFEST_UNVERIFIABLE,
585+
Some(pkg) if blocks.iter().any(|b| !b.installs_dependencies_of(&pkg)) => {
586+
DEPENDENCY_NOT_INSTALLED
587+
}
588+
Some(_) => continue,
589+
};
590+
gaps.insert(purl.to_string(), gap);
591+
}
592+
gaps
593+
}
594+
595+
/// Omission tag: a hosted yarn classic pin attested from the lock alone
596+
/// whose patch changes `package.json`, which could not be read to check
597+
/// the dependencies the lock installs beside it (#1538).
598+
const DEPENDENCY_MANIFEST_UNVERIFIABLE: &str = "dependency_manifest_unverifiable";
599+
600+
/// Largest patched `package.json` blob [`patched_blob`] reads.
601+
const MAX_PATCHED_MANIFEST_BYTES: usize = 4 * 1024 * 1024;
602+
603+
/// The patched file whose git-sha256 is `after_hash`: from the local
604+
/// `.socket/blobs` store, else (unless `--offline`) from the patch API.
605+
/// `None` when neither has it intact.
606+
async fn patched_blob(
607+
common: &GlobalArgs,
608+
params: &VexBuildParams,
609+
after_hash: &str,
610+
) -> Option<Vec<u8>> {
611+
use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes;
612+
// A record's hash names a file in the blob store: never a path.
613+
if after_hash.len() != 64 || !after_hash.bytes().all(|b| b.is_ascii_hexdigit()) {
614+
return None;
615+
}
616+
let intact = |bytes: &Vec<u8>| {
617+
bytes.len() <= MAX_PATCHED_MANIFEST_BYTES
618+
&& compute_git_sha256_from_bytes(bytes).eq_ignore_ascii_case(after_hash)
619+
};
620+
let local = common.socket_dir().join("blobs").join(after_hash);
621+
let small = tokio::fs::metadata(&local)
622+
.await
623+
.is_ok_and(|meta| meta.len() <= MAX_PATCHED_MANIFEST_BYTES as u64);
624+
if small {
625+
if let Some(bytes) = socket_patch_core::utils::fs::read_regular_to_bytes(&local)
626+
.await
627+
.ok()
628+
.filter(intact)
629+
{
630+
return Some(bytes);
631+
}
632+
}
633+
if common.offline {
634+
return None;
635+
}
636+
let overrides = common.api_client_overrides();
637+
let client = params
638+
.api_client
639+
.get_or_init(|| async {
640+
socket_patch_core::api::client::get_api_client_with_overrides(overrides)
641+
.await
642+
.0
643+
})
644+
.await;
645+
let mut body = client.fetch_blob(after_hash).await.ok()??;
646+
let mut bytes = Vec::new();
647+
while let Some(chunk) = body.chunk().await.ok()? {
648+
bytes.extend_from_slice(chunk.as_ref());
649+
if bytes.len() > MAX_PATCHED_MANIFEST_BYTES {
650+
return None;
651+
}
652+
}
653+
Some(bytes).filter(intact)
654+
}
655+
544656
/// Core VEX pipeline shared by the standalone `vex` command and the
545657
/// embedded `apply`/`vendor`/`scan` `--vex` paths: resolve the product, verify the
546658
/// plan's record view against disk (unless `no_verify`), build the OpenVEX
@@ -738,17 +850,34 @@ async fn generate_vex(
738850
};
739851
let hidden =
740852
|purl: &str| purl.starts_with("pkg:npm/") && (npm_store_hidden || pnp_unconsumed(purl));
853+
let excusable: HashSet<String> = outcome
854+
.failed
855+
.iter()
856+
.filter(|f| {
857+
f.reason == "package_not_found"
858+
&& plan.lockfile_basis.contains(&f.purl)
859+
&& !mirror_refused(&f.purl)
860+
&& crawled(&f.purl)
861+
&& !hidden(&f.purl)
862+
})
863+
.map(|f| f.purl.clone())
864+
.collect();
865+
// A yarn classic pin attests from the lock only when the lock
866+
// installs the patched package's dependencies with it (#1538).
867+
let excusable_purls: Vec<&str> = excusable.iter().map(String::as_str).collect();
868+
let dependency_gaps =
869+
yarn_classic_lock_dependency_gaps(common, params, &plan, &excusable_purls).await;
741870
let mut lockfile_attested = Vec::new();
742-
outcome.failed.retain(|f| {
743-
let excused = f.reason == "package_not_found"
744-
&& plan.lockfile_basis.contains(&f.purl)
745-
&& !mirror_refused(&f.purl)
746-
&& crawled(&f.purl)
747-
&& !hidden(&f.purl);
748-
if excused {
749-
lockfile_attested.push(f.purl.clone());
871+
outcome.failed.retain_mut(|f| {
872+
if f.reason != "package_not_found" || !excusable.contains(&f.purl) {
873+
return true;
874+
}
875+
if let Some(gap) = dependency_gaps.get(&f.purl) {
876+
f.reason = gap.to_string();
877+
return true;
750878
}
751-
!excused
879+
lockfile_attested.push(f.purl.clone());
880+
false
752881
});
753882
outcome.applied.extend(lockfile_attested);
754883
withhold_unpatched_jvm_copies(
@@ -1786,6 +1915,11 @@ fn omission_phrase(reason: &str) -> &'static str {
17861915
"the patched package.json declares a dependency that is not installed beside \
17871916
the package"
17881917
}
1918+
DEPENDENCY_MANIFEST_UNVERIFIABLE => {
1919+
"the patch changes package.json, and the patched package.json could be read \
1920+
neither from .socket/blobs nor from the patch API to check the dependencies the \
1921+
lockfile installs with it"
1922+
}
17891923
"mirror_overrides_source" => {
17901924
"a Bundler mirror overrides the hosted source and installed bytes were not verified"
17911925
}
@@ -1992,6 +2126,7 @@ mod tests {
19922126
"stale_install",
19932127
"mirror_overrides_source",
19942128
DEPENDENCY_NOT_INSTALLED,
2129+
DEPENDENCY_MANIFEST_UNVERIFIABLE,
19952130
RECORD_UNAVAILABLE,
19962131
RECORD_MISMATCH,
19972132
VENDOR_UNWIRED,

‎crates/socket-patch-cli/tests/apply/apply_silent_staging_errors.rs‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,8 @@
1818
//! Under `--json` both diagnostics stay off stderr — the envelope is the
1919
//! machine channel, and `apply_invariants.rs` pins its exact shape for
2020
//! this path (partialFailure, empty events, NO top-level error record —
21-
//! deliberately distinct from vendor's `no_local_source` hard error).
21+
//! unlike vendor, which records a `failed` / `apply_failed` event for the
22+
//! patch; see `vendor/vendor_silent_errors.rs`).
2223
2324
use std::path::Path;
2425

‎crates/socket-patch-cli/tests/cli/envelope_helper_copies.rs‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@ use crate::common::envelope::{
2020
/// and delete its entry; a stale entry is not an error, so a PR that
2121
/// migrates one never turns another red.
2222
const PENDING_PRIVATE_READERS: &[&str] = &[
23-
"covgap_commands_rollback.rs",
2423
"e2e_redirect_gradle_build.rs",
2524
"e2e_redirect_pnpm_build.rs",
2625
"e2e_socket_yml_policy.rs",

0 commit comments

Comments
 (0)