Skip to content

Commit 4d64cfc

Browse files
committed
Merge origin/main into agent/fix-pypi-out-of-root-include-pylock
Resolve the pypi_requirements.rs conflict with #1796 (extras pin un-hosted beside a pylock): both sides added new helpers after sibling_pin_wirable, so keep this PR's pin_outside_root / installs_outside_vendor / outside_root_refusal and main's extras_refusal / fresh_sibling_pin side by side. pypi.rs auto-merged: the out-of-root refusal in detect_pypi_flavor and main's fresh_sibling_pin wiring are independent. Co-Authored-By: Claude <noreply@anthropic.com>
2 parents 70b2147 + cbbbdb9 commit 4d64cfc

10 files changed

Lines changed: 904 additions & 39 deletions

File tree

‎crates/socket-patch-cli/CLI_CONTRACT.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1606,7 +1606,7 @@ command's integration tests check its output.
16061606
| `redirect_unconfirmed` | `skipped` (`details.mode: "hosted"`) | hosted `scan` / `get` (v5.0, additive): the patch was granted but no lockfile entry pinning it could be rewritten. The status and exit code are unchanged for now, pending the open hosted exit-policy decision (#704); `--silent` hides the human line. |
16071607
| `lockfile_unreadable` / `lockfile_unparseable` / `patched_ref_invalid` / `patched_ref_unattributable` | run-level `warnings[]` | vex (every form): lockfile-discovery diagnostics — see "Manifest-less VEX (lockfile discovery)". Never flip the exit on their own. |
16081608
| `vex_npm_shrinkwrap_only` | run warning and `failed[].reason` | vex (every form, #899): the patch is wired only in a root `npm-shrinkwrap.json` with no `package-lock.json` twin, which npm >= 12 never reads; no statement until the twin exists. `list` / `rollback` / `remove` still manage the wiring. |
1609-
| `vendor_multiple_lockfiles` / `pypi_multiple_lockfiles` | `skipped` (warning) | vendor: a sibling lockfile of another package manager (for PyPI, also a root `requirements.txt` that pins the package beside the wired tool lock) will still install UNPATCHED bytes; names the wired winner + the ignored locks. Beside `Pipfile.lock`, `uv.lock` or a wired PEP 751 / script lock, an exact registry pin of the package in `requirements.txt` or an in-root `-r` include (`pipenv requirements` or `uv export` output, or the file `pip lock -r requirements.txt` read) is wired with the lock instead and never named here (#612, #1368, #1758): both files then refer to the committed wheel, the ledger entry records both, every revert restores both, a superseding patch re-wires both, and a re-run over a project whose lock is already wired (a vendor from before, or an export made since) wires the export too (`pypi_requirements_sibling_wired`). A pin the requirements wiring cannot rewrite (a range, extras, an include outside the root) stays a loser. Likewise, a `pylock.toml` / `pylock.<name>.toml` that `pipenv lock` writes beside `Pipfile.lock` under `[pipenv] use_pylock = true` is wired with it (#1368), with the same ledger, revert, supersede and re-run behavior (`pypi_pipenv_pylock_wired`); a pylock the wiring cannot rewrite in place (a symlink, a non-registry source) stays a loser. |
1609+
| `vendor_multiple_lockfiles` / `pypi_multiple_lockfiles` | `skipped` (warning) | vendor: a sibling lockfile of another package manager (for PyPI, also a root `requirements.txt` that pins the package beside the wired tool lock) will still install UNPATCHED bytes; names the wired winner + the ignored locks. Beside `Pipfile.lock`, `uv.lock` or a wired PEP 751 / script lock, an exact registry pin of the package in `requirements.txt` or an in-root `-r` include (`pipenv requirements` or `uv export` output, or the file `pip lock -r requirements.txt` read) is wired with the lock instead and never named here (#612, #1368, #1758): both files then refer to the committed wheel, the ledger entry records both, every revert restores both, a superseding patch re-wires both, and a re-run over a project whose lock is already wired (a vendor from before, or an export made since) wires the export too (`pypi_requirements_sibling_wired`). An exact pin of the package with extras (`six[foo]==1.16.0`), or a file's only line for it when that line is an unpinned extras requirement (`six[foo]`, which hosted mode also rewrites), cannot be expressed as a wheel path line. It refuses a fresh wiring of the lock `pypi_extras_unsupported`, as the requirements-only wiring does, rather than leave `pip install -r` unpatched; a hosted → vendored switch then keeps both files' hosted pins (#1794). A lock already wired re-runs in sync with the extras pin still named here. Any other pin the requirements wiring cannot rewrite (a range, an include outside the root) stays a loser. Likewise, a `pylock.toml` / `pylock.<name>.toml` that `pipenv lock` writes beside `Pipfile.lock` under `[pipenv] use_pylock = true` is wired with it (#1368), with the same ledger, revert, supersede and re-run behavior (`pypi_pipenv_pylock_wired`); a pylock the wiring cannot rewrite in place (a symlink, a non-registry source) stays a loser. |
16101610
| `vendor_npm_shrinkwrap_only` | `skipped` (warning) | vendor / scan / get `--mode vendored` (npm, #899): the package was wired into `npm-shrinkwrap.json`, the only npm lock (also on an in-sync re-run). npm >= 12 never reads the shrinkwrap and installs the unpatched registry bytes; rename the lock to `package-lock.json` (or commit a copy under that name) and re-run. |
16111611
| `vendor_npm_deno_lock` | `skipped` (warning) | vendor / scan / get `--mode vendored` (npm, every lockfile flavor, #406): the package was vendored (also on an in-sync re-run), but the project root's `deno.lock` locks the same `name@version`. `deno install` installs a `package.json` project's npm dependencies from `deno.lock` and never reads the rewired lock, so installs Deno runs keep the registry copy and `vex` omits the patch (`vex_deno_lock_copy`); npm / pnpm / yarn / bun installs are patched. To patch what Deno installs, run `deno install` and then `socket-patch scan --mode agent`. |
16121612
| `vendor_workspace_member_skipped` | `skipped` (warning) | vendor / scan / get `--mode vendored` (npm, #688): a lock entry with the package's `name@version` is the project's own source (a workspace member or a `file:` directory), so it is left alone while the lock's registry copies are vendored; patch that source directly if it needs the fix. When it is the only instance, vendoring refuses instead. |

‎crates/socket-patch-cli/src/commands/vendored_backend/repair.rs‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -551,6 +551,67 @@ impl VendoredBackend<'_> {
551551
continue;
552552
}
553553
}
554+
// A stray file in the tree (a Finder `.DS_Store`) fails
555+
// the Gradle build (#656); the index says what belongs
556+
// there, so it goes offline too.
557+
if vendor::jvm::apply::is_jvm_entry(&entry) {
558+
let indexed = vendor::jvm::apply::recorded_tree_files(&entry);
559+
let swept =
560+
match vendor::jvm::apply::unindexed_tree_entries(&common.cwd, &indexed)
561+
.and_then(|found| {
562+
vendor::jvm::apply::refuse_linked(&found).map(|()| found)
563+
}) {
564+
Ok(stray) if stray.is_empty() || common.dry_run => Ok(stray),
565+
Ok(stray) => vendor::jvm::apply::sweep_unindexed(&stray)
566+
.await
567+
.map(|()| stray),
568+
Err(e) => Err(e),
569+
};
570+
match swept {
571+
Ok(stray) if stray.is_empty() => {}
572+
Ok(stray) if common.dry_run => {
573+
env.record(
574+
PatchEvent::new(PatchAction::Verified, purl.clone())
575+
.with_details(serde_json::json!({
576+
"vendorArtifact": true,
577+
"wouldRemoveUnindexed": stray
578+
.iter()
579+
.map(|s| s.rel.as_str())
580+
.collect::<Vec<_>>(),
581+
})),
582+
);
583+
}
584+
Ok(stray) => {
585+
if !quiet {
586+
for s in &stray {
587+
println!("Removed unindexed vendored file {}", s.rel);
588+
}
589+
}
590+
env.record(
591+
PatchEvent::new(PatchAction::Rebuilt, purl.clone())
592+
.with_details(serde_json::json!({
593+
"path": entry.artifact.path,
594+
"unindexedRemoved": stray
595+
.iter()
596+
.map(|s| s.rel.as_str())
597+
.collect::<Vec<_>>(),
598+
"artifactRebuilt": false,
599+
})),
600+
);
601+
repaired += 1;
602+
}
603+
Err(e) => {
604+
fail(
605+
env,
606+
common.json,
607+
purl,
608+
"vendor_artifact_unrepairable",
609+
format!("cannot remove unindexed vendored files: {e}"),
610+
);
611+
continue;
612+
}
613+
}
614+
}
554615
// Dir-shaped gem artifacts from pre-inventory vendors:
555616
// the health check could only verify the PATCHED members
556617
// — unpatched-file drift is invisible until a re-vendor

‎crates/socket-patch-cli/tests/e2e_vendor_gradle_build.rs‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -573,6 +573,44 @@ fn gradle_vendor_428_subproject_refuses() {
573573
assert_victim(&out, "pristine", None, "`cd app && gradle` still builds");
574574
}
575575

576+
// ── #656 ────────────────────────────────────────────────────────────────
577+
578+
/// #656: a Finder `.DS_Store` in the vendored tree fails the build with
579+
/// "unindexed files … re-run `socket-patch vendor`". That re-run, and
580+
/// `repair`, now remove it, and the build consumes the patched jar again.
581+
#[test]
582+
#[ignore = "real Gradle (the fake Central); run with --ignored"]
583+
fn gradle_vendor_656_stray_tree_file() {
584+
const SUITE: &str = "e2e_vendor_gradle_build::656";
585+
let Some(world) = World::new(SUITE) else {
586+
return;
587+
};
588+
let proj = world.root.join("proj");
589+
simple_project(&proj);
590+
prepare(&world, &proj, "printRuntimeClasspath");
591+
world.vendor(&proj);
592+
let out = world.build(&proj, &["printRuntimeClasspath"]);
593+
assert_victim(&out, "patched", Some(&proj), "vendored build");
594+
let stray = proj
595+
.join(tree_rel(VICTIM, VICTIM_VERSION))
596+
.join(".DS_Store");
597+
for remedy in [&["vendor"][..], &["repair"]] {
598+
std::fs::write(&stray, b"\0\0\0\x01Bud1").unwrap();
599+
let out = world.build(&proj, &["printRuntimeClasspath"]);
600+
assert!(!ok(&out), "{remedy:?}: the stray file fails the build");
601+
assert!(
602+
dump(&out).contains("unindexed files"),
603+
"{remedy:?}:\n{}",
604+
dump(&out)
605+
);
606+
world.socket_ok(&proj, remedy);
607+
assert!(!stray.exists(), "{remedy:?} removes the stray file");
608+
world.socket_ok(&proj, &["vendor", "--check"]);
609+
let out = world.build(&proj, &["printRuntimeClasspath"]);
610+
assert_victim(&out, "patched", Some(&proj), &format!("after {remedy:?}"));
611+
}
612+
}
613+
576614
// ── #429 ────────────────────────────────────────────────────────────────
577615

578616
/// #429: a `core.autocrlf=true` clone of a vendored checkout (buildSrc

‎crates/socket-patch-cli/tests/mode_migration_pypi.rs‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3402,6 +3402,46 @@ async fn pip_lock_rerun_wires_a_requirements_pin_added_later() {
34023402
);
34033403
}
34043404

3405+
/// #1794: the extras refusal guards a FRESH lock wiring only. A pylock
3406+
/// already wired to the vendored wheel (by a run before the extras pin
3407+
/// was added, or by a release before the refusal) re-runs in sync instead
3408+
/// of failing every later `vendor`, and the extras pin stays a named
3409+
/// loser as before.
3410+
#[tokio::test]
3411+
async fn pip_lock_rerun_keeps_a_wired_lock_beside_an_extras_pin() {
3412+
let (_tmp, root) = project();
3413+
stage_pip_lock(&root);
3414+
std::fs::write(root.join("requirements.txt"), "idna==3.7\n").unwrap();
3415+
stage_manifest(&root);
3416+
let (code, env) = run_cli(&root, &["vendor"], &[]);
3417+
assert_eq!(code, 0, "{env:#}");
3418+
let wired = std::fs::read_to_string(root.join("pylock.toml")).unwrap();
3419+
assert!(
3420+
wired.contains(&format!(".socket/vendor/pypi/{UUID}/")),
3421+
"{wired}"
3422+
);
3423+
3424+
std::fs::write(
3425+
root.join("requirements.txt"),
3426+
"idna==3.7\nsix[foo]==1.16.0\n",
3427+
)
3428+
.unwrap();
3429+
let (code, env) = run_cli(&root, &["vendor"], &[]);
3430+
assert_eq!(code, 0, "an in-sync lock is not refused: {env:#}");
3431+
assert!(
3432+
!env.to_string().contains("pypi_extras_unsupported"),
3433+
"{env:#}"
3434+
);
3435+
assert_eq!(
3436+
std::fs::read_to_string(root.join("pylock.toml")).unwrap(),
3437+
wired
3438+
);
3439+
assert_eq!(
3440+
std::fs::read_to_string(root.join("requirements.txt")).unwrap(),
3441+
"idna==3.7\nsix[foo]==1.16.0\n"
3442+
);
3443+
}
3444+
34053445
/// #1758: hosted mode pins six in both `requirements.txt` and the
34063446
/// `pip lock` pylock.toml. The hosted → vendored switch (plain `vendor`,
34073447
/// the eject, and `scan --mode vendored`) restores both files to their
@@ -3478,6 +3518,86 @@ async fn pip_lock_hosted_to_vendored_keeps_the_requirements_patched() {
34783518
}
34793519
}
34803520

3521+
/// #1794: a `requirements.txt` pin with extras (`six[foo]==1.16.0`) can't
3522+
/// be written as a vendored wheel path line, so the pylock beside it can't
3523+
/// be vendored alone: the switch's upstream restore had just un-hosted the
3524+
/// requirements pin, and `pip install -r requirements.txt` would then go
3525+
/// from patched to unpatched. The switch refuses the purl with the
3526+
/// requirements wiring's own code, and the restore is rolled back, so both
3527+
/// files keep their hosted pin byte for byte. Before the fix both files
3528+
/// were un-hosted, only pylock.toml was wired, and the run exited 0.
3529+
#[tokio::test]
3530+
async fn pip_lock_hosted_to_vendored_refuses_an_extras_pin() {
3531+
for takeover in [false, true] {
3532+
let server = MockServer::start().await;
3533+
let hosted_url = mount_hosted_api(&server, true).await;
3534+
mount_patch_view(&server).await;
3535+
let uri = server.uri();
3536+
let switch: Vec<&str> = if takeover {
3537+
vec![
3538+
"scan",
3539+
"--mode",
3540+
"vendored",
3541+
"--yes",
3542+
"--api-url",
3543+
&uri,
3544+
"--org",
3545+
ORG,
3546+
"--api-token",
3547+
"fake-token",
3548+
"--patch-server-url",
3549+
&uri,
3550+
"--vendor-url",
3551+
&uri,
3552+
]
3553+
} else {
3554+
let mut args = vec!["vendor", "--patch-server-url", &uri, "--vendor-url", &uri];
3555+
args.extend(api_args(&uri));
3556+
args
3557+
};
3558+
let (_tmp, root) = project();
3559+
std::fs::write(
3560+
root.join("requirements.txt"),
3561+
"idna==3.7\nsix[foo]==1.16.0\n",
3562+
)
3563+
.unwrap();
3564+
std::fs::write(root.join("pylock.toml"), PIP_LOCK).unwrap();
3565+
let files = ["requirements.txt", "pylock.toml"];
3566+
let (code, env) = hosted_scan(&root, &server);
3567+
assert_eq!(code, 0, "hosted scan: {env:#}");
3568+
let hosted: Vec<String> = files
3569+
.iter()
3570+
.map(|file| std::fs::read_to_string(root.join(file)).unwrap())
3571+
.collect();
3572+
for (file, text) in files.iter().zip(&hosted) {
3573+
assert!(
3574+
text.contains(&hosted_url),
3575+
"hosted mode pins {file}:\n{text}\n{env:#}"
3576+
);
3577+
}
3578+
3579+
stage_manifest(&root);
3580+
prebuilt_common::mount_project(&server, &root).await;
3581+
let (code, env) = run_cli(&root, &switch, &[]);
3582+
assert_eq!(code, 1, "{switch:?}: the purl is refused: {env:#}");
3583+
assert!(
3584+
env.to_string().contains("pypi_extras_unsupported"),
3585+
"{switch:?}: refused with the requirements wiring's code: {env:#}"
3586+
);
3587+
for (file, before) in files.iter().zip(&hosted) {
3588+
let text = std::fs::read_to_string(root.join(file)).unwrap();
3589+
assert_eq!(
3590+
&text, before,
3591+
"{switch:?}: {file} keeps its hosted pin byte for byte\n{env:#}"
3592+
);
3593+
}
3594+
assert!(
3595+
!root.join(".socket/vendor/pypi").exists(),
3596+
"{switch:?}: nothing is vendored"
3597+
);
3598+
}
3599+
}
3600+
34813601
/// The `pipenv requirements > requirements.txt` export of [`stage_pipenv`]'s
34823602
/// lock: the file Docker / plain-pip installs read.
34833603
const PIPENV_EXPORT: &str = "-i https://pypi.org/simple\nsix==1.16.0 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2'\n";

0 commit comments

Comments
 (0)