Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
delthas
force-pushed
the
wip-yarn-berry
branch
from
September 18, 2026 10:16
88d4cd6 to
0d660a1
Compare
delthas
changed the title
delthas
force-pushed
the
wip-yarn-berry
branch
from
September 21, 2026 12:34
0d660a1 to
e636e34
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## development/8.6 #2706 +/- ##
===================================================
+ Coverage 74.76% 74.77% +0.01%
===================================================
Files 227 227
Lines 18650 18650
Branches 3864 3894 +30
===================================================
+ Hits 13943 13945 +2
+ Misses 4702 4700 -2
Partials 5 5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
delthas
force-pushed
the
wip-yarn-berry
branch
from
September 22, 2026 13:57
e636e34 to
f499aad
Compare
delthas
force-pushed
the
wip-yarn-berry
branch
3 times, most recently
from
September 22, 2026 14:39
d7056cc to
7d5797a
Compare
node-fcntl is now published to npm (scality/node-fcntl#23), so there is no reason to keep pulling it from GitHub. A registry tarball is immutable and carries an integrity hash, where the codeload tarball carried none; nothing else changes, as it is the same code under a scoped name. Only the import path differs, in lib/storage/data/file/utils.js. Yarn 1 resolves it identically -- the addon builds and loads as before.
Yarn 1 is unmaintained and its installs have become unreliable; this moves Arsenal to Yarn 4.18.0. Yarn is pinned through the packageManager field, with its integrity hash, rather than by committing a release binary: CI enables Corepack explicitly, and anyone without it gets a clear error naming the version to use rather than a silently different Yarn. nodeLinker is node-modules rather than the default PnP, which cannot load this package's native addons from inside a zip. It is also what scality/bench-vault chose. approvedGitRepositories is required, not optional hardening: Yarn defaults to an empty allowlist that blocks every Git fetch, so a package with Git dependencies cannot install without it. Scoped to the Scality org rather than the "**" Yarn writes during migration. npmPreapprovedPackages exempts @scality/* from the one-day quarantine Yarn applies to newly published versions. That guard is aimed at compromised third-party publishes; our own packages come from our CI, so waiting a day to consume them buys nothing. Everything else keeps it. networkConcurrency: 1 replaces Yarn 1's --network-concurrency flag, which Berry rejects on the CLI but still honours as a setting. Git dependencies that still carry Yarn 1 lockfiles are bootstrapped with Yarn Classic, and concurrent Classic installs race on its shared cache. It can be dropped once those dependencies are themselves on Berry. Note that Yarn rewrites .yarnrc.yml during the migration step itself, dropping comments and re-adding permissive security defaults; the file here is the reviewed version, and it stays put on subsequent installs.
delthas
force-pushed
the
wip-yarn-berry
branch
2 times, most recently
from
September 22, 2026 15:13
4310775 to
031aac6
Compare
eslint.config.mjs imports @eslint/js and @eslint/eslintrc, neither of which is declared. Both resolve today only because ESLint pulls them in and the node-modules linker hoists them to the top of the tree, so any stricter resolver breaks linting outright. They are pinned to the versions already resolved, so this is a no-op at install time.
Berry disables install/build scripts by default, where Yarn 1 let all ~1100 packages run them. Four genuinely need to compile: @scality/fcntl and ioctl are node-gyp addons, leveldown ships native code, and mongodb-memory-server downloads a server binary. The key must be the real package name. Keying it on an alias makes Berry silently skip the build (YN0004) -- the install still succeeds and the addon is simply never compiled. This is the clearest security improvement in the migration: install-time code execution goes from every dependency to four named ones.
delthas
force-pushed
the
wip-yarn-berry
branch
from
September 22, 2026 15:22
031aac6 to
211997a
Compare
| - name: Install NodeJS | ||
| uses: actions/setup-node@v6 | ||
| with: | ||
| node-version: '24' |
There was a problem hiding this comment.
cache: yarn and cache-dependency-path were removed from this job but kept in every other job. Without them, the test-workflows job re-downloads all dependencies on every CI run.
Suggested change
| node-version: '24' | |
| node-version: '24' | |
| cache: yarn | |
| cache-dependency-path: tests/workflows/yarn.lock |
Berry runs scripts through its own portable shell, which has no `export` builtin: both scripts failed immediately with "command not found: export". An inline environment prefix is equivalent and also valid in bash, so this works under either package manager. Append to NODE_OPTIONS rather than replacing it, so that a value set by the caller survives; Yarn passes `--require .pnp.cjs` through this variable under the PnP linker, and overwriting it leaves every script unable to resolve its own dependencies. The `:-` default is required because that same shell treats an unset variable as an error rather than as empty, and CI does not set NODE_OPTIONS.
Setting packageManager flips Yarn's Git-dependency bootstrap from the Yarn Classic branch to the Berry branch. Classic ran `yarn install`, which executes prepare; Berry runs `yarn pack --install-if-needed`, which executes prepack and not prepare. Without prepack, that bootstrap produces a 3-entry, 6969-byte tarball containing only LICENSE, README and package.json -- no build/ at all -- so every Berry consumer would install an empty Arsenal. Verified on a pristine `git archive HEAD` checkout; with prepack the same command produces 701 entries including the full build/ tree. Cloudserver already pins Yarn 4, so it is such a consumer today. # Conflicts: # package.json # Conflicts: # package.json
- corepack enable has to run before actions/setup-node. Its `cache: yarn` probe shells out to Yarn to locate the cache folder, and `yarn cache dir` exits 1 once packageManager pins Yarn 4 (actions/setup-node#1027). - --frozen-lockfile becomes --immutable, and --network-concurrency is dropped: Berry rejects it on the CLI and it now lives in .yarnrc.yml. - Berry removed the global --silent flag, so `yarn --silent X` becomes `yarn X`; Berry is quiet by default. - The tests/workflows job keeps Yarn 1 and therefore drops `cache: yarn`, since setup-node probes from the repo root. It also uses working-directory rather than `yarn --cwd`: Corepack resolves packageManager from the process CWD, so --cwd would pick up the root's Yarn 4 and ignore the nested pin. - Yarn 1 built the package as a side effect of install, via the root prepare script; Berry does not run prepare on install. Some tests spawn a process that requires build/, so it is now built explicitly. - release.spec.ts mocks workflow steps by name, so the new Corepack step needs an entry or the suite really executes it inside the act image.
This is a separate install root with its own lockfile, so it becomes its own Berry project rather than a workspace: keeping its dependency tree isolated from the root avoids any hoisting interaction, and Berry inherits the root .yarnrc.yml anyway, so only the linker is restated here. @kie/act-js and unrs-resolver need to compile, so they are allowlisted via dependenciesMeta; build scripts are otherwise off by default. CI enables Corepack for this job and uses --immutable in place of --frozen-lockfile. Verified locally on Node 24 against a live container socket: 7/7 suites pass, including the release workflow tests that really execute the new Corepack step through act.
This directory had a package.json but no lockfile, so Berry treated it
as part of the root project and refused to install ("doesn't seem to be
part of the project declared in ..."). It now carries its own lockfile
and is a separate Berry project, which also means CI stops re-resolving
its dependency ranges on every run.
Since the versions were being pinned for the first time, the 2018-era
ones are refreshed rather than frozen in place:
node-forge ^0.7.1 -> ^1.3.1 (matches the root)
async ~2.6.1 -> ~2.6.4 (matches the root)
mocha 5.2.0 -> ^12.0.2 (and moved to devDependencies)
The code only uses pki.privateKeyFromPem, util.decode64, md.sha256 and
async.waterfall, all unchanged across those majors. All 3 tests pass,
including the decryption path that exercises node-forge.
delthas
force-pushed
the
wip-yarn-berry
branch
2 times, most recently
from
September 22, 2026 15:50
0593c98 to
0437a71
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves Arsenal from Yarn 1 to Yarn 4.18.0 with the
node-moduleslinker, including the two nested install roots (tests/workflows,lib/executables/pensieveCreds).Yarn is pinned through the
packageManagerfield with its integrity hash, rather than by committing a release binary; CI enables Corepack, and contributors needcorepack enableonce.Install-time code execution drops from every dependency to four: Berry disables build scripts by default, and the packages that genuinely need one are allowlisted in
dependenciesMeta.Berry does not run the root
preparescript on install, sobuild/is no longer a side effect ofyarn install. Aprepackscript keeps consumers that install Arsenal from Git getting a built package, and CI builds explicitly.Why
node-modulesand not PnPPnP is Berry's default: no
node_modules, dependencies stay zipped, and a package may only require what it declares. That last property is the interesting one, because it turns every dependency that resolves by luck of hoisting into a hard failure.It does work here. Flipping
nodeLinkertopnpand fixing four things gets a fully greenyarn build, all 4433 tests, andyarn lint:packageExtensionsfor@aws-sdk/lib-storage, whose published.d.tsimports@smithy/typeswhile declaring it only as a devDependency — a recurring shape in that SDK (Missing@aws-sdk/smithy-clientdependency in@aws-sdk/lib-storageaws/aws-sdk-js-v3#3626, #5035, #5362).packageExtensionsforts-jest, which requiresjest-utilunconditionally but declares it as an optional peer.NODE_OPTIONSinstead of overwriting it (PnP passes--require .pnp.cjsthrough that variable), and declaring@eslint/js/@eslint/eslintrc.This PR still lands on
node-modules, because that keeps today's resolution semantics and makes the change purely a package-manager swap — reviewable and revertible on its own. PnP is a sensible follow-up, and the remaining work is twopackageExtensionsentries.Worth knowing before that follow-up: the same experiment on Cloudserver does not come out clean.
@scality/cloudserverclient@1.0.12declares 4 dependencies while its shipped code requires far more; 28 AWS/Smithy packages had to be added by hand before I stopped, and it had not converged. That one needs fixing at the source, not papering over downstream.Notes for reviewers
approvedGitRepositoriesis required, not hardening. Yarn's default is an empty allowlist that blocks every Git fetch, so a package with Git dependencies cannot install without it. It is scoped to the Scality org rather than the**Yarn writes during migration.networkConcurrency: 1replaces Yarn 1's--network-concurrency, which Berry rejects on the CLI but still honours as a setting. Git dependencies that still carry Yarn 1 lockfiles are bootstrapped by Yarn Classic, and concurrent Classic installs race on its shared cache. Droppable once those dependencies are on Berry.npmMinimalAgeGatestays at Yarn's 1-day default, so a dependency update touching a release published in the last 24 hours will fail.@scality/*is preapproved, so our own releases are usable immediately.Consumers are unaffected. Yarn 1 never reads a Git dependency's lockfile — it resolves Arsenal's tree into its own — and Berry consumers do the same.