Migrate to Yarn 4 (Berry)

[delthas]

Moves Cloudserver from Yarn 1 to Yarn 4.18.0 with the node-modules linker.

Outside the lockfile the change is small: four CI files, the Dockerfile, .yarnrc.yml, .gitignore and package.json. No application code changes.

Design decisions

• node-modules, not PnP. PnP resolves only what a package declares, and this tree does not. @scality/cloudserverclient@1.0.12 declares 4 dependencies while its shipped code requires far more; I added 28 AWS/Smithy packages by hand through packageExtensions and it still had not converged, each new code path finding more. That package needs fixing at the source before PnP is worth revisiting. (Arsenal is closer — see Migrate to Yarn 4 (Berry) Arsenal#2711 — but it is deferred there too.)

• Corepack and packageManager, not a committed release binary. No 4 MB blob in the tree and no yarnPath. The cost is that contributors run corepack enable once; CI does it explicitly before Yarn is first invoked.

• utapi is bumped first, on purpose. utapi required levelup, memdown and encoding-down without declaring them. They resolved only because Yarn 1's hoisting happened to place compatible versions on top; Berry hoists levelup@0.19.1 instead of 4.4.0 and the server dies at startup with "Must provide a location for the database". 8.2.7 declares them (UTAPI-126), so the bump lands before the migration and every commit here is independently installable.

• The non-workspace test suites now call mocha directly. tests/functional/{report,raw-node,healthchecks} each have a package.json but are not workspaces, so cd X && yarn test resolves to the root project. Yarn 1 tolerated it; Berry hard-errors with "the nearest package directory doesn't seem to be part of the project". They now invoke mocha directly, like the twenty sibling ft_* scripts already did.

• Build scripts stay off by default. Install-time code execution drops from every dependency to eight, allowlisted individually in dependenciesMeta.

• yarn workspaces focus --production needs a guard. It replaces --production in the image build, but it cannot enforce lockfile immutability — it silently re-resolves. A cheap build-less yarn install --immutable runs first to validate the lockfile; together they preserve what --frozen-lockfile gave us.

• The runtime image drops Yarn entirely. packageManager makes the bundled Yarn 1 refuse to run, and the obvious fix — baking the Corepack-pinned release in, since downloading it on every container start would break air-gapped deployments — buys nothing. The runtime needs a package manager only to expand start into its three node processes, and npm ships with the base image, so CMD becomes npm run start. Federation already bypasses this entirely and runs node index.js from supervisord. Verified by building the production target and booting it.

  One downstream consumer does run yarn inside this image: scality/sorbet's docker-compose.yml. It pins cloudserver:8.7.9, so it is unaffected today; SOR-295 tracks switching it before that pin moves.

• approvedGitRepositories is required, not hardening. Yarn's default is an empty allowlist that blocks every Git fetch, so a repo with Git dependencies cannot install at all without it. Scoped to the Scality org rather than the ** Yarn writes during migration.

• networkConcurrency: 1 replaces Yarn 1's --network-concurrency, which Berry rejects on the CLI but still honours as a setting. Git dependencies that still carry Yarn 1 lockfiles are bootstrapped by Yarn Classic, and concurrent Classic installs race on its shared cache. Droppable once those dependencies are on Berry.

Issue: CLDSRV-1004

[bert-e]

Hello delthas,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/bypass_source_branch_lineage	Bypass the cross-branch contamination check	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request.
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[codecov]

❌ 3 Tests Failed:

Tests completed	Failed	Passed	Skipped
9913	3	9910	0

View the full list of 3 ❄️ flaky test(s)

"after all" hook for "should batch delete objects where requester has permission"::Multi-Object Delete Access "after all" hook for "should batch delete objects where requester has permission"

Flake rate in main: 100.00% (Passed 0 times, Failed 10 times)

Stack Traces | 1.4s run time

The bucket you tried to delete is not empty.

"before all" hook for "should not delete locked objects"::Multi-Object Delete with Object Lock "before all" hook for "should not delete locked objects"

Flake rate in main: 100.00% (Passed 0 times, Failed 10 times)

Stack Traces | 0.006s run time

Object Lock configuration cannot be enabled on existing buckets

"before all" hook for "should return access denied error for each object where no acl permission"::Multi-Object Delete Access "before all" hook for "should return access denied error for each object where no acl permission"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 5.1s run time

Socket timed out without establishing a connection within 5000 ms

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[francoisferrand]

upgrading yarn is great, but best delay for a few days at least:

• there are many dependency bumps / cleanups in progress : so really duplicated work to do this at the same time, and try to fix these problems
• same for prettier interaction: best let it land first
• impact on developers is not trivial: enabling corepack is required and global, so it will likely require combined deployment accross all repos. Could be its own project, even with yarn v1, btw → need communication & coordination across repos
• if we switch but don't use the features of yarn (pnp), why stick to yarn? No saying I am against it, but worth evaluating if yarn is still needed (and maybe we still want to keep it anyway, for muscle memory 🤷‍♂️)