Skip to content

test(release): complete credential-free brand pilot #436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AprilNEA wants to merge 3 commits into
base: xuan/code-559
Choose a base branch
from
Open

test(release): complete credential-free brand pilot #436

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9f41721
test(release): validate credential-free brand pilot
AprilNEA Aug 9, 2026
a7f15df
fix(release): harden pilot matrix gates
AprilNEA Aug 9, 2026
e37d4a1
fix(release): use protected release environment
AprilNEA Aug 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
203 changes: 203 additions & 0 deletions .github/release/brand-matrices/code-561-pilot.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
{
"brandBuildMatrixVersion": 1,
"brands": [
{
"brandId": "acme",
"channel": "canary",
"compliance": {
"android": {
"checklist": {
"configurableFeaturesDisclosed": true,
"dataPracticesReviewed": true,
"noExecutableCode": true,
"permissionsReviewed": true,
"storeMetadataReviewed": true
},
"disclosedFeatures": [
"feature.aiAssist",
"feature.newEditor",
"modules.messaging.enabled",
"modules.terminal.enabled",
"modules.workspace.enabled"
]
},
"desktop": {
"checklist": {
"configurableFeaturesDisclosed": true,
"dataPracticesReviewed": true,
"noExecutableCode": true,
"permissionsReviewed": true,
"storeMetadataReviewed": true
},
"disclosedFeatures": [
"feature.aiAssist",
"feature.newEditor",
"modules.messaging.enabled",
"modules.terminal.enabled",
"modules.workspace.enabled"
]
},
"ios": {
"checklist": {
"configurableFeaturesDisclosed": true,
"dataPracticesReviewed": true,
"noExecutableCode": true,
"permissionsReviewed": true,
"storeMetadataReviewed": true
},
"disclosedFeatures": [
"feature.aiAssist",
"feature.newEditor",
"modules.messaging.enabled",
"modules.terminal.enabled",
"modules.workspace.enabled"
]
}
},
"distribution": {
"desktop": null,
"mobile": null
Comment on lines +57 to +59

@pullfrog pullfrog Bot Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A null distribution makes this matrix unusable for any build=true dispatch — brand-matrix.cjs:232 fails with desktop and mobile delivery inputs are required when build=true, which I reproduced against this exact file. Since matrix_json is rejected for builds (release-brand-matrix.yml:77-80), this is the only committed matrix a build could use, so release-environment-preflight, render, desktop-validation and mobile-validation are all unreachable and only credential-free-validation ever runs.

Technical details 
# The committed pilot matrix cannot drive any `build=true` job

## Affected sites
- `.github/release/brand-matrices/code-561-pilot.json:57-59,156-158` — `distribution.desktop` and `distribution.mobile` are `null` for both brands.
- `.github/scripts/brand-matrix.cjs:232-237` — `if (options.build && (distribution.desktop === null || distribution.mobile === null)) fail(...)`.
- `.github/workflows/release-brand-matrix.yml:77-80` — `matrix_json is plan-only; builds require a reviewed matrix_file`, so there is no alternative input.

## Reproduction
```
node .github/scripts/brand-matrix.cjs \
  --matrix-file .github/release/brand-matrices/code-561-pilot.json \
  --build true --sign false --upload false
# TypeError: matrix.brands[0].distribution: desktop and mobile delivery inputs are required when build=true
```

## Required outcome
- Either the pilot matrix carries real `distribution` blocks (pointing at pilot R2 prefixes and a pilot EAS project) so `build=true, sign=false` can exercise `render` + `desktop-validation` + `mobile-validation`, or the PR states that the pilot is plan-only and that the `build`-gated jobs are validated some other way.

## Open questions for the human
- Was `build=true, sign=false` ever run against this matrix, or was the Electron/Xvfb evidence in the PR description produced locally rather than through this workflow?

},
"releaseManifests": {
"android": {
"brandId": "acme",
"channel": "canary",
"configRevisionId": "code-561-operational",
"expectedSnapshotSha256": "0675b1b33e81d4898f75233fdf9bda7243348e286ebfd3b06f807d82fff8818f",
"platform": "android",
"publicKeyringsSha256": "1a674a4c47d1ef57e51f7e50e8f044f32cffa0450574f12f23ddeb5cb619d445",
"publisherGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"releaseManifestFormatVersion": 1,
"revisionSha256": "e8389e2edc8273c5ec1b029c7101ae18af6e5b55218f0c8fd2dfe640cf695c5b",
"sourceGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"telemetryEndpoint": "https://acme.example.invalid/telemetry"
},
"desktop": {
"brandId": "acme",
"channel": "canary",
"configRevisionId": "code-561-operational",
"expectedSnapshotSha256": "936250a3ef922cede3a200b5dc401cc7697ee1db90dc3efd0f873358524f01e3",
"platform": "desktop",
"publicKeyringsSha256": "1a674a4c47d1ef57e51f7e50e8f044f32cffa0450574f12f23ddeb5cb619d445",
"publisherGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"releaseManifestFormatVersion": 1,
"revisionSha256": "e8389e2edc8273c5ec1b029c7101ae18af6e5b55218f0c8fd2dfe640cf695c5b",
"sourceGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"telemetryEndpoint": "https://acme.example.invalid/telemetry"
},
"ios": {
"brandId": "acme",
"channel": "canary",
"configRevisionId": "code-561-operational",
"expectedSnapshotSha256": "a689a8d95f74d9cb00b5d9850af3ecfd50edb23d2496c71805c9ffe4659d56ae",
"platform": "ios",
"publicKeyringsSha256": "1a674a4c47d1ef57e51f7e50e8f044f32cffa0450574f12f23ddeb5cb619d445",
"publisherGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"releaseManifestFormatVersion": 1,
"revisionSha256": "e8389e2edc8273c5ec1b029c7101ae18af6e5b55218f0c8fd2dfe640cf695c5b",
"sourceGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"telemetryEndpoint": "https://acme.example.invalid/telemetry"
}
}
},
{
"brandId": "zenith",
"channel": "canary",
"compliance": {
"android": {
"checklist": {
"configurableFeaturesDisclosed": true,
"dataPracticesReviewed": true,
"noExecutableCode": true,
"permissionsReviewed": true,
"storeMetadataReviewed": true
},
"disclosedFeatures": [
"feature.aiAssist",
"feature.newEditor",
"modules.messaging.enabled",
"modules.terminal.enabled",
"modules.workspace.enabled"
]
},
"desktop": {
"checklist": {
"configurableFeaturesDisclosed": true,
"dataPracticesReviewed": true,
"noExecutableCode": true,
"permissionsReviewed": true,
"storeMetadataReviewed": true
},
"disclosedFeatures": [
"feature.aiAssist",
"feature.newEditor",
"modules.messaging.enabled",
"modules.terminal.enabled",
"modules.workspace.enabled"
]
},
"ios": {
"checklist": {
"configurableFeaturesDisclosed": true,
"dataPracticesReviewed": true,
"noExecutableCode": true,
"permissionsReviewed": true,
"storeMetadataReviewed": true
},
"disclosedFeatures": [
"feature.aiAssist",
"feature.newEditor",
"modules.messaging.enabled",
"modules.terminal.enabled",
"modules.workspace.enabled"
]
}
},
"distribution": {
"desktop": null,
"mobile": null
},
"releaseManifests": {
"android": {
"brandId": "zenith",
"channel": "canary",
"configRevisionId": "code-561-operational",
"expectedSnapshotSha256": "a0ef5196645ae3b857343784f7a5ab5d6f5184b15c7cb646d8e86c93ff5384b0",
"platform": "android",
"publicKeyringsSha256": "1a674a4c47d1ef57e51f7e50e8f044f32cffa0450574f12f23ddeb5cb619d445",
"publisherGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"releaseManifestFormatVersion": 1,
"revisionSha256": "e8389e2edc8273c5ec1b029c7101ae18af6e5b55218f0c8fd2dfe640cf695c5b",
"sourceGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"telemetryEndpoint": "https://zenith.example.invalid/telemetry"
},
"desktop": {
"brandId": "zenith",
"channel": "canary",
"configRevisionId": "code-561-operational",
"expectedSnapshotSha256": "99a93cec0ca5381faa15a5def6727736f220b5d7d111e1fce04afda1d321aef2",
"platform": "desktop",
"publicKeyringsSha256": "1a674a4c47d1ef57e51f7e50e8f044f32cffa0450574f12f23ddeb5cb619d445",
"publisherGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"releaseManifestFormatVersion": 1,
"revisionSha256": "e8389e2edc8273c5ec1b029c7101ae18af6e5b55218f0c8fd2dfe640cf695c5b",
"sourceGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"telemetryEndpoint": "https://zenith.example.invalid/telemetry"
},
"ios": {
"brandId": "zenith",
"channel": "canary",
"configRevisionId": "code-561-operational",
"expectedSnapshotSha256": "e1b93b64973e0192ed2e1d8ba9a4cca27ae2bb5521ef6011392c2d86b510b95b",
"platform": "ios",
"publicKeyringsSha256": "1a674a4c47d1ef57e51f7e50e8f044f32cffa0450574f12f23ddeb5cb619d445",
"publisherGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"releaseManifestFormatVersion": 1,
"revisionSha256": "e8389e2edc8273c5ec1b029c7101ae18af6e5b55218f0c8fd2dfe640cf695c5b",
"sourceGitSha": "e4a0624abbc8ed1cac4948fa90239176a83cb96e",
"telemetryEndpoint": "https://zenith.example.invalid/telemetry"
}
}
}
]
}
140 changes: 140 additions & 0 deletions .github/scripts/brand-matrix.test.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ const RE_DIVERGENT_SOURCE = /all platforms must share sourceGitSha/;
const RE_SHARED_DESTINATION = /R2 prefixes in one bucket must not overlap/;
const RE_SHARED_CREDENTIALS = /credentialSecretPrefix: must be unique/;
const RE_SHARED_APP_STORE_APP = /ios\.ascAppId: must be unique/;
const ACTIONS_EXPRESSION = String.fromCodePoint(36);

function sha(character) {
return character.repeat(64);
Expand Down Expand Up @@ -81,6 +82,31 @@ function matrix(...brands) {
}

describe('parseBrandBuildMatrix', () => {
it('pins the CODE-561 credential-free pilot to two brands and all platforms', async () => {
const pilot = JSON.parse(
await readFile(
new URL('../release/brand-matrices/code-561-pilot.json', import.meta.url),
'utf8',
),
);
const plan = buildMatrixPlan(pilot);

@pullfrog pullfrog Bot Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildMatrixPlan(pilot) passes no options, so options.build is undefined and the build=true distribution check at brand-matrix.cjs:232 is never reached — the test goes green on a matrix that cannot actually build. Line 106 then asserts the all-null distribution as though it were the desired state. Adding expect(() => buildMatrixPlan(pilot, { build: true })).toThrow(/delivery inputs are required/) would at least make the constraint explicit and intentional.


expect(plan.targets.include.map(({ brandId, platform }) => `${brandId}/${platform}`)).toEqual([
'acme/desktop',
'acme/ios',
'acme/android',
'zenith/desktop',
'zenith/ios',
'zenith/android',
]);
expect(
new Set(pilot.brands.map((entry) => entry.releaseManifests.desktop.publisherGitSha)),
).toEqual(new Set(['e4a0624abbc8ed1cac4948fa90239176a83cb96e']));
expect(
pilot.brands.every((entry) => Object.values(entry.distribution).every((x) => x === null)),
).toBe(true);
});

it('builds the complete brand by platform plan', () => {
const input = matrix(brand('acme'), brand('zenith'));
const plan = buildMatrixPlan(input);
Expand Down Expand Up @@ -226,3 +252,117 @@ describe('parseBrandBuildMatrix', () => {
);
});
});

describe('release brand matrix workflow', () => {
it('keeps local runtime validation independent of provider and signing inputs', async () => {
const workflow = await readFile(
new URL('../workflows/release-brand-matrix.yml', import.meta.url),
'utf8',
);
const validation = workflow.slice(
workflow.indexOf(' credential-free-validation:'),
workflow.indexOf(' release-environment-preflight:'),
);

expect(validation).toContain('needs: prepare');
expect(validation).toContain(
`matrix: ${ACTIONS_EXPRESSION}{{ fromJSON(needs.prepare.outputs.targets) }}`,
);
expect(validation).toContain('xvfb-run -a pnpm -F @linkcode/desktop e2e:config-canary');
expect(validation).toContain('pnpm -F @linkcode/mobile smoke:export');
expect(validation).toContain(
`expo prebuild --clean --no-install --platform '${ACTIONS_EXPRESSION}{{ matrix.platform }}'`,
);
expect(validation).toContain("matrix.platform == 'desktop'");
expect(validation).toContain("matrix.platform != 'desktop'");
expect(validation).toContain(
`credential-free-${ACTIONS_EXPRESSION}{{ matrix.brandId }}-${ACTIONS_EXPRESSION}{{ matrix.platform }}`,
);
expect(validation).toContain('"local-static-origin"');
expect(validation).toContain('providerDeploymentId:null');
expect(validation).not.toContain('environment: release');
expect(validation).not.toContain('secrets.');

@pullfrog pullfrog Bot Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not.toContain('secrets.') misses the indexed form, which this same workflow already uses at line 548 (secrets[format('{0}_R2_ACCESS_KEY_ID', …)]) — so a future edit could wire brand credentials into the credential-free job and this guard would stay green. expect(validation).not.toMatch(/secrets[.[]/) covers both spellings.

expect(validation).not.toContain('release-environment-preflight');
});

it('fails closed unless the release environment is protected', async () => {
const workflow = await readFile(
new URL('../workflows/release-brand-matrix.yml', import.meta.url),
'utf8',
);
const preflight = workflow.slice(
workflow.indexOf(' release-environment-preflight:'),
workflow.indexOf(' render-inputs:'),
);

expect(preflight).not.toContain('environment:');
expect(preflight).toContain('protection_rules');
expect(preflight).toContain('required_reviewers');
expect(preflight).toContain('deployment_branch_policy');
expect(preflight).toContain('gh api "repos/$GITHUB_REPOSITORY/environments/release"');
expect(preflight).toContain('secrets.RELEASE_ENVIRONMENT_ADMIN_TOKEN');
expect(preflight).toContain('inputs.build');
const renderInputs = workflow.slice(
workflow.indexOf(' render-inputs:'),
workflow.indexOf(' signing-inputs:'),
);
expect(renderInputs).toContain('needs: [prepare, release-environment-preflight]');
expect(renderInputs).toContain('environment: release');
const signingInputs = workflow.slice(
workflow.indexOf(' signing-inputs:'),
workflow.indexOf(' render:'),
);
expect(signingInputs).toContain('needs: [prepare, release-environment-preflight]');
expect(signingInputs).toContain('environment: release');
expect(workflow.split(' environment: release')).toHaveLength(7);
expect(workflow.split('release_environment: release')).toHaveLength(3);
});

it('passes the release environment through reusable signing workflows', async () => {
const [desktop, mobile] = await Promise.all([
readFile(new URL('../workflows/build-desktop.yml', import.meta.url), 'utf8'),
readFile(new URL('../workflows/build-mobile.yml', import.meta.url), 'utf8'),
]);

expect(desktop).toContain('release_environment:');
expect(desktop).toContain(
`environment: ${ACTIONS_EXPRESSION}{{ inputs.release_environment || 'release' }}`,
);
expect(desktop).toContain(
`environment: ${ACTIONS_EXPRESSION}{{ inputs.sign && (inputs.release_environment || 'release') || '' }}`,
);
expect(mobile).toContain('release_environment:');
expect(
mobile.split(
`environment: ${ACTIONS_EXPRESSION}{{ inputs.release_environment || 'release' }}`,
),
).toHaveLength(4);
});

it('binds credential-free desktop recovery evidence to immutable release inputs', async () => {
const workflow = await readFile(
new URL('../workflows/release-brand-matrix.yml', import.meta.url),
'utf8',
);
const desktopValidation = workflow.slice(
workflow.indexOf(' desktop-validation:'),
workflow.indexOf(' mobile-validation:'),
);

expect(desktopValidation).toContain('inputs.build && !inputs.sign');
expect(desktopValidation).toContain('xvfb-run -a pnpm -F @linkcode/desktop e2e:config-canary');
expect(desktopValidation).toContain(
'54ce1fc855e12295a8dd1490463c9afac8e84a526f1e16340bcefe4f0fec8e39',

@pullfrog pullfrog Bot Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing in the suite ties this literal to the actual fixture. The test asserts the workflow contains the hash, and the workflow asserts the file matches it — but that second check only runs on a manual release dispatch, which per the PR description is not happening yet. So editing apps/desktop/e2e/fixtures/pilot-e2e-v1.json stays green through pnpm test and pnpm check:ci and only surfaces months later as a red release job.

createHash and readFile are already imported at the top of this file, so closing it is two lines:

const fixture = await readFile(
  new URL('../../apps/desktop/e2e/fixtures/pilot-e2e-v1.json', import.meta.url),
);
expect(createHash('sha256').update(fixture).digest('hex')).toBe(
  '54ce1fc855e12295a8dd1490463c9afac8e84a526f1e16340bcefe4f0fec8e39',
);

);
expect(desktopValidation).toContain('"normal":["1","2","3","4"]');
expect(desktopValidation).toContain('"emergency":["1","2","3"]');
expect(desktopValidation).toContain('"kind":"local-static-origin"');
expect(desktopValidation).toContain('"providerDeploymentId":null');
expect(desktopValidation).toContain('--expected-delivery-sha256');
expect(desktopValidation).toContain(
'--release-manifest release-inputs/release-manifest.desktop.json',
);
expect(desktopValidation).toContain('--out release-provenance.desktop.json');
expect(desktopValidation).not.toContain('environment: release');
});
});
9 changes: 7 additions & 2 deletions .github/workflows/build-desktop.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ on:
type: string
required: false
default: ""
release_environment:
description: GitHub environment that owns signing and publisher credentials
type: string
required: false
default: release
# CI builds on PRs — unsigned.
# pull_request:
# paths:
Expand Down Expand Up @@ -87,7 +92,7 @@ jobs:
name: Render immutable config
if: ${{ inputs.sign && inputs.rendered_artifact == '' }}
runs-on: ${{ vars.CI_RUNNER_LINUX || 'ubuntu-latest' }}
environment: release
environment: ${{ inputs.release_environment || 'release' }}
steps:
- name: Checkout
uses: actions/checkout@v7
Expand Down Expand Up @@ -132,7 +137,7 @@ jobs:
if: ${{ !cancelled() && (needs.render-config.result == 'success' || needs.render-config.result == 'skipped') }}
runs-on: ${{ matrix.os }}
# Signing is gated by the `release` environment (secrets + tag policy); '' = no environment.
environment: ${{ inputs.sign && 'release' || '' }}
environment: ${{ inputs.sign && (inputs.release_environment || 'release') || '' }}
strategy:
fail-fast: false
# Runners resolve through repo/org `vars` (ArcBox pins Blacksmith labels there); forks
Expand Down
Loading