Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 28 additions & 7 deletions actions/Validate/Validate.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,14 @@ appends a Markdown summary to -SummaryPath, writes the errors and warnings outpu
{ ExitCode, Findings, Diff, Summary, Annotations, DiffRef, UpdateCheck }. ExitCode is 1 when there are errors, or
warnings with -FailOnWarning. The script never calls exit, so tests run it in-process; action.yaml exits with ExitCode.

-CheckForUpdates runs the template update check in check mode (Rulebook.Update): the template of the settings at the
head of its branch, downloaded with GITHUB_TOKEN (never the write token), or -TemplatePath and -InstalledTemplatePath
as local folders. One notice (no updates, templateSha not recorded) or warning (updates available, check skipped) and
the section '## Template update check' in the summary. Neither counts towards warnings= or -FailOnWarning, and the
check never fails the step. UpdateCheck is { Status (none, sha-only, available, skipped), Reason, Plan }. The check works
-CheckForUpdates 'true' or 'false' (case-insensitive; $true and $false work too) runs or skips the template update
check; '' (the default) follows update.check of the settings, true when absent. Omitting -CheckForUpdates follows
update.check of the settings; pass 'false' to skip the check. Any other value turns the check off and says so in the
log. The check runs in check mode (Rulebook.Update): the template of the settings at the head of its branch,
downloaded with GITHUB_TOKEN (never the write token), or -TemplatePath and -InstalledTemplatePath as local folders.
One notice (no updates, templateSha not recorded) or warning (updates available, check skipped) and the section
'## Template update check' in the summary. Neither counts towards warnings= or -FailOnWarning, and the check never
fails the step. UpdateCheck is { Status (none, sha-only, available, skipped), Reason, Plan }. The check works
in -UpdateWorkPath when given (left in place afterwards), else in a temporary folder it removes.

The effective diff compares against -DiffRef. Without -DiffRef it is origin/<GITHUB_BASE_REF> on a pull_request or
Expand All @@ -24,7 +27,7 @@ event, a ref that does not resolve, or a diff that fails is a note in the summar
param(
[string]$RepositoryRoot = '.',
[switch]$FailOnWarning,
[switch]$CheckForUpdates,
[AllowEmptyString()][string]$CheckForUpdates = '',
[AllowEmptyString()][string]$DiffRef,
[string]$SummaryPath = $env:GITHUB_STEP_SUMMARY,
[string]$JsonPath,
Expand Down Expand Up @@ -57,6 +60,24 @@ $workspace = if ([string]::IsNullOrEmpty($WorkspaceRoot)) { $root } else { (Reso
$separator = [System.IO.Path]::DirectorySeparatorChar
$relativeRoot = [System.IO.Path]::GetRelativePath($workspace, $root).Replace($separator, '/')

# The update check: an explicit -CheckForUpdates wins; '' follows update.check of the settings, read leniently (a
# missing or unreadable settings file, or a value that is not a boolean, leaves the check on; the checks report them).
$runUpdateCheck = $true
if (-not [string]::IsNullOrWhiteSpace($CheckForUpdates)) {
$runUpdateCheck = $CheckForUpdates.Trim() -ieq 'true'
if ($CheckForUpdates.Trim() -inotin 'true', 'false') { Write-Host "checkForUpdates '$CheckForUpdates' is not 'true' or 'false'; the update check is off" }
} else {
try {
$settingsForCheck = Get-Content -LiteralPath (Join-Path $root '.github' 'Rulebook-Settings.json') -Raw -ErrorAction Stop | ConvertFrom-Json -AsHashtable -ErrorAction Stop
if ($settingsForCheck -is [System.Collections.IDictionary] -and $settingsForCheck['update'] -is [System.Collections.IDictionary] -and $settingsForCheck['update']['check'] -is [bool]) {
$runUpdateCheck = $settingsForCheck['update']['check']
}
} catch {
Write-Verbose "Settings not readable for update.check: $($_.Exception.Message)"
}
if (-not $runUpdateCheck) { Write-Host 'Update check off (update.check is false)' }
}

# 1. Checks
$testParameters = @{ RepositoryRoot = $root }
if ($JsonPath) { $testParameters.Json = $JsonPath }
Expand Down Expand Up @@ -176,7 +197,7 @@ Write-Text -Path $SummaryPath -Text $summaryText

# 5. Update check (WP07): check mode only, never counted, never failing.
$updateCheck = $null
if ($CheckForUpdates) {
if ($runUpdateCheck) {
$updateCheck = [pscustomobject]@{ Status = 'skipped'; Reason = $null; Plan = $null }
$command = 'warning'
$message = $null
Expand Down
6 changes: 3 additions & 3 deletions actions/Validate/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ inputs:
required: false
default: 'false'
checkForUpdates:
description: "'true' runs the template update check in check mode: one notice or warning, never counted towards warnings or failOnWarning."
description: "'true' or 'false' runs or skips the template update check; empty follows update.check of the settings (default true)."
required: false
default: 'true'
default: ''

outputs:
errors:
Expand All @@ -41,7 +41,7 @@ runs:
$parameters = @{
RepositoryRoot = $env:INPUT_REPOSITORYROOT
FailOnWarning = $env:INPUT_FAILONWARNING -eq 'true'
CheckForUpdates = $env:INPUT_CHECKFORUPDATES -eq 'true'
CheckForUpdates = $env:INPUT_CHECKFORUPDATES
}
# The result object is the last pipeline output; anything a module emits before it is ignored.
$result = @(& (Join-Path $env:GITHUB_ACTION_PATH 'Validate.ps1') @parameters)[-1]
Expand Down
6 changes: 3 additions & 3 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,15 +242,15 @@ A rule is `id` (`^[A-Z]{2,3}[0-9]{4}i?$`), `action` (`Error`, `Warning`, `Info`,

**Validate action**

`actions/Validate/action.yaml` is a composite action. Inputs: `repositoryRoot` (default `.`), `failOnWarning` (default `'false'`), `checkForUpdates` (default `'true'`, the template update check of section 7.3 in check mode). Outputs: `errors`, `warnings`. One `pwsh` step receives the inputs and the read-only `GITHUB_TOKEN` through `env:` and runs `Validate.ps1` from `GITHUB_ACTION_PATH`, which imports the modules from `../../modules` of the same engine checkout, and exits with its `ExitCode`. Like the entry scripts of the other actions, `Validate.ps1` writes its annotations, summary and outputs through `modules/Rulebook.Action` (#58), a leaf module without engine imports.
`actions/Validate/action.yaml` is a composite action. Inputs: `repositoryRoot` (default `.`), `failOnWarning` (default `'false'`), `checkForUpdates` (default `''`: the template update check of section 7.3 in check mode runs unless `update.check` of the settings is `false`; `'true'` or `'false'` overrides the setting). Outputs: `errors`, `warnings`. One `pwsh` step receives the inputs and the read-only `GITHUB_TOKEN` through `env:` and runs `Validate.ps1` from `GITHUB_ACTION_PATH`, which imports the modules from `../../modules` of the same engine checkout, and exits with its `ExitCode`. Like the entry scripts of the other actions, `Validate.ps1` writes its annotations, summary and outputs through `modules/Rulebook.Action` (#58), a leaf module without engine imports.

`Validate.ps1` (`-RepositoryRoot`, `-FailOnWarning`, `-CheckForUpdates`, `-DiffRef`, `-SummaryPath`, `-JsonPath`, `-WorkspaceRoot`, and the test seams `-TemplatePath`, `-InstalledTemplatePath`, `-ApiUrl`, `-UpdateWorkPath`) never calls `exit` and returns `{ ExitCode, Findings, Diff, Summary, Annotations, DiffRef, UpdateCheck }`:

1. `Test-Rulebook`; one workflow command per finding, `::error file=<path>,title=<Rule>::<Id>: <Message>` (or `::warning`), the path relative to the workspace. The runner shows at most 10 error and 10 warning annotations per step; the job summary has the full list.
2. The effective diff against `-DiffRef`. The default is the pull request's base branch on a pull request (fetched when absent); on a push, the commit before the push from the event payload, else the last commit (`HEAD~1`); other events get no diff. `-DiffRef ''` disables it. A ref that does not resolve, or a failing comparison, is a note in the summary, never a failure.
3. The job summary: `## Rulebook validation` with the counts and a `| Rule | Severity | File | Id | Message |` table (or "No findings."), then `## Effective diff against <ref>` with one `| Id | Before | After | Decided by |` table per changed endpoint (or "No effective change.").
4. `errors=` and `warnings=` to `GITHUB_OUTPUT`. `ExitCode` is 1 on any error, or on any warning with `failOnWarning`.
5. With `-CheckForUpdates`: the update plan against the template of the settings at the head of its branch (downloaded with `GITHUB_TOKEN`, never the write token), one notice (no updates, templateSha not recorded) or warning (updates available, or `update check skipped: <reason>`) titled `Update check`, and the section `## Template update check` in the summary. Neither annotation counts towards `warnings=` or `failOnWarning`, and the check never fails the step. `UpdateCheck` is `{ Status (none, sha-only, available, skipped), Reason, Plan }`. Details: [reference/update-mechanics.md](reference/update-mechanics.md) section 7.
5. With the update check on (`-CheckForUpdates 'true'`, or `''` and `update.check` not `false`; when the setting turns it off the log says `Update check off (update.check is false)`): the update plan against the template of the settings at the head of its branch (downloaded with `GITHUB_TOKEN`, never the write token), one notice (no updates, templateSha not recorded) or warning (updates available, or `update check skipped: <reason>`) titled `Update check`, and the section `## Template update check` in the summary. Neither annotation counts towards `warnings=` or `failOnWarning`, and the check never fails the step. `UpdateCheck` is `{ Status (none, sha-only, available, skipped), Reason, Plan }`. Details: [reference/update-mechanics.md](reference/update-mechanics.md) section 7.

### 5.6 Template generators

Expand Down Expand Up @@ -518,7 +518,7 @@ A change set is `{ version, note?, changes[] }` with `set` (write an override en

`site` (D31, D35, D36): `enabled` builds and publishes the dashboard (default true; ignored with a warning for `dist-repo` and `gist`); `includeJustifications` publishes the organization's override and quarantine justification text (default false); `updateMode` is `skip` (keep locally changed site files on update) or `overwrite`.

`update` (WP07) and `scan` (WP08): `schedule` is a five-field cron string that the update writes into `UpdateRulebookSystemFiles.yaml` (`update.schedule`, shipped `null`: no schedule) or `ScanDiagnostics.yaml` (`scan.schedule`, shipped `"17 4 * * *"`: daily at 04:17 UTC) as its `schedule:` trigger, or `null` for none. An absent `update.schedule` removes the update schedule; an absent `scan` key or `scan.schedule` keeps the scan schedule the template ships (an organization from before WP08 has no `scan` key), and only an explicit `null` removes it. `unusedRulebookFiles` lists shipped files the update must not re-add and removes when present, by their repository-relative path with `/` (`stages/vnext.json`); the schema rejects a bare file name, a backslash and a leading slash. `templateSha` is written by every update; `templateUrl` is the template the next update pulls from.
`update` (WP07) and `scan` (WP08): `schedule` is a five-field cron string that the update writes into `UpdateRulebookSystemFiles.yaml` (`update.schedule`, shipped `null`: no schedule) or `ScanDiagnostics.yaml` (`scan.schedule`, shipped `"17 4 * * *"`: daily at 04:17 UTC) as its `schedule:` trigger, or `null` for none. An absent `update.schedule` removes the update schedule; an absent `scan` key or `scan.schedule` keeps the scan schedule the template ships (an organization from before WP08 has no `scan` key), and only an explicit `null` removes it. `update.check` (boolean, default `true`, not shipped) set to `false` turns the template update check of the Validate action off; an explicit `checkForUpdates` input still wins (#67). `unusedRulebookFiles` lists shipped files the update must not re-add and removes when present, by their repository-relative path with `/` (`stages/vnext.json`); the schema rejects a bare file name, a backslash and a leading slash. `templateSha` is written by every update; `templateUrl` is the template the next update pulls from.

The schema is closed: an unknown key at the top level or in `publish`, `quarantine`, `commitOptions`, `site`, `update` or `scan` is an error, and a work package that needs a new key adds it to the schema in its own pull request. Required are `templateUrl`, `baseUrl` (empty, or `https://` with a DNS host name and an optional numeric port, no user info, and without a query, a fragment, `.` or `..` segments, quotes, backslashes, control characters or a trailing slash), `publish`, `quarantine` with both keys, `levels` and `stages`. `publish.target` requires its own fields: `dist-repo` needs `repository` (`owner/name`) and `branch`, `azure-blob` needs `storageAccount` and `container`, `gist` needs `gistId`; fields of another target are allowed and ignored. `templateSha` is empty or a 40-character commit sha. A level or stage entry has a `name` matching `^[A-Za-z0-9-]+$` and an optional `description`; a level may have `basedOn` with the same pattern. Beyond the schema, C5 checks that slugs are unique per array after lowercasing, that every `basedOn` resolves to an existing `base/<slug>.ruleset.json` without a cycle, and that quarantine values are stage slugs from the settings.

Expand Down
4 changes: 3 additions & 1 deletion docs/reference/update-mechanics.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Line endings become LF. Keys the engine does not know survive; the schema reject

## 7. Check mode

The Validate action runs the check when `checkForUpdates` is `'true'` (the default); CheckForUpdates runs it with `update` other than `'Y'`. Validate downloads the template of the settings at the head of its branch, and both read it with `GITHUB_TOKEN` only (CheckForUpdates passes no write token in check mode); a template that token cannot read (a private template, or a pull request from a fork with a restricted token) gives "update check skipped".
The Validate action runs the check when `checkForUpdates` is `'true'`, or when it is empty (the default) and `update.check` of the settings is not `false`; CheckForUpdates runs it with `update` other than `'Y'`. Validate downloads the template of the settings at the head of its branch, and both read it with `GITHUB_TOKEN` only (CheckForUpdates passes no write token in check mode); a template that token cannot read (a private template, or a pull request from a fork with a restricted token) gives "update check skipped".

| Outcome | Annotation |
|---|---|
Expand All @@ -122,6 +122,8 @@ The Validate action runs the check when `checkForUpdates` is `'true'` (the defau

Neither annotation counts towards `warnings=` or `failOnWarning`, and the check never fails the step. The engine CI passes `checkForUpdates: 'false'` on its fixture steps.

**Turning it off (#67).** `update.check: false` in `.github/Rulebook-Settings.json` turns the check off for every Validate run; the log says `Update check off (update.check is false)` and the summary has no update section. The key lives in the settings, so it survives updates, where an edited `checkForUpdates` input in `Validate.yaml` (a system file) would be overwritten. The template does not ship the key; absent means `true`. Precedence: an explicit `checkForUpdates` input (`'true'` or `'false'`) wins over the setting, an empty input follows it. Validate reads the setting leniently: a missing or unreadable settings file, or a value that is not a boolean, leaves the check on (the checks report the settings themselves). A caller of `Validate.ps1` that omits `-CheckForUpdates` follows the setting too; pass `'false'` to skip the check. Any other non-empty input turns the check off with the log line `checkForUpdates '<value>' is not 'true' or 'false'; the update check is off`.

## 8. Update mode

1. No token: `failure=token`, error `The <secretName> secret is needed to update system files. Read https://github.com/ALCops/rulebook/blob/main/docs/ghtokenworkflow.md`, before any request.
Expand Down
3 changes: 2 additions & 1 deletion schemas/rulebook-settings.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,8 @@
"description": "The update workflow (WP07): schedule is a five-field cron expression the update writes into UpdateRulebookSystemFiles.yaml as its schedule trigger, or null for none.",
"type": "object",
"properties": {
"schedule": { "type": ["string", "null"], "pattern": "^\\S+( \\S+){4}(?![\\s\\S])" }
"schedule": { "type": ["string", "null"], "pattern": "^\\S+( \\S+){4}(?![\\s\\S])" },
"check": { "description": "false turns the template update check of the Validate action off; an explicit checkForUpdates input still wins.", "type": "boolean", "default": true }
},
"additionalProperties": false
},
Expand Down
2 changes: 1 addition & 1 deletion template/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ System files are replaced when you update from the template; settings are kept;

## Updating

The workflow **Update Rulebook System Files** (Actions > Update Rulebook System Files > Run workflow) pulls the newest version of the template into a pull request: the system files, your settings with the new `templateSha`, and `rulesets/` and `skeletons/` regenerated under your overrides and quarantine, with the effective change of every endpoint in the pull request. Your own files are never touched. It needs a secret `GHTOKENWORKFLOW` (a GitHub App or a personal access token, the same secret AL-Go uses), because the workflow token cannot change workflow files. While `checkForUpdates` is on (the default), every Validate run also tells you when an update is available; when the template cannot be read with the workflow token (a private template, or a pull request from a fork) it says "update check skipped" instead. How it works: [docs/updating.md](https://github.com/ALCops/rulebook/blob/main/docs/updating.md); the secret: [docs/ghtokenworkflow.md](https://github.com/ALCops/rulebook/blob/main/docs/ghtokenworkflow.md).
The workflow **Update Rulebook System Files** (Actions > Update Rulebook System Files > Run workflow) pulls the newest version of the template into a pull request: the system files, your settings with the new `templateSha`, and `rulesets/` and `skeletons/` regenerated under your overrides and quarantine, with the effective change of every endpoint in the pull request. Your own files are never touched. It needs a secret `GHTOKENWORKFLOW` (a GitHub App or a personal access token, the same secret AL-Go uses), because the workflow token cannot change workflow files. Every Validate run also tells you when an update is available (set `"update": { "check": false }` in `.github/Rulebook-Settings.json` to turn that off); when the template cannot be read with the workflow token (a private template, or a pull request from a fork) it says "update check skipped" instead. How it works: [docs/updating.md](https://github.com/ALCops/rulebook/blob/main/docs/updating.md); the secret: [docs/ghtokenworkflow.md](https://github.com/ALCops/rulebook/blob/main/docs/ghtokenworkflow.md).

## First steps

Expand Down
Loading
Loading