Skip to content

Latest commit

 

History

History
141 lines (98 loc) · 11.7 KB

setup.md

File metadata and controls

141 lines (98 loc) · 11.7 KB

Setup

tamis runs as a scheduled GitHub Actions workflow. You need a GitHub App, a private repository and a model provider key.

Setup in one command

The tamis binary of a release does steps 1 to 3 and step 5, as an organisation owner:

gh auth refresh -s project,workflow
GH_TOKEN=$(gh auth token) LLM_API_KEY=sk-... tamis init -repository your-org/your-project -references "master origin/release-branch"

It opens your browser to confirm the GitHub App, then creates the private your-org/tamis-runner repository with its tamis environment, the App credentials and the provider key as secrets, and the workflow pinned to the latest release. -runner names another repository, -engine openai takes an OpenAI key, -app-name replaces the your-org-tamis App name, and -packs php-src turns on the guidance packs of the project. Install the App from the address it prints, then go on with the first run. The sections below do the same by hand.

1. Create the GitHub App

Open this address with YOUR-ORG replaced by your organisation name. The form comes pre-filled with the "Repository security advisories" permission in read and write, and no webhook. Add the organisation "Projects" permission in read and write, which the project board needs.

https://github.com/organizations/YOUR-ORG/settings/apps/new?name=YOUR-ORG-tamis&description=First-pass%20triage%20of%20new%20security%20advisories&url=https://github.com/php/tamis&public=false&webhook_active=false&repository_advisories=write

Once the App exists:

  1. Note its client ID, and its slug from its public page address.
  2. Generate a private key.
  3. Install it on the project repository only, with "Only select repositories".

tamis posts as <slug>[bot] and leaves its own comments out of the report it reads.

A personal access token of a repository security manager with the repository_advisories:write scope and write access to the project board also works. tamis then posts under that account, and you can omit bot-login.

2. Create a private repository

Run the workflow from a private repository such as your-org/tamis-runner, not from the public project. A public repository exposes its workflow logs, step inputs and run times, and tamis handles embargoed reports.

In that repository, create a tamis environment and add:

Kind Name Value
Variable TAMIS_CLIENT_ID The App client ID
Secret TAMIS_PRIVATE_KEY The App private key
Secret ANTHROPIC_API_KEY or OPENAI_API_KEY The model provider key, or a Codex access token

Restrict the environment to the default branch so other workflows cannot read its secrets.

Use a dedicated provider project or account with a hard spending or credit limit, because alerts alone do not stop charges. Approve the configured models and their retention and residency terms. Resource budgets lists the limits tamis enforces itself.

3. Add the workflow

Copy the workflow from the README, then:

  • replace your-org, your-project, your-app-slug and the commit placeholder,
  • pin the full SHA of a reviewed commit, since the action refuses a branch or tag, and review later updates in the private repository,
  • set since to today's date, so tamis only processes advisories in the triage state created after it,
  • set project to the project board,
  • set references to every branch your security policy covers, default branch first. tamis analyses the report and locates the cited code on the first one.

Keep persist-credentials: false on the checkout so no token lands in .git/config, and fetch-depth: 0 so the agent can read the history.

4. First run

Trigger the workflow by hand with the advisory input set to a test GHSA. tamis triages that advisory whatever its state and date, and puts it on the board.

Download the private tamis-reports artifact and check that the board item names embargoed advisories only as exact duplicates. Then let the schedule take over. Artifact access follows the permissions of the private repository, so grant it to the security team only.

GitHub Actions has no trigger for security advisories, so the schedule polls every 15 minutes. A run with no new advisory neither starts the agent nor calls the model.

5. Project board

tamis puts its triages on a private organisation project, one item per advisory, so the security team sorts them by priority. Give the App the organisation "Projects" permission in read and write, then create the project with the tamis binary of a release, as an organisation member allowed to create projects:

gh auth refresh -s project
GH_TOKEN=$(gh auth token) tamis create-board your-org

The command prints the project as your-org/NUMBER. It creates a private project named tamis with the fields below and a single Triage view of the advisories in triage or draft that are no duplicate, sorted by Severity then Reported. To lay out a project by hand instead, add these fields:

Field Type Options, in this order
GHSA Text
Duplicate of Text
Duplicates Number
Reported Date
Priority Single select To triage, Duplicate, Analysis failed, Analysis failed, retrying, Analysis pending
Severity Single select Critical, High, Medium, Low
State Single select triage, draft, published, closed, withdrawn
Policy exclusion Single select Yes, No
Located Single select Yes, No
Component Text
Upstream Text
Original report Text
Attempts Number

Then set project to your-org/NUMBER. When the project is public, or a field or an option is missing or renamed, tamis leaves the board out of the run, skips every advisory, and fails the run so the error shows.

The code sets Priority:

  • To triage for a report that is no duplicate, which the views order by Severity,
  • Duplicate behind the oldest high-confidence exact duplicate the triage shows, which Duplicate of names,
  • Analysis pending while an analysis runs, Analysis failed, retrying after a failed attempt with another to come, and Analysis failed once tamis gives up. The draft body then explains the status instead of holding the triage.

The model estimates Severity from the security policy and its CVSS vector, and ignores the one the reporter claims. A policy or a guidance pack can set it below the vector's rating, as the php-src pack does for crashes. Duplicates counts the items that name the advisory in Duplicate of, or with issues-repository the sub-issues its issue gets, the duplicates of its duplicates included. Component is the directory of the extension, SAPI or subsystem the model names, such as sapi/fpm for FastCGI code a report reaches through FPM, once tamis checks the repository has that directory. Without one, tamis takes the directory of the faulty statement two levels deep, and leaves it empty when the statement was not located. Upstream names the project the faulty code is bundled from, such as lexbor in php-src, when the fix belongs to that project, and stays empty for the repository's own code. A project created before Upstream existed needs that field before upgrading tamis: gh project field-create NUMBER --owner your-org --name Upstream --data-type TEXT. Original report holds the oldest report of a group of duplicates, on that report and on each of its duplicates, so a view grouped by it shows each original with its duplicates.

At the end of each run, tamis copies the state of every advisory to State, recounts Duplicates and ranks every triage again from Duplicate of, so a new priority rule reaches the board with a release. None of it calls the model. A Priority changed by hand comes back at the next run, while a change to Duplicate of lasts. The REST API cannot archive items, so filter the views on State instead, for example a table on State:triage,draft -Priority:Duplicate sorted by Severity then Reported. A board laid out before To triage existed needs that option before upgrading tamis, and the next run moves to it the Look first and When possible items that have Policy exclusion and Located set. Its Triage view still sorts on Priority, where those items now tie, so sort it by hand on Severity then Reported and filter it on State:triage,draft -Priority:Duplicate.

Deleting an item by hand triggers a new triage of its advisory on the next run.

With issues-repository set, for example to your-org/your-project-security, new items are issues of that repository instead of drafts, and each duplicate becomes a sub-issue of its Original report. Once that report leaves triage and draft, its duplicates still waiting move under the oldest of them, so a view filtered on State shows them, and that issue gets a comment naming the report it left. tamis closes an issue when its advisory is published, closed or withdrawn. Everyone with access to that repository reads the reports, and each new issue notifies its watchers. tamis refuses a public repository and the repository of the advisories, and then leaves the board out of the run.

tamis writes the issues and adds them to the project with issues-token, a second installation token that reaches the issues repository only, and refuses it when it also reaches repository. github-token updates the fields of the items, so it reads the issues repository too, but keep its "Issues" permission at read, so neither token can open an issue on the project:

- uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
  id: app
  with:
    client-id: ${{ vars.TAMIS_CLIENT_ID }}
    private-key: ${{ secrets.TAMIS_PRIVATE_KEY }}
    owner: your-org
    repositories: your-project,your-project-security
    permission-repository-advisories: write
    permission-organization-projects: write
    permission-issues: read
- uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
  id: issues
  with:
    client-id: ${{ vars.TAMIS_CLIENT_ID }}
    private-key: ${{ secrets.TAMIS_PRIVATE_KEY }}
    owner: your-org
    repositories: your-project-security
    permission-issues: write
    permission-organization-projects: write

Then pass issues-token: ${{ steps.issues.outputs.token }}. The App needs the "Issues" permission in read and write and an installation on the issues repository.

With close-duplicates: true, closing an issue as a duplicate decides its report. At the start of the next run, tamis credits the reporter as reporter on the Original report of the issue when that report is still in triage or draft, comments on the advisory to name that report, and closes it. A note on the issue then names the original and the login credited on it. Reopening the advisory afterwards reopens its issue, and tamis leaves both open. The original is always the one of Original report, whichever issue the duplicate was closed against, and an advisory without one stays as it is. The comment names the original even under embargo, and whoever can close an issue of that repository can then close an advisory.

With comment left at false, tamis posts nothing on the advisories but the comment of close-duplicates. With comment: true and issues-repository, each advisory also gets a comment linking it to the issue tamis creates for it, which notifies its reporter. The item of an advisory says it was triaged, Attempts counts its failed attempts, and deleting an item triages its advisory again on the next run.