Engineering Metrics CLI

Engineering Metrics CLI is a TypeScript command-line tool for analyzing GitHub organization activity and producing engineering delivery metrics over a selected time period.

Current status

The project is in an early MVP stage with repository discovery, PR metrics, and deployment frequency proxy available.

Implemented in the current slice:

• project scaffold with TypeScript, pnpm, and Vitest configuration
• commander-based CLI entrypoint
• gh process wrapper with availability and authentication checks
• repository discovery via gh repo list
• repository filtering with --min-prs
• merged pull request discovery via gh pr list
• merged PR throughput counts by repository and bucket
• PR lead time proxy from createdAt -> mergedAt
• lead time summaries with average, median, and p90 in hours
• successful workflow run discovery via gh run list
• deployment frequency proxy from successful workflow runs
• period parsing for values such as 90d, 12w, and 3m
• UTC analysis window generation
• calendar-aligned bucket generation for day, week, and month
• runtime validation of repository and pull request JSON payloads with zod
• terminal output with cli-table3
• JSON output for repository, throughput, lead time, and deployment proxy summaries
• JSON artifact persistence for every CLI run under artifacts/
• fixture-driven tests for repository parsing, pull request parsing, period parsing, and bucket generation
• CLI flow tests for repository filtering and output-focused formatter coverage

Not implemented yet:

• advanced statistical aggregation beyond the current avg / median / p90 summaries
• broader live validation across more organizations and edge cases

Metric assumptions

The MVP is intended to report the following metrics:

• PR throughput = count of merged pull requests in the selected period
• lead time proxy = PR cycle time from createdAt to mergedAt
• deployment frequency = proxy derived from successful workflow runs when --include-runs is enabled

Current status:

• merged PR throughput counts are implemented
• lead time proxy is implemented
• deployment frequency proxy is implemented

These are practical engineering metrics, not compliance-grade DORA measurements.

In particular:

• lead time is a proxy based on PR cycle time, not formal DORA lead time for changes
• deployment frequency is a proxy based on successful workflow runs, not authoritative production deployment events

Requirements

• Node.js 20+
• gh CLI installed and authenticated
• pnpm via Corepack or a direct installation

Setup

corepack enable
corepack use pnpm@10.6.5
pnpm install

Scripts

pnpm dev -- --org github --top 10
pnpm build
pnpm typecheck
pnpm test

CLI behavior in the current slice

The current implementation ranks repositories by recent activity (updatedAt), applies optional merged PR filtering, fetches metrics for the qualifying repositories, and prints either:

• a terminal table rendered with cli-table3
• or JSON output with metadata, selected repositories, and merged PR summaries

Every CLI run also writes the full report as a JSON artifact under artifacts/. The artifacts/ directory is gitignored so local org data does not get staged by default.

Current repository discovery notes:

• repositories are fetched with gh repo list
• the CLI requests up to 1000 organization repositories and sorts them locally by updatedAt
• --repo supports either repo-name or owner/repo-name
• --min-prs filters repositories by merged PR count inside the analysis window
• --top applies after --min-prs filtering in org mode
• single-repo mode fails clearly if the selected repository does not meet --min-prs
• --period is applied to a rolling UTC analysis window
• --bucket is applied to generated time buckets that are clipped to the analysis window
• weekly buckets use ISO week alignment in UTC (Monday-start)
• every run writes a JSON artifact file under artifacts/eng-metrics-*.json
• --json changes stdout to JSON, but artifact persistence happens regardless of stdout format
• merged PRs are fetched per selected repository with gh pr list
• merged PR bucket counts use mergedAt as the event timestamp
• lead time proxy uses PR cycle time from createdAt to mergedAt
• lead time bucket summaries are grouped by the PR mergedAt bucket
• successful workflow runs are fetched with gh run list --status success when --include-runs is enabled
• deployment frequency proxy uses successful workflow runs and buckets them by run updatedAt

The CLI already accepts these options:

• --org
• --period
• --bucket
• --top
• --repo
• --json
• --include-runs
• --min-prs

Current limitation:

• merged PR counts, lead time proxy, and deployment frequency proxy are connected to buckets

Example commands

pnpm dev -- --org github --top 10

pnpm dev -- --org github --repo cli --json

pnpm dev -- --org github --repo cli --period 90d --bucket week --include-runs

Example artifact path:

artifacts/eng-metrics-github-cli-2026-03-17t08-44-36-315z.json

Example JSON output

{
  "implementationStatus": "delivery-metrics-discovery",
  "generatedAt": "2026-03-17T00:00:00.000Z",
  "analysisWindow": {
    "period": "3m",
    "bucketSize": "week",
    "alignment": "iso-week-utc",
    "timezone": "UTC",
    "startAt": "2025-12-17T00:00:00.000Z",
    "endAt": "2026-03-17T00:00:00.000Z"
  },
  "buckets": [
    {
      "key": "2025-12-17T00:00:00.000Z",
      "label": "2025-12-17 00:00 -> 2025-12-22 00:00 UTC",
      "bucketSize": "week",
      "startAt": "2025-12-17T00:00:00.000Z",
      "endAt": "2025-12-22T00:00:00.000Z"
    }
  ],
  "options": {
    "org": "github",
    "period": "3m",
    "bucket": "week",
    "top": 10,
    "includeRuns": false,
    "minPrs": 0
  },
  "repositories": [
    {
      "name": "cli",
      "nameWithOwner": "github/cli",
      "description": "GitHub’s official command line tool",
      "updatedAt": "2026-03-16T18:10:00Z",
      "isPrivate": false,
      "url": "https://github.com/github/cli"
    }
  ],
  "pullRequests": {
    "organization": {
      "mergedPullRequestCount": 12,
      "repositoriesWithMergedPullRequests": 1,
      "bucketCounts": []
    },
    "repositories": [
      {
        "repositoryNameWithOwner": "github/cli",
        "mergedPullRequestCount": 12,
        "firstMergedAt": "2026-01-03T10:00:00Z",
        "lastMergedAt": "2026-03-16T18:10:00Z",
        "bucketCounts": []
      }
    ]
  },
  "leadTime": {
    "organization": {
      "repositoriesWithLeadTimeData": 1,
      "sampleCount": 12,
      "averageHours": 18.4,
      "medianHours": 12.25,
      "p90Hours": 43.5,
      "bucketStats": []
    },
    "repositories": [
      {
        "repositoryNameWithOwner": "github/cli",
        "sampleCount": 12,
        "averageHours": 18.4,
        "medianHours": 12.25,
        "p90Hours": 43.5,
        "bucketStats": []
      }
    ]
  },
  "deploymentFrequency": {
    "organization": {
      "successfulWorkflowRunCount": 8,
      "repositoriesWithSuccessfulWorkflowRuns": 1,
      "repositoriesWithUnavailableRunData": 0,
      "bucketCounts": []
    },
    "repositories": [
      {
        "repositoryNameWithOwner": "github/cli",
        "availability": "available",
        "unavailableReason": null,
        "successfulWorkflowRunCount": 8,
        "firstSuccessfulRunAt": "2026-01-10T09:00:00Z",
        "lastSuccessfulRunAt": "2026-03-16T19:00:00Z",
        "bucketCounts": []
      }
    ]
  }
}

Architecture

src/
  cli/
  gh/
  metrics/
  buckets/
  output/
  config/
  types/
  utils/

test/
  fixtures/

Current module responsibilities:

• src/cli/ defines the command entrypoint and top-level execution flow
• src/gh/ wraps gh execution and validates GitHub JSON responses
• src/metrics/ summarizes merged PR counts across repositories and buckets
• src/buckets/ parses period strings and generates clipped UTC time buckets
• src/output/ formats terminal and JSON output
• src/config/ stores defaults and CLI validation rules
• src/types/ defines shared domain and option types

Testing

Vitest is configured from the start.

Current test coverage includes:

• fixture-driven validation of repository payload parsing
• fixture-driven validation of merged pull request payload parsing
• fixture-driven validation of workflow run payload parsing
• repository selection by recent activity
• CLI-level --min-prs filtering behavior
• merged pull request counting across repositories and buckets
• lead time calculation and avg/median/p90 summaries
• deployment frequency proxy aggregation and unavailable-run handling
• period parsing
• UTC bucket generation and event-to-bucket lookup
• combined terminal formatter behavior
• artifact persistence and filename sanitization

Limitations

• The current slice computes merged PR throughput counts, PR lead time proxy, and deployment frequency proxy.
• Repository selection starts from updatedAt ordering and can be narrowed further by --min-prs using merged PR counts in the selected window.
• Deployment frequency will remain a workflow-run proxy in the MVP even after the metric phase is implemented.
• Bucket intervals are generated in UTC and use ISO week alignment for week.
• gh pr list fetching currently uses a simple per-repository query with a limit of 1000 merged PRs per repository per run.
• Lead time values are reported in hours and are bucketed by PR mergedAt, while the duration itself is calculated from createdAt -> mergedAt.
• Deployment frequency remains a proxy based on successful workflow runs and may not reflect authoritative production deployments.
• Workflow runs that cannot be fetched are marked as unavailable instead of failing the entire report when --include-runs is enabled.
• JSON artifacts are written to the local workspace under artifacts/ for every run.