Skip to content

feat(scm): Add a git endpoint that relays through scm-platform - #128444

Open
vaind wants to merge 18 commits into
masterfrom
ivan/scm-git-endpoint-view
Open

vaind wants to merge 18 commits into
masterfrom
ivan/scm-git-endpoint-view

Conversation

@vaind

@vaind vaind commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Seer's coding jobs need to clone, fetch and push customer repositories without ever holding a GitHub or GitLab token. This adds an internal endpoint that speaks git's HTTP protocol and relays each request to the repository's provider, adding the org's integration token on the way:

/api/0/internal/scm-git/repositories/{repository_id}.git/{info/refs | git-upload-pack | git-receive-pack}

Each request carries a grant in X-Scm-Git-Grant: a short-lived token Seer signs for one org, one operation (fetch or push) and a list of repositories. The view verifies it, checks the organizations:scm-git-endpoint flag for that org, and hands the request to scm-platform's scm.git module (sentry-scm 2.0.0), which does the remaining checks and streams the relay. Anything the grant doesn't cover is refused before Sentry looks up the repository or mints a token. Those refusals are plain-text 403s, which git shows as remote: <reason>, never 401s, which would make git ask for a password. Only the credential proxy Seer's jobs go through can reach the endpoint, so a refusal means Sentry or Seer is misconfigured; each one is also reported as a Sentry issue, grouped by reason.

  • Tokens come from integration_service.get_access_token (feat(integrations): Add a provider access token RPC for cells #128490, feat(integrations): Add GitLab to the access token RPC #128762): control mints or refreshes them and sends only the token to the cell. GitHub, GitHub Enterprise, GitLab and Cursor Origin work.
  • Requests to the provider go through BlacklistAdapter, so a GitHub Enterprise or self-managed GitLab host that resolves to an internal address gets a 503. A self-managed GitLab served over plain http gets a 502, so the token never crosses the network unencrypted.
  • grant joins the SDK's request-header denylist, so the grant isn't recorded on spans or error events.
  • A push is held in worker memory up to nginx's body limit, because Endpoint.dispatch reads bodies sent with a Content-Length whole, as it already does for objectstore and debug-file uploads.

Before an org gets the flag:

  • SCM_GIT_GRANT_SECRETS (a list, so secrets can rotate) has to be set in both Sentry and Seer. Without it, every request is refused.
  • The cells' outbound addresses have to be on the published IP allow list. The cell calls the provider itself, because control's integration proxy reads whole bodies and rewrites Authorization, so customer allow lists on GitHub Enterprise, self-managed GitLab and GitHub organizations will see the cell's addresses instead of control's.
  • Each clone or push holds a web worker for the whole transfer, since the relay's 300-second timeout only limits how long the provider may stay silent, not the total time. We need to know which worker pool serves /api/0/internal/scm-git/, that nginx and load-balancer timeouts allow transfers of several minutes, and whether many coding jobs cloning large repositories at once could starve that pool's other API traffic.

Before merging, once sentry-scm ships a release with these two scm-platform changes, bump it here and:

Refs CW-2222

@linear-code

linear-code Bot commented Oct 21, 2026

Copy link
Copy Markdown

CW-2222

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 21, 2026
@vaind
vaind changed the base branch from master to ivan/integration-access-token-rpc October 22, 2026 07:13
@vaind
vaind added this pull request to stack #128492 October 22, 2026 07:39
Base automatically changed from ivan/integration-access-token-rpc to master October 22, 2026 12:45
@github-actions github-actions Bot added the Scope: Frontend Automatically applied to PRs that change frontend components label Oct 26, 2026
@sentry

sentry Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Sentry Snapshot Testing

Name Added Removed Changed Renamed Unchanged Skipped Status
sentry-frontend
sentry-frontend
0 0 0 0 766 0 ✅ Unchanged

⚙️ sentry-frontend Snapshot Settings

@vaind
vaind removed this pull request from stack #128492 October 27, 2026 18:57
@vaind
vaind changed the base branch from master to ivan/api-url-codegen-unescape October 27, 2026 19:04
@vaind
vaind added this pull request to stack #128946 October 27, 2026 20:50
Base automatically changed from ivan/api-url-codegen-unescape to master October 27, 2026 21:32
vaind added a commit that referenced this pull request Oct 27, 2026
…8751)

CI turns every Django API route into a TypeScript string in
`knownSentryApiUrls.generated.ts`, so frontend calls to `getApiUrl()`
are checked against real routes. The generator stripped regex groups and
character classes but copied escapes through, so a route that matches a
literal dot, such as `\.git` in the git endpoint added in #128444, came
out as `'/internal/scm-git/repositories/$repositoryId\.git/$route'`.
oxlint rejects that string as a useless escape, which fails lint on any
PR that adds such a route.

Escaped punctuation is now written as the character itself
(`$repositoryId.git`). No route on master has an escape, so the
generated file doesn't change here.

Refs
[CW-2222](https://linear.app/getsentry/issue/CW-2222/git-endpoint-view-in-sentry)
vaind added 15 commits October 27, 2026 23:01
Coding jobs need to clone, and Seer needs to push, without holding a provider token. The view
verifies the grant Seer signs, picks the grant's repository that the request path names, and
hands the request to scm-platform's GitServer. GitServer relays it to the provider with the
integration's token and streams both bodies.

The view is behind organizations:scm-git-endpoint and has no route yet, because the URL layout
is still being settled; its tests mount it on a stand-in route. It imports scm.git, which no
released sentry-scm has yet. GitLab and Cursor Origin are refused until a cell can get their
tokens from control.

Refs CW-2222
The settled layout is /api/0/internal/scm-git/repositories/{repository_id}.git/{route}. The
path carries the Sentry Repository.id that the grant must list, so GitServer's own grant check
refuses any other repository before the database, the token or the provider. The view no longer
matches the path against repository names, which collide between GitHub, GitHub Enterprise and
Cursor Origin. scm-git/hosts/ stays free for a later route that names repositories by host.

Refs CW-2222
Review fixes for the git endpoint:
- Add "grant" to the SDK's request header denylist. None of its terms matched X-Scm-Git-Grant,
  so the grant reached spans and error events.
- Answer a GitHub token that can't be minted (e.g. a suspended app) with GitServer's text/plain
  502 rather than a JSON 500, which git shows only as "HTTP 500".
- Limit the route's repository id to 19 ASCII digits. \d also matches other scripts' digits,
  and int() refuses an id of 4,301 digits with a 500.
- Test org scoping with a repository that has a working integration, and test unmatched URLs at
  the resolver, where they reach the API catch-all.

Refs CW-2222
SCM errors always carry a detail, so the refusal helper no longer falls back to the error code.
The tests share a helper for the info/refs request that opens every fetch, and the inactive
repository test now checks the refusal like its siblings.

Refs CW-2222
Which token a repository's provider takes is per-provider knowledge, like which adapter to
build, so the lookup moves from the git endpoint into sentry.scm.private.helpers next to
fetch_service_provider. The GitLab and Cursor Origin branches then land in the same place as
their adapters.

Refs CW-2222
scm-platform's GitServer now takes an emit_error callback and tags its counts with the referrer
Seer puts in each grant. The view logs each failure with what the request had resolved, because
git shows the client only the message. Its own refusals (no organization, flag off) carry the same
referrer tag and log line, so they read like the ones GitServer answers.

Refs CW-2222
integration_service.get_access_token mints or refreshes the token in control and sends only the
token to the cell, for GitHub, GitHub Enterprise, GitLab and Cursor Origin. The git endpoint drops
its own switch on provider, GitLab and Cursor Origin repositories now relay instead of being
refused, and the cell no longer receives the full integration metadata just to read a token. The
git server also builds its repository and provider lookups with make_fetchers, so the two share
one integration fetch per request.

Refs CW-2222
The token RPC now leaves GitLab out until GitLab's client refreshes tokens before they expire, and
answers None for it. The git server turns None into a 502, which tells git there was an outage when
Sentry simply can't relay GitLab yet, so the endpoint refuses GitLab repositories as not supported
before asking for a token, as it did before the switch to the RPC.

Refs CW-2222
sentry-scm 2.0.0 takes signing secrets as scm.Secret, which shows as [Filtered] in error reports.
Held as plain strings in a list, a failure while verifying a grant would have sent every
SCM_GIT_GRANT_SECRETS entry to Sentry in the frame's locals, and GitServer now refuses to be built
with them. The setting stays a list of strings and each entry is wrapped where it's read.

Refs CW-2222
The grant is an HS256 JWT. The comment named only the HMAC it costs, which read as if grants were bare HMAC signatures.

Refs CW-2222
A GitHub Enterprise or self-managed GitLab host that resolves to an internal address was answered
with a 503 but never logged, and GitServer counts it as an unhandled_exception. Nothing named the
organization or repository, so a customer's misconfigured host looked like a bug in the endpoint.
The view now logs it as restricted_ip_address with the blocked address and the request's context.

Refs CW-2222
When control answered the token RPC with something other than an RPC response, such as a load
balancer's error page, RpcResponseException escaped fetch_access_token and git showed a bare
"HTTP 500". It's now handled like an unreachable control silo: reported to Sentry and answered as
the 502 that says Sentry couldn't get a token.

Refs CW-2222
Only the credential proxy Seer's coding jobs go through can reach the git
endpoint, so a refused request is never a stranger knocking: a missing or
unsigned grant, a disabled flag or a grant that doesn't match the request
means Sentry or Seer is misconfigured. A warning log doesn't surface that,
and Sentry's logging integration never turns a log into an issue, so the
endpoint now captures each refusal explicitly and logs it at error level.

The view's own refusals, the ones GitServer hands back through emit_error,
and a provider host on an internal address share one helper, so all three
report the same way. Issues group by refusal code, since the detail varies
within a code, and carry the repository, organization, referrer and
whatever else the request had resolved as tags, which the message leaves
out.
The endpoint refused GitLab only because the token RPC left it out.
#128762 added GitLab to the RPC, and sentry-scm already
relays to a GitLab remote with the oauth2 user, so the refusal now only
blocks repositories that work.
@vaind
vaind force-pushed the ivan/scm-git-endpoint-view branch from cfdef2d to f214b2b Compare October 27, 2026 22:08
vaind added 2 commits October 27, 2026 23:12
Anyone can write a referrer into a grant they sign themselves, and the
referrer tags Datadog metrics and Sentry issues. Only a missing grant was
checked for leaving it out; a forged one that carries a referrer was not.
Sentry's SDK config sets data_collection, which turns off the SDK's
event scrubber while every frame's local variables are still sent. A
refusal raised after the token lookup has GitServer.handle and
GitRelay.forward on its stack, both holding the provider token, and a
host on an internal address raises from inside requests, whose frames
hold the Authorization header. capture_exception sent all of that.

Each refusal is now a message grouped and tagged as before, which
carries no stack. The internal-address report also leaves out the
address, and an exception that isn't an SCMCodedError is reported by
its name, since its message could hold a URL or a credential.
@vaind
vaind marked this pull request as ready for review October 27, 2026 22:54
@vaind
vaind requested review from a team as code owners October 27, 2026 22:54
@vaind
vaind marked this pull request as draft October 27, 2026 22:55
@vaind
vaind marked this pull request as ready for review October 27, 2026 22:58
@vaind
vaind marked this pull request as draft October 27, 2026 22:59
@vaind
vaind marked this pull request as ready for review October 27, 2026 23:05

This branch was successfully deployed

1 active (outdated) deployment
Preview — cfdef2dc Deployed Oct 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant