Distinguish GitHub rate limiting from an invalid token - #107
Merged
Conversation
A 403 with an exhausted rate limit now raises RateLimitError instead of InvalidTokenError, so a throttled student is told to wait rather than to restart their codespace.
Students report these failures by screenshot, so the message carries their GitHub login, numeric id, and the GitHub request id.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_validate_github_token()treated 401 and 403 as the same failure:GitHub returns 403 for an exhausted rate limit as well as for a revoked token, so a student whose hourly API budget is spent is told their token is broken, sent to
https://cs50.dev/restart, and has their cached credentials discarded bylogout().That advice makes things worse.
/restartis the most API-expensive route on cs50.dev, so each attempt spends more of an already-empty budget — and because GitHub counts rejected requests too, retrying inflates usage further. The workaround only appears to work because revoking, re-authorizing, and waiting for a codespace restart takes long enough for the hourly window to roll over, which is why students report repeating it before every submission.This is not rare. Production telemetry over one 26-minute window showed 30 students blocked, 16 of them at 2x or worse against a 5,000/hour limit, peaking at 39,558 requests used.
Changes
1. A 403 with an exhausted rate limit now raises
RateLimitError.logout()The retry time comes from
x-ratelimit-reset, falling back to a message without a time when that header is missing or unparseable. Crucially this path does not calllogout()— the credential is valid, and discarding it forces an unnecessary re-authentication.2. Both messages now identify the student, since these failures are reported by screenshot:
The login comes from
CS50_GH_USER, the numeric id is parsed from GitHub's own "API rate limit exceeded for user ID …" message (the only place it appears on a failed request), and the request id fromx-github-request-idfor escalating to GitHub. The id falls back toRepositoryName, which in a CS50 codespace is the student's numeric id. Each field is omitted when unavailable, so no bareDetails:line appears, and the fallback is guarded with.isdigit()so local development does not print a project name as an id.3. Version bumped to 3.2.2.
Tests
Suite goes 73 -> 79, all passing.
test_forbidden_token_403now sets an explicitx-ratelimit-remainingheader so it still assertsInvalidTokenErrorfor a genuine authorization failure. New tests cover the exhausted case, the secondaryretry-aftercase, reset-time formatting with both fallbacks, identity capture, and_identify()returning empty rather than a partial line.Note
This reaches students only through the codespace image, so it needs a rebuild after release.