Repository navigation
Payload problem - #13
Merged
Merged
Conversation
Groq reports a tokens-per-minute overrun as HTTP 413 with `code: "rate_limit_exceeded"`, not as 429. The OpenAI-compatible client read that as a transport error, so it skipped the backoff, ignored `Retry-After`, and the raw exception string reached the reader through `retrieval_path` as "413 Payload Too Large" — which reads as "your question was too big" and sends them off shortening a question that was never the problem. On a free-tier key this was every second question inside a minute. * Classify 413-with-a-rate-limit-marker as `RateLimitError` in both `chat` and `stream`, so it gets the backoff and the failover chain. A 413 without the marker is a genuinely oversized body and stays a hard error: waiting never shrinks it. * Give the sufficiency judge chunk heads instead of the answer's full 24k chars. It only decides whether to spend another hop, but it ran on the same model against the same quota window, so one question cost roughly twice what the answer needed. * Stop putting `str(exc)` in front of the reader; report the shape of the failure and leave the detail in the log line above it. * Document `CEREBRAS_API_KEY`, which was missing from both deployment env blocks — without it the resolution chain has no second tier to fail over to. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Indexing is the expensive half of this product — clone, chunk, summarise, embed — and the free-repository path ran it on the platform's key, so a deployment could not bound its own cost or rate limits. Hugging Face was also optional, which left `llm_hf_chat_fallback` off and the third failover tier unreachable for the readers who most needed it. * Both keys are now mandatory at connect time, each validated against its provider before anything is stored, and HF chat fallback is always on for a reader-built provider. A credential row saved before this reports as not connected rather than quietly falling back to the platform token. * `POST /repos` resolves the reader's provider first and refuses with `PROVIDER_KEY_REQUIRED` without one. The web app already opens the key dialog on that code, so the reader sees a prompt, not an error. * The arq worker no longer builds a platform-keyed provider at startup. The job carries a session id and resolves that reader's keys from the encrypted store itself, so no API key is ever written to the job queue. It is used only for account-bound keys, the one case another process can rebuild them; anonymous sessions keep indexing in-process as they already did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.