Conversation
|
@ydah broadly looks good. I have some concern about memory bloat though, especially if for any reason the extra copy of the data doesn't get garbage collected promptly. Any thoughts on how we could mitigate that or at least identify if it is a real concern? |
|
Thanks for raising this. The change I pushed removes the chunk array and avoids constructing a second full-body string, but it still buffers the entire response until validation completes. I haven’t measured peak RSS or verified how long Rack may retain the buffer after close, so I can’t say whether the remaining memory overhead is acceptable. A subprocess RSS benchmark using representative stream sizes would help quantify that cost. For long-lived or unbounded streams, we would need either incremental parsing or an explicit size limit. |
Streaming response validation runs from a
Rack::BodyProxyclose callback and currently re-enumerates the original response body. Rack closes that body before invoking the callback, so one-shot bodies validate as empty and close-sensitive bodies raiseIOErrorafter the response has already been sent.Wrap streaming enumeration to collect the chunks yielded to the client, then validate those captured chunks from the close callback. The original response body is still closed before validation.