[John-Cusack]

Hoist the nine multipart callback lookups out of the per-byte loop in MultipartParser._internal_write: each is now resolved once per write() with a _noop fallback, and the data_callback helper takes the resolved function directly. This removes the per-event "on_" + name concatenation, dict lookup, and cast — the same shape as the QuerystringParser optimization in #316, extended to the multipart parser.

Behavior is unchanged: the full suite passes, and a differential oracle comparing the complete callback event stream (every event name plus every data payload) between base and patched trees reports zero divergences across whole, multi-chunk, and 1-byte-chunked parses of all benchmark forms.

Proof

Wall time, interleaved best-of-9, fresh parser per iteration, CPython 3.14.2, base = main @ 51ff1a4. Workloads use the builders in tests/test_benchmarks.py (plus an 8 MiB upload written in 64 KiB chunks):

Workload	Base	Patched	Speedup (2 runs)
100-part form (LARGE_FORM)	432.2 / 426.0 µs	334.4 / 336.9 µs	1.26–1.29x
2-part form (SIMPLE_FORM)	13.9 / 13.5 µs	11.4 / 11.4 µs	1.18–1.22x
8 MiB upload, 64 KiB chunks	779.8 / 722.9 µs	675.0 / 680.0 µs	1.06–1.16x (noisy at this scale)
1 MiB CRLF boundary-char worst case	108.3 / 101.2 µs	105.3 / 99.1 µs	~1.02x (data-copy bound, unchanged)
8 MiB upload, single write()	—	—	~1.00x (data-copy bound, unchanged)

Peak memory (tracemalloc, same workloads): 3.5 KiB before and after on both the 100-part form and the 8 MiB upload — no change.

Reproduction: check out this branch and main side by side, then time MultipartParser(BOUNDARY, callbacks).write(LARGE_FORM) from tests/test_benchmarks.py in alternating best-of-9 loops (each run needs uv sync --frozen for the dev venv; python --version must print 3.14.2 to match these figures).

Notes

• Event-heavy parses (many parts/headers per byte) gain the most; bulk data movement is untouched, so single-write() throughput is flat — reported above rather than omitted.
• Like Speed up querystring callback dispatch #316, callbacks are resolved once per write(), so a set_callback issued from inside a callback takes effect on the next write() rather than mid-chunk. No test in the suite depends on mid-chunk re-registration.
• Tests: new TestMultipartParser with test_no_callbacks / test_none_callbacks (mirroring the querystring ones from Speed up querystring callback dispatch #316) plus test_events_match_across_chunkings, which pins the joined payload per data channel and the notification sequence across 1/3/7/64-byte chunkings. Gates green: 161 passed, ruff format --check, ruff check, mypy clean.

[codspeed]

Merging this PR will improve performance by 30.55%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 4 untouched benchmarks
⏩ 5 skipped benchmarks¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	test_parse_multipart_large_form	7.2 ms	5.5 ms	+30.55%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing John-Cusack:rs-upstream-perf (e7af72a) with main (51ff1a4)}

Footnotes

1. 5 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[cubic-dev-ai]

2 issues found and verified against the latest diff

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="tests/test_multipart.py">

<violation number="1" location="tests/test_multipart.py:725">
P2: This comparison drops callback ordering: moving `part_data` after `part_end` still produces the same summary. Normalize adjacent data fragments while preserving every event’s position, then compare the normalized sequences.</violation>
</file>

<file name="python_multipart/multipart.py">

<violation number="1" location="python_multipart/multipart.py:1144">
P3: This comment says there is “one dict lookup per write”, but the block performs nine `.get()` calls per write. Say “one lookup per callback per write” to keep the performance description accurate.</violation>
</file>

<sub>Reply to a comment to ask cubic a question or push back. It learns from your replies.

View guided diff | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: This comparison drops callback ordering: moving part_data after part_end still produces the same summary. Normalize adjacent data fragments while preserving every event’s position, then compare the normalized sequences.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At tests/test_multipart.py, line 725:

<comment>This comparison drops callback ordering: moving `part_data` after `part_end` still produces the same summary. Normalize adjacent data fragments while preserving every event’s position, then compare the normalized sequences.</comment>

<file context>
@@ -644,6 +644,101 @@ def test_invalid_max_size(self) -> None:
+        # Chunk boundaries legitimately split data callbacks, so compare the
+        # joined payload per channel plus the notification-event sequence.
+        def summarize(events: list[tuple[Any, ...]]) -> tuple[Any, ...]:
+            joined: dict[str, list[bytes]] = {}
+            notifications: list[str] = []
+            for event in events:
</file context>

Suggested change

	joined: dict[str, list[bytes]] = {}
	notifications: list[str] = []
	for event in events:
	if len(event) == 2:
	joined.setdefault(event[0], []).append(event[1])
	else:
	notifications.append(event[0])
	return ({name: b"".join(payloads) for name, payloads in joined.items()}, notifications)
	normalized: list[tuple[Any, ...]] = []
	for event in events:
	if (
	len(event) == 2
	and normalized
	and len(normalized[-1]) == 2
	and normalized[-1][0] == event[0]
	):
	previous = normalized[-1]
	normalized[-1] = (event[0], previous[1] + event[1])
	else:
	normalized.append(event)
	return tuple(normalized)

[cubic-dev-ai]

P3: This comment says there is “one dict lookup per write”, but the block performs nine .get() calls per write. Say “one lookup per callback per write” to keep the performance description accurate.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At python_multipart/multipart.py, line 1144:

<comment>This comment says there is “one dict lookup per write”, but the block performs nine `.get()` calls per write. Say “one lookup per callback per write” to keep the performance description accurate.</comment>

<file context>
@@ -1140,6 +1140,39 @@ def _internal_write(self, data: bytes, length: int) -> int:
         current_header_size = self._current_header_size
 
+        # Hoist callback dispatch out of the per-byte loop (same shape as the
+        # QuerystringParser optimization in #316): one dict lookup per write
+        # instead of a `"on_" + name` concatenation, dict lookup and `cast`
+        # per emitted event.
</file context>

Suggested change

	# QuerystringParser optimization in #316): one dict lookup per write
	# QuerystringParser optimization in #316): one lookup per callback per write

[cubic-dev-ai]

2 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="python_multipart/multipart.py">

<violation number="1" location="python_multipart/multipart.py:1147">
P3: This snapshot changes `BaseParser.set_callback()` semantics without documenting the operation boundary: a callback registered during `on_part_begin` no longer affects later events in the same `write()`. Document that callback updates made during a write take effect from the next `write()` (while updates between writes apply normally).\n\n(Based on your team's feedback about callback update timing.) [d68ae4b8-e1a0-4dee-a221-824cce4f1a50]</violation>
</file>

<file name="tests/test_multipart.py">

<violation number="1" location="tests/test_multipart.py:732">
P2: This comparison can miss a broken callback stream: it derives its expected result from the optimized parser and discards ordering between notifications and data callbacks. Compare against an explicit expected sequence, merging only adjacent chunks from the same data callback.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: This comparison can miss a broken callback stream: it derives its expected result from the optimized parser and discards ordering between notifications and data callbacks. Compare against an explicit expected sequence, merging only adjacent chunks from the same data callback.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At tests/test_multipart.py, line 732:

<comment>This comparison can miss a broken callback stream: it derives its expected result from the optimized parser and discards ordering between notifications and data callbacks. Compare against an explicit expected sequence, merging only adjacent chunks from the same data callback.</comment>

<file context>
@@ -644,6 +644,101 @@ def test_invalid_max_size(self) -> None:
+                    joined.setdefault(event[0], []).append(event[1])
+                else:
+                    notifications.append(event[0])
+            return ({name: b"".join(payloads) for name, payloads in joined.items()}, notifications)
+
+        whole = self.parse()
</file context>

[cubic-dev-ai]

P3: This snapshot changes BaseParser.set_callback() semantics without documenting the operation boundary: a callback registered during on_part_begin no longer affects later events in the same write(). Document that callback updates made during a write take effect from the next write() (while updates between writes apply normally).\n\n(Based on your team's feedback about callback update timing.) [d68ae4b8-e1a0-4dee-a221-824cce4f1a50]

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At python_multipart/multipart.py, line 1147:

<comment>This snapshot changes `BaseParser.set_callback()` semantics without documenting the operation boundary: a callback registered during `on_part_begin` no longer affects later events in the same `write()`. Document that callback updates made during a write take effect from the next `write()` (while updates between writes apply normally).\n\n(Based on your team's feedback about callback update timing.) [d68ae4b8-e1a0-4dee-a221-824cce4f1a50]</comment>

<file context>
@@ -1140,6 +1140,39 @@ def _internal_write(self, data: bytes, length: int) -> int:
+        # QuerystringParser optimization in #316): one dict lookup per write
+        # instead of a `"on_" + name` concatenation, dict lookup and `cast`
+        # per emitted event.
+        callbacks = cast("MultipartCallbacks", self.callbacks)
+        on_part_begin = callbacks.get("on_part_begin")
+        on_part_data = callbacks.get("on_part_data")
</file context>