Skip to content

fix: stop re-sending unchanged MJPEG frames and speed up the first frame - #12

Merged
azarouski merged 2 commits into
developfrom
fix/mjpeg-frames-processing
Oct 21, 2026
Merged

azarouski merged 2 commits into
developfrom
fix/mjpeg-frames-processing

Conversation

@azarouski

Copy link
Copy Markdown

Summary

Fixes in the MJPEG screenshots broadcaster (FBMjpegServer, FBImageProcessor).

  • Unchanged frames were re-sent when scaling is enabled. The new screenshot was compared with the last sent frame, which differs from the captured data once it has been scaled, so with mjpegScalingFactor < 100 a static screen was rescaled and streamed at the full framerate. It is now compared with the last captured frame, the scaling factor and the orientation setting.
  • Scaled frames were larger than full-size ones. Scaled frames were recompressed with a quality of at least 0.9 regardless of mjpegServerScreenshotQuality. With the default quality of 10 a 50% frame came out bigger than the original. The configured quality is used now.
  • Data races. The last frame and the new client flag were accessed from several queues without synchronization.
  • First frame for a new client. The last frame is now resent right after the client subscribes, instead of waiting for the 0.5s timer (or up to 20s if the timer fired before the client was subscribed).

Measurements (simulator, static screen)

Before After
Frames in 10s at mjpegScalingFactor=50, 10 fps 95 frames, ~10 MB 2 frames, ~54 KB
Time to the first frame for a new client 120–370 ms 1–5 ms

Testing

  • FBImageProcessorTests pass.
  • Manual runs on the iPhone 16 Pro simulator: frame counts at 50% and 100% scaling, first frame delivery for a late client.

azarouski and others added 2 commits October 20, 2026 19:13
- Compare a new screenshot with the previous captured one instead of the
  previous sent one. With mjpegScalingFactor < 100 the sent frame never
  matched the captured data, so a static screen was rescaled and streamed
  at the full framerate.
- Recompress scaled frames with the configured mjpegServerScreenshotQuality.
  The forced minimum of 0.9 made scaled frames larger than full-size ones
  for low quality settings.
- Fix data races on the last frame and the new client flag, which were
  accessed from several queues without synchronization.
- Resend the last frame once the client is subscribed to the broadcast,
  so a new client on a static screen does not wait for the periodic frame.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A client joining a static screen waited for the next tick of the 0.5s
first-frame timer. Trigger the resend as soon as the client is subscribed;
the timer still retries if no frame has been captured yet.

Time to the first frame on a static screen dropped from 120-370ms to 1-5ms
on the simulator.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@azarouski
azarouski merged commit ba43d6b into develop Oct 21, 2026
39 of 44 checks passed
@azarouski
azarouski deleted the fix/mjpeg-frames-processing branch October 21, 2026 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant