Skip to content

Rgb24 JPEG decode/encode in LibJpegWrap and JpegCodec (for v1.5.0) - #2

Merged
rafalmaciag merged 2 commits into
masterfrom
feat/rgb24-codec
Oct 26, 2026
Merged

rafalmaciag merged 2 commits into
masterfrom
feat/rgb24-codec

Conversation

@rafalmaciag

Copy link
Copy Markdown
Contributor

Why

The simulator is dropping SixLabors.ImageSharp (Split License, owner ruling 2026-10-26) for this package. Until now JpegCodec could only decode to Gray8/I420 and encode I420/Gray8, so it had no Rgb24 path.

What

Native (LibJpegWrap.cpp; not compiled for Emscripten, so the WASM build stays BGRA-only)

  • DecodeJpegToRgb24: JCS_RGB, full resolution, libjpeg-turbo's default (fancy) chroma upsampling.
  • EncodeRgb24ToJpeg(rgb, w, h, stride, quality, subsampling, out, outSize): jpeg_set_defaults + jpeg_set_quality(q, TRUE) (baseline, integer DCT). subsampling is 0 = 4:2:0 (libjpeg's default) or 1 = 4:4:4.
  • Errors recover through setjmp/longjmp (libjpeg's documented pattern; error_exit must not return), and the warning output is silenced. Encode uses jpeg_mem_dest and copies into the caller's buffer, so it returns 0 if that buffer is too small.

Managed

  • IJpegCodec/JpegCodec: FrameHeader DecodeRgb24(jpeg, output), FrameImage DecodeRgb24(jpeg) (pooled), int EncodeRgb24(in frame, quality, output, JpegSubsampling = Ratio420), FrameImage EncodeRgb24(in frame, quality, JpegSubsampling = Ratio420) (pooled; uses the worst-case tjBufSize bound).
  • New JpegSubsampling { Ratio420, Ratio444 }.
  • The Rgb24 calls are stateless: they ignore Quality/DctMethod and are safe to call concurrently.
  • Breaking for anyone implementing IJpegCodec themselves: four new members. In this repo only the test mock implements it, and the mock is updated.
  • README: Rgb24 section, plus the native build recipe and the libjpeg-turbo8 runtime dependency. I also fixed the Usage example, which called Encode on an Rgb24 frame; that call throws.

Native binaries

  • linux-x64: rebuilt in ubuntu:22.04 against libjpeg-turbo8-dev 2.1.2, then stripped. Highest GLIBC symbol is 2.14. NEEDED: libjpeg.so.8, libstdc++.so.6, libgcc_s.so.1, libc.so.6. ldd inside the simulator runtime image (rocketwelder/simulator:10f8d00, Ubuntu 24.04) resolves libjpeg.so.8 to libjpeg-turbo8, which is already installed there.
  • linux-arm64 and win-x64: unchanged. The host has no aarch64 cross toolchain or qemu binfmt, and no Windows toolchain. On those platforms the Rgb24 methods turn EntryPointNotFoundException into a clear NotSupportedException. A follow-up rebuild of the arm64 .so (on a Jetson or with a cross toolchain) would remove that limit.

Tests (JpegCodecRgb24Tests, 14 new)

  • The reference pair in TestData/ was made with libjpeg-turbo 2.1.2's own tools: cjpeg -quality 90 of a 48x32 synthetic image, and djpeg -ppm of that JPEG. DecodeRgb24 matches the djpeg bytes exactly, for both overloads.
  • Round trip on a smooth 96x64 image: 4:2:0 q90 PSNR 31.97 dB (bound 30); 4:4:4 q92 PSNR 46.42 dB (bound 40).
  • On a red/blue 4-px checker, 4:4:4 scores a higher PSNR than 4:2:0.
  • A padded stride encodes to the same bytes as a tight one. Also covered: output buffer too small, a corrupt header (the longjmp path; the codec stays usable afterwards), non-Rgb24 input and quality out of range.
  • Gray8 and I420 decodes of the reference are unchanged (header checks, and the Gray8 output equals the I420 Y plane).
  • Locally (linux-x64): 121 passed / 7 failed. The 7 failures are the HttpFrameGrabberTests/MjpegProviderTests live-stream socket tests, and they fail the same way on master (107 passed / 7 failed). Release build: 0 warnings.

Release

The csproj <Version> is untouched because publish-nuget.yml takes the version from the tag. Tag v1.5.0 after merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_014wH5aL3TZhJNvLqa8YTku8

Native (LibJpegWrap.cpp, non-WASM): DecodeJpegToRgb24 (JCS_RGB, full resolution,
libjpeg-turbo default upsampling) and EncodeRgb24ToJpeg (jpeg_set_defaults +
jpeg_set_quality baseline, 4:2:0 default or 4:4:4, row stride honoured); errors
recover through setjmp/longjmp. linux-x64 LibJpegWrap.so rebuilt in ubuntu:22.04
(GLIBC <= 2.14 symbols, NEEDED libjpeg.so.8). linux-arm64 and win-x64 binaries are
unchanged (no cross toolchain); the managed Rgb24 calls throw NotSupportedException there.

Managed: IJpegCodec/JpegCodec gain DecodeRgb24 (buffer + pooled overloads) and
EncodeRgb24(frame, quality, [output], JpegSubsampling); new JpegSubsampling enum.

Tests: decode of a cjpeg reference equals djpeg's Rgb24 bytes exactly; round trip
PSNR (4:2:0 q90 > 30 dB, 4:4:4 q92 > 40 dB); 4:4:4 beats 4:2:0 on a chroma checker;
padded stride, buffer-too-small, corrupt input, format/quality validation; Gray8/I420
decode of the reference unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wH5aL3TZhJNvLqa8YTku8

@rafalmaciag rafalmaciag left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST CHANGES: two small, cheap fixes on the public API surface; the rest is solid.

Verified at 35689a8: Release build 0 warnings; tests 121 passed / 7 failed (the 7 are the HttpFrameGrabberTests/MjpegProviderTests live-stream socket tests, pre-existing); all 14 Rgb24 tests pass. linux-x64 .so: ldd -> libjpeg.so.8, max GLIBC_2.14, GCC 11.4 (22.04); nm -D shows DecodeJpegToRgb24/EncodeRgb24ToJpeg added, no old export removed (CreateBgraDecoder/DecoderDecodeBGRA/CloseBgraDecoder now also exported, additive). WASM/arm64/win-x64 untouched. P/Invoke signatures match the exports. Native longjmp recovery works: probed with ctypes, a bad SOF precision, truncated header and truncated scan all return cleanly from DecodeJpegToRgb24.

Blocking

  1. Pooled DecodeRgb24(ReadOnlyMemory<byte>) (JpegCodec.cs:334-339) gets dimensions via GetJpegImageInfo, whose native GetJpegInfo (LibJpegWrap.cpp:804) still uses jpeg_std_error -> exit(). Reproduced: the reference JPEG with SOF precision set to 5 makes GetJpegImageInfo print "Unsupported JPEG data precision 5" and kill the process (exit 1); the same bytes into DecodeJpegToRgb24 return 0 cleanly. So the pooled overload does not meet "corrupt JPEG -> clean failure" (the test only covers the buffer overload). Fix: give GetJpegInfo the new jump_error_mgr (also fixes the pre-existing DecodeI420/Decode pooled paths), or get the dimensions from DecodeJpegToRgb24 (it fills info before the size check). Add a pooled corrupt-header test.
  2. EncodeRgb24 (ValidateRgb24Encode, JpegCodec.cs ~399) never checks frame.Data.Length >= Stride*(Height-1) + Width*3; native then reads past the pinned buffer (EncodeRgb24, LibJpegWrap.cpp:~788). Public package -> a wrong header becomes an out-of-bounds read. Add the check (in long arithmetic) and a test.

Nits (follow-up OK)

  • LibJpegWrap.cpp: whole file converted CRLF -> LF (diff shows 1681 lines; 125 real). Restore CRLF or add a .gitattributes in a separate commit.
  • EncodeRgb24 error path: jpeg_mem_dest only updates *outbuffer in term_destination; after empty_mem_output_buffer has grown the buffer, mem is stale and free(mem) would double-free. Unreachable today (all managed-reachable errors fire in jpeg_start_compress), but writing straight into the caller buffer with the existing jpeg_memory_dest (plus a longjmp-safe empty_output_buffer) removes the malloc+memcpy and the hazard.
  • Truncated scan data decodes "successfully" (warnings silenced); consider failing when jerr.pub.num_warnings > 0, or document it.
  • Pooled decode header.Width * header.Height * 3 and Stride*Height are int math; use checked/long.
  • README: list the four new IJpegCodec members in the API block (README.md ~146-170) and state the breaking change for implementers (it is only in the PR body). Add <exception cref="NotSupportedException"> (non-linux-x64) and InvalidOperationException to the doc comments.
  • Gray8AndI420Decode_..._AreUnchanged compares the two decodes with each other, not with a master-built baseline; fine as a smoke test, not a regression pin.

…b24 checks data length

- GetJpegInfo (behind GetJpegImageInfo, used by every pooled decode) now uses the
  longjmp error handler: a malformed header (e.g. SOF precision 12) returns 0 and the
  managed side throws InvalidOperationException instead of libjpeg calling exit().
  Test: pooled DecodeRgb24 of a JPEG with SOF precision 12 throws; with the previous
  .so the same test aborts the test host ("Unsupported JPEG data precision 12").
- EncodeRgb24 throws ArgumentException before the P/Invoke when frame data is shorter
  than Stride * (Height - 1) + Width * 3 (both overloads; test).
- Gray8/I420 test compares against outputs stored from master add0195.
- Doc comments name NotSupportedException / InvalidOperationException / ArgumentException;
  README states the four new IJpegCodec members are breaking for custom implementations.
- linux-x64 LibJpegWrap.so rebuilt the same way (ubuntu:22.04); exports, NEEDED and
  GLIBC floor (2.14) unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wH5aL3TZhJNvLqa8YTku8
@rafalmaciag

Copy link
Copy Markdown
Contributor Author

Review fixes are in 4c4e8d1:

  • GetJpegInfo now uses the longjmp error handler. A JPEG with SOF precision 12 now throws InvalidOperationException; with the previous .so the same test crashes the test host.
  • EncodeRgb24 checks the frame data length before calling into native code.
  • The Gray8/I420 test now compares against outputs stored from master add0195.
  • Doc comments name the exceptions, and the README states the breaking change.
  • The linux-x64 .so was rebuilt the same way. Exports, NEEDED and the GLIBC floor are identical, and ldd in simulator:10f8d00 finds every dependency.
  • Tests: 123 passed / 7 failed. The 7 are the pre-existing live-stream socket tests.

Follow-ups are in #3.

@rafalmaciag rafalmaciag left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE: re-checked at 4c4e8d1. (1) GetJpegInfo (LibJpegWrap.cpp:804-819) now uses jump_error_mgr under the same #ifndef EMSCRIPTEN guard; my ctypes probe (reference JPEG with SOF precision 5) that exited the process before now returns 0 and the process lives; the new pooled DecodeRgb24 bad-precision test covers it. (2) ValidateRgb24Encode (JpegCodec.cs:399-409) checks Data.Length >= Stride*(Height-1)+Width*3 in long arithmetic, used by both overloads before any P/Invoke; the test exercises both. Release build 0 warnings; tests 123 passed / 7 failed (the 7 pre-existing HttpFrameGrabber/MjpegProvider live-stream socket tests). .so: exports identical to the previous head, NEEDED libjpeg.so.8, max GLIBC_2.14. New TestData blobs contain no LF/CR bytes, so no EOL-conversion risk. Remaining nits (CRLF->LF churn in LibJpegWrap.cpp, the jpeg_mem_dest stale-pointer free on the encode error path, silenced warnings on truncated scans, int size math) belong to #3.

@rafalmaciag
rafalmaciag merged commit 04f8f2d into master Oct 26, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant