Repository navigation
Fade Restoration: Remove Stain, and an Estimate that reads more frames - #1588
Merged
marcinz606 merged 9 commits intoNov 1, 2026
Merged
Conversation
Bounds read each channel's 0.1/99.9 percentiles over cells with no decoded zero, a confidence below 0.4 fails closed, and the subject sectors fall back to the unfiltered set. On the NORAD VueScan scans this turns 6 of 9 refused frames into estimates; the estimates still carry the highlight stain (see the report).
Faded slides often carry a per-channel highlight stain: blue 0.24-0.69 D darker than red at the brightest cells. A scale about D = 0 cannot remove an offset, so Fade Restoration left yellow whites and teal shadows. Remove Stain (fade_white_anchor, off by default) renders (D - w)/a: the fade slot becomes a 3x4 affine M(L - w), with the offset in the unused w lanes of the GPU unmix rows. Estimate measures each channel's white (the 99.9th percentile cell), reads the grays above it when the toggle is on, and stores it with the survivals on the fade roll card. With the toggle off, Estimate suggests it when the brightest white is within 0.1 D of full scale and blue is darkest by 0.2 D or more. Off is bit-identical to before. NORAD rms R-G/B-G against the archive edits, Fade off -> on: 106-03 .22/.26 -> .13/.13, 106-04 .11/.16 -> .09/.12, 106-21 .20/.10 -> .13/.08, 106-25 .25/.27 -> .14/.09. The suggestion fires on all four and on no SilverFast (2.2) or toppenhaug frame tested.
…ounds The robust Estimate (0.1 percentile bounds, 0.4 gate) moved or refused frames the ungated Estimate read well: toppenhaug 35 went from g0.94 b0.55 to g0.72 b0.41, and 1aa71353e6cc (a strongly faded slide whose midtone and shadow casts disagree) was refused. The Estimate now reads the frame first like PR marcinz606#1575 did (extreme-cell bounds, sectors excluded), with decoded zeros dropped and a 0.2 gate that catches colored content. Only when that fails does it use the robust bounds and their 0.4 gate, with and without the sectors. Over 137 frames this matches the PR wherever its first reading passes, keeps the frames the robust bounds rescued (NORAD 106-03, -10, -12, -14, -16, -6), and agrees with the A/B sheet the user reviewed. Remove Stain results move by 0.02 or less.
…n on On the transfer curve Cast Removal fits a per-channel line through the midtone and shadow refs of the neutral axis. With Remove Stain on, the white is already neutral, and the line through a shadow ref off the midtone's line tilted it: 106-03 went pink-magenta at Cast Removal 1.0 (B-G rms vs the archive correction .129 at 0, .217 at 1.0), 106-12 pink (.155 -> .210), 106-16 cyan (.115 -> .158). transfer_cast_affine, shared by the CPU and the GPU, puts the anchored white in place of the shadow ref when Remove Stain acts. At Cast Removal 1.0: 106-03 .113, 106-12 .128, 106-16 .113, CPU and GPU equal. Using the highlight ref instead scored close but turned the whites visibly green-cyan. Without Remove Stain nothing changes.
The restored white landed at full scale, so the frame got brighter and clear areas clipped. The affine offset now adds a neutral target t = min(max(w), -FADE_WHITE_HEADROOM), with the headroom at 0.1 D. A bright white lands 0.1 D under full scale. A dim scan keeps its cleanest channel's level, so the toggle does not move its exposure. The fade still acts on L - w, so the Estimate is unchanged. NORAD, CPU, Positive path, Cast Removal 0, Midtone Estimate, before -> after: 106-25 clipped 4.35% -> 0.019% luma 0.470 -> 0.374 rms .144/.089 -> .144/.089 106-03 clipped 3.29% -> 0.016% luma 0.218 -> 0.173 rms .134/.129 -> .132/.129 106-04 clipped 2.86% -> 0.015% luma 0.268 -> 0.213 rms .091/.113 -> .091/.113 106-21 clipped 1.72% -> 0.002% luma 0.290 -> 0.230 rms .142/.071 -> .142/.071 (clipped = any channel >= 0.98; luma = mean linear Rec. 709 luma; rms = R-G/B-G log chroma against the archive's corrected files.) Toggle off with a stored white stays bit-identical on all 10 files of the harness. CPU/GPU with the toggle on: mean < 1e-6, max <= 0.0013 on 106-25, -03 and -21 (106-04 keeps its known 908/907 row mismatch). A headroom-shifted Estimate grid would bias the survivals toward 1 (synthetic b 0.60 read as 0.62-0.63), so the Estimate keeps reading density above the white. Cast Removal on the transfer curve pivots on that same neutral target t (fade_white_target), not on full scale, so the white it holds neutral is the one Remove Stain placed.
Cast Removal pivots on the anchored white only when the green midtone sits STAIN_PIVOT_MIN_ARM (0.5 D) or more under it. A shorter arm made a small midtone cast a large gain on every darker tone: at a midtone of -0.4 a +0.08 D red cast gave red gain 1.36 (shadows cyan), at -0.25 the 2.0 limit. Closer midtones keep the shadow line. NORAD 106-03/-04/-21/-25 at Cast Removal 1.0 are unchanged (all pivot on the white). measure_white skips cells at full scale in every channel (a mount gap, a leak, a blown specular) and decoded zeros, and returns None with no cell left. A 3% gap at 1.0 measured the white as exactly (0, 0, 0), the "not measured" sentinel, so the toggle did nothing and the hint looped. An anchored Estimate with no white says so. A failed Estimate writes the roll's white only when it reads as a stain on clear film (stain_suggested); a black frame stored (-6, -6, -6). The Estimate hint belongs to its frame: another frame clears it, and so does turning Remove Stain on or off. Comments and docs say Remove Stain; PIPELINE maps it to White Anchor once and splits the fade bullet into restoration, Estimate and White Anchor paragraphs. A sample note leaves the STAIN_BLUE_MIN comment.
marcinz606#1587 moved the fade composition to features/process/fade.py (unmix_with_fade, TRANSFER render only) and made Estimate read the render's meter grid. Remove Stain follows it: - The white offset is built in unmix_with_fade, so it acts on a Slide as captured only; fade_white_target gates on the same render path. - fade_estimate_grid renders with Remove Stain off as well as the fade at identity; estimate_fade(grid, white_anchor) measures the white on that grid. - unmix_log_image runs negpy_core.unmix_log on the 3x3 and adds a (3, 4) matrix's offset after it. - The Remove Stain handlers drop invalidate_local_bounds, as the other fade handlers did. - The Calibration guide panel stays under its word budget: a short Remove Stain bullet, with the Fade intro and the Green/Blue bullet shortened, in English and the four localized guides.
Contributor
Author
|
@marcinz606 Thanks for doing some follow-ups and keeping it separated on the slide path. The one thing #1587 changed that might have been unintentional is that the fade now only works on the Transfer path and not on Positive. But Transfer is not meant for scanner files, and scanner files open as Positive by default. This is what #1574 was about: the Transfer path seems to have been designed for camera raws, and it does not work correctly for scanner TIFFs, either raw linear or gamma encoded. I developed and tested Fade Restoration mainly on the Positive path, with many scanner TIFFs and even JPEGs, and it works perfectly fine there. Would it be OK if I widened it to the Positive path as well? It is only the gate: fade on any slide (Transfer or Positive) instead of Transfer only, so the negative path stays untouched. I can put it in marcinz606/NegPy#1588 · Fade Restoration: Remove Stain… or in a separate small PR, whichever you prefer. I’ve also updated marcinz606/NegPy#1588 · Fade Restoration: Remove Stain… onto your #1587 changes. |
Owner
|
it was not intentional, go ahead and change it in either another commit of this or separate PR |
The fade was gated to the TRANSFER render, but a scanner file opens Positive, so the fade and its card were gone for scanner TIFFs and JPEGs. The fade and Remove Stain now apply on any slide; the negative path is unchanged. Marcin confirmed the Transfer-only gate from marcinz606#1587 was not intended.
…n suggested The Estimate row is now Estimate, the method combo, then Remove Stain. When Estimate finds a yellow stain, the Remove Stain icon turns warn_amber and the suggestion shows on its own warning line. Turning Remove Stain on runs Estimate on the open frame, so it measures this frame's own white and reads the survivals above it in one click. When it stores no white, the hint says that no stain was found.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some faded slides also turn yellow in their clear areas. Fade Restoration then scales the stain with the dyes, so the whites stay tinted and Estimate reads the stain as fade. This adds Remove Stain to the Fade Restoration controls, for slides only, and makes Estimate read frames that it refused before.
faderoll card): Estimate measures each layer's white in the clear areas and stores it with the survivals. The fade is then restored above that white.With Remove Stain off, renders are bit-identical, even with a stored white. Defaults are unchanged, so I did not bump
THUMBNAIL_RENDER_VERSIONorBOUNDS_VERSION.On four NORAD slides, the color error against the archive's corrected files (R-G/B-G rms) went down with Remove Stain on:
The suggestion fires on all the NORAD frames and on none of the SilverFast or Nikon Scan frames I tried. A CPU/GPU parity test covers it.
The guide,
PIPELINE.md, the shortcut registry and the four translation catalogs are updated. The Fade Restoration guide section is not in the localized in-app guides, and it was not before this PR either.make allpasses locally.