diff --git a/README.md b/README.md index f9b89ed..34585e9 100644 --- a/README.md +++ b/README.md @@ -110,38 +110,40 @@ previews, and chain in either order without extra dependencies. ## content-aware elastic collapse -`collapse` lives under **SHAPE & MOTION**. The default `elastic` mode evolves -seeded, spatially nonuniform displacement fields. Gradient-rich regions resist -deformation; motion can amplify it elsewhere. Fields are analyzed at no more -than 192 pixels on the longest side, smoothed in time, then resized into -full-frame maps for CPU `cv2.remap`. This is not a uniform zoom. Reflective -sampling keeps edges filled and output dimensions never change. +`collapse` lives under **SHAPE & MOTION**. The default `seam` mode behaves like +Photoshop content-aware scaling: each frame starts at its original composition, +then its content-aware shrink grows linearly from the first frame to the clip's +end. Low-energy seams are removed from a reduced analysis frame, and the carved +coordinate map is stretched back over the original canvas, so output dimensions +never change. High-gradient regions resist shrinking. Analysis is capped at 192 +pixels on the longest side and updates at most five times per second; the +displacement field is smoothed between updates. `mode=elastic` keeps the earlier +seeded, spatially nonuniform elastic warp available as an alternative. | option | range / values | default | | --- | --- | --- | -| `intensity` | 0-100%, displacement strength | 65% | -| `horizontal` | 0-100%, horizontal deformation scale | 65% | -| `vertical` | 0-100%, vertical deformation scale | 45% | +| `intensity` | 0-100%, overall scale strength (elastic warp strength in elastic mode) | 65% | +| `horizontal` | 0-100%, share of the maximum width shrink / horizontal warp | 65% | +| `vertical` | 0-100%, share of the maximum height shrink / vertical warp | 45% | | `preserve` | 0-100%, content/edge protection | 75% | -| `mode` | elastic, seam | elastic | -| `seam_strength` | 0-100%, offline seam removal amount | 60% | +| `mode` | seam, elastic | seam | +| `seam_strength` | 0-100%, maximum content-aware shrink by clip end | 60% | | `motion` | 0-100%, elastic motion influence | 50% | | `smoothness` | 0-100%, spatial field smoothness | 70% | | `speed` | 0-4 elastic cycles per second | 0.35 | | `coherence` | 0-100%, elapsed-time field smoothing | 80% | -| `seed` | 0-2147483647; 0 derives phases from the source | 0 | +| `seed` | 0-2147483647; elastic mode only, 0 derives phases from the source | 0 | -The optional `seam` mode is an offline content-aware carve, not a full-resolution -seam search on every frame. A gradient-energy dynamic program removes at most -four seams per axis from the reduced frame, at most once every 0.2 seconds of -video. Source coordinate maps are carved alongside the image, resized to the -original dimensions, and temporally smoothed before sampling the current frame. -This smooths seam decisions rather than crossfading old pixels. Seam mode uses -`seam_strength`, the two axis scales, preservation, smoothness, coherence and -intensity. Elastic motion, speed and random phases do not drive seam selection. +In seam mode, shrink is capped at 50% per axis. The target reduction is +`50% × intensity × seam_strength × horizontal/vertical control`; `preserve` +steers seams away from strong edges. The final displacement is multiplied by +elapsed clip progress, producing a linear ramp while each output frame remains +at source dimensions. `smoothness` and `coherence` steady the reduced-resolution +seam map; motion, speed, and random phases apply only to elastic mode. Zero intensity is an exact passthrough; two zero axis scales disable both modes. -Zero seam strength disables seam mode. The first frame is deformed immediately. +Zero seam strength disables seam mode. Seam mode leaves the first frame +unchanged and reaches its configured scale at the end of the clip. State resets on a new input/shape, explicit reset, rewind, frame discontinuity, explicit scene cut, or a large RGB scene change. Persistent analysis state is bounded independently of clip length; full-frame remap buffers are transient. diff --git a/ROADMAP.md b/ROADMAP.md index 7e884c5..0498bda 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -373,28 +373,22 @@ Shared rules for the stateful effects: - On a motion event, queue a bounded frame or patch snapshot with a due timestamp. Release delayed frames or short repayment bursts without blocking the render loop. Handle clip end when queued debt remains. - Options: motion threshold/sensitivity, min/max delay, debt gain, release rate, burst size, freeze chance, patch mode, queue capacity, seed. - Test that static input creates little debt, a sudden moving shape triggers delayed output, sensitivity and release controls change behavior, fixed seeds repeat, and queue memory stays bounded. -- [x] E2.46 - content-aware elastic collapse (prompt effect 05): deform the image nonuniformly while keeping the output dimensions fixed. - - Make real-time elastic remapping the default: evolve a smooth displacement/scale field, then sample with a CPU remap such as `cv2.remap` if OpenCV is justified. Do not reduce this to uniform zoom. - - Keep seam carving as an optional offline mode: use a gradient/energy map, work at reduced resolution, limit seams per update, smooth decisions over time, then resize to the required frame size. Do not carve every full-resolution frame without measuring flicker and cost. - - Options: intensity, horizontal/vertical scale, content preservation, seam strength/mode, motion influence, field smoothness/speed, temporal coherence, seed. - - Test fixed output size, stronger deformation at higher intensity, repeatability for a fixed seed, and less incidental flicker with temporal smoothing. Benchmark real-time and seam modes separately. - _Done: `collapse` defaults to seeded, smooth nonuniform elastic fields with - gradient-based content protection and motion weighting, sampled by CPU - `cv2.remap`. Offline `mode=seam` uses gradient-energy minimum-cost seams and - carved source-coordinate maps at <=192px, <=4 seams per axis, <=5 updates/s; - displacement decisions are smoothed over elapsed time. Exposes intensity, - axis scales, preservation, mode/strength, motion, smoothness/speed, coherence - and seed through the registry and menu. Reuses the stateful adapter, preserving - dimensions/audio and supporting sediment in either adjacent chain order. - Optional headless OpenCV is isolated in requirements-collapse.txt. Tests cover - deformation strength, content protection, repeatability, resets/cuts/rewinds, - tiny/short/static inputs, no mutation, passthrough, bounds, bounded state, - flicker reduction in both modes and real CLI/menu renders. All nine suites pass. - Separate 120-frame CPU benchmarks at 640x360 and 1280x720 are saved in - benchmarks/collapse.json, including frame time, update cost and peak process - RSS; README records the measured results without an end-to-end real-time claim. - Existing stateful VFR normalization, adjacent-processor and preview limits - remain explicit._ +- [x] E2.46 - content-aware scaling and elastic warp (prompt effect 05): change image composition while preserving output dimensions. + - Default to progressive content-aware scale: linearly ramp seam-carved shrink from identity at clip start to the configured target at clip end, then remap each frame to its original canvas. + - Analyze at reduced resolution, preserve high-energy regions, cap shrink to half per axis, and smooth cached decisions. Keep the earlier seeded elastic warp as an alternate mode. + - Options: intensity, horizontal/vertical shrink, content preservation, seam strength/mode, elastic motion influence, field smoothness/speed, temporal coherence, seed. + - Test linear progress against clip duration, fixed output size, content preservation, repeatability, bounded state and real CLI/menu renders. + _Done: `collapse` defaults to a content-aware seam scale that begins at the + source composition and grows linearly to its target over probed clip duration. + The low-resolution energy map removes seams from low-detail regions, then the + source-coordinate map is stretched back to the original frame dimensions. + Analysis is capped at 192px; shrink is limited to 50% per axis and refreshed + at <=5Hz with temporal smoothing. The seeded elastic warp remains selectable. + Duration flows through the shared stateful adapter. Tests cover exact start, + linear midpoint field, end-frame deformation, output shape/audio, resets, + preservation, optional dependencies and TUI/CLI integration. Optional headless + OpenCV remains isolated in requirements-collapse.txt. Existing stateful VFR + normalization, adjacent-processor and preview limits remain explicit._ - [ ] E2.47 - pixel archaeology (prompt effect 09): transplant patches from earlier frames using content matching. - Keep a bounded reservoir of old frames or patches. Cache inexpensive descriptors such as color mean/variance, gradient histograms, or downsampled pixels; sample candidates instead of searching every patch in every frame. - Match current regions to historical patches by color, texture, or edge compatibility. Include a controlled mismatch mode. Support hard copy and soft/feathered boundaries; every replacement must come from actual video history, not random rectangles. diff --git a/effex.py b/effex.py index e3183d3..1a8b997 100755 --- a/effex.py +++ b/effex.py @@ -1925,12 +1925,13 @@ def build_temporal_sediment(ctx: EffectContext) -> Plan: "percent", 45, minimum=0, maximum=100, step=5), Param("preserve", "content preservation", "protect strong edges from collapse", "percent", 75, minimum=0, maximum=100, step=5), - Param("mode", "mode", "fast elastic field or bounded offline seam carving", - "choice", "elastic", choices=( - ("elastic", "elastic", "smooth content-weighted CPU remapping"), - ("seam", "seam (offline)", "reduced-resolution energy-based seams"), + Param("mode", "mode", "progressive content-aware scaling or elastic warping", + "choice", "seam", choices=( + ("seam", "content-aware scale", + "linearly shrink by seams, then resize to the original frame"), + ("elastic", "elastic warp", "smooth content-weighted CPU remapping"), )), - Param("seam_strength", "seam strength", "seams removed per offline update", + Param("seam_strength", "shrink amount", "maximum content-aware shrink by the end of the clip", "percent", 60, minimum=0, maximum=100, step=5), Param("motion", "motion influence", "amplify elastic deformation in moving regions", "percent", 50, minimum=0, maximum=100, step=5), @@ -1947,14 +1948,14 @@ def build_temporal_sediment(ctx: EffectContext) -> Plan: def build_collapse(ctx: EffectContext) -> Plan: width, height = _source_dims(ctx.clip, ctx.frame) - mode = str(ctx.opts.get("mode", "elastic")) + mode = str(ctx.opts.get("mode", "seam")) return Plan( vf=f"stateful elastic collapse ({mode}) frame pass", out_suffix=ctx.suffix or "_COLLAPSED", width=width, height=height, notes=[ - "nonuniform content-aware displacement with fixed output dimensions", - "elastic mode uses seeded smooth fields and CPU OpenCV remapping", - "offline seam mode analyzes at most 192px, up to 4 seams per axis at 5Hz", + "content-aware mode linearly shrinks over the clip and remaps to original dimensions", + "seam analysis is capped at 192px; target scales remove at most half of either axis", + "elastic mode remains available for seeded smooth fields and CPU OpenCV remapping", "needs requirements-collapse.txt; no stateful preview or original VFR PTS", ], ) @@ -2111,8 +2112,8 @@ def build_inversion(ctx: EffectContext) -> Plan: "pass 2: ffmpeg -f concat -i list.txt -c copy .mp4", )), Effect(key="collapse", name="elastic collapse", - blurb="content-aware elastic deformation or offline seam collapse", - tag="stateful CPU displacement and remap", + blurb="progressive content-aware scaling or elastic warp", + tag="stateful seam carving and CPU remap", build=build_collapse, params=COLLAPSE_PARAMS, frame_processor=ElasticCollapse, stages=( @@ -2601,6 +2602,8 @@ def run_stateful_chain(chain: Chain, ctx: EffectContext, clip: Clip, tools: Tool factory = effect.frame_processor processor = factory() opts = opts_of(ctx.opts, effect) + if factory is ElasticCollapse: + opts["_clip_duration"] = clip.duration seed = int(opts.get("seed", 0) or 0) if not seed: seed = 20261023 + (clip.size % 1000003) diff --git a/seam_carving.py b/seam_carving.py index a5c8651..5798d5c 100644 --- a/seam_carving.py +++ b/seam_carving.py @@ -10,9 +10,9 @@ numpy alone. """ import math -from typing import List +from typing import List, Optional -_MAX_SEAMS = 4 +_MAX_SEAMS = 96 def _coerce_scalar(value: float) -> float: @@ -26,8 +26,10 @@ def _coerce_scalar(value: float) -> float: return min(1.0, max(0.0, number)) -def _coerce_seams(value) -> int: - """Clamp max_seams to an integer in [0, 4].""" +def _coerce_seams(value) -> Optional[int]: + """Clamp max_seams to [0, 96], or keep None as an uncapped request.""" + if value is None: + return None try: number = int(value) except (TypeError, ValueError, OverflowError): @@ -126,17 +128,17 @@ def _remove_horizontal_seam(array, seam: List[int]): def build_seam_field(frame, horizontal, vertical, strength, preservation, - max_seams: int = 4): + max_seams: Optional[int] = 4, max_shrink: float = 1.0): """Return a normalized HxWx2 displacement field from seam carving. ``frame`` is a reduced-resolution uint8 RGB array (H, W, 3). The scalar controls are clamped to [0, 1] (non-finite becomes 0). ``max_seams`` is - clamped to [0, 4]. Up to ``strength * horizontal * max_seams`` vertical - seams (one pixel per row) are removed, producing x displacement, and up to - ``strength * vertical * max_seams`` horizontal seams (one pixel per - column) are removed, producing y displacement. Each seam is the min-cost - path through the preservation-weighted gradient energy, found by dynamic - programming. + ``max_seams`` caps removals at [0, 96]; None derives the count from the + axis length and ``max_shrink``. ``max_shrink`` is clamped to [0, 1]. With + an explicit cap, up to ``strength * horizontal * max_seams`` vertical + seams and ``strength * vertical * max_seams`` horizontal seams are removed, + subject to each axis's ``max_shrink`` limit. Each seam is the min-cost path + through preservation-weighted gradient energy, found by dynamic programming. Source coordinate maps are carved alongside the energy, then resized back to the original HxW via cv2.resize. The returned field is the carved @@ -152,6 +154,7 @@ def build_seam_field(frame, horizontal, vertical, strength, preservation, strength = _coerce_scalar(strength) preservation = _coerce_scalar(preservation) max_seams = _coerce_seams(max_seams) + max_shrink = _coerce_scalar(max_shrink) frame = np.asarray(frame) if frame.ndim != 3 or frame.shape[2] != 3: @@ -165,10 +168,16 @@ def build_seam_field(frame, horizontal, vertical, strength, preservation, or (horizontal == 0.0 and vertical == 0.0)): return zero - n_vertical = min(int(round(strength * horizontal * max_seams)), - width - 1) - n_horizontal = min(int(round(strength * vertical * max_seams)), - height - 1) + vertical_limit = min(width - 1, int(round((width - 1) * max_shrink))) + horizontal_limit = min(height - 1, int(round((height - 1) * max_shrink))) + if max_seams is None: + n_vertical = int(round(strength * horizontal * vertical_limit)) + n_horizontal = int(round(strength * vertical * horizontal_limit)) + else: + n_vertical = int(round(strength * horizontal * max_seams)) + n_horizontal = int(round(strength * vertical * max_seams)) + n_vertical = min(n_vertical, vertical_limit) + n_horizontal = min(n_horizontal, horizontal_limit) if n_vertical == 0 and n_horizontal == 0: return zero diff --git a/stateful_effects.py b/stateful_effects.py index b702812..2323ea0 100644 --- a/stateful_effects.py +++ b/stateful_effects.py @@ -283,17 +283,18 @@ def process(self, frame, frame_index: int, timestamp: Optional[float], class ElasticCollapse: - """Content-weighted CPU displacement with bounded low-resolution state. + """Progressive content-aware scaling or elastic warp with bounded state. - OpenCV is loaded only on initialization. Full-size maps are transient; - evolving fields and scene/motion analysis never exceed MAX_WORK_SIDE. + Seam mode linearly scales a reduced-resolution carved field over clip time; + elastic mode evolves a seeded nonuniform field. OpenCV is loaded only on + initialization; full-size maps are transient and analysis stays bounded. """ MAX_WORK_SIDE = 192 MAX_STATE_BYTES = 4 * 1024 * 1024 DEFAULTS = { "intensity": 65, "horizontal": 65, "vertical": 45, "preserve": 75, - "mode": "elastic", "seam_strength": 60, "motion": 50, + "mode": "seam", "seam_strength": 60, "motion": 50, "smoothness": 70, "speed": 0.35, "coherence": 80, "seed": 0, } @@ -304,6 +305,8 @@ def __init__(self): self._shape = (0, 0, 0) self._work_shape = (0, 0) self._fps = 30.0 + self._clip_duration: Optional[float] = None + self._progress = 1.0 self._seed = 0 self._x: Any = None self._y: Any = None @@ -352,6 +355,8 @@ def initialize(self, config: dict, frame_shape: tuple, fps: float, number = self._number(config.get(key, default), default, 0, high) self._config[key] = number if key == "speed" else int(number) self._fps = self._number(fps, 30.0, 0.001, 1000) + duration = self._number(config.get("_clip_duration"), None, 0, 1e12) + self._clip_duration = duration if duration and duration > 0 else None self._seed = int(self._number(seed, 0, 0, 2147483647)) height, width = self._shape[:2] scale = min(1.0, self.MAX_WORK_SIDE / max(height, width)) @@ -378,7 +383,9 @@ def displacement(self) -> Any: """A copy of the applied normalized (dx, dy) field for diagnostics.""" if self._field is None: return None - return self._field * (self._config["intensity"] / 100.0) + amount = (self._progress if self._config["mode"] == "seam" + else self._config["intensity"] / 100.0) + return self._field * amount def serialize_config(self) -> dict: return dict(self._config) @@ -392,6 +399,7 @@ def reset(self) -> None: self._previous_timestamp = None self._next_index = None self._clock = 0.0 + self._progress = 1.0 self._seam_target = None self._next_seam_time = 0.0 self.seam_updates = 0 @@ -399,13 +407,15 @@ def reset(self) -> None: def _seam_field(self, current): from seam_carving import build_seam_field - # Never perform catch-up carving after a gap: one bounded update only. + # Update the target at most 5Hz; frame rendering scales it by clip progress. if self._seam_target is None or self._clock + 1e-9 >= self._next_seam_time: target = build_seam_field( current, self._config["horizontal"] / 100.0, self._config["vertical"] / 100.0, - self._config["seam_strength"] / 100.0, - self._config["preserve"] / 100.0, max_seams=4, + (self._config["intensity"] * self._config["seam_strength"] + / 10000.0), + self._config["preserve"] / 100.0, + max_seams=None, max_shrink=0.5, ) self._seam_target = self._cv.GaussianBlur( target, (0, 0), 0.5 + 3 * self._config["smoothness"] / 100.0) @@ -461,6 +471,13 @@ def process(self, frame, frame_index: int, timestamp: Optional[float], if ((self._next_index is not None and index != self._next_index) or rewind or cut or (context or {}).get("scene_cut")): self.reset() + if self._config["mode"] == "seam": + self._progress = ( + min(1.0, max(0.0, stamp / self._clip_duration)) + if stamp is not None and self._clip_duration else 1.0 + ) + else: + self._progress = 1.0 dt = (max(0.0, stamp - self._previous_timestamp) if stamp is not None and self._previous_timestamp is not None else 1.0 / self._fps) @@ -478,11 +495,16 @@ def process(self, frame, frame_index: int, timestamp: Optional[float], self._previous_timestamp = stamp self._next_index = index + 1 intensity = self._config["intensity"] / 100.0 - if intensity == 0 or not (self._config["horizontal"] or self._config["vertical"]): + if (intensity == 0 + or (self._config["mode"] == "seam" + and self._config["seam_strength"] == 0) + or not (self._config["horizontal"] or self._config["vertical"])): return frame.copy() full = cv.resize(self._field, (width, height), interpolation=cv.INTER_LINEAR) - map_x = full[:, :, 0] * (intensity * max(0, width - 1)) - map_y = full[:, :, 1] * (intensity * max(0, height - 1)) + amount = (self._progress if self._config["mode"] == "seam" + else intensity) + map_x = full[:, :, 0] * (amount * max(0, width - 1)) + map_y = full[:, :, 1] * (amount * max(0, height - 1)) map_x += np.arange(width, dtype=np.float32)[None, :] map_y += np.arange(height, dtype=np.float32)[:, None] return cv.remap(frame, map_x, map_y, interpolation=cv.INTER_LINEAR, diff --git a/tests/collapse_regression.py b/tests/collapse_regression.py index 4f5bc7c..fa2415a 100644 --- a/tests/collapse_regression.py +++ b/tests/collapse_regression.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 """Elastic collapse processor and shared-renderer integration tests.""" import json +import subprocess import sys import tempfile import unittest @@ -36,7 +37,7 @@ def frame(shape=(64, 96, 3)): def test_elastic_deformation_preserves_rgb_shape_without_mutation(self): frame = self.frame() original = frame.copy() - processor = self.processor(intensity=100, preserve=0) + processor = self.processor(mode="elastic", intensity=100, preserve=0) out = processor.process(frame, 0, 0.0, {}) self.assertEqual(out.shape, frame.shape) self.assertEqual(out.dtype, np.uint8) @@ -49,8 +50,8 @@ def test_elastic_deformation_preserves_rgb_shape_without_mutation(self): def test_strength_scales_displacement_and_pixel_change(self): frame = self.frame() - low = self.processor(intensity=15, preserve=0) - high = self.processor(intensity=100, preserve=0) + low = self.processor(mode="elastic", intensity=15, preserve=0) + high = self.processor(mode="elastic", intensity=100, preserve=0) a = low.process(frame, 0, 0.0, {}) b = high.process(frame, 0, 0.0, {}) self.assertGreater(float(np.linalg.norm(high.displacement)), @@ -68,7 +69,8 @@ def test_off_and_disabled_axes_are_exact_passthrough(self): def test_seed_repeatability_and_reset(self): frame = self.frame() - a, b = self.processor(), self.processor() + a = self.processor(mode="elastic") + b = self.processor(mode="elastic") first = a.process(frame, 0, 0.0, {}) b.process(frame, 0, 0.0, {}) for i in range(8): @@ -76,14 +78,15 @@ def test_seed_repeatability_and_reset(self): b.process(frame, i + 1, (i + 1) / 30, {})) a.reset() np.testing.assert_array_equal(a.process(frame, 0, 0.0, {}), first) - b.initialize({}, frame.shape, 30, 99) + b.initialize({"mode": "elastic"}, frame.shape, 30, 99) self.assertFalse(np.array_equal(b.process(frame, 0, 0.0, {}), first)) def test_temporal_coherence_reduces_static_flicker(self): frame = self.frame() measurements = [] for coherence in (0, 95): - p = self.processor(coherence=coherence, speed=4, preserve=0, motion=0) + p = self.processor(mode="elastic", coherence=coherence, speed=4, + preserve=0, motion=0) outputs = [p.process(frame, i, i / 30, {}).astype(np.float32) for i in range(30)] measurements.append(float(np.abs(np.diff(outputs, axis=0)).mean())) @@ -91,8 +94,8 @@ def test_temporal_coherence_reduces_static_flicker(self): def test_content_preservation_reduces_edge_displacement(self): frame = self.frame() - a = self.processor(preserve=0) - b = self.processor(preserve=100) + a = self.processor(mode="elastic", preserve=0) + b = self.processor(mode="elastic", preserve=100) a.process(frame, 0, 0.0, {}) b.process(frame, 0, 0.0, {}) self.assertLess(float(np.linalg.norm(b.displacement)), @@ -100,19 +103,21 @@ def test_content_preservation_reduces_edge_displacement(self): def test_axes_motion_speed_and_smoothness_controls(self): frame = self.frame() - p = self.processor(horizontal=100, vertical=0) + p = self.processor(mode="elastic", horizontal=100, vertical=0) p.process(frame, 0, 0.0, {}) self.assertEqual(float(np.abs(p.displacement[:, :, 1]).max()), 0) fields = [] for motion in (0, 100): - p = self.processor(motion=motion, preserve=0, coherence=0, speed=0) + p = self.processor(mode="elastic", motion=motion, preserve=0, + coherence=0, speed=0) p.process(frame, 0, 0.0, {}) p.process(np.roll(frame, 5, axis=1), 1, 1 / 30, {}) fields.append(float(np.linalg.norm(p.displacement))) self.assertGreater(fields[1], fields[0]) gradients = [] for smoothness in (0, 100): - p = self.processor(smoothness=smoothness, preserve=0, speed=0, motion=0) + p = self.processor(mode="elastic", smoothness=smoothness, + preserve=0, speed=0, motion=0) first = p.process(frame, 0, 0.0, {}) np.testing.assert_array_equal(p.process(frame, 1, 1.0, {}), first) gradients.append(float(np.abs(np.diff(p.displacement, axis=0)).mean())) @@ -135,7 +140,8 @@ def test_scene_cut_rewind_and_discontinuity_reset(self): def test_timestamp_cadence_not_frame_count_drives_field(self): frame = self.frame() - a, b = self.processor(coherence=0, motion=0), self.processor(coherence=0, motion=0) + a = self.processor(mode="elastic", coherence=0, motion=0) + b = self.processor(mode="elastic", coherence=0, motion=0) a.process(frame, 0, 0.0, {}) b.process(frame, 0, 0.0, {}) a.process(frame, 1, 1.0, {}) @@ -165,11 +171,12 @@ def test_collapse_is_registered_with_all_controls(self): effect = effex.EFFECT_BY_KEY["collapse"] self.assertEqual({p.key for p in effect.params}, set(stateful_effects.ElasticCollapse.DEFAULTS)) self.assertIs(effect.frame_processor, stateful_effects.ElasticCollapse) + self.assertEqual(effex.default_opts(effect)["mode"], "seam") def test_seam_mode_is_distinct_and_zero_strength_disables_it(self): frame = self.frame() seam = self.processor(mode="seam", seam_strength=100) - elastic = self.processor() + elastic = self.processor(mode="elastic") out = seam.process(frame, 0, 0.0, {}) self.assertFalse(np.array_equal(out, elastic.process(frame, 0, 0.0, {}))) self.assertEqual(out.shape, frame.shape) @@ -178,6 +185,26 @@ def test_seam_mode_is_distinct_and_zero_strength_disables_it(self): off = self.processor(mode="seam", seam_strength=0) np.testing.assert_array_equal(off.process(frame, 0, 0.0, {}), frame) + def test_content_aware_scale_grows_linearly_over_clip_duration(self): + frame = self.frame() + processor = self.processor( + mode="seam", intensity=100, horizontal=100, vertical=0, + preserve=0, seam_strength=100, coherence=0, _clip_duration=2, + ) + first = processor.process(frame, 0, 0.0, {}) + first_field = processor.displacement.copy() + middle = processor.process(frame, 30, 1.0, {}) + middle_field = processor.displacement.copy() + final = processor.process(frame, 60, 2.0, {}) + final_field = processor.displacement.copy() + + np.testing.assert_array_equal(first, frame) + np.testing.assert_allclose(first_field, 0) + np.testing.assert_allclose(middle_field, final_field * 0.5, atol=1e-6) + self.assertEqual(middle.shape, frame.shape) + self.assertEqual(final.shape, frame.shape) + self.assertGreater(float(np.abs(final.astype(float) - frame).mean()), 4) + def test_seam_updates_are_rate_limited_and_deterministic(self): frame = self.frame() a, b = self.processor(mode="seam"), self.processor(mode="seam") @@ -205,6 +232,41 @@ def test_seam_decisions_are_temporally_smoothed(self): self.assertGreater(changes[0], 0) self.assertLess(changes[1], changes[0] * 0.5, changes) + def test_cli_seam_scale_starts_at_source_and_grows_toward_clip_end(self): + with tempfile.TemporaryDirectory(prefix="effex-collapse-progress-") as directory: + root = Path(directory) + out_dir = root / "out" + out_dir.mkdir() + clip = root / "clip.mp4" + self.assertTrue(make_clip(clip, seconds=1, size="160x90", audio=False)) + env = env_for(root, out_dir) + proc = run([ + EFFEX, "collapse", clip, + "--set", "mode=seam", "--set", "intensity=100", + "--set", "horizontal=100", "--set", "vertical=100", + "--set", "preserve=0", "--set", "seam_strength=100", + "--json", "--preset", "ultrafast", + ], env_extra=env) + self.assertEqual(proc.returncode, 0, proc.stderr.decode()) + output = Path(json.loads(proc.stdout)["output"]) + + def frames(path): + decoded = subprocess.run( + ["ffmpeg", "-v", "error", "-i", str(path), "-vsync", "0", + "-f", "rawvideo", "-pix_fmt", "rgb24", "pipe:1"], + capture_output=True, check=False) + self.assertEqual(decoded.returncode, 0, decoded.stderr.decode()) + return np.frombuffer(decoded.stdout, np.uint8).reshape((-1, 90, 160, 3)) + + source_frames = frames(clip) + output_frames = frames(output) + differences = np.abs(output_frames.astype(np.int16) + - source_frames.astype(np.int16)).mean(axis=(1, 2, 3)) + self.assertEqual(len(output_frames), len(source_frames)) + self.assertLess(float(differences[0]), 8.0, differences.tolist()) + self.assertGreater(float(differences[-1]), + float(differences[0]) + 3.0, differences.tolist()) + def test_cli_bounds_and_real_renders(self): with tempfile.TemporaryDirectory(prefix="effex-collapse-") as directory: root = Path(directory) diff --git a/tests/menu_smoke.py b/tests/menu_smoke.py index 7e7ce3b..234d01e 100644 --- a/tests/menu_smoke.py +++ b/tests/menu_smoke.py @@ -16,6 +16,7 @@ make_clip, require_tools, temp_folders, video_dims) from pty_driver import PtyDriver # noqa: E402 +UP = b"\x1bOA" DOWN = b"\x1bOB" RIGHT = b"\x1bOC" ENTER = b"\r" @@ -827,7 +828,7 @@ def scenario_psychedelic(c: Checker): def scenario_collapse(c: Checker): - """elastic and offline seam modes share controls and the normal renderer""" + """progressive content-aware scaling and elastic warp share the renderer""" import importlib.util from harness import newest_mp4 @@ -849,15 +850,23 @@ def scenario_collapse(c: Checker): c.check(f"collapse {mode}: controls present", all( label in last_frame(d) for label in ("horizontal deformation amount", "vertical deformation amount", - "protect strong edges", "seam strength", + "protect strong edges", "shrink amount", "smooth field changes", "random seed"))) + if mode == "elastic": + for _ in range(4): + press_and_wait(d, DOWN) + press_and_wait(d, RIGHT) + for _ in range(4): + press_and_wait(d, UP) + c.check("collapse elastic: legacy warp mode can be selected", + "elastic warp" in last_frame(d)) press_and_wait(d, RIGHT) c.check(f"collapse {mode}: intensity is editable", "70%" in last_frame(d)) if mode == "seam": for _ in range(4): press_and_wait(d, DOWN) - press_and_wait(d, RIGHT) - c.check("collapse seam: offline mode can be selected", "seam (offline)" in last_frame(d)) + c.check("collapse seam: content-aware scale is the default", + "content-aware scale" in last_frame(d)) c.check(f"collapse {mode}: no unsupported preview key", "p preview" not in last_frame(d)) d.send(b"r") c.check(f"collapse {mode}: confirm opens", wait_last(d, "render this video?"))