Skip to content

NeoPic: select capture backends per object and isolate benchmark subclasses - #106

Draft
defisym wants to merge 23 commits into
masterfrom
codex/neopic-capture-backends
Draft

defisym wants to merge 23 commits into
masterfrom
codex/neopic-capture-backends

Conversation

@defisym

@defisym defisym commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Modifying existing stuff

NeoPic's GPU capture experiments now use per-object backend selection instead of build macros. The production Capture Frame Area action captures once; benchmark timing and iteration state live in an optional derived class.

Backend selection and synchronous capture

The Display Settings → Frame Capture → Capture Backend property follows the existing hotspot combobox pattern:

  • Window DC (default): the existing rhHEditWin capture and cloned bitmap.
  • Secondary Surface (DX11): the valid Frame secondary region.
  • Windows Graphics Capture: the root capture cropped to the same child client area.

All backends coexist in one MFX. The stored property selects a factory-created subclass. Invalid stored values normalize to DC. In a Fusion DX9 runtime, selecting Secondary or the current DX11-device-based WGC path resolves to DC during construction, with a one-time log. The ordinary and optional benchmark factories share this resolver; the saved property remains unchanged so DX11 runs retain the user's choice. DX11 capture/source failures keep the existing explicit-failure behavior.

FrameCapture owns the common capture-and-publish transaction. Concrete DC/Secondary/WGC sources supply an owned Surface; the GPU sources share normalization and query completion. A virtual destructor ensures deletion through the RUNDATA base pointer releases derived resources. WGC drains pending work before its borrowed-frame owner is destroyed.

Successful captures publish through LoadFromPointer(..., L"_TempCapture", ..., TakeOwnership) before the action returns. The DC bitmap is cloned once; GPU RTTs are transferred directly. Capture failure preserves the previous snapshot. Update only prepares a source and never schedules deferred publication.

The existing normalization behavior is preserved: source allocation padding is excluded, RGB is sampled into the logical output viewport with opaque alpha and the application's filter preference, and DX11.1 context state restores Fusion's pipeline. The full valid source rectangle is stretched to the destination; client/source/output sizes and aspect ratios need not match. The author accepted the preceding image/resize correction; application retesting after this refactor is still needed.

The resolver reads the window Surface's driver metadata, with the app Surface prototype as a startup fallback when the window Surface does not exist yet. It does not copy a current-layer canvas. WGC as an OS API can capture DX9 windows; this implementation depends on Fusion's DX11 device and does not implement DX9 image interop.

Saved-data compatibility

Main.h uses the versioned layout in EditData.h. V1 is frozen; V2 adds a four-byte CaptureBackend field and 64 reserved bytes, with a Win32 payload-budget assertion for future additions.

UpdateEditStructure reuses the same UpdateEditData helper as Fini. The callback preserves every old setting, defaults the new selector to DC, clears the reserve and updates the maximum allocation size. New objects initialize the selector and reserve explicitly. Existing ACE IDs, object identity and events are unchanged.

Optional derived benchmark

Production sources include neither FrameCaptureBenchmark.h nor the statistics runner. There are no backend/benchmark enable or iteration-count macros.

For measurement, explicitly include FrameCaptureBenchmark.h in Runtime.cpp and construct via CreateFrameCaptureBenchmark(rdPtr, selectedBackend, {100, 10}). The template derives from the selected source class and overrides the common transaction:

  • Constructor parameters supply measured iterations and warmup calls.
  • QPC reports successful-sample percentiles separately from failures; warmup and per-capture diagnostics are excluded.
  • DC uses the logical comparison size, flushes GDI before cloning and waits for DX11 HWA upload.
  • WGC retains a ready input during the batch; exception-safe session cleanup restores release behavior.
  • The last successful image remains under _TempCapture. Disk saving is outside measurement.

This remains snapshot-processing throughput: the action blocks events/Display, and WGC may reuse a ready frame.

Limits retained

Secondary is a conditional runtime target, not a documented final-window buffer. Unsupported Frame effects/transforms/origins and unavailable/invalid sources remain explicit failures. Old secondary surfaces belong to frame transitions and are not substituted as a previous Display image. No black-pixel heuristic or unrelated canvas is used.

WGC startup waits for visible windows and logs its failure stage/HRESULT. Its producer is asynchronous, so completed copying does not establish current-Display freshness. No ready frame returns E_PENDING. The Windows yellow capture indicator remains; no borderless-capture consent is requested.

D3D live-object investigation

The author reports exit-time live-object warnings appearing only after GPU capture was introduced. This remains an integration regression to investigate; the anonymous shutdown report is not dismissed as harmless.

With a debug-layer device, normalization objects are named NeoPic.Capture.*. Immutable state creation can return shared interfaces, so a state name alone does not establish exclusive ownership. A real debug-layer WARP test observes named objects while in use and verifies no remaining external references after one capture, 32 captures and repeated destruction, without ClearState. That test passed; it does not prove Fusion released its SDK image cache or every extension instance.

The native pass now has explicit cache reset, used on device changes and after GPU-source teardown. A used source logs "native capture cache released (published snapshots have separate ownership)" after its cache is released. The expanded lifetime test covers idempotent reset and reinitialization. The application's actual exit warnings still need correlation with this marker, published snapshot lifetime and a detailed in-process live-object report.

Validation

Portable statistics and V1/V2 upgrade/default/reserve tests pass with GCC/ASan/UBSan; existing crop/completion regressions also pass.

Passed on Windows Server 2022 / MSVC / WARP: Windows checks. The explicit reset/reinitialize lifetime cases and DX9 ordinary/benchmark factory fallback tests passed too. The workflow now checks the real independent WGC implementation once (it is no longer a macro-disabled translation unit), the production WARP pixel/resize pass, real debug-layer native-resource destruction, statistics, V1 payload preservation/defaults/reserve, and capture-class inheritance/ownership/exception cleanup with test doubles.

Public CI does not contain proprietary Fusion SDK headers. Its SDK stand-ins test production class dispatch and ownership, not real Fusion layout/rendering or MFX linking. The default local script additionally syntax-checks Main/Runtime/Edittime/General/capture/optional headers against the real SDK and compiles the property resources:

./Extensions/NeoPic/tests/CompileCapture.ps1 -RunGpuSmokeTest

See CaptureTesting.md for installation, V1 MFA upgrade/save/reopen/export checks, all three property choices and the optional subclass setup.

The existing Encryption/build imports and native/legacy wchar_t separation are preserved. No SDK exports or generated MFX binaries are included. Draft pending the author's property/upgrade and post-refactor runtime checks.

@defisym defisym changed the title NeoPic: add macro-selected Secondary Surface and WGC capture tests NeoPic: test synchronous child-client capture backends Nov 11, 2026
@defisym defisym changed the title NeoPic: test synchronous child-client capture backends NeoPic: normalize synchronous child-client captures on GPU Nov 11, 2026
@defisym defisym changed the title NeoPic: normalize synchronous child-client captures on GPU NeoPic: normalize GPU captures and benchmark the capture action Nov 11, 2026
@defisym defisym changed the title NeoPic: normalize GPU captures and benchmark the capture action NeoPic: select capture backends per object and isolate benchmark subclasses Nov 12, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant