Skip to content

feat: PyVNCServer 3.3 — native DXGI metadata and encoding interoperability - #13

Merged
xulek merged 20 commits into
mainfrom
v3.3-dxgi-interoperability
Sep 7, 2026
Merged

xulek merged 20 commits into
mainfrom
v3.3-dxgi-interoperability

Conversation

@xulek

@xulek xulek commented Sep 7, 2026 •

Copy link
Copy Markdown
Owner

PyVNCServer 3.3 — DXGI metadata + interoperability

This PR implements the v3.3 capture/interoperability milestone.

Native DXGI metadata

  • reads IDXGIOutputDuplication::GetFrameDirtyRects and GetFrameMoveRects while the DXCam frame is still acquired
  • reuses DXCam's existing Desktop Duplication object instead of starting a second capture stack
  • validates and clips native dirty/move rectangles before exposing them to the framebuffer pipeline
  • treats a successful DXGI no-new-frame result as an authoritative empty update
  • falls back to the existing software framebuffer differ whenever native metadata cannot be read or validated safely
  • automatically disables native metadata for scaled, rotated or cropped DXCam captures until coordinate translation is implemented
  • exposes DXGI move rectangles as CopyRect hints
  • conservatively includes move destinations in dirty pixel regions as well, preserving correctness for clients without CopyRect and clients that skip producer generations

UltraVNC / encoding interoperability

Adds real TCP/RFB end-to-end regressions that keep one connection open while changing the client's advertised encoding set.

The CI matrix parses the actual wire payload for:

Raw → RRE → Hextile → Zlib → ZRLE → Tight → Raw

There is also an explicit RRE → Tight regression matching the failure mode previously observed with UltraVNC. These tests validate both the rectangle encoding ID and the corresponding payload framing/decoding.

DXCam contract protection

  • performance extra now requires dxcam>=0.3.0
  • a dedicated Windows CI job installs .[dev,performance]
  • the job verifies the actual DXCam private integration points and the Desktop Duplication dirty/move methods
  • the ctypes/COM vtable path is exercised by a synthetic ABI test without requiring an interactive desktop

Diagnostics and docs

  • adds benchmarks/benchmark_dxgi_metadata.py
  • documents native metadata, fallback semantics and conservative move handling
  • documents the automated and manual UltraVNC compatibility matrix
  • bumps package version to 3.3.0

Run the real Windows diagnostic with:

python -m pip install -e ".[performance]"
python benchmarks/benchmark_dxgi_metadata.py --frames 300 --fps 60

Validation

Current GitHub Actions result is fully green:

  • Ubuntu / Python 3.11: ✅
  • Ubuntu / Python 3.12: ✅
  • Ubuntu / Python 3.13: ✅
  • Windows / Python 3.13: ✅
  • Windows DXCam [performance] contract: ✅
  • coverage: ✅
  • wheel + sdist build: ✅
  • distribution validation + wheel install smoke test: ✅

Remaining manual validation before release

A GitHub-hosted Windows runner can validate the DXCam ABI but cannot provide a representative interactive desktop/GPU capture workload. Before merging/releasing v3.3, run the DXGI metadata benchmark and an UltraVNC session on a real Windows desktop, especially window movement, scrolling, Tight and RRE switching.

@xulek xulek changed the title feat: add native DXGI dirty/move metadata for v3.3 feat: PyVNCServer 3.3 — native DXGI metadata and encoding interoperability Sep 7, 2026
@xulek
xulek merged commit b963962 into main Sep 7, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant