3D memory constellation usability, a11y & performance pass (reduced-motion, WebGL fallback, keyboard, mobile) #25

Activity

1. harrymove-ctrl

   added this to the Sui Overflow milestone on Jun 16, 2026
2. harrymove-ctrl

   added

   P1Important: hardens the demo and core productImportant: hardens the demo and core product

   ui-uxWeb app UI/UX, accessibility, and responsive designWeb app UI/UX, accessibility, and responsive design

   on Jun 16, 2026
3. harrymove-ctrl commented on Jun 19, 2026

   

   harrymove-ctrl

   on Jun 19, 2026

   OwnerAuthor

   More actions

   Implementation Plan

   Approach
   Split the monolithic NamespaceMemoryConstellation.tsx (575 lines) into a thin parent that owns shared chrome (search box, legend, fullscreen button, detail panel, selected state) and chooses a renderer, plus a Constellation3D child that owns the THREE scene, and a new MemoryGraph2D child (DOM/SVG list) used when WebGL is unavailable. This makes the "dispose on fallback switch" guarantee free (React unmounts the 3D child → its existing cleanup runs), keeps hooks unconditional, and isolates the WebGL/reduced-motion/mobile logic. Reduced-motion is read via a live matchMedia ref (reusing the main.tsx:953 + staggered-text.tsx:129-141 patterns) and branches the animate loop / controls.autoRotate / bloom; on reduced-motion or static state we render on-demand (controls change + resize) instead of a permanent requestAnimationFrame loop.

   Prerequisites & dependencies

   • No blocking issues — the issue states "Dependencies: None"; the 2D fallback reuses the already-imported facts-to-graph.ts (MemoryGraph prop is already passed in) and related-neighbors.ts. This is roadmap §4 P0 ("Reduced-motion + WebGL/low-power fallback for the constellation").
   • No new creds/bindings — pure client-side React/three work in apps/web. No Worker/D1/Harbor/Sui touch.
   • Libs already present: three@0.184 (incl. three/examples/jsm/capabilities/WebGL.js), motion/react (exports useReducedMotion), lucide-react, @testing-library not installed but jsdom@27 + vitest@4 are available at the root (env is node; use a // @vitest-environment jsdom pragma for DOM-touching tests).
   • Test runner: root vitest run over apps/**/*.test.ts (env node). Existing sibling tests: facts-to-graph.test.ts, related-neighbors.test.ts, entity-detail.test.ts.

   Step-by-step

   1. Add capability/preference helpers — new apps/web/src/components/namespace-memory/capabilities.ts:

      • webglSupported(): boolean — create a throwaway <canvas> and test getContext('webgl2') || getContext('webgl') inside a try/catch (or wrap three/examples/jsm/capabilities/WebGL.js's isWebGLAvailable). DOM-touching → covered by a jsdom-pragma test.
      • prefersReducedMotion(): boolean — window.matchMedia('(prefers-reduced-motion: reduce)').matches (mirror main.tsx:953).
      • isCompactViewport(): boolean — window.matchMedia('(max-width: 640px)').matches.
      • Pure reducer nextActiveIndex(current, key, length) for arrow/Home/End keyboard nav (unit-testable in node).

   2. Refactor the parent NamespaceMemoryConstellation (NamespaceMemoryConstellation.tsx):

      • Lift selected/setSelected and the search/legend/panel/fullscreen JSX (currently lines 548-574) into the parent so they work in both render modes.
      • Add state: webglOk (init webglSupported()), reducedMotion, compact; subscribe to both media queries with the addEventListener('change', …) pattern from staggered-text.tsx:129-141 (update state on change).
      • Add a single keydown handler on rootRef (replaces the missing Esc support): if focus is in the search input let the search box own Esc (clear query); else Escape → if selected deselect (setSelected(null)), else if fullscreen call toggleFullscreen(). (Keep existing toggleFullscreen at lines 286-295 and the fullscreen change effect at 273-284.)
      • Render {webglOk ? <Constellation3D … /> : <MemoryGraph2D … />}. Pass graph, selected, onSelect={setSelected}, reducedMotion, compact, and onContextLost={() => setWebglOk(false)}.

   3. Create Constellation3D.tsx — move the THREE setup verbatim from the current one-time scene effect (lines 322-453), the selected-autoRotate effect (456-459), and the mesh-rebuild effect (462-539), then apply the gating:

      • Reduced-motion gate (AC Walrus storage + Memory via Tatum, plus CI deploy #1): keep reducedMotion in a ref updated each render. In the scene effect, set controls.autoRotate = !reducedMotionRef.current (replaces hard autoRotate = true at line 345). Replace the unconditional s.raf = requestAnimationFrame(animate) (line 362): when reduced-motion, do not schedule the loop — instead define renderOnce() (freeze uTime at a fixed value, run controls.update() + composer.render() / direct renderer.render, then reproject labels) and call it on controls.addEventListener('change', renderOnce) and from the ResizeObserver. When motion is allowed, keep the existing RAF loop. Gate bloom: under reduced-motion either skip composer.addPass(bloom) (render via RenderPass+OutputPass only) or set bloom.strength = 0. Re-evaluate when the media query flips (re-run effect via reducedMotion dep, or toggle inside a small effect).
      • WebGL context loss (AC Memory knowledge graph: 3D neural constellation + grounded entity panel #2/Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6): add renderer.domElement.addEventListener('webglcontextlost', e => { e.preventDefault(); onContextLost(); }) so a runtime loss flips the parent to the 2D fallback.
      • Mobile perf (AC Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5): replace setPixelRatio(Math.min(devicePixelRatio, 2)) (line 333) with setPixelRatio(compact ? 1 : Math.min(devicePixelRatio, 2)); construct UnrealBloomPass (line 349, currently strength 1.6) with reduced strength/radius under compact (e.g. compact ? 0.6 : 1.6). Recompute both in the ResizeObserver (lines 431-438) so rotation/resize across the breakpoint updates them.
      • Focus-on-select: instead of the parent calling s.focusRec, have the child watch the selected prop in an effect → fly the camera (existing focusRec, lines 423-428). Under reduced-motion, jump the camera/target directly (set, don't lerp via s.desired) then renderOnce(), since there's no RAF loop to run the lerp.
      • Dispose composer (AC Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6): extend the cleanup (lines 442-452) — it currently disposes meshes/stars/controls/renderer but not the composer/bloom. Add composer.dispose() and bloom.dispose() (and remove controls/canvas listeners) so unmounting to the fallback leaks no GL context or render targets.
      • Keep the projected DOM labels host but make it keyboard/SR-friendly (see step 5).

   4. Create MemoryGraph2D.tsx (AC Memory knowledge graph: 3D neural constellation + grounded entity panel #2) — a no-WebGL fallback rendered from the same MemoryGraph prop (output of factsToMemoryGraph, facts-to-graph.ts) plus relatedNeighbors (related-neighbors.ts):

      • A grouped, scrollable list/SVG of nodes sorted by salience, colored via colorForType/labelForType from lib/entity-colors.ts (signatures at entity-colors.ts:87/:91), each row a focusable <button> (roving tabindex, arrow/Enter nav using nextActiveIndex) calling onSelect(node). Selecting reuses the same shared MemoryChunkPanel/relatedNeighbors path as 3D.
      • Render inside the existing .nmc-overlay-msg/.nmc-canvas box so layout/fullscreen still work; add styles in namespace-memory.css.

   5. Make search + nodes keyboard-navigable (AC docs: add product roadmap (Harbor storage + Talus on-chain direction) #4) — MemorySearchBox.tsx (currently a plain input + button list, 31 lines):

      • Input → role="combobox", aria-expanded, aria-controls="nmc-results", aria-autocomplete="list", aria-activedescendant = active option id; results container → role="listbox" id="nmc-results"; each result → role="option" with stable id + aria-selected.
      • Add onKeyDown: ArrowUp/Down move active index (nextActiveIndex), Enter selects the active option (onSelect + clear), Esc clears query. Mouse onClick still works.
      • For 3D nodes specifically: add a visually-hidden focusable node list (or remove aria-hidden="true" from the labels host at line 560 and make labels focusable buttons that call onSelect) so screen-reader/keyboard users can reach nodes without a pointer. The 2D fallback already satisfies node keyboard-nav inherently.

   6. CSS (namespace-memory.css):

      • Style the combobox/listbox active option ([aria-selected="true"]) and focus rings on .nmc-results button, .nmc-related, fallback rows.
      • Mobile bottom-sheet (AC Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5): add @media (max-width: 640px) for .nmc-panel (currently top:0;right:0;height:100% at lines 23-28) → left:0;right:0;bottom:0;top:auto;height:auto;max-height:75vh;border-left:none;border-top:1px solid …;border-radius:12px 12px 0 0. Tighten .nmc-search/.nmc-legend for small widths.
      • Style MemoryGraph2D rows/groups (reuse --cm-* tokens; keep brand lime accents).

   7. Call-site sanity — no change needed at FactsPanel (main.tsx:7253, constellation rendered in .nmc-host at main.tsx:7358-7360); the component keeps the same { graph, entityDetail } props. Verify constellationGraph (main.tsx:7260) still feeds both modes.

   New / changed files

   • apps/web/src/components/namespace-memory/capabilities.ts (new)
   • apps/web/src/components/namespace-memory/Constellation3D.tsx (new — extracted THREE scene)
   • apps/web/src/components/namespace-memory/MemoryGraph2D.tsx (new — 2D/list fallback)
   • apps/web/src/components/namespace-memory/NamespaceMemoryConstellation.tsx (changed — thin parent: capability/RM/compact state, Esc handler, mode switch, shared chrome)
   • apps/web/src/components/namespace-memory/MemorySearchBox.tsx (changed — combobox/listbox a11y + keyboard nav)
   • apps/web/src/components/namespace-memory/namespace-memory.css (changed — listbox/focus styles, mobile bottom-sheet, 2D fallback rows)
   • apps/web/src/components/namespace-memory/capabilities.test.ts + MemorySearchBox.keyboard.test.ts (new tests)

   Test plan

   • Unit (node env): nextActiveIndex wrap/clamp/Home/End; search filter/ranking still matches MemorySearchBox haystack behavior. Reuse existing facts-to-graph.test.ts/related-neighbors.test.ts to confirm the 2D fallback consumes the same graph shape.
   • Unit (jsdom pragma): webglSupported() returns false when getContext throws/returns null; MemorySearchBox keyboard: typing → listbox opens with aria-activedescendant, ArrowDown/Enter fires onSelect, Esc clears.
   • Integration: render NamespaceMemoryConstellation with a stubbed webglSupported → false and assert the MemoryGraph2D list renders the expected nodes (no canvas, no THREE import path hit) — proves AC Memory knowledge graph: 3D neural constellation + grounded entity panel #2.
   • Manual (proves it works): (a) DevTools → Rendering → "Emulate prefers-reduced-motion: reduce": confirm no autorotate, no continuous RAF (Performance panel shows idle, not a steady 60fps loop), bloom off/reduced, scene still visible and orbit-draggable. (b) chrome://gpu disable WebGL (or stub) → 2D list renders, and in DevTools Memory/Performance the WebGL context count drops to 0 after switch (AC Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6 — no leaked context). (c) Esc deselects an open node and exits fullscreen (AC fix(ci): force bun + pin wrangler in Deploy workflow #3). (d) Tab to search, arrow/enter to select; Tab through 2D list (AC docs: add product roadmap (Harbor storage + Talus on-chain direction) #4). (e) Resize to 375px width: pixelRatio 1, reduced bloom, detail panel as bottom-sheet, graph usable (AC Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5).
   • bun run -C apps/web typecheck + root vitest run green; vite build succeeds.

   Risks & gotchas

   • No leaked GL context: the win from the parent/child split is that unmounting Constellation3D runs cleanup — but only if cleanup is complete. The current cleanup (lines 447-451) misses composer.dispose()/bloom.dispose(); add them or the fallback switch still leaks render targets (directly violates AC Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6). Also remove all listeners (controls change, webglcontextlost, pointer down/up).
   • Reduced-motion without an RAF loop: camera fly (s.desired lerp) and click ripple pulses depend on the loop. Under reduced-motion, jump the camera directly and accept static pulses; render on controls 'change' + resize so dragging/resizing still repaint (otherwise the canvas appears frozen after the first user input).
   • State that must survive without re-mounting the scene: put reducedMotion/compact in refs read inside the loop to avoid tearing down/rebuilding the whole THREE scene on every media-query flip; only re-run the effect for changes that truly require pass reconfiguration (bloom add/remove).
   • Label projection cost: 2D fallback for webglcontextlost must not also keep the (now dead) RAF running — ensure the child fully unmounts.
   • No mainnet/testnet/Worker/migration concerns — this is client-only; the only platform risk is the three example-module import paths (three/examples/jsm/...) which already resolve under Vite.
   • aria-hidden labels: removing it (line 560) to make labels focusable can flood the a11y tree for dense graphs (up to 40 nodes via MAX_NODES in facts-to-graph.ts); prefer a single visually-hidden listbox of nodes over 40 focusable label divs.

   Acceptance-criteria mapping

   • Autorotate + bloom + RAF gated on prefers-reduced-motion → Step 2 (RM state/listener) + Step 3 (autoRotate at line 345, RAF at line 362, bloom at line 349, on-demand renderOnce).
   • 2D/list fallback from facts-to-graph.ts (+ related-neighbors.ts) when WebGL unavailable → Step 1 (webglSupported) + Step 2 (mode switch) + Step 4 (MemoryGraph2D).
   • Esc deselects active node and exits fullscreen → Step 2 (single keydown handler).
   • Search results and nodes keyboard-navigable (focus + arrow/enter, aria-activedescendant) → Step 5 (MemorySearchBox combobox/listbox) + Step 4 (focusable 2D rows) + visually-hidden node list for 3D.
   • Small viewports (<640px): reduced pixelRatio + bloom, usable graph + detail panel → Step 1 (isCompactViewport) + Step 3 (pixelRatio at line 333, bloom strength) + Step 6 (bottom-sheet @media).
   • Switching to fallback disposes renderer/composer (no leaked context) → Step 2 (unmount on !webglOk) + Step 3 (extended cleanup incl. composer.dispose()/bloom.dispose(), webglcontextlost).

   Effort
   M — ~2.5-3.5 person-days. The component split + extraction is the bulk (~1.5 pd) but mechanical; reduced-motion/on-demand-render and the 2D fallback are ~1 pd; search a11y, Esc, mobile CSS, and tests ~0.5-1 pd. Risk concentrated in the THREE lifecycle (dispose + on-demand render), which the manual GL-context-count check validates.

   ---

   Implementation plan generated for execution (planning phase). Part of the roadmap (#4).