diff --git a/README.md b/README.md index 93cf5b3..1ac1cf0 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,11 @@ patchstory render ./pr-walkthrough.json --out ./site `fetch` required). Zip-friendly. - **AI optional.** A heuristic generator always works with no API key. AI *improves* the story; it is never required. +- **Narrated play mode.** Press **▶ Play** to turn the walkthrough into a + self-playing screencast: each chapter becomes a scene that pans the actual + diff and spotlights the lines it references, narrated aloud via the browser's + built-in speech synthesis (captions included). No ffmpeg, no API key, no + network — the same single `.html`, just playing itself. --- @@ -172,6 +177,7 @@ Commands file raw unified diff file github GitHub PR (uses `gh` if available, else public .diff) render render an existing pr-walkthrough.json + video render a narrated .mp4 screencast of the walkthrough serve [dir|file] serve an output folder/file on your LAN schema print the pr-walkthrough.json JSON Schema @@ -190,8 +196,13 @@ Options --serve serve the result on your LAN after generating --open open the result in a browser --port port for --serve / serve (default 8137) - --diff (render only) raw diff to fill the diff explorer + --diff (render/video) raw diff to fill the diff explorer --zip also write .zip + --tts (video) auto | elevenlabs | espeak-ng | flite | say | none + --voice (video) voice id (elevenlabs) or name (espeak-ng/say) + --chrome (video) Chrome/Chromium used to rasterize scenes + --fps (video) frames per second (default 30) + --keep (video) keep the intermediate working dir -h, --help show help --version show version ``` @@ -207,6 +218,36 @@ produces output. others can open. `--redact` masks secrets (token shapes, `KEY=value`, private keys) in the diff before it's embedded *or* sent to an AI generator. +### Narrated video (opt-in MP4) + +`patchstory video --diff -o walkthrough.mp4` renders the +walkthrough into a real, shareable `.mp4`: a title card, one **animated** scene per +chapter (the diff reveals line-by-line and the referenced lines light up as they're +narrated), and an outro. Unlike everything else here, this shells out to **system +tools** — it adds no npm runtime deps, and they're only touched when you ask for a video. + +Two engines (`--engine`): + +- **`hyperframes`** (default) — generates a [HyperFrames](https://hyperframes.heygen.com) + composition (HTML + GSAP) and renders it frame-by-frame in headless Chrome via + `npx hyperframes`. This is the animated one. Needs network for `npx` on first use. +- **`pan`** — a fully local fallback: rasterizes each scene with Chromium and pans it + with **ffmpeg**. No `npx`/network; lower production value. + +**Text-to-speech** (`--tts`, default `auto`): `elevenlabs` (`ELEVENLABS_API_KEY`, best +quality), `kokoro` (local neural TTS via HyperFrames — no key, the keyless default), +local `espeak-ng` / `flite` / macOS `say`, or `none` (silent; captions still shown). + +**ffmpeg/ffprobe** are resolved from `PATH`, then `/usr/bin`, then +`PATCHSTORY_FFMPEG` / `PATCHSTORY_FFPROBE` — each validated by actually running it, so a +broken or shadowing PATH entry is skipped (and the working one is handed to HyperFrames). + +Narration is the audio track — nothing is burned into or captioned over the frame, so the +code and motion graphics stay unobstructed. + +It's slower and heavier than the HTML — the in-page play mode is the local-first default; +the MP4 is for when you need a file to drop in Slack or a release thread. + ### Interactive UI Syntax-highlighted diffs (highlight.js, bundled at build time — lazily applied @@ -217,7 +258,9 @@ dark, copy-summary, a "Start here" guide and recurring-theme detection on the overview, related commits per chapter, and a footer build stamp. Keyboard: `j`/`k` next/prev chapter · `/` search · `e`/`c` expand/collapse all · -`r` toggle reviewed · `t` theme · `?` shortcuts · `Esc` close. +`r` toggle reviewed · `p` play narrated walkthrough · `t` theme · `?` shortcuts · +`Esc` close. In play mode: `space` play/pause · `←`/`→` prev/next scene · `m` +mute (captions only) · `Esc` close. --- diff --git a/integrations/claude-code/patchstory/skills/patchstory/SKILL.md b/integrations/claude-code/patchstory/skills/patchstory/SKILL.md index 9a59841..17b02f1 100644 --- a/integrations/claude-code/patchstory/skills/patchstory/SKILL.md +++ b/integrations/claude-code/patchstory/skills/patchstory/SKILL.md @@ -6,7 +6,9 @@ description: > open PR if there is one, otherwise the branch vs its default base — or takes an explicit PR number / PR URL / git range. The agent authors the narrative itself (chapters with intent, risk, reviewer questions, and verification steps), then patchstory renders it as - one self-contained .html with secrets redacted and opens it in the browser. + one self-contained .html with secrets redacted and opens it in the browser. The rendered + page also has a narrated "play" mode — a self-playing screencast that pans the diff while + reading each chapter aloud — so author a short spoken `narration` per chapter. Triggers: "/patchstory", "patchstory this", "make a walkthrough of this PR", "tell the story of this PR", "PR walkthrough", "explain this PR for a human", "patchstory #123", "patchstory the current branch". @@ -89,6 +91,10 @@ who has never seen the change: one chapter when they tell one sub-story. Each chapter: - **`intent`** — *why* this exists / what problem it solves (the most valuable field). - **`summary`** — what the diff in this chapter does. + - **`narration`** — 1–4 sentences of *spoken* prose for the page's "play" mode: plain and + conversational, what you'd say out loud while walking someone through this chapter. Avoid + symbols/paths that sound bad read aloud. Optional, but author it — without it, play mode + falls back to reading `intent` then `summary`. - **`risk_level`** — `low|medium|high`. Raise for auth, payments, migrations, money math, deletions, or anything externally observable. - **`review_notes`** — sharp reviewer questions. @@ -120,7 +126,8 @@ Authoritative copy: `patchstory schema`. Required: `version`, `title`, `summary` (+ `source.type` ∈ `github_pr|git_diff|commit_range|diff_file`), `stats` (`files_changed`, `additions`, `deletions` — numbers), `chapters`. Each chapter needs a **unique** `id`, `title`, `summary`, `risk_level` (`low|medium|high`), and `files`. `diff_hunks` items need `file`, -`start_line`, `end_line` (line numbers in the **new** file). Everything else is optional. +`start_line`, `end_line` (line numbers in the **new** file). Everything else is optional, +including the chapter's `narration` (spoken script for "play" mode). ```jsonc { @@ -139,6 +146,7 @@ Authoritative copy: `patchstory schema`. Required: `version`, `title`, `summary` "title": "Detect multiple faces in uploaded media", "summary": "Adds metadata and detection logic for multi-face media.", "intent": "Determine whether creator approval is needed before publishing.", + "narration": "When media is uploaded, we now count the faces in it. If there's more than one person, the upload can't auto-publish — it routes to the creator for approval first. This chapter adds the detection logic and the fields that track that state.", "risk_level": "medium", "files": ["app/models/media.rb", "app/services/face_detection_service.rb"], "diff_hunks": [ @@ -166,5 +174,13 @@ Authoritative copy: `patchstory schema`. Required: `version`, `title`, `summary` "$WORK/site"`, drop `--single-file`) and then `patchstory serve "$WORK/site"` in the background — it binds `0.0.0.0` and prints a URL other devices can open. The `--port` is a starting hint; if taken, serve picks the next free port and prints the real one. +- **Want a shareable video instead of HTML?** `patchstory video "$WORK/pr-walkthrough.json" + --diff "$WORK/pr.diff" --redact -o "$WORK/walkthrough.mp4"` renders a narrated, **animated** + `.mp4` from the same JSON (a title card + one scene per chapter where the diff reveals and + the referenced lines light up as they're narrated — this is why authoring `narration` is + worth it). Default engine `hyperframes` (animated, via `npx hyperframes`; needs network); + `--engine pan` is a local ffmpeg fallback. TTS `--tts auto` picks ElevenLabs if + `ELEVENLABS_API_KEY` is set, else local `kokoro` (no key). Slower/heavier than the HTML — + only reach for it when a video file is the deliverable. - **Private PRs** need `gh` (authenticated). The public `.diff` fallback is public-repos-only. - The work dir under `~/.cache/patchstory/` persists; old runs can be deleted freely. diff --git a/packages/cli/src/args.ts b/packages/cli/src/args.ts index 16ec5f8..4c17095 100644 --- a/packages/cli/src/args.ts +++ b/packages/cli/src/args.ts @@ -9,7 +9,7 @@ export interface ParsedArgs { /** Flags that take no value (presence = true). */ const BOOLEAN_FLAGS = new Set([ "zip", "help", "version", "no-open", "single-file", "serve", "open", "redact", - "scaffold", + "scaffold", "keep", ]); export function parseArgs(argv: string[]): ParsedArgs { diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 9ee12dd..a42c8b0 100644 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -23,7 +23,8 @@ import type { ResolvedSource, WalkthroughBundle, } from "@patchstory/core"; -import { renderWalkthrough, renderSingleFile } from "@patchstory/renderer"; +import { renderWalkthrough, renderSingleFile, renderVideo } from "@patchstory/renderer"; +import type { TtsProvider } from "@patchstory/renderer"; import { parseArgs, flagStr } from "./args.ts"; import { zipDirectory } from "./zip.ts"; import { serveStatic, findFreePort, openBrowser, lanIp } from "./serve.ts"; @@ -45,6 +46,7 @@ Commands: file Walkthrough of a raw unified diff file github Walkthrough of a GitHub pull request render Render an existing pr-walkthrough.json + video Render a narrated .mp4 screencast of the walkthrough serve [dir|file] Serve an output folder/file on your LAN schema Print the pr-walkthrough.json JSON Schema @@ -64,8 +66,14 @@ Options: --serve Serve the result on your LAN after generating --open Open the result in a browser --port Port for --serve / serve (default: 8137) - --diff (render only) raw diff to populate the diff explorer + --diff (render/video) raw diff to populate the diff explorer --zip Also write .zip + --engine (video) hyperframes (animated, default) | pan (static) + --tts (video) auto | elevenlabs | kokoro | espeak-ng | flite | say | none + --voice (video) voice id (elevenlabs) or name (kokoro/espeak-ng/say) + --chrome (video) Chrome/Chromium binary for the pan engine + --fps (video) frames per second (default: 30) + --keep (video) keep the intermediate working dir -h, --help Show this help --version Show version @@ -136,6 +144,38 @@ async function main() { return; } + // `video` renders a narrated .mp4 from an existing walkthrough JSON (+ diff). + if (command === "video") { + const redactV = !!flags.redact; + const result = await renderCommand(positionals, flags, redactV); + const bundle: WalkthroughBundle = { walkthrough: result.walkthrough, diff: result.diff }; + const outFlag = flagStr(flags, "out") ?? "./walkthrough.mp4"; + const outFile = /\.mp4$/i.test(outFlag) ? outFlag : `${outFlag}.mp4`; + process.stdout.write(`\nRendering video → ${resolve(outFile)}\n`); + try { + const res = await renderVideo(bundle, { + out: outFile, + engine: (flagStr(flags, "engine") as "hyperframes" | "pan" | undefined), + tts: flagStr(flags, "tts") as TtsProvider | undefined, + voice: flagStr(flags, "voice"), + chrome: flagStr(flags, "chrome"), + fps: flags.fps ? Number(flagStr(flags, "fps")) : undefined, + keep: !!flags.keep, + onProgress: (m) => process.stdout.write(` ${m}\n`), + }); + process.stdout.write( + `\n✓ Video written to ${res.file}\n` + + ` ${res.sceneCount} scenes · ~${Math.round(res.durationSec)}s · tts: ${res.ttsProvider}\n`, + ); + if (redactV) { + process.stdout.write("🛈 Redaction on: secrets masked in the diff shown on-screen.\n"); + } + } catch (err) { + fail(err instanceof Error ? err.message : String(err)); + } + return; + } + const generator = flagStr(flags, "generator") ?? "none"; const model = flagStr(flags, "model"); const redact = !!flags.redact; diff --git a/packages/core/src/schema.ts b/packages/core/src/schema.ts index 4e460b3..a02b6f8 100644 --- a/packages/core/src/schema.ts +++ b/packages/core/src/schema.ts @@ -56,6 +56,7 @@ export const WALKTHROUGH_JSON_SCHEMA = { title: { type: "string" }, summary: { type: "string" }, intent: { type: "string" }, + narration: { type: "string" }, risk_level: { type: "string", enum: ["low", "medium", "high"] }, files: { type: "array", items: { type: "string" } }, diff_hunks: { diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 7d25a95..8feceeb 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -54,6 +54,14 @@ export interface Chapter { summary: string; /** Why this part of the change exists. */ intent?: string; + /** + * Spoken narration for this chapter, used by the in-page "play" mode (a + * narrated, auto-advancing screencast). One to four sentences of plain, + * conversational prose — what a reviewer would say out loud while walking + * someone through this change. Optional: when absent, play mode falls back + * to `intent` then `summary`. + */ + narration?: string; risk_level: RiskLevel; files: string[]; diff_hunks: DiffHunkRef[]; diff --git a/packages/renderer/src/index.ts b/packages/renderer/src/index.ts index 67bb7c8..1663d53 100644 --- a/packages/renderer/src/index.ts +++ b/packages/renderer/src/index.ts @@ -16,6 +16,10 @@ import { dirname, join } from "node:path"; import type { WalkthroughBundle } from "@patchstory/core"; import { WEB_CSS, WEB_HTML, WEB_JS } from "./assets.generated.ts"; +// Opt-in MP4 export (drives system ffmpeg + headless Chromium + a TTS engine). +export { renderVideo } from "./video/index.ts"; +export type { VideoOptions, VideoResult, TtsProvider } from "./video/index.ts"; + export interface RenderOptions { /** ISO timestamp stamped into the document. */ generatedAt?: string; diff --git a/packages/renderer/src/video/composition.ts b/packages/renderer/src/video/composition.ts new file mode 100644 index 0000000..7177572 --- /dev/null +++ b/packages/renderer/src/video/composition.ts @@ -0,0 +1,242 @@ +/** + * Generate a complete HyperFrames composition (one self-contained index.html + * with a single paused GSAP timeline) from a walkthrough's scenes. + * + * This is the animated video engine: a title card, then one scene per chapter + * that reveals the actual diff line-by-line and lights up the referenced + * ("spotlight") lines as they're narrated, with sentence-beat captions timed to + * the measured voiceover. HyperFrames seeks this timeline frame-by-frame in + * headless Chrome, so everything obeys the determinism contract: no wall-clock + * logic, captions are static DOM text, animate `autoAlpha`/transforms, and every + * faded-out element is hard-cleared with `tl.set(..., {autoAlpha:0})`. + */ + +export interface CodeRow { + sign: " " | "+" | "-"; + num: number | null; + /** Pre-highlighted, escaped HTML for the line content. */ + html: string; + spot: boolean; +} + +export interface VideoScene { + kind: "title" | "chapter" | "outro"; + eyebrow: string; + title: string; + risk?: "low" | "medium" | "high"; + intent?: string; + subtitle?: string; // title/outro only + filePath?: string; + rows?: CodeRow[]; + hasSpots: boolean; + /** Absolute timeline position (seconds). */ + start: number; + dur: number; + voStart: number; + voDur: number; + /** Audio file (relative to the composition), or null for silent scenes. */ + audio: string | null; +} + +const FRAME_W = 1920; +const FRAME_H = 1080; + +/* timing constants (seconds) */ +const FADE = 0.5; +const HDR_IN = 0.3; +const CODE_IN = 0.8; +const ROW_STAGGER = 0.085; +const ROW_DUR = 0.4; + +function esc(s: string): string { + return s.replace(/&/g, "&").replace(//g, ">").replace(/"/g, """); +} + +const CSS = ` +* { margin: 0; padding: 0; box-sizing: border-box; } +:root { + --bg:#0d1117; --panel:#161b22; --panel2:#1c2430; --line:#30363d; + --text:#e6edf3; --muted:#768390; --accent:#4493f8; + --add:#3fb950; --add-wash:rgba(63,185,80,0.13); --del:#f85149; +} +html,body { width:${FRAME_W}px; height:${FRAME_H}px; overflow:hidden; background:var(--bg); } +body { font-family: system-ui,-apple-system,"Segoe UI",Roboto,Helvetica,Arial,sans-serif; color:var(--text); letter-spacing:-0.01em; -webkit-font-smoothing:antialiased; } +.mono { font-family: ui-monospace,"JetBrains Mono","DejaVu Sans Mono",Menlo,Consolas,monospace; } +.scene { position:absolute; inset:0; opacity:0; } +#bg { position:absolute; inset:0; background: radial-gradient(1400px 880px at 32% 34%, #151b24 0%, #0f141b 48%, var(--bg) 100%); } + +/* header */ +.hdr { position:absolute; left:120px; top:92px; width:1680px; } +.eyebrow-row { display:flex; align-items:center; gap:20px; margin-bottom:16px; } +.eyebrow { font-size:26px; font-weight:800; letter-spacing:0.34em; text-transform:uppercase; color:var(--muted); } +.pill { font-size:22px; font-weight:900; letter-spacing:0.06em; text-transform:uppercase; padding:6px 18px; border-radius:999px; } +.pill-low { background:var(--add-wash); color:var(--add); } +.pill-medium { background:rgba(210,153,34,0.15); color:#d29922; } +.pill-high { background:rgba(248,81,73,0.15); color:var(--del); } +.title { font-size:68px; font-weight:900; letter-spacing:-0.03em; line-height:1.04; + display:-webkit-box; -webkit-line-clamp:2; -webkit-box-orient:vertical; overflow:hidden; } +.intent { margin-top:14px; font-size:30px; font-weight:500; color:var(--muted); max-width:1500px; + display:-webkit-box; -webkit-line-clamp:1; -webkit-box-orient:vertical; overflow:hidden; } + +/* code panel */ +.code { position:absolute; left:120px; top:300px; width:1680px; background:var(--panel); + border-radius:20px; box-shadow:0 24px 80px rgba(0,0,0,0.45), inset 0 0 0 1px var(--line); overflow:hidden; } +.code-head { display:flex; align-items:center; gap:14px; height:64px; padding:0 26px; background:var(--panel2); box-shadow:inset 0 -1px 0 var(--line); } +.dot { width:15px; height:15px; border-radius:50%; } +.dot.r{background:#ff5f57;} .dot.y{background:#febc2e;} .dot.g{background:#28c840;} +.code-path { margin-left:12px; font-size:24px; color:var(--muted); font-weight:600; } +.code-body { padding:18px 0; } +.row { position:relative; display:flex; align-items:center; height:44px; font-family: ui-monospace,"JetBrains Mono","DejaVu Sans Mono",Menlo,Consolas,monospace; font-size:26px; line-height:44px; white-space:pre; } +.row .sb { position:absolute; left:0; top:0; width:6px; height:100%; background:var(--add); transform:scaleY(0); transform-origin:top center; } +.row:not(.spot) .sb { display:none; } +.row .g { width:92px; flex:none; text-align:right; padding-right:22px; color:#586069; } +.row .s { width:30px; flex:none; text-align:center; } +.row.add .s { color:var(--add); } .row.del .s { color:var(--del); } +.row.add { background:rgba(63,185,80,0.07); } .row.del { background:rgba(248,81,73,0.07); } +.row .c { overflow:hidden; padding-right:24px; } + +/* title / outro takeover */ +.takeover { position:absolute; inset:0; display:flex; flex-direction:column; align-items:center; justify-content:center; text-align:center; gap:30px; padding:0 200px; } +.tk-eyebrow { font-size:30px; font-weight:800; letter-spacing:0.4em; text-transform:uppercase; color:var(--accent); } +.tk-title { font-size:128px; font-weight:900; letter-spacing:-0.04em; line-height:0.98; + display:-webkit-box; -webkit-line-clamp:3; -webkit-box-orient:vertical; overflow:hidden; } +.tk-sub { font-size:42px; font-weight:500; color:var(--muted); max-width:1380px; line-height:1.3; } + +/* hljs (GitHub dark) */ +.hljs-keyword,.hljs-selector-tag,.hljs-literal,.hljs-section,.hljs-doctag{color:#ff7b72} +.hljs-string,.hljs-regexp,.hljs-meta .hljs-string,.hljs-addition{color:#a5d6ff} +.hljs-comment,.hljs-quote{color:#8b949e;font-style:italic} +.hljs-number,.hljs-symbol,.hljs-bullet,.hljs-selector-id{color:#79c0ff} +.hljs-title,.hljs-title.function_,.hljs-title.class_{color:#d2a8ff} +.hljs-type,.hljs-class .hljs-title,.hljs-built_in{color:#ffa657} +.hljs-attr,.hljs-attribute,.hljs-variable,.hljs-template-variable,.hljs-property,.hljs-params,.hljs-selector-attr,.hljs-selector-class{color:#79c0ff} +.hljs-name,.hljs-tag,.hljs-selector-pseudo{color:#7ee787} +.hljs-meta{color:#8b949e} +`; + +function rowHtml(r: CodeRow): string { + const cls = r.sign === "+" ? "add" : r.sign === "-" ? "del" : "ctx"; + return ( + `
` + + `` + + `${r.num != null ? r.num : ""}` + + `${r.sign === " " ? "" : r.sign}` + + `${r.html || " "}` + + `
` + ); +} + +function sceneHtml(s: VideoScene, idx: number, z: number): string { + const id = `sc${idx}`; + if (s.kind === "title" || s.kind === "outro") { + return ( + `
` + + `
` + + `
${esc(s.eyebrow)}
` + + `
${esc(s.title)}
` + + (s.subtitle ? `
${esc(s.subtitle)}
` : "") + + `
` + ); + } + const rows = (s.rows ?? []).map(rowHtml).join(""); + return ( + `
` + + `
` + + `
` + + `${esc(s.eyebrow)}` + + (s.risk ? `${esc(s.risk)} risk` : "") + + `
` + + `
${esc(s.title)}
` + + (s.intent ? `
${esc(s.intent)}
` : "") + + `
` + + (rows + ? `
${esc(s.filePath ?? "")}
${rows}
` + : "") + + `
` + ); +} + +/** Emit the GSAP calls for one scene, at absolute times. */ +function sceneTimeline(s: VideoScene, idx: number): string[] { + const id = `#sc${idx}`; + const t = s.start; + const end = s.start + s.dur; + const L: string[] = []; + const at = (x: number) => x.toFixed(3); + + // container in / out + L.push(`tl.fromTo("${id}",{autoAlpha:0},{autoAlpha:1,duration:${FADE}},${at(t)});`); + L.push(`tl.to("${id}",{autoAlpha:0,duration:${FADE}},${at(end - FADE)});`); + L.push(`tl.set("${id}",{autoAlpha:0},${at(end)});`); + + if (s.kind === "title" || s.kind === "outro") { + L.push(`tl.from("${id}-eb",{autoAlpha:0,y:18,duration:0.6},${at(t + 0.3)});`); + L.push(`tl.from("${id}-ti",{autoAlpha:0,y:40,scale:0.96,duration:0.8},${at(t + 0.5)});`); + if (s.subtitle) L.push(`tl.from("${id}-sub",{autoAlpha:0,y:22,duration:0.6},${at(t + 1.2)});`); + return L; + } + + // header + L.push(`tl.from("${id}-hdr",{autoAlpha:0,y:22,duration:0.6},${at(t + HDR_IN)});`); + + // code panel + staggered row reveal + const nRows = s.rows?.length ?? 0; + let revealEnd = t; + if (nRows) { + const codeAt = t + CODE_IN; + L.push(`tl.from("${id}-code",{autoAlpha:0,y:26,duration:0.6},${at(codeAt)});`); + L.push(`tl.from("${id} .row",{autoAlpha:0,x:-24,duration:${ROW_DUR},stagger:${ROW_STAGGER}},${at(codeAt + 0.3)});`); + revealEnd = codeAt + 0.3 + nRows * ROW_STAGGER + ROW_DUR; + + if (s.hasSpots) { + // spotlight moment: after reveal, but tied to the narration's middle + let spotAt = Math.max(revealEnd + 0.2, s.voStart + s.voDur * 0.32); + spotAt = Math.min(spotAt, end - 1.2); + L.push(`tl.to("${id} .row:not(.spot)",{opacity:0.28,duration:0.5},${at(spotAt)});`); + L.push(`tl.to("${id} .row.spot",{backgroundColor:"rgba(63,185,80,0.16)",duration:0.5},${at(spotAt)});`); + L.push(`tl.fromTo("${id} .row.spot .sb",{scaleY:0},{scaleY:1,duration:0.45,stagger:0.05,ease:"power2.out"},${at(spotAt)});`); + } + } + + // Narration is delivered as a soft subtitle track (see hyperframes.ts), not + // burned into the frame — so there are no on-screen caption tweens here. + return L; +} + +export function buildComposition(scenes: VideoScene[], totalDur: number): string { + let z = 10; + const sceneEls = scenes.map((s, i) => sceneHtml(s, i, z++)).join("\n "); + const audioEls = scenes + .filter((s) => s.audio) + .map( + (s, i) => + ``, + ) + .join("\n "); + + const tlCalls = scenes.flatMap((s, i) => sceneTimeline(s, i)).join("\n "); + + return ` + + + + + + + + +
+
+ ${sceneEls} + ${audioEls} +
+ + + +`; +} diff --git a/packages/renderer/src/video/hyperframes.ts b/packages/renderer/src/video/hyperframes.ts new file mode 100644 index 0000000..0551703 --- /dev/null +++ b/packages/renderer/src/video/hyperframes.ts @@ -0,0 +1,229 @@ +/** + * The animated video engine: generate a HyperFrames composition from the + * walkthrough and render it to MP4. HyperFrames (HTML→video via headless Chrome, + * frame-accurate) and a TTS engine are invoked through `npx` — no npm runtime + * dependency is added to patchstory itself. + */ + +import { spawnSync } from "node:child_process"; +import { mkdirSync, writeFileSync, existsSync, rmSync } from "node:fs"; +import { join, resolve, dirname, basename } from "node:path"; +import type { WalkthroughBundle, DiffFile, Chapter } from "@patchstory/core"; +import { highlightLine, langForPath } from "../../web/highlight.ts"; +import { + resolveTool, + ffprobeDuration, + synth, + orderedChapters, + narrationFor, + estimateDuration, + HYPERFRAMES_VERSION, + type VideoOptions, + type VideoResult, + type TtsProvider, +} from "./index.ts"; +import { buildComposition, type VideoScene, type CodeRow } from "./composition.ts"; + +const MAX_ROWS = 13; + +function escHtml(s: string): string { + return s.replace(/&/g, "&").replace(//g, ">"); +} + +/** Pick the chapter's most relevant file and window its lines around the spotlight. */ +function rowsForChapter( + c: Chapter, + fileByPath: Map, +): { rows: CodeRow[]; filePath: string; hasSpots: boolean } { + const refs = new Map>(); + for (const r of c.diff_hunks ?? []) { + const arr = refs.get(r.file) ?? []; + arr.push([r.start_line, r.end_line]); + refs.set(r.file, arr); + } + + // Prefer the first file with referenced ranges, else the first with hunks. + const candidate = + c.files.find((p) => refs.has(p) && fileByPath.get(p)?.hunks.length) ?? + c.files.find((p) => fileByPath.get(p)?.hunks.length); + const f = candidate ? fileByPath.get(candidate) : undefined; + if (!f) return { rows: [], filePath: c.files[0] ?? "", hasSpots: false }; + + const fileRefs = refs.get(f.path) ?? null; + const lang = langForPath(f.path); + const overlapping = fileRefs + ? f.hunks.filter((h) => fileRefs.some(([s, e]) => h.newStart <= e && h.newStart + h.newLines >= s)) + : f.hunks; + const hunks = overlapping.length ? overlapping : f.hunks; + + const all: CodeRow[] = []; + for (const h of hunks) { + for (const l of h.lines) { + const spot = + !!fileRefs && l.newNumber != null && fileRefs.some(([s, e]) => l.newNumber! >= s && l.newNumber! <= e); + all.push({ + sign: l.type === "add" ? "+" : l.type === "del" ? "-" : " ", + num: l.newNumber, + html: lang ? highlightLine(l.content, lang) : escHtml(l.content), + spot, + }); + } + } + + // Window MAX_ROWS lines centered on the first spotlight (or first add). + let rows = all; + if (all.length > MAX_ROWS) { + let focus = all.findIndex((r) => r.spot); + if (focus < 0) focus = all.findIndex((r) => r.sign === "+"); + if (focus < 0) focus = 0; + let start = Math.max(0, focus - Math.floor(MAX_ROWS / 2)); + start = Math.min(start, all.length - MAX_ROWS); + rows = all.slice(start, start + MAX_ROWS); + } + return { rows, filePath: f.path, hasSpots: rows.some((r) => r.spot) }; +} + +export async function renderVideoHyperframes( + bundle: WalkthroughBundle, + opts: VideoOptions, +): Promise { + const log = opts.onProgress ?? (() => {}); + const outFile = resolve(opts.out); + const fps = opts.fps ?? 30; + + // Resolve a *working* ffmpeg/ffprobe and make HyperFrames' child processes use + // them (PATH prefix), with a Homebrew lib fallback for its loader. + const ffmpeg = resolveTool("ffmpeg", opts.ffmpeg); + const ffprobe = resolveTool("ffprobe", opts.ffprobe); + const env: NodeJS.ProcessEnv = { + ...process.env, + PATH: `${dirname(ffmpeg)}:${process.env.PATH ?? ""}`, + }; + const brewLib = "/home/linuxbrew/.linuxbrew/opt/ffmpeg/lib"; + if (existsSync(brewLib)) { + env.LD_LIBRARY_PATH = [brewLib, process.env.LD_LIBRARY_PATH].filter(Boolean).join(":"); + } + + const provider: TtsProvider = + opts.tts && opts.tts !== "auto" + ? opts.tts + : process.env.ELEVENLABS_API_KEY + ? "elevenlabs" + : "kokoro"; + log(`engine: hyperframes · tts: ${provider}`); + + const w = bundle.walkthrough; + const fileByPath = new Map(); + for (const f of bundle.diff.files) fileByPath.set(f.path, f); + + const projDir = join(dirname(outFile), `.ps-hf-${basename(outFile).replace(/\W+/g, "")}`); + rmSync(projDir, { recursive: true, force: true }); + const audioDir = join(projDir, "audio"); + mkdirSync(audioDir, { recursive: true }); + + // 1) Assemble scene descriptors with their narration text. + type Pending = { scene: Omit; voText: string }; + const pending: Pending[] = []; + + pending.push({ + scene: { + kind: "title", + eyebrow: "PatchStory walkthrough", + title: w.title, + subtitle: w.summary, + hasSpots: false, + }, + voText: w.summary, + }); + + const chapters = orderedChapters(w); + chapters.forEach((c, i) => { + const { rows, filePath, hasSpots } = rowsForChapter(c, fileByPath); + const narration = narrationFor(c) || c.summary; + pending.push({ + scene: { + kind: "chapter", + eyebrow: `Chapter ${i + 1} of ${chapters.length}`, + title: c.title, + risk: c.risk_level, + intent: c.intent, + filePath, + rows, + hasSpots, + }, + voText: narration, + }); + }); + + pending.push({ + scene: { + kind: "outro", + eyebrow: "PatchStory", + title: "That's the walkthrough.", + subtitle: "Generated by PatchStory · patchstory render for the interactive version", + hasSpots: false, + }, + voText: "", + }); + + // 2) Synthesize narration, measure, and lay scenes out on the timeline. + const scenes: VideoScene[] = []; + let cursor = 0; + for (let i = 0; i < pending.length; i++) { + const p = pending[i]; + log(`[${i + 1}/${pending.length}] ${p.scene.title}`); + let audioRel: string | null = null; + let voDur = 0; + if (p.voText.trim()) { + const audioPath = await synth(provider, p.voText, join(audioDir, `line${i}`), opts.voice, env); + if (audioPath) { + audioRel = `audio/${basename(audioPath)}`; + voDur = ffprobeDuration(ffprobe, audioPath); + } + if (!voDur) voDur = estimateDuration(p.voText); + } + + const intro = 0.6; + const tail = p.scene.kind === "chapter" ? 1.0 : 0.9; + const dur = p.voText.trim() ? intro + voDur + tail : 3.4; + scenes.push({ ...p.scene, start: cursor, dur, voStart: cursor + intro, voDur, audio: audioRel }); + cursor += dur; + } + const total = cursor; + + // 3) Write the composition project and render it. + writeFileSync(join(projDir, "index.html"), buildComposition(scenes, total)); + writeFileSync( + join(projDir, "hyperframes.json"), + JSON.stringify( + { + $schema: "https://hyperframes.heygen.com/schema/hyperframes.json", + paths: { blocks: "compositions", components: "compositions/components", assets: "assets" }, + }, + null, + 2, + ), + ); + writeFileSync(join(projDir, "meta.json"), JSON.stringify({ id: "patchstory", name: "patchstory" })); + + log(`rendering ${scenes.length} scenes (~${Math.round(total)}s) with hyperframes…`); + mkdirSync(dirname(outFile), { recursive: true }); + + const r = spawnSync( + "npx", + ["--yes", `hyperframes@${HYPERFRAMES_VERSION}`, "render", projDir, "-o", outFile, "-q", "high", "-f", String(fps)], + { encoding: "utf8", env, maxBuffer: 64 * 1024 * 1024, stdio: ["ignore", "pipe", "pipe"] }, + ); + if (r.status !== 0 || !existsSync(outFile)) { + const tail = ((r.stderr ?? "") + (r.stdout ?? "")).split("\n").slice(-12).join("\n"); + throw new Error( + `hyperframes render failed.\n${tail}\n\n` + + "Tip: ensure network access for `npx hyperframes`, or use --engine pan for the offline ffmpeg renderer.", + ); + } + + if (!opts.keep) rmSync(projDir, { recursive: true, force: true }); + else log(`project kept in ${projDir}`); + + return { file: outFile, sceneCount: scenes.length, durationSec: total, ttsProvider: provider }; +} diff --git a/packages/renderer/src/video/index.ts b/packages/renderer/src/video/index.ts new file mode 100644 index 0000000..fde63fe --- /dev/null +++ b/packages/renderer/src/video/index.ts @@ -0,0 +1,524 @@ +/** + * Render a walkthrough to a narrated MP4 — the opt-in counterpart to the + * in-page "play" mode. Same scene model, but produced as a real, shareable + * video file using *system* tools (headless Chromium + ffmpeg + a TTS engine), + * so it adds no npm runtime dependencies. The heavy tools are only touched when + * someone asks for a video. + * + * Pipeline, per scene: + * 1. Build a deterministic HTML scene (fixed header/caption bands). + * 2. Headless-screenshot it to one tall PNG. + * 3. Synthesize narration audio (elevenlabs | espeak-ng | flite | say | none). + * 4. ffmpeg: slice the PNG into a fixed header, a vertically-panning code + * region, and a fixed caption, compose onto a frame, mux the audio. + * Then concat the per-scene clips into the final MP4. + */ + +import { spawnSync } from "node:child_process"; +import { + mkdirSync, + writeFileSync, + existsSync, + rmSync, +} from "node:fs"; +import { join, resolve, dirname, basename } from "node:path"; +import type { WalkthroughBundle, DiffFile, Chapter } from "@patchstory/core"; +import { + buildSceneHtml, + blocksForChapter, + sceneHeight, + FRAME_W, + FRAME_H, + HEADER_H, + CAPTION_H, + MID_H, + type SceneSpec, +} from "./scene-html.ts"; + +export type TtsProvider = + | "auto" + | "elevenlabs" + | "kokoro" + | "espeak-ng" + | "flite" + | "say" + | "none"; + +/** Pinned so renders are reproducible. */ +export const HYPERFRAMES_VERSION = "0.7.11"; + +export type VideoEngine = "hyperframes" | "pan"; + +export interface VideoOptions { + /** Output .mp4 path. */ + out: string; + /** "hyperframes" = animated GSAP scenes (default); "pan" = static screenshot pan. */ + engine?: VideoEngine; + tts?: TtsProvider; + /** Voice id (elevenlabs) or voice name (espeak-ng/say). */ + voice?: string; + /** Chrome/Chromium binary override. */ + chrome?: string; + /** ffmpeg / ffprobe binary overrides (else resolved from PATH / system). */ + ffmpeg?: string; + ffprobe?: string; + fps?: number; + /** Max diff lines shown per scene (keeps pans sane). */ + maxLinesPerScene?: number; + /** Keep the intermediate working directory. */ + keep?: boolean; + onProgress?: (msg: string) => void; +} + +export interface VideoResult { + file: string; + sceneCount: number; + durationSec: number; + ttsProvider: TtsProvider; +} + +/* ------------------------------ tool helpers ----------------------------- */ + +function has(bin: string): boolean { + const r = spawnSync(bin, ["--version"], { stdio: "ignore" }); + return !r.error && (r.status === 0 || r.status === 1); // some print version on stderr/exit 1 +} + +/** + * Resolve a tool to a binary that actually *runs* ` -version` cleanly. + * Guards against a broken/shadowing PATH entry (e.g. a half-upgraded Homebrew + * ffmpeg missing shared libs) by validating each candidate and falling back to + * the system location. Honors an explicit override / env var first. + */ +export function resolveTool(name: string, override?: string): string { + const candidates = [ + override, + process.env[`PATCHSTORY_${name.toUpperCase()}`], + name, + `/usr/bin/${name}`, + `/usr/local/bin/${name}`, + `/opt/homebrew/bin/${name}`, + ].filter((c): c is string => !!c); + for (const c of candidates) { + const r = spawnSync(c, ["-version"], { stdio: "ignore" }); + if (!r.error && r.status === 0) return c; + } + throw new Error( + `${name} not found or not runnable (required for \`patchstory video\`). ` + + `Install ffmpeg, or set PATCHSTORY_${name.toUpperCase()} to a working binary.`, + ); +} + +function hasFlatpakChromium(): boolean { + const r = spawnSync("flatpak", ["info", "org.chromium.Chromium"], { stdio: "ignore" }); + return !r.error && r.status === 0; +} + +interface ChromeRunner { + label: string; + argv: (args: string[]) => { cmd: string; args: string[] }; +} + +function resolveChrome(workDir: string, override?: string): ChromeRunner { + const candidates = override + ? [override] + : [ + process.env.PATCHSTORY_CHROME ?? "", + "google-chrome", + "google-chrome-stable", + "chromium", + "chromium-browser", + "chrome", + ].filter(Boolean); + + for (const c of candidates) { + const r = spawnSync(c, ["--version"], { stdio: "ignore" }); + if (!r.error) return { label: c, argv: (args) => ({ cmd: c, args }) }; + } + if (hasFlatpakChromium()) { + return { + label: "flatpak org.chromium.Chromium", + argv: (args) => ({ + cmd: "flatpak", + args: ["run", `--filesystem=${workDir}`, "org.chromium.Chromium", ...args], + }), + }; + } + throw new Error( + "no Chrome/Chromium found. Install Chromium (or pass --chrome , " + + "or set PATCHSTORY_CHROME). On Linux, `flatpak install org.chromium.Chromium` also works.", + ); +} + +const CHROME_BASE = [ + "--headless=new", + "--no-sandbox", + "--disable-gpu", + "--hide-scrollbars", + "--force-device-scale-factor=1", + "--no-first-run", + "--no-default-browser-check", + "--disable-extensions", +]; + +function chromeMeasureHeight( + chrome: ChromeRunner, + htmlPath: string, + profileDir: string, + fallback: number, +): number { + const { cmd, args } = chrome.argv([ + ...CHROME_BASE, + `--user-data-dir=${profileDir}`, + `--window-size=${FRAME_W},200`, + "--virtual-time-budget=2500", + "--dump-dom", + `file://${htmlPath}`, + ]); + const r = spawnSync(cmd, args, { encoding: "utf8", maxBuffer: 96 * 1024 * 1024 }); + const m = (r.stdout ?? "").match(/(\d+)<\/title>/); + const h = m ? parseInt(m[1], 10) : NaN; + return Number.isFinite(h) && h > HEADER_H + CAPTION_H ? h : fallback; +} + +function chromeScreenshot( + chrome: ChromeRunner, + htmlPath: string, + outPng: string, + height: number, + profileDir: string, +): void { + const { cmd, args } = chrome.argv([ + ...CHROME_BASE, + `--user-data-dir=${profileDir}`, + `--window-size=${FRAME_W},${height}`, + "--virtual-time-budget=2500", + `--screenshot=${outPng}`, + `file://${htmlPath}`, + ]); + const r = spawnSync(cmd, args, { encoding: "utf8" }); + if (!existsSync(outPng)) { + throw new Error( + `headless screenshot failed (${chrome.label}). ${(r.stderr ?? "").split("\n").slice(-3).join(" ").trim()}`, + ); + } +} + +/* --------------------------------- ffmpeg -------------------------------- */ + +export function ffprobeDuration(ffprobe: string, path: string): number { + const r = spawnSync( + ffprobe, + ["-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", path], + { encoding: "utf8" }, + ); + const d = parseFloat((r.stdout ?? "").trim()); + return Number.isFinite(d) ? d : 0; +} + +function ffprobeHeight(ffprobe: string, path: string): number { + const r = spawnSync( + ffprobe, + ["-v", "error", "-select_streams", "v:0", "-show_entries", "stream=height", "-of", "csv=p=0", path], + { encoding: "utf8" }, + ); + const h = parseInt((r.stdout ?? "").trim(), 10); + return Number.isFinite(h) ? h : 0; +} + +/** Build one scene clip: pan the code between a fixed header and caption. */ +function buildClip( + ffmpeg: string, + scenePng: string, + pngHeight: number, + audioPath: string | null, + durationSec: number, + fps: number, + clipOut: string, +): void { + const W = FRAME_W; + const codeH = Math.max(0, pngHeight - HEADER_H - CAPTION_H); + const D = durationSec.toFixed(3); + + // Crop the panning code window out of the screenshot's middle band. + let midCrop: string; + if (codeH > MID_H) { + const panRange = codeH - MID_H; + // y travels from the top of the code (HEADER_H) down by panRange over D. + midCrop = `crop=${W}:${MID_H}:0:'${HEADER_H}+${panRange}*min(t/${D}\\,1)'`; + } else { + // Short scene: no pan; show the code at the top of the gap. + midCrop = `crop=${W}:${Math.max(2, codeH)}:0:${HEADER_H}`; + } + + const midOverlayY = HEADER_H; + const filter = + `[0:v]split=3[a][b][c];` + + `[a]crop=${W}:${HEADER_H}:0:0[hd];` + + `[b]crop=${W}:${CAPTION_H}:0:${pngHeight - CAPTION_H}[cap];` + + `[c]${midCrop}[mid];` + + `color=c=0x0d1117:s=${W}x${FRAME_H}:r=${fps}:d=${D}[bg];` + + `[bg][mid]overlay=0:${midOverlayY}[m1];` + + `[m1][hd]overlay=0:0[m2];` + + `[m2][cap]overlay=0:${FRAME_H - CAPTION_H},format=yuv420p[v]`; + + const args: string[] = ["-y", "-loop", "1", "-framerate", String(fps), "-i", scenePng]; + if (audioPath) { + args.push("-i", audioPath); + } else { + // Silent track so every clip has uniform streams (clean concat -c copy). + args.push("-f", "lavfi", "-i", "anullsrc=channel_layout=stereo:sample_rate=44100"); + } + args.push( + "-filter_complex", filter, + "-map", "[v]", + "-map", "1:a", + "-t", D, + "-r", String(fps), + "-c:v", "libx264", + "-pix_fmt", "yuv420p", + "-profile:v", "high", + "-preset", "veryfast", + "-c:a", "aac", + "-b:a", "160k", + "-ar", "44100", + clipOut, + ); + + const r = spawnSync(ffmpeg, args, { encoding: "utf8" }); + if (r.status !== 0 || !existsSync(clipOut)) { + throw new Error(`ffmpeg failed building a scene clip:\n${(r.stderr ?? "").split("\n").slice(-6).join("\n")}`); + } +} + +function concatClips(ffmpeg: string, clips: string[], outFile: string, workDir: string): void { + const listPath = join(workDir, "clips.txt"); + writeFileSync(listPath, clips.map((c) => `file '${c.replace(/'/g, "'\\''")}'`).join("\n") + "\n"); + const r = spawnSync( + ffmpeg, + ["-y", "-f", "concat", "-safe", "0", "-i", listPath, "-c", "copy", "-movflags", "+faststart", outFile], + { encoding: "utf8" }, + ); + if (r.status !== 0 || !existsSync(outFile)) { + throw new Error(`ffmpeg concat failed:\n${(r.stderr ?? "").split("\n").slice(-6).join("\n")}`); + } +} + +/* ----------------------------------- TTS --------------------------------- */ + +function pickTts(requested: TtsProvider | undefined): TtsProvider { + if (requested && requested !== "auto") return requested; + if (process.env.ELEVENLABS_API_KEY) return "elevenlabs"; + if (has("espeak-ng")) return "espeak-ng"; + if (has("flite")) return "flite"; + if (process.platform === "darwin") return "say"; + return "none"; +} + +export async function synth( + provider: TtsProvider, + text: string, + outBase: string, + voice: string | undefined, + env?: NodeJS.ProcessEnv, +): Promise<string | null> { + if (provider === "none" || provider === "auto" || !text.trim()) return null; + + if (provider === "elevenlabs") { + const key = process.env.ELEVENLABS_API_KEY; + if (!key) throw new Error("ELEVENLABS_API_KEY is not set (needed for --tts elevenlabs)."); + const voiceId = voice || "21m00Tcm4TlvDq8ikWAM"; // Rachel + const res = await fetch(`https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`, { + method: "POST", + headers: { "xi-api-key": key, "content-type": "application/json", accept: "audio/mpeg" }, + body: JSON.stringify({ text, model_id: "eleven_multilingual_v2" }), + }); + if (!res.ok) { + throw new Error(`ElevenLabs API error ${res.status}: ${(await res.text()).slice(0, 200)}`); + } + const out = `${outBase}.mp3`; + writeFileSync(out, Buffer.from(await res.arrayBuffer())); + return out; + } + + if (provider === "kokoro") { + // Local neural TTS via hyperframes' bundled Kokoro — no API key. + const out = `${outBase}.wav`; + const r = spawnSync( + "npx", + ["--yes", `hyperframes@${HYPERFRAMES_VERSION}`, "tts", text, "-o", out, "-v", voice || "af_heart", "-s", "0.97"], + { encoding: "utf8", env: env ?? process.env, maxBuffer: 16 * 1024 * 1024 }, + ); + if (r.status !== 0 || !existsSync(out)) { + throw new Error(`kokoro (hyperframes tts) failed: ${(r.stderr ?? "").slice(-300)}`); + } + return out; + } + + // Local engines read the text from a file (avoids any shell-quoting issues). + const txtPath = `${outBase}.txt`; + writeFileSync(txtPath, text); + const opt = { encoding: "utf8" as const, env: env ?? process.env }; + + if (provider === "espeak-ng") { + const out = `${outBase}.wav`; + const r = spawnSync("espeak-ng", ["-v", voice || "en-us", "-s", "165", "-w", out, "-f", txtPath], opt); + if (r.status !== 0 || !existsSync(out)) throw new Error(`espeak-ng failed: ${r.stderr ?? ""}`); + return out; + } + if (provider === "flite") { + const out = `${outBase}.wav`; + const r = spawnSync("flite", ["-f", txtPath, "-o", out], opt); + if (r.status !== 0 || !existsSync(out)) throw new Error(`flite failed: ${r.stderr ?? ""}`); + return out; + } + if (provider === "say") { + const out = `${outBase}.aiff`; + const a = ["-o", out, "-f", txtPath]; + if (voice) a.unshift("-v", voice); + const r = spawnSync("say", a, opt); + if (r.status !== 0 || !existsSync(out)) throw new Error(`say failed: ${r.stderr ?? ""}`); + return out; + } + return null; +} + +/* --------------------------------- scenes -------------------------------- */ + +export function orderedChapters(w: WalkthroughBundle["walkthrough"]): Chapter[] { + if (w.reviewer_path?.length) { + const byId = new Map(w.chapters.map((c) => [c.id, c])); + const seen = new Set<string>(); + const out: Chapter[] = []; + for (const id of w.reviewer_path) { + const c = byId.get(id); + if (c && !seen.has(id)) { + out.push(c); + seen.add(id); + } + } + for (const c of w.chapters) if (!seen.has(c.id)) out.push(c); + return out; + } + return w.chapters; +} + +export function narrationFor(c: Chapter): string { + if (c.narration && c.narration.trim()) return c.narration.trim(); + return [c.intent, c.summary].filter(Boolean).join(" ").trim(); +} + +export function estimateDuration(text: string): number { + const words = text.trim() ? text.trim().split(/\s+/).length : 0; + return Math.min(18, Math.max(3.5, words / 2.6)); +} + +/* --------------------------------- driver -------------------------------- */ + +export async function renderVideo( + bundle: WalkthroughBundle, + opts: VideoOptions, +): Promise<VideoResult> { + const engine = opts.engine ?? "hyperframes"; + if (engine === "pan") return renderVideoPan(bundle, opts); + // Dynamic import keeps the static module graph acyclic (hyperframes.ts pulls + // its shared helpers from here). + const { renderVideoHyperframes } = await import("./hyperframes.ts"); + return renderVideoHyperframes(bundle, opts); +} + +/** The static-screenshot + ffmpeg-pan engine (`--engine pan`). */ +async function renderVideoPan( + bundle: WalkthroughBundle, + opts: VideoOptions, +): Promise<VideoResult> { + const log = opts.onProgress ?? (() => {}); + const fps = opts.fps ?? 30; + const maxLines = opts.maxLinesPerScene ?? 80; + const outFile = resolve(opts.out); + + const ffmpeg = resolveTool("ffmpeg", opts.ffmpeg); + const ffprobe = resolveTool("ffprobe", opts.ffprobe); + + const w = bundle.walkthrough; + const fileByPath = new Map<string, DiffFile>(); + for (const f of bundle.diff.files) fileByPath.set(f.path, f); + + const workDir = join(dirname(outFile), `.ps-video-${basename(outFile).replace(/\W+/g, "")}`); + rmSync(workDir, { recursive: true, force: true }); + mkdirSync(workDir, { recursive: true }); + const profileDir = join(workDir, "chrome-profile"); + + const chrome = resolveChrome(workDir, opts.chrome); + log(`chrome: ${chrome.label}`); + const provider = pickTts(opts.tts); + log(`tts: ${provider}`); + + // Build the scene list: a title card, then one scene per chapter. + const chapters = orderedChapters(w); + const specs: SceneSpec[] = []; + + specs.push({ + eyebrow: "PatchStory walkthrough", + title: w.title, + caption: w.summary, + blocks: [], + hasSpots: false, + }); + + chapters.forEach((c, i) => { + const refs = new Map<string, Array<[number, number]>>(); + for (const r of c.diff_hunks ?? []) { + const arr = refs.get(r.file) ?? []; + arr.push([r.start_line, r.end_line]); + refs.set(r.file, arr); + } + const { blocks, hasSpots } = blocksForChapter(c.files, refs, fileByPath, maxLines); + specs.push({ + eyebrow: `Chapter ${i + 1} of ${chapters.length}`, + title: c.title, + risk: c.risk_level, + intent: c.intent, + caption: narrationFor(c) || c.summary, + blocks, + hasSpots, + }); + }); + + const clips: string[] = []; + let total = 0; + + for (let i = 0; i < specs.length; i++) { + const spec = specs[i]; + const tag = `scene-${String(i).padStart(2, "0")}`; + log(`[${i + 1}/${specs.length}] ${spec.title}`); + + // 1) HTML → screenshot + const htmlPath = join(workDir, `${tag}.html`); + writeFileSync(htmlPath, buildSceneHtml(spec)); + const measured = chromeMeasureHeight(chrome, htmlPath, profileDir, sceneHeight(spec)); + const pngPath = join(workDir, `${tag}.png`); + chromeScreenshot(chrome, htmlPath, pngPath, measured, profileDir); + const pngH = ffprobeHeight(ffprobe, pngPath) || measured; + + // 2) narration → audio (+ duration) + const audioBase = join(workDir, `${tag}-audio`); + const audio = await synth(provider, spec.caption, audioBase, opts.voice); + const dur = (audio ? ffprobeDuration(ffprobe, audio) : estimateDuration(spec.caption)) + 0.6; + + // 3) compose the clip + const clip = join(workDir, `${tag}.mp4`); + buildClip(ffmpeg, pngPath, pngH, audio, dur, fps, clip); + clips.push(clip); + total += dur; + } + + log(`concatenating ${clips.length} clips…`); + mkdirSync(dirname(outFile), { recursive: true }); + concatClips(ffmpeg, clips, outFile, workDir); + + if (!opts.keep) rmSync(workDir, { recursive: true, force: true }); + else log(`intermediates kept in ${workDir}`); + + return { file: outFile, sceneCount: specs.length, durationSec: total, ttsProvider: provider }; +} diff --git a/packages/renderer/src/video/scene-html.ts b/packages/renderer/src/video/scene-html.ts new file mode 100644 index 0000000..4f01780 --- /dev/null +++ b/packages/renderer/src/video/scene-html.ts @@ -0,0 +1,244 @@ +/** + * Static, self-contained HTML for a single *video* scene. + * + * Unlike the interactive web client, these pages are deterministic and meant to + * be rasterized once by a headless browser. The layout is deliberately built + * from FIXED pixel heights: a header band of exactly {@link HEADER_H}px at the + * top, a caption band of exactly {@link CAPTION_H}px at the bottom, and the code + * in between. Because those two bands are pixel-exact, the video pipeline can + * slice the one tall screenshot into "fixed header / panning code / fixed + * caption" with simple top/bottom crops — no per-element measurement needed. + * + * Theme is always dark (it reads best as video). Syntax highlighting reuses the + * same per-line highlighter as the interactive renderer. + */ + +import { highlightLine, langForPath } from "../../web/highlight.ts"; +import type { DiffFile } from "@patchstory/core"; + +/* --- Fixed geometry. The pipeline imports these to compute crop offsets. --- */ +export const FRAME_W = 1920; +export const FRAME_H = 1080; +export const HEADER_H = 230; +export const CAPTION_H = 210; +export const MID_H = FRAME_H - HEADER_H - CAPTION_H; // 640 — the code viewport +export const LINE_H = 40; +export const FILE_HEAD_H = 58; +export const CODE_PAD_V = 28; +export const FILE_GAP = 16; + +export interface SceneLine { + sign: " " | "+" | "-"; + /** New-file line number, or null for deletions. */ + num: number | null; + /** Raw (un-highlighted) line content. */ + content: string; + spot: boolean; +} + +export interface SceneFileBlock { + path: string; + status: string; + lines: SceneLine[]; +} + +export interface SceneSpec { + eyebrow: string; + title: string; + risk?: "low" | "medium" | "high"; + intent?: string; + /** Narration text, shown as the fixed caption band. */ + caption: string; + blocks: SceneFileBlock[]; + hasSpots: boolean; +} + +/** Deterministic content height of the code area, given its blocks. The + * pipeline reads the *actual* height from the screenshot, but this is a good + * window-size hint so the headless capture isn't clipped. */ +export function codeHeight(blocks: SceneFileBlock[]): number { + let h = CODE_PAD_V * 2; + blocks.forEach((b, i) => { + h += FILE_HEAD_H + b.lines.length * LINE_H; + if (i > 0) h += FILE_GAP; + }); + return h; +} + +export function sceneHeight(spec: SceneSpec): number { + return HEADER_H + (spec.blocks.length ? codeHeight(spec.blocks) : MID_H) + CAPTION_H; +} + +function esc(s: string): string { + return s.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">"); +} + +function statusTag(status: string): string { + return `<span class="tag tag-${esc(status)}">${esc(status)}</span>`; +} + +function lineRow(l: SceneLine, lang: string | null): string { + const cls = l.sign === "+" ? "add" : l.sign === "-" ? "del" : "ctx"; + const html = lang ? highlightLine(l.content, lang) : esc(l.content); + return ( + `<div class="row ${cls}${l.spot ? " spot" : ""}">` + + `<span class="gut">${l.num != null ? l.num : ""}</span>` + + `<span class="sgn">${l.sign === " " ? "" : l.sign}</span>` + + `<span class="txt">${html || " "}</span>` + + `</div>` + ); +} + +function fileBlock(b: SceneFileBlock): string { + const lang = langForPath(b.path); + const rows = b.lines.map((l) => lineRow(l, lang)).join(""); + return ( + `<div class="fileblock">` + + `<div class="fhead">${statusTag(b.status)}<span class="fpath">${esc(b.path)}</span></div>` + + `<div class="rows">${rows}</div>` + + `</div>` + ); +} + +const CSS = ` +*{box-sizing:border-box;margin:0;padding:0} +html,body{width:${FRAME_W}px;background:#0d1117;color:#e6edf3; + font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif; + -webkit-font-smoothing:antialiased} +.mono{font-family:ui-monospace,"SF Mono",Menlo,Consolas,"DejaVu Sans Mono",monospace} + +.header{height:${HEADER_H}px;padding:34px 64px 0;overflow:hidden; + background:linear-gradient(180deg,#161b22 0%,#0d1117 100%);border-bottom:1px solid #21262d} +.eyebrow-row{display:flex;align-items:center;gap:16px;margin-bottom:14px} +.eyebrow{font-size:22px;letter-spacing:2px;text-transform:uppercase;color:#768390;font-weight:700} +.pill{font-size:18px;font-weight:800;text-transform:uppercase;letter-spacing:1px;padding:5px 14px;border-radius:14px} +.pill-low{background:#12261e;color:#3fb950} +.pill-medium{background:#272115;color:#d29922} +.pill-high{background:#25171c;color:#f85149} +.title{font-size:48px;line-height:1.12;font-weight:800;letter-spacing:-0.5px; + display:-webkit-box;-webkit-line-clamp:2;-webkit-box-orient:vertical;overflow:hidden} +.intent{margin-top:10px;font-size:26px;line-height:1.3;color:#adbac7; + display:-webkit-box;-webkit-line-clamp:1;-webkit-box-orient:vertical;overflow:hidden} + +.code{padding:${CODE_PAD_V}px 64px} +.fileblock{border-radius:12px;overflow:hidden;background:#161b22} +.fileblock+.fileblock{margin-top:${FILE_GAP}px} +.fhead{height:${FILE_HEAD_H}px;display:flex;align-items:center;gap:14px;padding:0 20px;background:#1c2430} +.fpath{font-family:ui-monospace,Menlo,Consolas,"DejaVu Sans Mono",monospace;font-size:24px;color:#adbac7} +.tag{font-size:17px;font-weight:700;text-transform:uppercase;letter-spacing:.5px;padding:3px 9px;border-radius:6px} +.tag-added{background:#12261e;color:#3fb950} +.tag-deleted{background:#25171c;color:#f85149} +.tag-modified{background:#11243e;color:#4493f8} +.tag-renamed,.tag-copied{background:#272115;color:#d29922} + +.row{height:${LINE_H}px;display:flex;align-items:center; + font-family:ui-monospace,Menlo,Consolas,"DejaVu Sans Mono",monospace;font-size:24px;line-height:${LINE_H}px;white-space:pre} +.gut{width:84px;flex:none;text-align:right;padding-right:18px;color:#586069; + font-variant-numeric:tabular-nums;overflow:hidden} +.sgn{width:30px;flex:none;text-align:center} +.txt{flex:1;overflow:hidden;padding-right:24px} +.row.add{background:#12261e}.row.add .sgn{color:#3fb950} +.row.del{background:#25171c}.row.del .sgn{color:#f85149} +.code.spots .row{opacity:.38} +.code.spots .row.spot{opacity:1;box-shadow:inset 4px 0 0 #4493f8} + +.caption{height:${CAPTION_H}px;display:flex;align-items:center;justify-content:center; + text-align:center;padding:24px 120px;background:#161b22;border-top:1px solid #21262d} +.caption span{font-size:33px;line-height:1.34;color:#e6edf3; + display:-webkit-box;-webkit-line-clamp:4;-webkit-box-orient:vertical;overflow:hidden} + +/* GitHub-dark syntax tokens */ +.hljs-keyword,.hljs-selector-tag,.hljs-literal,.hljs-section,.hljs-doctag{color:#ff7b72} +.hljs-string,.hljs-regexp,.hljs-meta .hljs-string,.hljs-addition{color:#a5d6ff} +.hljs-comment,.hljs-quote{color:#8b949e;font-style:italic} +.hljs-number,.hljs-symbol,.hljs-bullet,.hljs-selector-id{color:#79c0ff} +.hljs-title,.hljs-title.function_,.hljs-title.class_{color:#d2a8ff} +.hljs-type,.hljs-class .hljs-title,.hljs-built_in{color:#ffa657} +.hljs-attr,.hljs-attribute,.hljs-variable,.hljs-template-variable,.hljs-property, +.hljs-params,.hljs-selector-attr,.hljs-selector-class{color:#79c0ff} +.hljs-name,.hljs-tag,.hljs-selector-pseudo{color:#7ee787} +.hljs-meta{color:#8b949e} +`; + +/** Build the full self-contained HTML for one scene. */ +export function buildSceneHtml(spec: SceneSpec): string { + const header = + `<div class="header">` + + `<div class="eyebrow-row">` + + `<span class="eyebrow">${esc(spec.eyebrow)}</span>` + + (spec.risk ? `<span class="pill pill-${spec.risk}">${esc(spec.risk)}</span>` : "") + + `</div>` + + `<div class="title">${esc(spec.title)}</div>` + + (spec.intent ? `<div class="intent">${esc(spec.intent)}</div>` : "") + + `</div>`; + + const code = spec.blocks.length + ? `<div class="code${spec.hasSpots ? " spots" : ""}">${spec.blocks.map(fileBlock).join("")}</div>` + : `<div class="code" style="height:${MID_H}px"></div>`; + + const caption = `<div class="caption"><span>${esc(spec.caption)}</span></div>`; + + return ( + `<!doctype html><html><head><meta charset="utf-8"><style>${CSS}</style></head>` + + `<body>${header}${code}${caption}` + + // Report the rendered height so the pipeline can capture without clipping. + `<script>document.title=String(document.documentElement.scrollHeight)</script>` + + `</body></html>` + ); +} + +/** + * Turn a chapter's referenced files into renderable code blocks, mirroring the + * interactive player's logic: prefer the hunks overlapping the referenced line + * ranges, and spotlight the exact referenced lines. + */ +export function blocksForChapter( + files: string[], + refs: Map<string, Array<[number, number]>>, + fileByPath: Map<string, DiffFile>, + maxLines: number, +): { blocks: SceneFileBlock[]; hasSpots: boolean; truncated: boolean } { + const blocks: SceneFileBlock[] = []; + let hasSpots = false; + let budget = maxLines; + let truncated = false; + + for (const path of files) { + if (budget <= 0) { + truncated = true; + break; + } + const f = fileByPath.get(path); + if (!f || f.binary || !f.hunks.length) continue; + const fileRefs = refs.get(path) ?? null; + const overlapping = fileRefs + ? f.hunks.filter((h) => fileRefs.some(([s, e]) => h.newStart <= e && h.newStart + h.newLines >= s)) + : f.hunks; + const hunks = overlapping.length ? overlapping : f.hunks; + + const lines: SceneLine[] = []; + for (const h of hunks) { + for (const l of h.lines) { + if (budget <= 0) { + truncated = true; + break; + } + const spot = + !!fileRefs && + l.newNumber != null && + fileRefs.some(([s, e]) => l.newNumber! >= s && l.newNumber! <= e); + if (spot) hasSpots = true; + lines.push({ + sign: l.type === "add" ? "+" : l.type === "del" ? "-" : " ", + num: l.newNumber, + content: l.content, + spot, + }); + budget--; + } + if (budget <= 0) break; + } + if (lines.length) blocks.push({ path: f.path, status: f.status, lines }); + } + return { blocks, hasSpots, truncated }; +} diff --git a/packages/renderer/web/app.ts b/packages/renderer/web/app.ts index 8efde07..9ef18eb 100644 --- a/packages/renderer/web/app.ts +++ b/packages/renderer/web/app.ts @@ -1076,6 +1076,9 @@ function onKey(e: KeyboardEvent) { case "e": setAllHunks(false); break; case "c": setAllHunks(true); break; case "r": toggleCurrentReviewed(); break; + case "p": openPlayer(parseRoute().name === "chapter" + ? Math.max(0, orderedChapters().findIndex((c) => c.id === parseRoute().param)) + : 0); break; case "?": setHelp(true); break; } } @@ -1106,6 +1109,7 @@ function buildHelpOverlay(): HTMLElement { kbdRow("/", "Focus search"), kbdRow("e / c", "Expand / collapse all hunks"), kbdRow("r", "Toggle reviewed (current chapter/file)"), + kbdRow("p", "Play narrated walkthrough"), kbdRow("t", "Toggle light / dark"), kbdRow("?", "Show this help"), kbdRow("Esc", "Close"), @@ -1179,6 +1183,17 @@ function mountChrome() { "Copy summary", ); + const playBtn = el( + "button", + { + class: "btn small playbtn", + type: "button", + title: "Play narrated walkthrough (p)", + onclick: () => openPlayer(0), + }, + "▶ Play", + ); + // Mobile drawer toggle (hidden on wide screens via CSS). const menuBtn = el( "button", @@ -1205,6 +1220,7 @@ function mountChrome() { el("div", { class: "topbar-spacer" }), search, el("div", { class: "progress", id: "progress" }), + playBtn, copyBtn, themeBtn, ); @@ -1232,6 +1248,470 @@ function mountChrome() { app.appendChild(buildHelpOverlay()); } +/* ----------------------------- Play mode -------------------------------- */ +/** + * "Play" turns the static walkthrough into a narrated, auto-advancing + * screencast: each chapter becomes a scene that pans through the actual diff + * (spotlighting the lines it references) while the browser's built-in speech + * synthesis reads the chapter's narration. No ffmpeg, no API key, no network — + * it's the same single .html, just playing itself. Narration text also shows as + * captions, so it works fully muted. + */ + +interface PlayerEls { + scene: HTMLElement; + caption: HTMLElement; + playBtn: HTMLButtonElement; + muteBtn: HTMLButtonElement; + counter: HTMLElement; + progressFill: HTMLElement; +} + +const player = { + idx: 0, + playing: false, + muted: false, + done: false, + overlay: null as HTMLElement | null, + els: null as PlayerEls | null, + code: null as HTMLElement | null, // the scrolling code surface of the current scene + raf: 0, + backup: 0, // safety timer id for the speaking case + token: 0, // bumped on every scene change to invalidate stale callbacks + // scene clock (drives both the pan and, when not speaking, auto-advance) + elapsed: 0, + duration: 0, + last: 0, + speaking: false, +}; + +function canSpeak(): boolean { + return typeof window !== "undefined" && "speechSynthesis" in window; +} + +function reduceMotion(): boolean { + return !!window.matchMedia?.("(prefers-reduced-motion: reduce)").matches; +} + +function narrationText(c: Chapter): string { + if (c.narration && c.narration.trim()) return c.narration.trim(); + const bits: string[] = []; + if (c.intent) bits.push(c.intent.trim()); + if (c.summary) bits.push(c.summary.trim()); + return bits.join(" "); +} + +/** Build the scrolling code surface for a scene: the chapter's diff hunks, with + * the specifically-referenced line ranges spotlighted. */ +function buildSceneCode(c: Chapter): HTMLElement { + const wrap = el("div", { class: "scene-code" }); + const refsByFile = new Map<string, Array<[number, number]>>(); + for (const r of c.diff_hunks ?? []) { + const arr = refsByFile.get(r.file) ?? []; + arr.push([r.start_line, r.end_line]); + refsByFile.set(r.file, arr); + } + + let any = false; + let spotted = false; + for (const path of c.files) { + const f = fileByPath.get(path); + if (!f || f.binary || !f.hunks.length) continue; + const lang = langForPath(f.path); + const refs = refsByFile.get(path) ?? null; + const card = el( + "div", + { class: "scene-file" }, + el("div", { class: "scene-file-head" }, statusTag(f), el("span", { class: "scene-file-path" }, f.path)), + ); + // Prefer the hunks that overlap the referenced ranges; fall back to all. + const overlapping = refs + ? f.hunks.filter((h) => refs.some(([s, e]) => h.newStart <= e && h.newStart + h.newLines >= s)) + : f.hunks; + const hunks = overlapping.length ? overlapping : f.hunks; + for (const h of hunks) { + const body = renderUnifiedHunk(h, lang); + if (refs) { + body.querySelectorAll<HTMLElement>(".dl").forEach((row) => { + const n = parseInt(row.querySelector(".ln-new")?.textContent ?? "", 10); + if (!Number.isNaN(n) && refs.some(([s, e]) => n >= s && n <= e)) { + row.classList.add("dl-spot"); + spotted = true; + } + }); + } + card.appendChild(body); + any = true; + } + wrap.appendChild(card); + } + // Only dim non-referenced lines when there's actually something to spotlight. + if (spotted) wrap.classList.add("has-spots"); + if (!any) wrap.appendChild(el("p", { class: "scene-nocode muted" }, "No code changes to show for this chapter.")); + + // Scenes are small — highlight syntax eagerly rather than on scroll. + wrap.querySelectorAll<HTMLElement>("[data-hl]").forEach((cell) => { + const lang = cell.getAttribute("data-hl")!; + const raw = cell.textContent ?? ""; + if (raw) cell.innerHTML = highlightLine(raw, lang); + cell.removeAttribute("data-hl"); + }); + return wrap; +} + +function renderScene() { + const els = player.els; + if (!els) return; + const chapters = orderedChapters(); + const c = chapters[player.idx]; + clear(els.scene); + + els.scene.appendChild( + el( + "div", + { class: "scene-head" }, + el("span", { class: "scene-eyebrow" }, `Chapter ${player.idx + 1} of ${chapters.length}`), + riskBadge(c.risk_level), + ), + ); + els.scene.appendChild(el("h2", { class: "scene-title" }, c.title)); + if (c.intent) els.scene.appendChild(el("div", { class: "scene-intent" }, c.intent)); + player.code = buildSceneCode(c); + els.scene.appendChild(player.code); + + // restart the fade-in animation + els.scene.classList.remove("scene-in"); + void els.scene.offsetWidth; + els.scene.classList.add("scene-in"); +} + +function easeInOut(t: number): number { + return t < 0.5 ? 2 * t * t : 1 - Math.pow(-2 * t + 2, 2) / 2; +} + +function stopSceneTimers() { + if (player.raf) cancelAnimationFrame(player.raf); + player.raf = 0; + if (player.backup) clearTimeout(player.backup); + player.backup = 0; + try { + if (canSpeak()) window.speechSynthesis.cancel(); + } catch { + /* ignore */ + } +} + +/** Start (or restart) the scene clock loop from the current `elapsed`. */ +function runClock() { + player.last = performance.now(); + const step = (now: number) => { + if (!player.playing) return; + const dt = now - player.last; + player.last = now; + player.elapsed += dt; + const t = player.duration > 0 ? Math.min(1, player.elapsed / player.duration) : 1; + + const code = player.code; + if (code && !reduceMotion()) { + const max = Math.max(0, code.scrollHeight - code.clientHeight); + if (max > 0) code.scrollTop = max * easeInOut(t); + } + updateProgressBar(t); + + if (t >= 1) { + // When speaking, the utterance's onend drives advancement; otherwise the + // clock does. (A backup timer covers browsers that never fire onend.) + if (!player.speaking) { + advance(1); + return; + } + } + player.raf = requestAnimationFrame(step); + }; + player.raf = requestAnimationFrame(step); +} + +function playScene() { + stopSceneTimers(); + const token = ++player.token; + player.elapsed = 0; + player.done = false; + renderScene(); + + const chapters = orderedChapters(); + const c = chapters[player.idx]; + const narration = narrationText(c); + if (player.els) { + player.els.caption.textContent = narration || "—"; + player.els.counter.textContent = `${player.idx + 1} / ${chapters.length}`; + } + updateControls(); + + // Estimate scene length from the narration (~2.6 words/sec), min 4.5s. + const words = narration.trim() ? narration.trim().split(/\s+/).length : 0; + player.duration = Math.max(4500, (words / 2.6) * 1000); + + player.speaking = false; + if (player.playing && !player.muted && canSpeak() && narration.trim()) { + try { + const u = new SpeechSynthesisUtterance(narration); + u.rate = 1; + u.onend = () => { + if (token === player.token && player.playing) advance(1); + }; + u.onerror = () => { + // Speech failed (no voices, blocked, headless...). Don't skip the scene — + // fall back to the timed clock, which advances at the end of the pan. + if (token === player.token) player.speaking = false; + }; + window.speechSynthesis.cancel(); + window.speechSynthesis.speak(u); + player.speaking = true; + // Safety net: some engines drop onend. Advance well after the estimate. + player.backup = window.setTimeout(() => { + if (token === player.token && player.playing) advance(1); + }, player.duration + 10000); + } catch { + player.speaking = false; + } + } + + if (player.playing) runClock(); +} + +function advance(delta: number) { + const chapters = orderedChapters(); + const next = player.idx + delta; + if (next < 0) { + player.idx = 0; + playScene(); + return; + } + if (next >= chapters.length) { + finishPlayback(); + return; + } + player.idx = next; + playScene(); +} + +function finishPlayback() { + stopSceneTimers(); + player.playing = false; + player.done = true; + const els = player.els; + if (!els) return; + clear(els.scene); + els.scene.appendChild( + el( + "div", + { class: "scene-end scene-in" }, + el("div", { class: "scene-end-mark" }, "✓"), + el("h2", {}, "Walkthrough complete"), + el("p", { class: "muted" }, `${orderedChapters().length} chapters · ${W?.title ?? ""}`), + el( + "div", + { class: "scene-end-actions" }, + el("button", { class: "btn", type: "button", onclick: () => replay() }, "↻ Replay"), + el("button", { class: "btn ghost", type: "button", onclick: () => closePlayer() }, "Close"), + ), + ), + ); + els.caption.textContent = "—"; + updateProgressBar(1); + updateControls(); +} + +function replay() { + player.idx = 0; + player.playing = true; + playScene(); +} + +function togglePlay() { + if (player.done) { + replay(); + return; + } + player.playing = !player.playing; + if (player.playing) { + // Resume: continue speech if we were mid-utterance, else re-speak nothing + // (the clock simply carries on driving the pan / advance). + if (player.speaking && canSpeak()) { + try { + window.speechSynthesis.resume(); + } catch { + /* ignore */ + } + } + if (player.speaking) { + const token = player.token; + const remaining = Math.max(2000, player.duration + 10000 - player.elapsed); + player.backup = window.setTimeout(() => { + if (token === player.token && player.playing) advance(1); + }, remaining); + } + runClock(); + } else { + if (player.raf) cancelAnimationFrame(player.raf); + player.raf = 0; + if (player.backup) clearTimeout(player.backup); + player.backup = 0; + if (player.speaking && canSpeak()) { + try { + window.speechSynthesis.pause(); + } catch { + /* ignore */ + } + } + } + updateControls(); +} + +function toggleMute() { + player.muted = !player.muted; + if (player.muted && canSpeak()) { + try { + window.speechSynthesis.cancel(); + } catch { + /* ignore */ + } + player.speaking = false; + } + updateControls(); + // Re-arm the current scene so muting/unmuting takes effect immediately. + if (player.playing && !player.done) playScene(); +} + +function updateProgressBar(t: number) { + const els = player.els; + if (!els) return; + const total = orderedChapters().length || 1; + const frac = Math.min(1, (player.idx + Math.min(1, Math.max(0, t))) / total); + els.progressFill.style.width = `${frac * 100}%`; +} + +function updateControls() { + const els = player.els; + if (!els) return; + els.playBtn.textContent = player.done ? "↻" : player.playing ? "❚❚" : "▶"; + els.playBtn.title = player.done ? "Replay" : player.playing ? "Pause (space)" : "Play (space)"; + els.muteBtn.textContent = player.muted ? "🔇" : "🔊"; + els.muteBtn.title = player.muted ? "Captions only — click for narration" : "Narration on — click to mute"; + els.muteBtn.classList.toggle("active", !player.muted); +} + +function playerKey(e: KeyboardEvent) { + if (!player.overlay) return; + switch (e.key) { + case "Escape": + e.preventDefault(); + e.stopPropagation(); + closePlayer(); + break; + case " ": + e.preventDefault(); + e.stopPropagation(); + togglePlay(); + break; + case "ArrowRight": + e.preventDefault(); + e.stopPropagation(); + advance(1); + break; + case "ArrowLeft": + e.preventDefault(); + e.stopPropagation(); + advance(-1); + break; + case "m": + e.stopPropagation(); + toggleMute(); + break; + } +} + +function ctrlBtn(label: string, title: string, cls: string, onClick: () => void): HTMLButtonElement { + return el( + "button", + { class: `pbtn ${cls}`, type: "button", title, onclick: onClick }, + label, + ) as HTMLButtonElement; +} + +function openPlayer(startIdx = 0) { + if (!W || !orderedChapters().length) return; + if (player.overlay) closePlayer(); + player.idx = Math.max(0, Math.min(orderedChapters().length - 1, startIdx)); + player.playing = true; + player.done = false; + + const scene = el("div", { class: "player-scene" }); + const caption = el("div", { class: "player-caption", "aria-live": "polite" }); + const progressFill = el("div", { class: "player-progress-fill" }); + const counter = el("span", { class: "player-counter" }); + + const playBtn = ctrlBtn("❚❚", "Pause (space)", "pbtn-play", () => togglePlay()); + const muteBtn = ctrlBtn("🔊", "Mute narration", "pbtn-mute", () => toggleMute()); + const prevBtn = ctrlBtn("⏮", "Previous chapter (←)", "", () => advance(-1)); + const nextBtn = ctrlBtn("⏭", "Next chapter (→)", "", () => advance(1)); + const closeBtn = ctrlBtn("✕", "Close (Esc)", "pbtn-close", () => closePlayer()); + + player.els = { scene, caption, playBtn, muteBtn, counter, progressFill }; + + const bar = el( + "div", + { class: "player-bar" }, + el( + "span", + { class: "player-brand" }, + el("span", { class: "brand-mark" }, "❯_"), + el("span", {}, "PatchStory"), + el("span", { class: "player-tag" }, "playing"), + ), + el("span", { class: "player-bar-title" }, W.title), + el("span", { class: "player-bar-spacer" }), + closeBtn, + ); + + const controls = el( + "div", + { class: "player-controls" }, + prevBtn, + playBtn, + nextBtn, + counter, + el("div", { class: "player-progress" }, progressFill), + muteBtn, + ); + + const overlay = el( + "div", + { class: "player", role: "dialog", "aria-label": "Walkthrough playback" }, + bar, + el("div", { class: "player-body" }, scene), + caption, + controls, + ); + + player.overlay = overlay; + document.body.appendChild(overlay); + window.addEventListener("keydown", playerKey, true); + playScene(); +} + +function closePlayer() { + stopSceneTimers(); + player.playing = false; + window.removeEventListener("keydown", playerKey, true); + if (player.overlay) { + player.overlay.remove(); + player.overlay = null; + } + player.els = null; + player.code = null; +} + /* -------------------------------- Boot ---------------------------------- */ function boot() { diff --git a/packages/renderer/web/styles.css b/packages/renderer/web/styles.css index 22f95e5..29ad005 100644 --- a/packages/renderer/web/styles.css +++ b/packages/renderer/web/styles.css @@ -400,6 +400,109 @@ code { font-family: var(--mono); font-size: 0.85em; } } .help-row span { color: var(--text-soft); font-size: 13px; } +/* ------------------------------ Play mode ------------------------------ */ +.playbtn { background: var(--bg-soft); color: var(--text); border-color: var(--border); } +.playbtn:hover { background: var(--border-soft); filter: none; } + +.player { + position: fixed; inset: 0; z-index: 60; + display: flex; flex-direction: column; + background: var(--bg); +} + +.player-bar { + display: flex; align-items: center; gap: 10px; + padding: 10px 18px; border-bottom: 1px solid var(--border); + background: var(--bg-elev); +} +.player-brand { display: flex; align-items: center; gap: 7px; font-weight: 700; } +.player-tag { + font-size: 10px; text-transform: uppercase; letter-spacing: 1px; font-weight: 700; + color: var(--accent); background: var(--accent-soft); + padding: 2px 7px; border-radius: 10px; +} +.player-bar-title { color: var(--muted); font-size: 13px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.player-bar-spacer { flex: 1; } + +.player-body { flex: 1; min-height: 0; display: flex; justify-content: center; padding: 24px 24px 0; } +.player-scene { + width: min(960px, 100%); display: flex; flex-direction: column; min-height: 0; +} +.scene-in { animation: sceneIn 0.45s ease both; } +@keyframes sceneIn { from { opacity: 0; transform: translateY(8px); } to { opacity: 1; transform: none; } } + +.scene-head { display: flex; align-items: center; gap: 12px; margin-bottom: 6px; } +.scene-eyebrow { font-size: 12px; text-transform: uppercase; letter-spacing: 0.6px; color: var(--muted); font-weight: 600; } +.scene-title { font-size: 26px; margin: 0 0 8px; letter-spacing: -0.3px; overflow-wrap: anywhere; } +.scene-intent { font-size: 15px; color: var(--text-soft); margin-bottom: 14px; max-width: 760px; } + +.scene-code { + flex: 1; min-height: 0; overflow: hidden; /* panned programmatically */ + border: 1px solid var(--border); border-radius: 12px; background: var(--bg-elev); + box-shadow: var(--shadow); + scroll-behavior: auto; +} +.scene-file + .scene-file { border-top: 1px solid var(--border); } +.scene-file-head { + display: flex; align-items: center; gap: 10px; padding: 9px 14px; + background: var(--bg-soft); border-bottom: 1px solid var(--border); + position: sticky; top: 0; z-index: 1; +} +.scene-file-path { font-family: var(--mono); font-size: 13px; } +.scene-nocode { padding: 28px; text-align: center; } + +/* Spotlighted lines — the ranges a chapter explicitly references. Non-spotlight + lines dim so the eye lands on what's being narrated. Dimming only kicks in + when the scene actually has spotlights (the `.has-spots` gate). */ +.scene-code.has-spots .dl { opacity: 0.4; transition: opacity 0.3s; } +.scene-code.has-spots .dl-spot { opacity: 1; box-shadow: inset 2px 0 0 var(--accent); } + +.player-caption { + flex-shrink: 0; text-align: center; + padding: 16px 24px; max-width: 900px; margin: 0 auto; + font-size: 17px; line-height: 1.5; color: var(--text); + min-height: 3.2em; display: flex; align-items: center; justify-content: center; +} + +.player-controls { + display: flex; align-items: center; gap: 14px; + padding: 12px 22px; border-top: 1px solid var(--border); background: var(--bg-elev); +} +.pbtn { + border: 1px solid var(--border); background: var(--bg-soft); color: var(--text); + height: 38px; min-width: 38px; padding: 0 10px; border-radius: 9px; + cursor: pointer; font-size: 14px; font-family: inherit; + display: inline-flex; align-items: center; justify-content: center; +} +.pbtn:hover { background: var(--border-soft); } +.pbtn-play { background: var(--accent); color: #fff; border-color: var(--accent); font-size: 13px; } +.pbtn-play:hover { filter: brightness(1.08); background: var(--accent); } +.pbtn-mute.active { color: var(--accent); } +.pbtn-close { margin-left: 0; } +.player-counter { font-variant-numeric: tabular-nums; color: var(--muted); font-size: 13px; min-width: 54px; text-align: center; } +.player-progress { flex: 1; height: 6px; background: var(--border-soft); border-radius: 4px; overflow: hidden; } +.player-progress-fill { height: 100%; width: 0; background: var(--accent); transition: width 0.15s linear; } + +.scene-end { text-align: center; margin: auto; padding: 40px 0; } +.scene-end-mark { + width: 56px; height: 56px; margin: 0 auto 14px; border-radius: 50%; + background: var(--risk-low-bg); color: var(--risk-low-fg); + display: flex; align-items: center; justify-content: center; font-size: 28px; font-weight: 700; +} +.scene-end h2 { margin: 0 0 6px; } +.scene-end-actions { display: flex; gap: 10px; justify-content: center; margin-top: 18px; } + +@media (prefers-reduced-motion: reduce) { + .scene-in { animation: none; } +} + +@media (max-width: 800px) { + .player-body { padding: 16px 12px 0; } + .scene-title { font-size: 21px; } + .player-bar-title { display: none; } + .player-caption { font-size: 15px; padding: 12px 16px; } +} + @media (max-width: 800px) { /* Topbar: brand + menu + theme on row 1, full-width search wraps to row 2. */ .topbar { flex-wrap: wrap; gap: 8px 10px; padding: 10px 14px; }