Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,16 @@ jobs:
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.14
- name: Block source-only Subtitle Studio releases
if: startsWith(github.ref_name, 'plugin-subtitle-studio-v')
run: |
echo "Subtitle Studio's reviewed native runtime bundle and packaged smoke gate are not complete." >&2
exit 1
- name: Install companion development dependencies
run: |
bun --cwd=tools/xiaoyunque-mcp install --frozen-lockfile
bun --cwd=tools/ffmpeg-mcp install --frozen-lockfile
bun --cwd=tools/subtitle-studio-mcp install --frozen-lockfile
- name: Verify official FFmpeg source
if: startsWith(github.ref_name, 'plugin-ffmpeg-tools-v')
run: |
Expand Down Expand Up @@ -67,6 +73,8 @@ jobs:
bun --cwd=tools/xiaoyunque-mcp test
bun --cwd=tools/ffmpeg-mcp run typecheck
bun --cwd=tools/ffmpeg-mcp test
bun --cwd=tools/subtitle-studio-mcp run typecheck
bun --cwd=tools/subtitle-studio-mcp test
bun tooling/build-companions.mjs --tag "$GITHUB_REF_NAME"
bun test
bun tooling/pack.mjs --tag "$GITHUB_REF_NAME"
Expand All @@ -92,6 +100,7 @@ jobs:
run: |
bun --cwd=tools/xiaoyunque-mcp install --frozen-lockfile
bun --cwd=tools/ffmpeg-mcp install --frozen-lockfile
bun --cwd=tools/subtitle-studio-mcp install --frozen-lockfile
- name: Verify official FFmpeg source
if: startsWith(github.ref_name, 'plugin-ffmpeg-tools-v')
run: |
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,7 @@ jobs:
run: bun --cwd=tools/xiaoyunque-mcp install --frozen-lockfile
- name: Install FFmpeg companion development dependencies
run: bun --cwd=tools/ffmpeg-mcp install --frozen-lockfile
- name: Install Subtitle Studio companion development dependencies
run: bun --cwd=tools/subtitle-studio-mcp install --frozen-lockfile
- name: Validate, test, and reproduce packages
run: bun run check
31 changes: 29 additions & 2 deletions docs/plugin-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,21 @@ omit `runtime` and `contributes.generation`. Declaring a runtime does not grant
Web surface caller authority, and granting `generation.execute` does not let the
iframe start processes or send arbitrary MCP requests.

A v3 Web surface may choose between two generic execution modes. `generation.canvas.execute`
commits admitted output directly to Canvas. `generation.workspace.execute` keeps admitted
output detached so the Plugin can preview it before the user explicitly publishes or
exports it. Detached text is returned inline; detached media is represented by a short-lived
opaque artifact id and playback URL. It never exposes a Project or native path. The owning
frame may publish, export, release, or cancel only its own artifacts in the current
Project/Canvas/node scope. Closing the frame, changing scope, expiry, or publishing releases
the temporary authorization and cleans unclaimed managed assets.

Workspace calls still select only a declared generation tool, use schema-validated scalar
`toolInput`, and derive media references from direct incoming Canvas edges. They are not an
arbitrary MCP or filesystem bridge. `generation.workspace.publish` may attach bounded SRT or
VTT text tracks; the host validates and stores them as portable managed VTT sidecars so the
Canvas player and downstream Project restore keep the same soft subtitles.

## Plugin service contribution

A v2 or v3 executable Plugin may expose bounded account/service state through the same
Expand Down Expand Up @@ -215,10 +230,20 @@ arrive as `{"protocol":"convax.plugin-host/1","type":"command","command":"refres
| `host.context.get` | none | current Project, Canvas, and owning node |
| `canvas.node.get` | `canvas.node.read` | owning node only |
| `canvas.node.updateState` | `canvas.node.write` | Plugin-namespaced node state |
| `canvas.connectedMedia.list` | `canvas.connectedMedia.read` | direct incoming managed audio/video nodes |
| `canvas.connectedMedia.playback.open` / `.close` | `canvas.connectedMedia.read` | opaque, expiring playback lease for one listed node |
| `project.file.readText` | `project.files.read` | current Project-relative text file |
| `agent.prompt` | `agent.prompt` | current Project and owning node resource |
| `generation.tools.list` | `generation.execute` | installed generation contracts in the current scope |
| `generation.canvas.execute` | `generation.execute` | shared scoped Canvas generation operation |
| `generation.workspace.execute` / `.cancel` | `generation.execute` | detached scoped operation using direct incoming references |
| `generation.workspace.publish` / `.export` / `.release` | `generation.execute` | owning frame's opaque detached artifact or direct source |
| `host.file.exportText` | `host.files.save` | Main save dialog for bounded UTF-8 text; no caller path |

Each `canvas.connectedMedia.list` item includes an opaque `sourceVersion`. Persist it
with any media-derived document, clear or explicitly migrate that document when the
version changes, and avoid reopening a playback lease when both node id and version
are unchanged. The token is an identity signal, not a Project path or content URL.

Request the smallest set. Arguments cannot select another Project, Canvas, or node.
Treat results as untrusted structured data, bound message sizes, handle errors, and
Expand All @@ -227,8 +252,10 @@ a failed optional view effect; do not report that as a reverted mutation.

## Forbidden behavior

No remote scripts/assets, iframe network APIs, popups, downloads, eval-generated
No remote scripts/assets, iframe network APIs, popups, browser-initiated downloads, eval-generated
code, native/WASM executables, packaged Node servers, filesystem paths, secrets,
telemetry, service workers, or generic method forwarding. Do not edit `.convax`
files. A v2 or v3 external runtime is a separately installed and authorized tool, never a
Plugin ZIP asset. Use host capabilities only.
Plugin ZIP asset. A declared `host.files.save` request may invoke only the host-owned
bounded text save dialog; it does not grant browser download or path access. Use host
capabilities only.
63 changes: 63 additions & 0 deletions docs/subtitle-studio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Subtitle Studio architecture

Subtitle Studio is split across the public Plugin Registry and the generic Convax
Plugin host. No Subtitle Studio UI, subtitle domain, model installer, native
process, or hard-coded Plugin identity belongs in the Convax application.

## Ownership

| Layer | Repository owner | Contents |
| --- | --- | --- |
| Static Canvas surface | `packages/plugins/subtitle-studio/package/` | player, operation buttons, local track state, translation orchestration |
| Portable subtitle domain | `tools/subtitle-studio-mcp/src/domain/` | document/SRT/translation/job/erase-plan validation |
| MCP companion | `tools/subtitle-studio-mcp/` | declared tools, staged-media execution, cancellation, runtime verification |
| Hard-erasure engine | `tools/subtitle-studio-mcp/native/subtitle-erasure/` | bounded OCR, temporal tracking, LaMa inpainting, validated remux |
| Generic host ports | `microvoid/convax` | direct connected media, opaque playback/artifacts, text-only Agent prompt, timed-text resources |

The static iframe never receives native paths, executable paths, model paths,
Project paths, shell access, or generic MCP access. Every media source is a direct
incoming Canvas edge and every native operation is one manifest-declared tool.

## Product flow

1. `canvas.connectedMedia.list` discovers the directly connected video and
`canvas.connectedMedia.playback.open` creates an opaque Range-capable lease. The
subtitle document is bound to the listed opaque `sourceVersion`, so replacing the
video cannot apply the previous video's tracks to the new source.
2. Transcription calls `subtitle.transcribe`. The companion selects the video's
audio stream itself; importing a separate audio file is not required.
3. Subtitle tracks remain editable soft subtitles in the Plugin node state.
Translation sends only cue ids and text through `agent.prompt` with
`mode: "text-only"`, which disables all Agent tools.
4. Soft erasure remuxes selected embedded text streams. Tracks created or imported
in Subtitle Studio are not part of that inspection and are never removed by it.
5. Hard erasure uses a normalized text-search region, detector geometry, temporal
tracking, and AI inpainting. It produces a detached artifact so the player can
switch to the exact processed result before publication.
6. “添加到画布” publishes the player video plus validated managed VTT sidecars.
“导出视频” exports the same current player video, muxing the current soft tracks
when required. Neither action accepts a caller-selected path.

## Runtime and release gate

The source package currently declares only `darwin-arm64`, matching the repository's
real companion CI coverage. The checked-in native source and protocol tests are not
an installable AI runtime by themselves.

A publishable `0.4.0` companion must still prove all of the following:

- one Registry-admitted executable no larger than 128 MiB;
- no Homebrew, PATH, ambient Python, or machine-local dynamic-library dependency;
- pinned FFmpeg/FFprobe/Whisper/OCR/LaMa inventory with exact sizes and SHA-256;
- complete third-party licenses and model notices;
- signed/notarized macOS artifact and minimum-OS verification;
- real transcription, soft-remux, preview, hard-erasure, mux, cancellation, and
cleanup smoke tests on packaged bytes;
- golden videos covering motion, scene cuts, no detection, corrupt media, audio
preservation, and outside-mask pixel identity.

Until that gate passes, the companion entrypoint fails closed instead of falling
back to arbitrary local programs or presenting source-only code as an installed AI
runtime. The publish workflow explicitly rejects `plugin-subtitle-studio-v*` tags;
remove that guard only in the reviewed change that supplies and validates the full
runtime bundle.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@
"render:showcases": "bun tooling/render-showcases.mjs",
"build:index": "bun tooling/build-index.mjs && bun tooling/build-showcase.mjs",
"build:companions": "bun tooling/build-companions.mjs",
"check": "bun run validate && bun --cwd=tools/xiaoyunque-mcp run typecheck && bun --cwd=tools/xiaoyunque-mcp test && bun --cwd=tools/ffmpeg-mcp run typecheck && bun --cwd=tools/ffmpeg-mcp test && bun run build:companions && bun test && bun run pack && bun run build:index && bun --cwd=tools/xiaoyunque-mcp run build && bun --cwd=tools/ffmpeg-mcp run build"
"check": "bun run validate && bun --cwd=tools/xiaoyunque-mcp run typecheck && bun --cwd=tools/xiaoyunque-mcp test && bun --cwd=tools/ffmpeg-mcp run typecheck && bun --cwd=tools/ffmpeg-mcp test && bun --cwd=tools/subtitle-studio-mcp run typecheck && bun --cwd=tools/subtitle-studio-mcp test && bun run build:companions && bun test && bun run pack && bun run build:index && bun --cwd=tools/xiaoyunque-mcp run build && bun --cwd=tools/ffmpeg-mcp run build && bun --cwd=tools/subtitle-studio-mcp run build"
}
}
28 changes: 28 additions & 0 deletions packages/plugins/subtitle-studio/convax-package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"schema": "convax.package/1",
"kind": "plugin",
"id": "subtitle-studio",
"name": "Subtitle Studio",
"description": "Local-first subtitle creation, multilingual adaptation, soft-subtitle handling, and AI hard-subtitle removal for directly connected Canvas video.",
"version": "0.4.0",
"license": "MIT",
"compatibility": {
"pluginSchema": "convax.plugin/3",
"pluginHost": "convax.plugin-host/3"
},
"companions": [
{
"command": "convax-subtitle-studio-mcp",
"version": "0.4.0",
"source": "tools/subtitle-studio-mcp",
"targets": [
{
"platform": "darwin",
"arch": "arm64",
"path": "dist/darwin-arm64/convax-subtitle-studio-mcp"
}
]
}
],
"yanked": false
}
21 changes: 21 additions & 0 deletions packages/plugins/subtitle-studio/package/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Microvoid contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Loading