fix(server): make agent-read local videos playable in the web UI - #3213
fix(server): make agent-read local videos playable in the web UI#32137Sageer wants to merge 6 commits into
Conversation
code-app: dd48d74f8e1eff00f744220109ec23e9acad752f
🦋 Changeset detectedLatest commit: 031d010 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b95401e9fd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (deps === undefined) return undefined; | ||
| const name = safePath.split(/[\\/]/).at(-1) ?? 'video'; | ||
| try { | ||
| const meta = await deps.files.save(Readable.from(data), name, { mimeType }); |
There was a problem hiding this comment.
Reclaim the app-scoped staging upload
Whenever ReadMediaFile handles a video with session media enabled, this call creates a new app-scoped file without expiresInSec, and nothing deletes it after materialize writes the session-canonical copy—or when materialization fails and the code falls back to direct upload. FileServiceImpl only prunes files with an expiration, so repeated reads permanently accumulate an extra copy of every video, up to 100 MB per invocation, even after the session is deleted. Give this staging file bounded ownership/lifetime or remove it once the session copy can serve the resolver and UI.
Useful? React with 👍 / 👎.
commit: |
… of failing the turn
…ailure degrades - delete the orphaned staged upload and warn when session media materialization fails, instead of silently falling back to eager upload - memoize the auth-failure degrade within the session so a rejected upload endpoint no longer triggers a full re-upload on every step - annotate degraded video path tags with the reason, so the model is not told a video was read when only the file path was delivered
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5374eecab5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (part.type !== 'image_url' && part.type !== 'video_url' && part.type !== 'audio_url') { | ||
| return part; | ||
| } | ||
| if (daemonFileRefFromPart(part) !== undefined) return part; |
There was a problem hiding this comment.
Route tool-result daemon refs through session media
When the app-scoped staged upload is deleted or otherwise unavailable while the canonical session copy still exists, returning daemon refs unchanged here leaves the tool result as a raw kimi-file:// part without its session identity. The committed web bundle extracts its fileId but no sessionId, so AuthMedia calls /files/{file_id} rather than /sessions/{session_id}/media/{file_id}; playback therefore currently depends on the extra app-scoped copy identified by the existing review comment and will break once that copy is reclaimed. Project these refs as session media or enrich the parsed media with the enclosing message's session ID.
Useful? React with 👍 / 👎.
…a store ReadMediaFile staging no longer writes an app-scoped IFileService blob: mint the file id locally (newFileId, shared with FileServiceImpl) and materialize the bytes straight into the session media store. Staged videos now follow the session lifecycle (removed on session delete) instead of accumulating in <home>/files/ forever, with a duplicate pair of copies per re-read. The resolver's mediaStore.read fallback covers request-time reads, and SessionMediaStoreService.read now reports the persisted original filename (matching open) so provider uploads keep the real name instead of the storage key.
Picks up the session-scoped media route for staged tool-result videos (kimi-code-app PR #334 head). Required now that staged videos no longer have an app-scoped files entry: the previous bundle fetched them from /api/v1/files/{id}, which would 404.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4012f8a90d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (materializedPath === undefined) { | ||
| throw new Error('session media materialization returned no path'); |
There was a problem hiding this comment.
Accept pathless session-media materialization
When ISessionMediaStore uses a non-filesystem backend, materialize() legitimately returns undefined even though the canonical bytes and metadata were stored and remain streamable (as demonstrated by the in-memory-backend case in sessionMediaStore.test.ts). Treating that optional display path as failure discards the valid session-media result and falls back to eager provider upload/inline data, so these hosts still persist a non-daemon tool result and cannot use the authenticated session-media route. Consider successful completion of the store call sufficient and return the daemon reference regardless of whether a local path exists.
AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L53-L62
Useful? React with 👍 / 👎.
A non-filesystem storage backend stores the canonical bytes and metadata but legitimately returns no local path from materialize(). Treating that as failure discarded the daemon reference and fell back to an eager provider upload, so those hosts kept persisting ms:// tool results. A resolved materialize call is success; the display path stays optional.
|
@codex review |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
Related Issue
No related issue — reported via internal feedback: on a
--host(non-loopback) server, an agent-generated local video renders as an unplayable placeholder in the web UI, and the console showsLoading media from 'ms://…' violates the following Content Security Policy directive: "default-src 'self'".Problem
When the agent reads a local video with ReadMediaFile (e.g. after generating one with a script), the video cannot be played in the web UI. Three reinforcing causes:
ms://…file reference in the tool result. That reference is only meaningful to the model API, but it flowed verbatim through the message projection to the web UI, which cannot load thems://scheme.media-src, so media loads fall back todefault-src 'self'— blocking not onlyms://but also the authenticated-fetchblob:playback used for regular attached videos, and inlinedata:videos.ms://tool results kept getting them passed through to clients.What changed
ReadMediaFilenow stages videos into the session media store (IFileService+ISessionMediaStore.materialize) and persists akimi-file://…daemon reference in the tool result. The media resolver still swaps in the providerms://reference at request build time, so the model sees the video in the next request exactly as before — the provider reference just never reaches persisted history or the UI. Falls back to the previous eager-upload path if staging fails.http(s):,data:,blob:,kimi-file://) to a text placeholder, covering histories written before this fix.media-src 'self' data: blob:, unblocking blob/data media playback on non-loopback binds.apps/kimi-code/dist-webrebuilt from code-appdd48d74f— the web UI renderskimi-file://tool results through the authenticated media route (MoonshotAI/kimi-code-app#334).Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update. (No user-facing doc surface for this fix)