Skip to content

Agent desktop: frame rate adapts to a slow network; files dropped on the tab are saved on the device - #69

Merged
GrischaE1 merged 2 commits into
mainfrom
agent-desktop-fps-dragdrop
Oct 23, 2026
Merged

GrischaE1 merged 2 commits into
mainfrom
agent-desktop-fps-dragdrop

Conversation

@GrischaE1

Copy link
Copy Markdown
Owner

Two items from the agent desktop's open list (docs/agent-desktop-draft.md §7): the frame rate now adapts to a slow network, and files can be dropped on the tab.

Frame rate on a slow network

Until now only the quality adapted. When the acks come late (a quarter of a 2 s window spent waiting), the device now takes one step every 2 s:

  1. quality by 10, down to 50 – text stays readable,
  2. then the frame rate by a third, down to 5 fps,
  3. then quality by 10, down to 20.

When the acks keep up, it goes back up in the reverse order. The encoder keeps the frame rate that was asked for, so a step doesn't force a key frame. Stats carry frameRateLimit, and RemoteHub's connection info says "Slow network: the device sends at most N fps to keep the picture sharp."

The existing slow-link case D-VID-05 (quality drops below 80) still holds.

Drag and drop onto the tab

Files and folders dragged onto the agent desktop tab are saved on the device's Desktop, in From RemoteHub. When a dropped name already exists there, they go into a dated folder inside it, so nothing is overwritten.

  • Protocol: a new transfer kind drop, behind the feature dropFiles. It is sent right away and never offered for pasting. The device never sends one, and an offer of one is refused.
  • Rights: the same as copying files (DesktopClipboard 3). Like the clipboard, it is closed while the lock screen is up. The service and the helper both check.
  • Written as the user: the helper finds the Desktop as LocalSystem (a redirected one too) but writes with the signed-in user's rights. A junction the user puts in the folder's place can only lead where they may write anyway.
  • Unfinished drops: a drop that doesn't finish removes only the files and folders it created (TransferReceiver with ownsFolder: false). Until now, a failed transfer deleted its whole target folder, which here would have taken earlier files with it.
  • Audit: a new agent event, 1108 DesktopFilesSaved, records the count, size and folder, but no file names.
  • In the tab: while dragging, a panel says where the files go or why they can't. Afterwards it says "Saved 3 files on the device's Desktop, in “From RemoteHub”."

Tests

  • Unit:
    • the quality and frame-rate steps (18 cases), and an end-to-end slow-ack run whose stats show the limit only once quality is at 50
    • the info line
    • drops end to end against an in-process agent: saved, a dated folder, a refused offer, refused by policy
    • an unfinished drop keeps earlier files; the folder choice; the tab's texts and drag state
  • Lab: a new harness step dropFiles, and new cases D-VID-14 and D-DROP-01 to 04 (drop, same names, planted junction, policy).
    • They passed on Server 2025 and Windows 11, together with D-VID-05 (runs 20261023-152043 and 20261023-153302).
    • D-DROP-01 first failed because of a bug in the test (the newest event comes first); I fixed the test and the case passed on the rerun.
  • Also fixed: JumpHostTests read the diagnostic log while other tests were writing to it and failed now and then. It now reads with sharing allowed.
  • Before this rebase onto main, the full unit suite had 2 failures: that flaky JumpHostTests test, and HelperClipboardFilesTests.A_reader_that_fails_or_hangs…, a timing test this PR doesn't touch, which most likely failed because the machine was busy with the lab.

Docs

  • agent-desktop-draft.md: §4.4, §4.7, the audit table, and the open list. I removed the items in it that are already done: ADMX templates, AllowAgentDesktop, the resume issue.
  • README, the lab case sheet and the test document.

… on the tab are saved on the device's Desktop

Frame rate (agent-desktop-draft.md 4.4):
- When the acks come late (a quarter of a 2 s window spent waiting), one step every 2 s: the quality by 10 to 50,
  then the frame rate by a third to 5 fps, then the quality by 10 to 20 - text stays readable before the picture
  moves less often. Back up in the reverse order when the acks keep up. The encoder keeps the asked rate, so no key
  frame per step. Stats carry frameRateLimit; RemoteHub's connection info says when the rate is lowered.

Drag and drop (4.7):
- Files and folders dragged onto the tab go as transfer kind "drop" (feature dropFiles): sent at once, never
  offered, saved in the signed-in user's Desktop\From RemoteHub - or a dated folder in it when a name is taken.
  Same right as copying files (DesktopClipboard 3), closed at the lock screen like the clipboard. The helper writes
  as the user, so a junction they plant leads only where they may write. An unfinished drop removes only what it
  created (TransferReceiver ownsFolder: false) - before, a failed transfer deleted its whole folder. The device
  never sends a drop; an offer of one is refused. Receipt names the folder; new audit event 1108 (counts, size,
  folder - no file names). While dragging, the tab says where the files go or why they can't.

Tests: unit tests for the steps, the info line, drops end to end (saved, dated folder, refused offer, policy,
unfinished drop, folder choice, texts and drag state). Lab: harness step dropFiles, D-VID-14 and D-DROP-01..04 -
passed on Server 2025 and W11 (runs 20261023-152043, 20261023-153302). Also a flaky JumpHostTests read of the
diagnostic log while it is written. Docs: agent-desktop-draft.md (4.4, 4.7, audit table, open list - stale items
removed), README, lab case sheet and test document.
@github-actions
github-actions Bot marked this pull request as draft October 23, 2026 14:03
@github-actions

Copy link
Copy Markdown

Pull requests start as drafts here. Mark this one Ready for review when it's done and all checks have passed.

…_locks_the_session_before_the_cover_ends

AutoSignInSessions ends the cover, then forgets the record; the test checked the record at once and CI (PR #69)
looked in between.
@GrischaE1
GrischaE1 marked this pull request as ready for review October 23, 2026 14:34
@GrischaE1
GrischaE1 merged commit 14c9802 into main Oct 23, 2026
13 checks passed
@GrischaE1
GrischaE1 deleted the agent-desktop-fps-dragdrop branch October 23, 2026 14:37
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 23, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant