Skip to content

fix(player): keep picture and sound level when coming back to the app - #32

Merged
houko merged 3 commits into
mainfrom
fix/player-resume-hitch
Nov 9, 2026
Merged

houko merged 3 commits into
mainfrom
fix/player-resume-hitch

Conversation

@houko

@houko houko commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Note: the third cause below, and joining the sound once the window shows, were pushed after this PR was merged; that change is in #35. The "this PR" rows of the second table are measured with #35.

Summary

Switching away from Kando and back made playback hitch. There were three causes:

  1. On every activation, fix(player): keep background playback smooth #30 reopened the picture and restarted the ffplay sound process, even when the window had stayed in sight and the picture had never fallen behind. Restarting the sound left a gap and jumped it back a little.
  2. When the window was out of sight, VideoPlayer stopped the picture while the sound played on. Coming back showed the frame the window was hidden on until the reopened picture was ready, then jumped.
  3. On macOS a window coming back reports itself shown a frame or more after the app is active, and VideoPlayer only counts time and starts decoding from then. A picture handed over at activation stood on its first frame through that and then for its whole length again, skipped the frames it was late on, and trailed the sound by about 50ms from then on.

Changes:

  • playerDidResignActive records the sound's clock and starts followSound, a goroutine that checks every 100ms whether the picture has fallen at least backgroundDrift (200ms) behind the sound. That only happens while the window is out of sight. Once it has, it opens a decoder at the sound's position and keeps it on the frame the sound is in.
  • On return, playerDidBecomeActive hands that frame and decoder to a new VideoPlayer (takeBackgroundPicture), which takes over the running sound and its volume. If the picture never fell behind, nothing happens. If the return comes before the stall was noticed, it falls back to the drift check and reopen, which also keeps the running sound.
  • The handed-over frame lasts 1ms, so VideoPlayer asks for the next one as soon as the window really shows. frameDecoder then gives the frame the sound is in at that moment, for what is left of it, so from the first moving frame the picture follows the sound clock.
  • Merged with the volume control (feat(player): add volume control and shortcuts #31): soundAt reads the sound clock through returning from the moment the app comes back until the picture takes over or fails. backgroundResumeAt becomes soundClock.
  • Replacing the playing film goes through playReplace, which also stops its background decoder. The background code lives in playback_background.go.

Screenshots

The real app ran with a fake HOME and a test clip that has a seconds counter. The app was hidden for about 4s and then brought back, and each pair below shows the first frame after the window appears and the frame 100ms later, at full size.

Before (main): the frame the window was hidden on (0:11) shows first, then the picture jumps to 0:16, where the sound is.

Before: stale frame at 0:11, then a jump to 0:16 100ms later

After: the first frame is already where the sound is (35, handed over at 35.67s), and the gradient strip has moved on 100ms later.

After: the first frame after the return is already at 35

Verification

For each run I recorded the window with screencapture around the return, measured per-frame change in the moving gradient strip, read the counter at 30fps, and checked whether the ffplay PID changed. Runs where someone clicked the test window, or where the hide did not take effect, were discarded.

Hidden for about 4s, then brought back:

Build Sound on return Old frame after return Window appears → picture moves
main ffplay restarted with -ss every time 100–170ms, then a jump –
background follow turned off (baseline for any window showing again) same process picture resumes from where it stood 34 / 79 / 50 / 75ms
handover before the last commit same process 0–1 frame 92 / 125 / 75 / 108 / 67 / 234 / 117 / 108ms, then a skip
this PR same process 0–1 frame 53 / 75 / 75 / 42 / 42 / 50ms

With this PR, 5 of 6 runs had no gap over 60ms in the 2s after the return. One run had gaps of 66–78ms, which steady playback with no switching shows too: two to five per 2s, from the screen recording's timing. Timing logs from a local build show the handover finishing 10–20ms after activation, and the decoder joining the sound 15–30ms later, as the window shows.

With the window kept in sight (4 runs), the picture played on, nothing was reopened, and the sound process stayed the same.

Tests:

  • TestPlaybackResyncsWhenAppReturns: the drift check, clamping at the end, and a user-paused film.
  • TestPlaybackHandsTheSoundToTheReopenedPicture: the reopen fallback keeps the same sound and volume.
  • TestPlaybackFollowsTheSoundBehindTheWindow: a stalled picture is noticed. Just after the return, soundAt follows the sound. The background picture takes over without reopening the film, keeping the same sound, and the return's clock is cleared.
  • TestHandedOverPictureJoinsTheSound: with real ffmpeg on a 10fps clip, the held frame lasts 1ms. With the sound at 450ms, the next frame given is the one from 400 to 500ms, lasting 50ms, and the frames after it last their full length.
  • make test, go vet, go tool mygo vet, and the player and volume tests with -race.

Coming back to the app reopened the picture and started the sound process again on every activation, even when the window stayed in sight and the picture had kept pace. Restarting ffplay left a gap and a small jump back in the sound, which showed as a hitch each time.

Now activation compares the sound's clock with the picture's playhead and reopens only when the picture trails by 200ms or more, which happens when the window was out of sight. The reopened picture takes over the sound that kept playing instead of opening a new one, and a deactivation during the reopen is carried to the new picture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added area: app The app shell and state in apps/kando bug size/M 100 to 499 changed lines conflicts Conflicts with its base branch labels Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

This pull request conflicts with main. Merge main into it or rebase it onto main, then push; the conflicts label goes away once it merges cleanly.

While the window is out of sight VideoPlayer stops the picture, but the sound plays on. Coming back reopened the picture there, so the frame the window was hidden on stayed up for the 200-400ms ffmpeg needed to seek, and then jumped.

Now, while the app is inactive, a goroutine checks whether the picture has fallen behind the sound. Once it has, it opens a decoder where the sound is and keeps it on the frame the sound is in. Coming back hands that frame and decoder to a new VideoPlayer, which takes the running sound over, so the picture shows the right frame at once. Returning before the picture has fallen behind falls back to the drift check and reopen from the previous commit.

The background code moves to playback_background.go, and replacing the playing film goes through playReplace, which also stops its background decoder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@houko houko changed the title fix(player): only resync the picture when it fell behind the sound fix(player): keep picture and sound level when coming back to the app Nov 9, 2026
Resolve the conflict with the volume control (#31). A picture taking over from the background keeps the sound and the volume keep had, and soundAt reads the sound clock through a returning field from the moment the app comes back until the picture takes over or fails, since backgroundAt is cleared then. backgroundResumeAt becomes soundClock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the conflicts Conflicts with its base branch label Nov 9, 2026
@houko
houko merged commit 290c34c into main Nov 9, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: app The app shell and state in apps/kando bug size/M 100 to 499 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant