Skip to content

audio_spectrum and waveform stay flat on a synthesised score #349

Description

@LeadcodeDev

A scenario whose soundtrack comes from audio.score (the synthesised voices from #331) renders with sound, but every audio-reactive component stays pinned at its minimum. The bars paint, they just never move.

Reproduction

A 12 s scenario, 120 BPM, a kick on every beat, plus an audio_spectrum with no track:

"audio": {
  "voices": { "kick": { "type": "sine", "freq": [150, 44], "sweep": 0.13, "decay": 0.40 } },
  "score": [{ "voice": "kick", "every": "1b" }]
},
...
{ "type": "audio_spectrum", "bars": 48, "mode": "bars", "color": "#A78BFA",
  "position": "absolute", "x": 240, "y": 560,
  "style": { "width": 800, "height": 110 } }

Sampling the 800×110 spectrum box on three stills, one just after a kick, one between kicks, one on the next kick:

t=0.05  lit pixels: 1252   topmost: 668
t=0.30  lit pixels: 1252   topmost: 668
t=0.55  lit pixels: 1252   topmost: 668

Identical at all three. topmost: 668 is the bottom 2 px of a box that runs from y=560 to y=670 — that is min_height, the floor the bars never leave. The sound itself is fine: volumedetect reads −7.8 dB at 0.5 s, 3.5 s, 6.5 s, 9.5 s and 11.5 s, and the muxed track is 12.000 s against a 12.000 s video.

What is not the cause

Two plausible explanations that the code rules out:

  • Not a missing track. synthesize_score_into_track runs at load time (loader.rs:16-23), inside load_scenario_with_vars, so by the time any encoder calls analyze_scenario_audio(scenario) the synth track is already in scenario.audio with a real src pointing at its temp WAV.
  • Not a missing track field on the component. rules/audio-reactive.md documents that when track is absent the first cached track is used, and there is only one track here.

So the analysis is running over a track that exists, and yielding nothing. analyze_scenario_audio documents graceful degradation — "if a track cannot be decoded (file not found, corrupt), it is silently skipped and all audio bindings fall back to their min value" — which is exactly the observed symptom, and exactly why nothing is reported. Worth checking source_fingerprint and decode_audio_file against the synth's own WAV format before looking further afield.

Why it matters more than it looks

These are two features that each work, and that cannot be combined. The synthesised score exists so an LLM can write a soundtrack without handing over a WAV; the audio-reactive components exist so the picture can follow the sound. A generated scenario that wants both — the obvious thing to want — gets a silent-looking visualiser and no diagnostic.

It is also the same shape as #339: a property that is declared, documented, accepted by the validator, and inert at render. The silence is the defect, not just the flat bars.

Found

While building a demo for #344's overlapping-scenes work. Not related to that change — reproduced on main at 6c7d779.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions