Skip to content
This repository was archived by the owner on Oct 18, 2026. It is now read-only.

feat: project editing, device knowledge, listening and live change events (Epics 9–15) - #21

Open
glassdimly wants to merge 17 commits into
fabb:mainfrom
glassdimly:claude/device-and-track-tools
Open

glassdimly wants to merge 17 commits into
fabb:mainfrom
glassdimly:claude/device-and-track-tools

Conversation

@glassdimly

@glassdimly glassdimly commented Oct 17, 2026 •

Copy link
Copy Markdown

Adds 75 MCP tools so an agent can build, shape and follow a project, not only read it and control transport. Existing tools are unchanged. Closes #3.

The work is organised as Epics 9–15 in _bmad-output/planning-artifacts (an epic per area, a story per group of tools, API reference entries for every tool), and each area has a live end-to-end test.

New tools

Epic Area Tools
9 Tracks and mixer create_track, delete_track, duplicate_track, get_track_mixer, set_track (name, volume, exact volume_db, pan, mute, solo, arm, activated, monitor mode, color, group expand, crossfade), set_send
9 Devices list_bitwig_devices, insert_device (Bitwig built-ins by name, CLAP / VST3 / VST2 id, or a preset file), copy_device, move_device, delete_device, set_device_enabled, set_device_window_open
9 Parameters get_device_parameters (all remote-control pages, plus the modulated value of controls a modulator moves), set_device_parameters (by name, or page + index) on any device on any track
9 Clips, transport, actions insert_file, create_clip, delete_clip, write_notes, set_transport, list_actions, invoke_action, undo, redo
10 Device chains, presets, containers get_device_chains and chain / chain_device_index on the device tools (a Vocoder's Modulator and Carrier, FX slots, layers); list_presets, replace_device, browser_open, browser_state, browser_filter, browser_commit, browser_cancel; list_drum_pads, set_drum_pad, insert_into_drum_pad, clear_drum_pads, list_layers, set_layer
11 Device knowledge list_installed_devices (every Bitwig device, and every CLAP / VST3 / VST2 plugin from Bitwig's own plugin cache), describe_device (controls measured on demand, with a scan history), fetch_device_docs (Bitwig's user-guide description of each device), catalog_status
12 Listening and auditioning set_engine_active, get_levels, get_playing_notes, play_notes, send_midi, show
13 Arrangement, scenes, clips, notes list_cue_markers, go_to_cue_marker, set_arrangement, stop_track; create_scene, rename_scene, set_scene, duplicate_scene, copy_scene, move_scene, delete_scene; copy_clip, move_clip, duplicate_clip, record_into_slot, set_clip, transform_clip; read_notes, edit_notes (velocity, release velocity, spread, expressions, chance, occurrence, recurrence, repeats); arranger: true on the clip tools and write_arranger_notes for the arranger clip the user selected
14 Project-wide control project_state, reset_project (clear solos, mutes, arms), set_groove, set_transport_options (punch, metronome volume, pre-roll, automation write mode, overdub, launch quantization, rewind, tap tempo, ...), set_cue
15 Following the user (#3) get_events (event log of transport, project, track, clip, selection and parameter changes, with long-poll), watch (streams changes as MCP progress and logging notifications while the call runs)

The server now also sends short instructions to clients on initialize, and docs/AGENT-GUIDE.md explains how the tools fit together.

Implementation notes

  • Device parameters, chains, pads and layers go through private cursor tracks and cursor devices (followSelection = false). Reading or editing never moves the user's selection; only show does, on purpose.
  • Bitwig ignores value().set() on remote controls (and on volume, pan, sends, groove, cue and accent) from an extension with no bound hardware control. setImmediately() works, so every value write uses it.
  • Bitwig applies changes asynchronously, so mutating calls poll for the result (2 s timeout) and return the new state.
  • The flat track bank lists regular tracks, then effect tracks, then the master. create_track counts these separately and checks the new track's type before renaming it. The master track can't be deleted or duplicated.
  • Moving a device between two chains of the same container uses the cursor itself as the source, because the selected slot belongs to the device, not the cursor.
  • Notes are read through addNoteStepObserver; the note map must not be cleared after selectChannel, or notes delivered by it are lost.
  • The built-in device list is read at runtime from Bitwig's own library (installationRoot), so every device of the running version is insertable; bitwig-devices.txt is now generated from that library.
  • Plugins are read from Bitwig's plugin metadata cache (~/.BitwigStudio/cache), so no plugin is ever loaded to list it. The format is Bitwig's own; PluginCache reads only the known tags and skips the rest.
  • watch and get_events exist because MCP Java SDK 0.11 sends progress and logging notifications during a call but does not handle resource subscriptions. watch was checked on the raw HTTP stream: each notification arrives when the change happens, before the result.
  • Stays on extension API 19.

Testing

  • ./gradlew test: 290 tests pass.
  • Six live scripts run over MCP against a running Bitwig. Each creates its own tracks and deletes them again, and refuses to run unless the open project is empty. Results on Bitwig 6.1.3, Linux:
Script Checks
scripts/live_test.py (Epic 9) 24/24
scripts/live_test_container.py (Epic 10) 19/19
scripts/live_test_notes.py (Epics 13, 14) 16/16
scripts/live_test_project.py (Epics 13, 14) 10/10
scripts/live_test_midi.py (Epic 12) 8/8
scripts/live_test_events.py (Epics 11, 15) 10/10

Docs

Follows the existing planning-artifact layout: prd/epic-9.md to prd/epic-15.md, stories 9.1 to 15.2, API reference entries for every new tool (Epic 9 with examples from a live session; Epics 10–15 generated from the tool schemas), and the live tests in testing/mcp-endpoints-verification.md. ROADMAP.md lists what is left and what the API cannot do, and docs/api-coverage.md (from scripts/api_coverage.py) lists which API methods are used.

Not covered

The Controller API does not allow these, so they stay manual:

  • audio and MIDI routing, sidechain sources, an Audio Receiver's source
  • placing clips on the arranger timeline (the arranger tools work on the clip the user selects)
  • moving or copying tracks (InsertionPoint.moveTracks/copyTracks are accepted and silently ignored in 6.1.3; duplicate_track works)
  • adding modulators, writing automation points
  • adding, duplicating or deleting layers, and emptying a drum pad (ignored in 6.1.3; host.deleteObjects on a drum pad crashes Bitwig, so it is never called)
  • capturing playing clips into a scene (does nothing from an extension in 6.1.3)

On @zsoltcs1123's question about tags on remote-control pages: the API (19, and 25 too) can only filter pages by a tag expression (createCursorRemoteControlsPage(name, count, filter), selectNextPageMatching). It cannot read or set a page's tags.

🤖 Generated with Claude Code

Jeremy John and others added 5 commits October 17, 2026 16:04
…actions)

Adds BitwigEditFacade and 24 MCP tools so an agent can build a mix, not just read it:

- tracks: create_track, delete_track, duplicate_track, get_track_mixer, set_track
  (name, volume, exact volume_db, pan, mute, solo, arm, activated, monitor mode, color), set_send
- devices: list_bitwig_devices, insert_device (Bitwig built-ins by name, CLAP or VST3 id),
  copy_device, move_device, delete_device, set_device_enabled, set_device_window_open
- parameters on any device, without moving the user's selection: get_device_parameters
  (all remote-control pages), set_device_parameters (by name or page+index)
- clips: insert_file, create_clip, delete_clip, write_notes
- transport: set_transport (tempo, position, loop, metronome, record, overdub)
- application: list_actions, invoke_action, undo, redo

The track bank now has 8 sends so send levels are reachable. The master track is never
counted as a regular track and cannot be deleted or duplicated through these tools.

Bitwig device UUIDs come from DrivenByMoss's device lists (LGPL-3.0).

Includes unit tests and scripts/live_test.py, which exercises every tool against a running
Bitwig and refuses to run unless an empty project is open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Write parameter, volume, pan and send values with setImmediately(); Bitwig ignores set()
  from an extension that has no bound hardware control.
- Count regular, effect and master tracks separately, and check a new track's type before
  renaming it, so create_track can never rename an effect or master track.
- Register send properties so get_track_mixer/set_track can read them.
- volume_db keeps the fader position whose displayed dB was closest to the target.
- README: list the new tools and how to run the tests.

Live test: 24/24 against Bitwig 6.1.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- prd/epic-9.md and prd/index.md: goal, background, user stories, tool map, cross-cutting rules
- stories 9.1-9.6: tracks and mixer, device chain editing, parameters of any device,
  clips and notes, transport settings and actions, live end-to-end test
- api-reference.md: request/response schemas for all 24 new tools; examples taken from a
  live Bitwig 6.1.3 session. project_getName marked as folded into status.
- testing/mcp-endpoints-verification.md: how to run scripts/live_test.py safely

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds BitwigEditFacadeBehaviourTest (12 tests) using small fakes that model the live
quirks: values written with set() are ignored (only setImmediately() applies) and
displays are rounded. Covers reading all/one remote-control page, setting parameters
by page+index and by name across pages, per-item failures, cursor focus failure,
volume_db landing within display rounding, clip creation and note writing on the grid,
file import into a slot, transport field isolation, device deletion and duplication.

Reintroducing set() for parameter or volume writes makes 4 of these fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bitwig splits a MIDI file with several tracks or channels into one new track each
instead of filling the target slot. insert_file used to report only inserted=false;
it now also returns new_tracks_created and a hint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zsoltcs1123

Copy link
Copy Markdown

Hey man this is good stuff, TY for this. Been wanting to do this but couldn't find the time...
What would also be great is to manage tags (get/set) on remote control pages, not sure if api supports that though

Jeremy John and others added 11 commits October 18, 2026 00:28
Adds get_device_chains, and a chain / chain_device_index option on insert_device,
copy_device, move_device, delete_device, set_device_enabled, get_device_parameters and
set_device_parameters, so devices inside a Vocoder's Modulator/Carrier, an FX device's FX
slot or an FX/Instrument Layer's layers can be listed, inserted, moved and configured.

Two independent cursor devices let copy/move address chains of the same device on both
ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d clip reading

New BitwigSessionFacade and tools: get_levels (peak/RMS per track and master),
list_cue_markers, go_to_cue_marker, set_arrangement (loop range, time signature),
rename_scene, stop_track (stop or return to arrangement), play_notes (audition notes into
a track's instrument without a clip), read_notes and set_clip (launcher clip notes, name,
loop).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_engine_active

The note map was cleared after selectChannel had already delivered the clip's notes.
set_engine_active hands Bitwig's audio engine to the current project; without it a
project cannot play, meter or audition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- list_installed_devices, describe_device (control scans on demand, with history),
  fetch_device_docs (Bitwig's own user-guide descriptions), catalog_status
- the built-in device list is read from Bitwig's own library at runtime, with a bundled
  copy generated from it (bitwig-devices.txt), so every device of the running version is insertable
- scripts/device_catalog.py for LV2-through-Carla inventory and batch scans
- ROADMAP.md and an API coverage report (scripts/api_coverage.py, docs/api-coverage.md)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-editing tools

- project_state, reset_project (unsolo/unmute/unarm all), create_scene
- set_groove, set_transport_options (punch in/out, metronome volume, pre-roll, automation
  write mode, launcher overdub, default launch quantization, post-recording action,
  rewind/fast-forward/tap tempo/continue/record)
- copy_clip, move_clip, duplicate_clip, record_into_slot
- edit_notes (velocity, length, gain, pan, timbre, pressure, micro-pitch, chance, mute,
  repeats on all notes or by pitch/time range), transform_clip (transpose, quantize,
  duplicate content, launch quantization and mode)

Track move/copy via InsertionPoint.moveTracks/copyTracks was tried and left out: Bitwig
6.1.3 accepts the calls from an extension but does nothing (request and control thread).

scripts/live_test_project.py exercises all of it against an empty project: 10/10 on
Bitwig 6.1.3. Roadmap and API coverage report updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- list_presets: search every .bwpreset on disk (installed packages, user library)
- insert_device takes preset_file; replace_device swaps a device, plugin or preset in place
- browser_open/state/filter/commit/cancel drive Bitwig's popup browser
  (content types, filter columns, paged results; replace mode lists a device's presets)
- list_drum_pads, set_drum_pad, insert_into_drum_pad (devices or samples), clear_drum_pads
- list_layers, set_layer for Instrument/FX Layer mixers
- device lists report the loaded preset
- roadmap: layer add/duplicate/delete and pad deletion are not possible;
  host.deleteObjects(drumPad) crashes Bitwig 6.1.3
- live test scripts/live_test_container.py (19 checks); fix stale EQ+ check in live_test.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t, arranger clips, cue mix

- edit_notes: release velocity, velocity spread, occurrence, recurrence, repeat curve and velocity shape
- read_notes reports every non-default expression and condition
- set_clip: loop on/off, play start/stop, shuffle, accent, color
- arranger: true on read_notes/edit_notes/set_clip/transform_clip and write_arranger_notes,
  working on the arranger clip the user selected
- set_scene, delete_scene, duplicate_scene, copy_scene, move_scene
- set_cue (cue volume and mix), set_track group_expanded and crossfade_mode
- roadmap: capture-playing-clips scene and layer duplicate/delete do nothing from an extension
- live test scripts/live_test_notes.py (16 checks)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- get_playing_notes: notes sounding right now on each track
- send_midi: cc, pitch bend, program, aftertouch or raw MIDI into a track's instrument
- get_device_parameters reports modulated_value/modulated_display for controls a modulator moves
- show: panel layout, and bring a track, launcher clip or device into view
- live test scripts/live_test_midi.py (8 checks)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ache, agent guide

- get_events: numbered log of transport, project, track, clip, selection and parameter changes,
  with long-poll (wait_ms) and coalescing of rapid changes
- watch: streams changes as MCP progress and log notifications for a duration (closes fabb#3)
- list_installed_devices / describe_device read CLAP, VST3 and VST2 plugins from Bitwig's own
  plugin cache (~/.BitwigStudio/cache), no Python step needed
- vst2_id on insert_device and replace_device
- server instructions for agents, docs/AGENT-GUIDE.md, README feature list and test list
- catalog_status no longer flags scans without a recorded Bitwig version as stale
- live test scripts/live_test_events.py (10 checks)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- prd/epic-10.md to epic-15.md: device chains/presets/containers, device knowledge,
  listening and auditioning, arrangement/scenes/clips/notes, project-wide control,
  following the user (issue fabb#3)
- stories 10.1 to 15.2 with acceptance criteria, implementation notes and live results
- api-reference.md: every new tool, generated from the tool schemas, plus additions to Epic 9 tools
- testing/mcp-endpoints-verification.md: the five new live test scripts
- prd/index.md epic overview; ROADMAP.md without fork-specific rows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@glassdimly glassdimly changed the title feat: project editing tools (tracks, devices, any-device parameters, clips, actions) feat: project editing, device knowledge, listening and live change events (Epics 9–15) Oct 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a tool that allows to listen for changes via SSE

2 participants