Skip to content

Live performance groups: relay Mirror → Python sidecar - #249

Open
jmacd867 wants to merge 6 commits into
mainfrom
LiveClustering
Open

jmacd867 wants to merge 6 commits into
mainfrom
LiveClustering

Conversation

@jmacd867

Copy link
Copy Markdown
Collaborator

Closes #245

Summary

Shows a pilot's Performance group while the Flight is under way. The relay's Mirror (#247) feeds a Python sidecar, live_groups. The sidecar scores the pilot against a Brief we write for our own flights and places them in the nearest group of a Reference cohort. No relay code changes. Design: docs/adr/0004.

sequenceDiagram
    participant Relay
    participant Sidecar as live_groups
    participant Operator
    Relay->>Sidecar: Mirror copy of every packet (reader thread)
    Operator->>Sidecar: Enter as each Instruction is spoken
    Sidecar->>Sidecar: attach Targets in force, blank inside the Settle window
    Sidecar->>Sidecar: Deviation features, nearest group (fitted once, never refit)
    Sidecar-->>Operator: status line every 5 s
    Sidecar->>Sidecar: CSV with target_* plus instruction log
Loading

Scoring rules:

on(instruction)
  targets = previous targets, overwritten by the ones it names     # blank carries over
  for each factor whose target changed (heading compared wrapped)
    not scored for --settle seconds (30)                           # others stay scored

on(sample)
  judge against the instruction in force when it arrived
  count scored time per factor
  name a group once every factor has --min-scored (60 s)
 inference/
+├── briefs/example_level_holds.csv
 └── pilot_inference/
     ├── capture/
+    │   └── replay.py          # sends a CSV in the relay's packet format
     └── performance/
         ├── clustering.py      # + fit_reference / CohortGroups.assign; fit() wraps it
+        ├── live.py            # Brief, carry-over, Settle window, per-factor scoring
+        └── live_groups.py     # the sidecar CLI

Docs: GLOSSARY.md (Instruction redefined; Brief, Settle window and Reference cohort added; climbs flagged as open), docs/adr/0004, docs/telemetry_schema.md, inference/README.md ("Live groups").

Evidence

  • Before: groups only offline, after the flight: data_logger → compute_features → cluster_pilots. No targets for any real flight.
    After: a synthetic pilot outside the cohort, replayed through the real CLI (--settle 2 --min-scored 30). Hidden tier 3; group 2 holds 4 of the cohort's 5 tier-3 pilots.

    Performance groups of the synthetic reference cohort (a demonstration, not a measurement):
      group 1 (5 pilots): altitude std 20.0 |bias| 27.7 ft, ...
      group 2 (4 pilots): altitude std 66.5 |bias| 103.2 ft, ...
    Relay connected from 127.0.0.1:62615
    [10:13:09] instruction 1/1 (settling), nothing scored yet
    [10:13:14] instruction 1/1, scored 5s, provisional, 30s scored needed for a group | ...
    [10:13:39] instruction 1/1, scored 30s, group 2 | altitude std 107.6 |bias| 116.9, ...
    
  • Tests: uv run pytest passes 113. New ones cover:

    blank target carries over; first instruction must set all four
    only changed targets settle; heading 0 after 360 does not
    sample handled after an instruction keeps the one it arrived under
    saved CSV (also appended) gives each flight its live features offline
    replayed packets -> samples; garbled connection doesn't stop the next
    port held the way data_logger holds it -> clear exit, nothing written (Windows: no silent sharing)
    CohortGroups.assign reproduces fit() and never moves the groups
    
  • Not verified: the live X-Plane → relay → Mirror → sidecar run. It needs the lab hardware. The checklist is in Live performance groups: relay → Mirror → Python sidecar #245.

Merge Danger

Door: two-way

The new modules are additive. clustering.fit() now goes through fit_reference(); a test checks that its assignments and centres are identical. data_logger.ensure_csv gains an optional fields argument with the old default.

Blast Radius: inference-only

The relay, plugin and baton are untouched. The live CSVs are git-ignored. On Windows, the sidecar now claims its port exclusively. A second listener on the same port, such as data_logger, is refused instead of silently sharing it. The glossary's new definition of Instruction changes the word's meaning for anyone reading older notes.

- Claim the listening port before the cohort fit and the CSVs, and stop
  with a clear message when it is taken. On Windows use
  SO_EXCLUSIVEADDRUSE: with SO_REUSEADDR the sidecar could silently share
  a port data_logger was already listening on.
- Keep accepting relay connections after one fails on garbled bytes or a
  socket error, since the mirror never reconnects by itself.
- Share one KMeans setup between the sweep and fit_reference so the two
  cannot drift.
- README: stop data_logger first, and keep the plugin window open.
…anges

Glossary: Instruction now means what the pilot is told rather than the
moment, and Brief, Settle window and Reference cohort are defined. Whether
a climb's altitude is scored before level-off is flagged as open.

Code to match:
- A blank target in a brief carries over from the instruction before;
  only the first instruction must set all four.
- Only targets whose value an instruction changes settle. The others,
  including ones restated at the same value, are scored through the
  turn, so altitude lost in a turn counts.
- Scored time is kept per factor, and a group waits until every factor
  has the minimum.
- The console and status line name the factors still settling and how
  long each has left; the instruction log records the targets in force.
- Rename the status line's "stage" to "progress", since the glossary
  keeps "stage" away from Flight phase.
- Heading 0 after 360 is the same north, so it no longer settles: a
  heading target counts as changed only if the wrapped difference is
  nonzero.
- Rename ReferenceGroups to CohortGroups and drop "reference group" from
  docstrings, help and console text, since the glossary keeps that term
  away from Reference cohort.
- Remove LiveFlight.given_at, which nothing used.
- Test that an appended CSV still gives each flight its live features,
  and that a named group stays shown while a later instruction settles.
- docs/telemetry_schema.md: targets now exist for flights we brief
  ourselves, written by live_groups; the client's are still missing.
- README: the live CSV is appended to, and a group stays shown while one
  factor settles. replay warns that the committed pre-#196 CSVs are
  mis-scaled.
@jmacd867

Copy link
Copy Markdown
Collaborator Author

Merge gate: hold the merge until the lab run in #245 passes (X-Plane → relay → Connect mirror → live_groups on a real flight, while iMotions records). Squash-merge and use the PR description as the commit message.

@jmacd867
jmacd867 requested review from Phlabry and a lite review from Copilot October 26, 2026 15:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved moderate findings affect replay selection, input validation, live processing performance, queue growth, shutdown ordering, and telemetry completeness.

2 open issues
What changed in this PR

This pull request adds a Python sidecar for live performance-group scoring. It applies flight instructions and target settle windows to mirrored relay telemetry, computes deviation features, assigns pilots to fixed reference cohorts, and records target-enriched CSV output.

Changes:

  • Added live brief parsing, target carry-over, settle-window handling, scoring, and feature calculation.
  • Added the live_groups sidecar, relay replay tooling, CSV logging, and fixed-cohort assignment.
  • Added tests, example briefs, telemetry documentation, operational guidance, and architecture documentation.
File Description Review notes
inference/​tests/​performance/​test_live.py Tests brief loading, target carry-over, settling, scoring, and live features. No review findings.
inference/​tests/​performance/​test_live_groups.py Tests sidecar packet handling, persistence, status reporting, port handling, and grouping. No review findings.
inference/​tests/​performance/​test_clustering.py Tests fixed reference-cohort fitting and assignment behavior. No review findings.
inference/​tests/​capture/​test_replay.py Tests relay packet replay, flight selection, and brief generation. No review findings.
inference/​README.md Documents live-group operation and replay usage. No review findings.
inference/​pilot_inference/​performance/​live.py Implements brief parsing, target attachment, settle windows, scoring, and live features. Moderate: reject non-finite targets. Moderate: avoid repeatedly rescanning the full sample history during status updates, which causes quadratic work and can delay live processing.
inference/​pilot_inference/​performance/​live_groups.py Implements the sidecar listener, recording, status output, and CSV logs. Moderate: coordinate reader shutdown before closing logs and drain queued samples. Moderate: avoid full-history recomputation while holding the recording lock. Moderate: bound or monitor the sample queue. Moderate: support or explicitly reject relay configurations that omit non-scored telemetry fields.
inference/​pilot_inference/​performance/​clustering.py Adds reusable fixed-cohort fitting and assignment APIs. No review findings.
inference/​pilot_inference/​capture/​replay.py Replays telemetry CSVs in relay packet format and emits briefs. Moderate: fail clearly when --flight is supplied without a flight_id column instead of ignoring the selector.
inference/​pilot_inference/​capture/​data_logger.py Allows custom CSV headers for live target-extended logs. No review findings.
inference/​briefs/​example_level_holds.csv Provides an example level-hold brief with carry-over targets. No review findings.
GLOSSARY.md Defines live-scoring terminology and clarifies instruction semantics. No review findings.
docs/​telemetry_schema.md Documents live target columns and sidecar-generated fields. No review findings.
docs/​adr/​0004-live-clustering-runs-in-a-python-sidecar.md Records the sidecar architecture and consequences. No review findings.
.gitignore Excludes generated live-flight and replay artifacts. No review findings.

Review effort: Lite

Comment thread inference/pilot_inference/performance/live.py
Comment thread inference/pilot_inference/performance/live_groups.py Outdated
@jmacd867

Copy link
Copy Markdown
Collaborator Author

On the overview findings without inline threads:

  • replay --flight with no flight_id column: agreed, a real bug; it currently sends the whole file. Will exit with an error instead.
  • Non-finite targets in a brief: agreed. inf passes validation today and becomes NaN deviations. load_brief will reject it.
  • Bounding the sample queue: not taking this. It only grows if processing falls behind, and the measured worst case is ~180 ms every 5 s (see the thread on live.py). A bounded queue would mean choosing which samples to drop, and the relay's Mirror already does that upstream (docs/adr/0003).
  • Relay configurations that omit telemetry fields: doesn't apply. The relay's field toggles only shape the iMotions XML and the test packet; the stream always carries all eight values from the plugin (packets_for_fields in relay/src/state.rs). data_logger makes the same assumption.

- Shutdown: RelayListener replaces serve_relay and owns the active
  connection. On stop it shuts down the server socket (closing alone does
  not wake accept() on Linux; checked under WSL), ends the connection in
  progress, and joins the reader, so nothing is queued after STOP and
  everything before it is written before the logs close.
- Status updates copy the flight under the lock with LiveFlight.snapshot()
  and recompute features outside it, so the recorder is not held up.
- replay --flight exits with a message when the CSV has no flight_id
  column, instead of sending the whole file.
- load_brief rejects infinite targets.
Windows is the lab OS; the Linux and macOS accept() wake-up is handled and was checked under WSL.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Live performance groups: relay → Mirror → Python sidecar

2 participants