Skip to content

Assign a pilot to a performance group live, during a flight #223

Open

Description

@jmacd867

Is your feature request related to a problem? Please describe.
#216 clusters pilots in batch, after their flights are finished and logged. The client eventually wants to see which performance group a pilot falls into while the pilot is still flying. Nothing in inference/ reads the stream live or assigns a pilot to an existing performance group. The batch model is not saved either, so the only option today is to refit the whole cohort.

Describe the solution you'd like
A live mode that taps the same TCP stream inference/Data/data_logger.py reads. It computes deviation features for the flight in progress at a fixed refresh rate and assigns the pilot to the nearest performance group of a saved batch fit, without refitting.

  • Save the scaler and group centres from a Build a baseline pilot clustering model #216 batch run (the reference cohort).
  • Recompute the flight's deviation features (std and |bias| per flight dynamics factor) from all samples so far. Reuse inference/features.py, not a second implementation.
  • Scale them with the saved scaler and assign the nearest centre.
  • Withhold an assignment until the flight has enough samples to make it meaningful.

Open questions (needs-info)

  • Refresh rate: how often to reassign (every N seconds, every N samples, or on demand).
  • Minimum flight length before the first assignment. std from a few seconds of data is noisy.
  • Window: all samples since flight start, or a rolling window.
  • Where the assignment is shown (console, relay GUI, an iMotions event, a file).
  • When to refit the reference cohort as more pilots are logged.

Describe alternatives you've considered

  • Refit the whole cohort on every refresh. Rejected because KMeans numbers groups differently on each fit, so a pilot's group could change label without any change in how they fly. It would also move every other pilot's assignment.
  • Assign only at the end of the flight. Simpler, but it is the batch case from Build a baseline pilot clustering model #216 and not what the client asked for.

Additional context
Blocked by #216 (batch fit to save), #211 (pilot_id/flight_id) and #213 (targets; without them there is no deviation to compute). Units must be post-#196, see docs/telemetry_schema.md.

Activity

  1. self-assigned this
    on Oct 7, 2026
  2. jmacd867 commented on Oct 15, 2026

    @jmacd867
    CollaboratorAuthor

    One more open question: the client's current data is from Prepar3D at about 1 Hz, and the move to X-Plane (our plugin, 20 Hz) comes later. Group centres saved from a Prepar3D batch fit may not apply to live X-Plane flights. The flight models differ, and std depends on the sample rate. Before live assignment uses a Prepar3D reference cohort, check that the same flying gives the same features in both simulators, or resample to a common rate.

  3. jmacd867 commented on Oct 16, 2026

    @jmacd867
    CollaboratorAuthor

    Answers from the client meeting (`docs/client_meeting_notes.md`):

    • Real-time clustering is wanted, and the client does not expect it to be computationally heavy.
    • It should run on the iMotions computer (the client already shows the pilots' views there), not on the simulator machine. That answers where the code runs. Where the result is shown is still open; we assume not to the pilot, since it could add stress.
    • Refresh rate is open. The client said it depends on the sensor rates: simulator about 90 Hz, EEG 256 Hz, EDA 128 Hz, eye tracking 51 Hz (as transcribed, unconfirmed). iMotions synchronizes timestamps. The client is not sure there is a common rate to cluster at.
    • The Prepar3D export is 1 Hz and the client cannot change it (closed software). That reinforces the earlier comment about a Prepar3D reference cohort not matching X-Plane at 20 Hz.
  4. Phlabry commented on Oct 23, 2026

    @Phlabry
    Collaborator

    Status update on the blockers listed here. #216 merged in #225 and #211 merged in #219, so the saved batch fit and the pilot_id/flight_id columns both exist now. #247 adds the other missing piece, a live stream the Python side can read while iMotions is recording.

    That leaves #213. Without targets there is no deviation to compute from real flights, so a live assignment can only be demonstrated on synthetic pilots. The open questions in this issue about refresh rate, minimum flight length and where the assignment is shown are still unanswered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:mlML / inference workenhancementNew feature or requestneeds-infoWaiting on reporter for more informationpython

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions