Skip to content

Collect pilot metadata and join it to telemetry for clustering #214

Open

Description

@Phlabry

Is your feature request related to a problem? Please describe.
The client wants pilot rating and flight hours to factor into clustering. Neither exists anywhere in the repo, and there is no place to put them.

Describe the solution you'd like
A small CSV keyed by pilot_id with certification_level, flight_hours and pilot_rating, plus a loader in inference/ that joins it to the telemetry at feature engineering time. Handle missing pilots without dropping their telemetry silently.

Additional context
Field definitions are in docs/telemetry_schema.md section 4b. Depends on #211 for the join key. The category set for certification_level (Student, Private, Commercial, ATP) still needs confirming, and so does whether pilot_rating is a licensing rating or an evaluative score, which changes its column type. Both questions are with the client.

Activity

  1. jmacd867 commented on Oct 3, 2026

    @jmacd867
    Collaborator
    • Validate certification_level against an ordered category list and map it to an ordinal 0-3 (Student, Private, Commercial, ATP). The set is still to be confirmed with the client.
    • Store pilot_rating as a string and keep it out of clustering until the client says whether it is a licensing rating or an evaluative score. Keep a single clearly named place to change that.
    • Pilots missing from the metadata are reported rather than silently dropped.
  2. added
    area:mlML / inference work
    needs-infoWaiting on reporter for more information
    on Oct 3, 2026
  3. Phlabry commented on Oct 5, 2026

    @Phlabry
    CollaboratorAuthor

    @jmacd867 the join key is in review as #219. Columns are pilot_id and flight_id, written by data_logger.py into every row, so you can build the join against those names now.

  4. jmacd867 commented on Oct 16, 2026

    @jmacd867
    Collaborator

    Answers from the client meeting (`docs/client_meeting_notes.md`):

    • Pilot rating is the type of certification, not a score. So it is not an evaluative score and the column can stay a string. Still unclear whether it differs from `certification_level`.
    • The client will send certification level, total flight hours and rating, for the pilots in the sample.
    • For performance groups, the client prioritizes total flight hours over certification level.
    • The client agreed to metadata being used at the end, to interpret groups and not to form them. This matches ADR 0001.
    • The client did clustering by hand last year and wants it automated.

    The `certification_level` category set (Student, Private, Commercial, ATP) is still to be confirmed.

  5. added this to the v1.0 milestone on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:mlML / inference workenhancementNew feature or requestneeds-infoWaiting on reporter for more informationpython

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions