Skip to content

Load the client's Prepar3D TSV exports into the telemetry schema #228

Open

Description

@Sanjana-Siddani

Is your feature request related to a problem? Please describe.
features.py expects the logger schema (pilot_id, flight_id, altitude, heading, vertical_speed, velocity, target_*). The client's files were exported from Prepar3D, not X-Plane, and their headers change from file to file. Nothing in inference/ can read them.

What the client files look like (checked across all 19):

  • 5 pilots (P001, P002, P003, P004, P007) and 19 flights, named P001_T1.tsv and so on
  • Tab-separated, sampled at about 1 Hz, latin-1 encoded (the degree sign is mis-encoded)
  • The header changes from file to file: Nose gear appears in 5 files (always 0), Flaps position percent is missing in P004_T1 and P004_T3, and 4 files carry a stray Engine N … column (Engine 2–4, every value 0)
  • Latitude/Longitude appear twice, and every file ends in 2 empty columns

Describe the solution you'd like
Add a pure inference/prepar3d_export.py and a thin CLI load_client_data.py.

  • Map columns by header name, never by position. Strip the unit suffix after the comma.
    • Altitude MSL → altitude (ft)
    • Heading (true) → heading (deg true; note that the schema doc says magnetic)
    • Indicated Airspeed → velocity (kts)
    • Vertical speed → vertical_speed (fpm)
    • Keep pitch, bank, throttle, elevator_input, aileron_input, rudder_input, elevator_trim and flaps (when present).
  • Derive pilot_id and trial from the file name (P001_T3 → P001, 3) and set flight_id to P001_T3.
  • Drop the duplicate Lat/Lon, the empty trailing columns and stray Engine N columns. Make flaps and nose_gear NaN when the column is missing, not 0.
  • These files come from Prepar3D, not through our plugin, so the Plugin applies metric conversion factors to pilot altitude and airspeed that are already in feet and knots #196 rescaling does not apply. Sanity-check the units: cruise IAS should be about 70–130 kt and the field elevation near 1,100 ft.

Acceptance criteria

  • All 19 files load into one long table with one row per sample
  • Tests cover each header variant: Nose gear present, Flaps missing, stray Engine column
  • A test checks that a missing required column raises an error naming it
  • docs/telemetry_schema.md gains a section on the client export format and its mapping

Update (source of the data): the client says these files "were derived from Prepar3D, as I currently do not have X-Plane data", and that the plan is to build the prototype on them and migrate to X-Plane later. Keep this loader separate from the plugin/logger path: it is a loader for Prepar3D exports only. The plugin will be the source once the client moves to X-Plane.

Activity

  1. changed the title [-]Load X-Plane Data Output TSVs into the telemetry schema[/-] [+]Load the client's Prepar3D TSV exports into the telemetry schema[/+] on Oct 15, 2026
  2. jmacd867 commented on Oct 15, 2026

    @jmacd867
    Collaborator

    Edited the body: the client's files come from Prepar3D, not X-Plane (her note: "they were derived from Prepar3D, as I currently do not have X-Plane data"). Also corrected the header counts after re-checking all 19 files: Nose gear is in 5 files, not 6, and always reads 0; stray Engine N columns are in 4 files, not 5, and are all zeros. The proposed module is renamed prepar3d_export.py.

  3. jmacd867 commented on Oct 16, 2026

    @jmacd867
    Collaborator

    Confirmed by the client (`docs/client_meeting_notes.md`):

    • The data is TSV because that is what Prepar3D exports. The export is 1 Hz and cannot be changed (closed software).
    • The route headings are true headings, so mapping `Heading (true)` to `heading` is right, and the schema doc's "magnetic" wording needs the heading-reference issue.
    • Units wanted: feet, knots and feet per minute.
    • Pilots 04 and 07: the meeting transcript says two pilots in the full set have missing data, but P004 and P007 are in our sample, so the transcript numbers are probably wrong. The client will say which pilots are affected.
    • The full set has 44 pilots, with 1 to 6 trials each. We hold 5 pilots.
  4. added this to the v1.0 milestone on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions