Industrial Automation Challenge Track 1 grader format clarification

[Harshul-24]

Ready-to-send note: Track 1 hosted grader issue

Subject: Industrial Automation Challenge Track 1 grader format clarification

Hello,

We have an AIF KG-governed Industrial Automation Challenge submission package.
Track 2 has been accepted and scored successfully:

• competition: industrial-automation-challenge-track-2
• submission ref: 55368732
• status: SubmissionStatus.COMPLETE
• public score: 1.00000

Track 1 appears blocked by the hosted grader rather than by local generation.
We tried the public starter-kit format, the visible Kaggle metadata-style
format, and numeric answer encodings. All variants reach Kaggle upload, but
the hosted evaluation fails.

Observed Track 1 refs:

Ref	Submission shape	Result
55368741	3,048-row full test file	Submission has 3048 rows but 2758 were expected
55368816	290-row multi_choice_question_answering tail only	Submission has 290 rows but 2758 were expected
55368833	2,758-row id,answer with option letters	Submission column answer must be a number
55368957	2,758-row numeric answer, A=0	Evaluation metric raised an unexpected error
55368978	2,758-row numeric answer, A=1	Evaluation metric raised an unexpected error
55408937	2,758-row metadata-preserving numeric answer, A=0	Submission column option_desc must be a number
55408938	2,758-row metadata-preserving numeric answer, A=1	Submission column option_desc must be a number
55488733	2,758-row public eight-column metadata schema with letter option	Submission column option must be a number
55488747	2,758-row all-numeric eight-column schema, A=0	Evaluation metric raised an unexpected error
55488774	2,758-row sequential-ID all-numeric eight-column schema, A=0	Solution and submission values for id do not match
55488794	2,758-row all-numeric eight-column schema, A=1	Evaluation metric raised an unexpected error
55488832	2,758-row minimal id,option numeric schema, A=0	Evaluation metric raised an unexpected error
55488849	2,758-row minimal id,option numeric schema, A=1	Evaluation metric raised an unexpected error

Could you please clarify the required Track 1 submission schema and answer
encoding, or check whether the hosted Track 1 metric has an internal error?

For reproducibility, a private GitHub handoff repository is available:

https://github.com/Harshul-24/aif-kg-industrial-automation-challenge

The private GitHub handoff repository is shared with biswasprateek. As of the
latest refresh on 2026-08-13 12:35 CDT, pending invitations were empty and
biswasprateek had write permission.

The main local documentation is:

• docs/research/industrial-automation-challenge/LEADERBOARD_SUBMISSION_REPORT.md
• docs/research/industrial-automation-challenge/COMPLETION_AUDIT.md
• docs/research/industrial-automation-challenge/AIF_KG_ASSETOPS_TRACK2_RESEARCH_SUMMARY.md
• docs/research/industrial-automation-challenge/runs/kaggle-live-status-20260813/README.md

Thank you.