Skip to content

Paper 1: official corrected-2009 audit + exploratory refit + fresh 2015 replication (author review) - #2

Draft
devissaputra wants to merge 31 commits into
research/kt-arxiv-verification-20261022from
research/kt-fresh-cohort-20261022
Draft

devissaputra wants to merge 31 commits into
research/kt-arxiv-verification-20261022from
research/kt-fresh-cohort-20261022

Conversation

@devissaputra

@devissaputra devissaputra commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Paper 1 — completed feasible scientific/technical work (22 October 2026)

Author-review working preprint, 11 pages. Draft PR; do not merge without the author reviewing claims and contributions.

1. Official corrected 2009 provenance: now directly established as NON-equivalent

Official maintainer-linked corrected ASSISTments 2009 release (SHA-256 162ef8d2d28bcbfea6591a282994062bd8d5eaa00636544292a0d268dca6e5da) has 4,163 learners and 283,105 skill-labeled response events. Frozen transformed Atomi mirror (SHA-256 2053b7ea63f1d5ea953834bf82ed89543e1f11bae1e3e7535c2f9d7f49d4dda7) contains 4,148 learners and 274,331 events.

All 4,148 mirror learner IDs exist in the corrected source. Among these, 3,331 have the same normalized skill-response sequence ordered by numeric order_id; 817 do not. Exact parity of the full source is therefore disproved. Student-level raw data have not been committed. Evidence: source comparison workflow 39773920168, aggregate report, runner.

2. Official corrected 2009 exploratory refit: completed

A new frozen, learner-disjoint 2009 corrected-source source-sensitivity experiment used 2,914 train / 624 validation / 625 test learners, 48,842 held-out interactions:

  • Enriched PFA AUC 0.69508, Brier 0.19675, log loss 0.58086; LBFGS converged in 930 iterations.
  • Extended GRU AUC 0.73243, Brier 0.18507, log loss 0.55124; best validation epoch 13.
  • Paired pooled Brier GRU-PFA −0.0116811, 95% learner-cluster interval [−0.0137180, −0.0098625], 2,000 resamples.
  • Exploratory because the 2009 environment and predictive direction were already studied; cannot prove architecture-specific causation.

Evidence: corrected source run 39774414969, source code, protocol, JSON.

3. ASSISTments 2015 independent-year replication: completed

A first nonconverged PFA pilot was retained as diagnostic/inconclusive. Stage 2 excluded its 5,000 learners, fit PFA and GRU on new training learners and evaluated 750 untouched held-out learners / 26,697 events:

  • Converged enriched PFA Brier 0.18695, AUC 0.68129;
  • Extended GRU Brier 0.18227, AUC 0.69968;
  • Brier delta −0.0046813, bootstrap 95% interval [−0.0060730, −0.0033457].
    Evidence: Actions 39754854426.

4. Final artifact integration and new QA

Remaining external gates, not delegated

  1. Author personally verifies and approves contribution, raw source inference and disclosures.
  2. Optional independent human scholarly review (arXiv moderation is not academic peer review). Not claimed as performed.
  3. Author's arXiv account/endorsement/license/preview and final submission. Not performed.

Limits: The 2009 strict parity is refuted. Findings compare unequal information representations and model capacities, do not identify a recurrence-only effect or teaching efficacy, and do not establish wide institutional transportability. These limitations are explicit in the manuscript.

@devissaputra devissaputra changed the title Empirical confirmation: preregistered 2015 external-cohort replication (draft) Research verification: fresh ASSISTments 2015 cohort confirms predictive Brier direction (draft) Oct 22, 2026
@devissaputra devissaputra changed the title Research verification: fresh ASSISTments 2015 cohort confirms predictive Brier direction (draft) Paper 1: integrate verified 2015 disjoint-cohort replication into scoped preprint (draft) Oct 22, 2026
@devissaputra devissaputra changed the title Paper 1: integrate verified 2015 disjoint-cohort replication into scoped preprint (draft) Paper 1: canonical 10-page manuscript updated with verified 2015 replication (draft) Oct 22, 2026
@devissaputra devissaputra changed the title Paper 1: canonical 10-page manuscript updated with verified 2015 replication (draft) Paper 1: official corrected-2009 audit + exploratory refit + fresh 2015 replication (author review) Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant