Skip to content

Add head-resolved four-head nanoGPT spectral analysis - #121

Merged
charlesmartin14 merged 1 commit into
mainfrom
codex/four-head-resolved-analysis
Aug 30, 2026
Merged

Add head-resolved four-head nanoGPT spectral analysis#121
charlesmartin14 merged 1 commit into
mainfrom
codex/four-head-resolved-analysis

Conversation

@charlesmartin14

Copy link
Copy Markdown
Member

Summary

  • adds a standalone post-hoc analyzer for the completed four-head nanoGPT campaign
  • splits packed Q/K/V projections into four output-row head matrices
  • splits packed O into four input-column head matrices
  • runs WeightWatcher separately on all 16 attention-head matrices per checkpoint
  • produces raw and clip-xmax alpha trajectories, full WeightWatcher CSV output, five-seed confidence intervals, paired optimizer differences, zoomed plots, and an HTML report
  • validates completion markers, permanent checkpoint inventory, tensor shapes, finiteness, and checkpoint SHA-256 identities
  • caches every checkpoint result so interrupted analysis resumes safely

Validation

  • python3 -m py_compile analyze_four_head_by_head.py
  • synthetic 128x128 slicing smoke test verified 16 distinct matrices: Q/K/V as 32x128 row slices and O as 128x32 column slices

This is read-only post-hoc analysis; it does not modify campaign results or require retraining.

@charlesmartin14
charlesmartin14 merged commit ad0d39d into main Aug 30, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant