Week 2 Day 4: compact RMSNorm, RoPE, and SwiGLU primitives - #320
Merged
Merged
Conversation
AI-Assisted: GPT-6 Sol + Sentinel
AI-Assisted: GPT-6 Sol + Sentinel
AI-Assisted: GPT-6 Sol + Sentinel
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Add Week 2 Day 4 compact model primitives on top of merged Day 3. The active lesson, starter seams, readable reference controls, supplied tests, CLI selectors, benchmarks, and shipped-day manifest progress cumulatively through
rmsnorm,rope, andswigluaftersimd-matmul. Learners implement the register-cached RMSNorm kernel and its wide fallback, RoPE, and SwiGLU; Tasks 1–3 also assign the three native CPU evaluators to report GPU-only errors. Day 5 tiled attention andselectedremain deferred.The chapter preserves the RMSNorm reduction stages, RoPE head-pair math, SwiGLU equation, cumulative model checks, and matched measurement method. Former Week 2 URLs not reused for active Day 4 remain historical. Required measurement uses the setup-cached Qwen3-0.6B model; a 4B repeat is optional. This PR makes no speed claim.
Local evidence
a4e49d7de3c281f8ada41b5c04c0482283bcd7bcat head18613b83bb4e743f1bae89bde02f8950b4947166, directly descended from merged Day 3 maindf51f93acf1ba24e3aa02d1460d870bd04dc7978. The cumulative diff is 31 Day 4 paths. The final lesson correction changed only one page; engineering and tests retain their accepted combined blobs.35979136982succeeded on this exact public head: 398 tests collected, 396 passed, 2 skipped. The hosted log does not name the skipped nodes.Open gates
This PR is Draft. Hosted macOS shipped-day CI has passed on this head. Fresh independent reviews of the exact public head, base, and body, interpretation of the hosted skips, and current repository rules remain gates before Ready or ordinary merge. No Day 4 performance benchmark was run; the separate full-course Draft #316 screen does not establish a Day 4 speedup.
Chi approved normal website updates for this five-day Week 2 batch. The ordinary main-branch Pages workflow may run after a separately gated merge. This PR does not authorize a release, tag, settings or visibility change, manual deployment, or later-day work.