Skip to content

Week 2 Day 4: compact RMSNorm, RoPE, and SwiGLU primitives - #320

Merged
skyzh merged 4 commits into
mainfrom
course/week2-day4-compact-primitives
Sep 24, 2026
Merged

skyzh merged 4 commits into
mainfrom
course/week2-day4-compact-primitives

Conversation

@skyzh

@skyzh skyzh commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Scope

Add Week 2 Day 4 compact model primitives on top of merged Day 3. The active lesson, starter seams, readable reference controls, supplied tests, CLI selectors, benchmarks, and shipped-day manifest progress cumulatively through rmsnorm, rope, and swiglu after simd-matmul. Learners implement the register-cached RMSNorm kernel and its wide fallback, RoPE, and SwiGLU; Tasks 1–3 also assign the three native CPU evaluators to report GPU-only errors. Day 5 tiled attention and selected remain deferred.

The chapter preserves the RMSNorm reduction stages, RoPE head-pair math, SwiGLU equation, cumulative model checks, and matched measurement method. Former Week 2 URLs not reused for active Day 4 remain historical. Required measurement uses the setup-cached Qwen3-0.6B model; a 4B repeat is optional. This PR makes no speed claim.

Local evidence

  • The frozen tree is a4e49d7de3c281f8ada41b5c04c0482283bcd7bc at head 18613b83bb4e743f1bae89bde02f8950b4947166, directly descended from merged Day 3 main df51f93acf1ba24e3aa02d1460d870bd04dc7978. The cumulative diff is 31 Day 4 paths. The final lesson correction changed only one page; engineering and tests retain their accepted combined blobs.
  • Both native extensions build. On the final head, independent review passed the Day 4 reference selector (7/7), the Day 4/RMSNorm boundary/interface set (41/41), and the unchanged extension interface test. Copied learner tests reach named Tasks 1–3 TODOs; unfinished cumulative model tasks remain expected learner failures. The shipped-day manifest collects 398 nodes with Day 5 deferred.
  • On the preceding combined author tree, the printed shipped-day gate passed 397 tests with one optional-model skip. The final one-page successor passed its focused, book, and integrity checks. Hosted macOS run 35979136982 succeeded on this exact public head: 398 tests collected, 396 passed, 2 skipped. The hosted log does not name the skipped nodes.
  • mdBook, sitemap, rendered local links, command syntax, diff hygiene, and strict Git integrity passed. Four fresh independent local reviews on the final head returned factual/preservation GO, correctness/test GO, learner WALKABLE, and static accessibility PASS. These are local verdicts only.

Open gates

This PR is Draft. Hosted macOS shipped-day CI has passed on this head. Fresh independent reviews of the exact public head, base, and body, interpretation of the hosted skips, and current repository rules remain gates before Ready or ordinary merge. No Day 4 performance benchmark was run; the separate full-course Draft #316 screen does not establish a Day 4 speedup.

Chi approved normal website updates for this five-day Week 2 batch. The ordinary main-branch Pages workflow may run after a separately gated merge. This PR does not authorize a release, tag, settings or visibility change, manual deployment, or later-day work.

@skyzh
skyzh marked this pull request as ready for review September 24, 2026 09:27
@skyzh
skyzh merged commit 68642f0 into main Sep 24, 2026
1 check passed
@skyzh
skyzh deleted the course/week2-day4-compact-primitives branch September 24, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant