Skip to content

Test pass: every suite green after phases 9b–12 - #72

Merged
LikelyCause merged 5 commits into
mainfrom
test-pass
Nov 16, 2026
Merged

LikelyCause merged 5 commits into
mainfrom
test-pass

Conversation

@LikelyCause

Copy link
Copy Markdown
Owner

The test pass after phases 9b–12, which merged without running their tests on purpose. Every suite is green.

Stale tests (the code changed on purpose)

  • "in its team": the 9b final review (M9) changed the GPU waiting copy on purpose, because "your team" was wrong for admins and other viewers outside the team. The unit tests, the e2e test and the user guide now follow the code.
  • Settings › Application is admin-only since phase 10, so it is gone from the member section list.
  • Password is the default auth mode (phase 11):
    • The helm case for pick mode now sets auth.mode=pick and auth.devPick, and a new case pins the password default.
    • The CI stack job starts its Compose stack with RAINFOREST_AUTH_MODE=pick and RAINFOREST_DEV_PICK=true, because the stack specs sign in by picking a user.
    • smoke.sh seed mode documents that it needs a pick stack. Its chart path already uses ci/dev-values.yaml.
    • test_auth sets the dev flag and stubs the sign-in audit and the credentials lookup. It also covers /me's new fields.
  • MLFLOW_LOGGING is now passed through docker-compose.yml and listed in .env.example.
  • test_access_lint: config.py reads meta.app_settings with its own connection. db.py's pool is built from get_settings(), which calls this read, so going through db.py would recurse. It is now in DRIVER_HOMES, with that reason.
  • Other stale tests:
    • oidc config fields.
    • Unique emails since migration 046 (test_mlflow).
    • A study's ends_by includes the last-trial margin.
    • A cluster blip on create does not burn a try.
    • The migration-replay tripwire moves to 046.
    • The route matrix's vault table keeps a route's own gate.
    • A running GPU job that is still starting keeps its waiting line.

New tests

  • Phase 10 (test_app_settings.py):
    • An override applies, and a reset brings the env value back.
    • A Lab choice above the deployment's ceiling is refused, and nothing is saved.
    • Every change is audited with its old and new value.
    • Bad values are refused, and members can't use the routes.
  • Phase 11 (test_passwords.py, 36 tests):
    • argon2id hashing and the password rules.
    • Lockout per account and per IP, with CLIENT_IP_HOPS, and the lock persisted before the refusal.
    • No user enumeration.
    • One-time links: stored hashed, single use, expiry, revoked by a newer link.
    • Must-change on first sign-in.
    • Pick mode refused without RAINFOREST_DEV_PICK.
  • 9b D1:
    • WaitingForQuota is a place in line.
    • The redaction filter keeps an untouched record's args (uvicorn's AccessFormatter).
    • The sweep leaves a Job's own pod alone.
    • Each test fails when its fix is reverted.
  • FakeKueue: a Job's pod now carries the template's GPU and job-id labels and an ownerReferences entry, and objects() reports owner_kind as the real client does. This catches the baa26aa class of bug.

Counts (local)

  • API:
    • make test-full: 3019 passed.
    • Serial run on a fresh database: see the comment.
  • UI:
    • typecheck, eslint and prettier are clean.
    • vitest: 750 passed.
    • The build passes.
    • nginx/test.sh passes.
  • Playwright fixture and gallery: 61 passed, run on dev-01 in the CI image.
  • helm-unittest: 247 passed.

No product code changed: no test showed a real bug.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LqNnEU86Q4fmrJnN2rXovS

LikelyCause and others added 5 commits November 16, 2026 10:27
…ings sections, pick mode no longer the default), MLFLOW_LOGGING in Compose, config.py's settings read allowed

- The 9b final review (M9) chose "first in its team" on purpose; the unit, e2e and user-guide copy follow it.
- Settings › Application is admin-only since phase 10; the member section list drops it.
- helm: pick mode's case sets auth.mode=pick and devPick; a new case pins the password default.
- CI stack job and smoke.sh seed mode: the stack specs sign in by picking, so the stack runs pick with the dev flag.
- docker-compose.yml passes MLFLOW_LOGGING (test_config_wiring).
- test_access_lint: config.py reads meta.app_settings with its own connection (db.py's pool is built from
  get_settings(), which calls it), listed in DRIVER_HOMES with that reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNnEU86Q4fmrJnN2rXovS
…, one-time links, must-change, pick needs the dev flag)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNnEU86Q4fmrJnN2rXovS
…e 10 and 9b D1

Stale tests (the code changed on purpose):
- test_auth: pick mode needs RAINFOREST_DEV_PICK; the in-memory store stubs the sign-in audit and credentials;
  /me's new password fields; the pick is audited.
- test_oidc: /api/auth/config names which sign-ins are on (password, oidc).
- test_mlflow: emails are unique since 046, so the trainer Ana is made once.
- test_gpu_waiting: a study's ends_by is policy.study_max_exec_s (its hours plus the last trial's margin).
- test_gpu_supervise: a cluster blip on create requeues without burning a try (counts=False).
- test_rls_app: the replay tripwire moves to 046 (045 and 046 replay cleanly).
- test_route_matrix: the vault table keeps a route's own gate (reset-link is a 404 in pick mode).

New:
- test_app_settings: an override applies at once and a reset drops the row; a Lab choice above
  NOTEBOOK_CPU_LIMIT/NOTEBOOK_MEMORY_LIMIT is refused with nothing saved; every change is audited; bad values
  and members refused.
- 9b D1: WaitingForQuota is in line (dispatch and kube.workload_state), the redaction filter keeps an untouched
  record's args (uvicorn's AccessFormatter), the sweep leaves a Job's own pod. Each fails with its fix reverted.
- FakeKueue: a Job's pod carries the template's GPU and job-id labels and an ownerReferences to its Job, and
  objects() reports owner_kind as the real client does (the baa26aa class of bug); wait_for_quota().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNnEU86Q4fmrJnN2rXovS
…s as an upgrade would

Run serially on one database, two replays broke later files: 031-033's replayed 041 (which drops every policy)
and then only 042-044, so 045 and 046's tables lost theirs (test_passwords, test_every_meta_table_has_rls); and
010's replay put back its permissive SELECT policies, which 041 drops on an upgrade. Every migration from 041 on
follows now, and 010's policies are dropped after its replay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNnEU86Q4fmrJnN2rXovS
@LikelyCause

Copy link
Copy Markdown
Owner Author

Serial API run on a fresh database (after 751972e, which fixed two migration replays that only broke a shared, serial database): 3019 passed, 0 failed, 0 skipped. Client tests: 79 passed. MLflow sidecar: 3 passed.

@LikelyCause
LikelyCause merged commit 8646a48 into main Nov 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant