[WIP] Ryot V11 - #1832
Draft
IgnisDa wants to merge 4221 commits into
Draft
[WIP] Ryot V11#1832IgnisDa wants to merge 4221 commits into
IgnisDa wants to merge 4221 commits into
Conversation
Re-export EntityDecoder and its entity maps from the sandbox-sdk fast-xml-parser module and build an equivalent decoder per parse in the MyAnimeList import. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move Effect language-service diagnostics from the tsconfig plugin (tsc mode)
to oxlint's type-aware mode via the @effect/tsgo oxlint patch.
Changes:
- Bump @effect/tsgo 0.45.0 -> 0.46.0; 0.45 does not support the installed
oxlint 1.85.0 / oxlint-tsgolint 7.0.2003.
- prepare: `effect-tsgo patch --no-typescript --oxlint`. No tsconfig loads
the plugin anymore, so patching TypeScript is unnecessary.
- Root .oxlintrc.json extends @effect/tsgo's `recommended` preset and sets
the 10 rules previously elevated in the backend tsconfigs to error
repo-wide: new-promise, async-function, node-builtin-import,
global-date-in-effect, process-env-in-effect, global-fetch-in-effect,
global-timers-in-effect, prefer-schema-over-json,
global-console-in-effect, global-error-in-effect-failure.
- e2e/.oxlintrc.json keeps its previous split: new-promise, async-function,
node-builtin-import, global-fetch-in-effect, prefer-schema-over-json off;
the other five at error.
- Remove the @effect/language-service plugin from the apps/server,
kernel/backend and e2e tsconfigs, and the tsgo `$schema` from those and
migrations/v10-rust (it only validated plugin options).
Fallout (each workspace linted like `check`, without --fix):
- Root `categories` (correctness: error, suspicious: warn) also enables
Effect rules the preset leaves off: any-unknown-in-error-context as error
and strict-effect-provide as warning (890).
- `check` now fails in 18 workspaces, all on Effect rules:
plugins/media 906 (any-unknown 504, async-function 193,
global-error-in-effect-failure 191)
kernel/client 538 (async-function 465, new-promise 32,
any-unknown 24)
packages/client-sdk 226 (async-function 188, new-promise 36)
kernel/backend 73 (any-unknown)
apps/website 54
packages/sandbox-sdk 49 (incl. 1 floating-effect)
plugins/fitness 37
packages/client-ui-sdk 32
packages/kernel-renderers 31
apps/browser-extension 30
plugins/fixture 18
packages/plugin-archive 16
packages/vite-compiler 11
packages/cli 8
packages/client-plugin-compiler 7
packages/testing 7
apps/docs 1
migrations/v10-rust 1
async-function accounts for ~1070 of these, mostly in React/browser code.
- Passing: apps/server, e2e, packages/{config, contract, plugin-kit,
transactional, ryotql, ryotql-recipes, sandbox-compiler, ts-utils,
typescript-compiler, client-plugin-contract}.
- ~1470 Effect warnings remain (strict-effect-provide, schema-number,
prefer-typed-schema-decoder, ...).
- Editors lose Effect hovers/quick fixes/refactors from the TS server;
diagnostics come from the oxlint extension only.
- oxlint is now a patched binary: CI must run `prepare` (bun install does),
and oxlint/oxlint-tsgolint upgrades are gated on @effect/tsgo support.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Make the category-enabled Effect severities explicit and add path-scoped warning overrides for existing async, Promise, and typed-error migration debt. Preserve the recommended preset and strict-effect-provide warnings so the remaining findings stay visible until their owning workspaces are migrated. Convert the docs configuration generator to Effect orchestration, encode migration report details through their schema, and remove a discarded Effect from the sandbox SDK type fixture. Document the temporary overrides and their removal criteria in docs/effect-lint-baseline.md. Verified with bun turbo --output-logs=full check and bun turbo --filter='!@ryot-app/e2e' --output-logs=full test.
…ature layers Introduce DatabaseSession with fiber-local transaction propagation, root and transaction state checks, typed state failures, and focused commit, rollback, interruption, and retry coverage. Capture the session in repositories and migrate services, workflows, test infrastructure, and the v10 data migration away from ambient Database injection. Move historical backup writes into restore-owned persistence, preserve transaction ownership and post-commit boundaries, and localize backend feature Layers with distinct runtime and migration variants. Repair service requirements and update backend guidance without changing product or sandbox policy.
…henticated mutations Add a contract-owned endpoint constructor that requires an explicit DemoAccessPolicy whenever an authenticated POST, PUT, PATCH, or DELETE route is defined. Migrate all applicable contract modules to the structural API while keeping public and admin endpoints on their appropriate construction path. Cover the assembled mutation policies and raw-route exceptions in contract tests, and document the construction boundary so a source-text architecture scanner is no longer required.
…ners Move generic sandbox runtime inputs, registry, payload, source walking, and process capture into the sandbox compiler. Keep kernel-specific runner generation under kernel/backend/tooling and move server assembly, development orchestration, cache warming, and runtime smoke verification to app-owned entrypoints without changing production payload, archive, or Deno behavior. Update package commands, Turbo inputs, and the Docker assembly path. Remove the obsolete backend architecture, cycle, duplicate-layer, and source-text checker scripts now that service Layers, restore persistence, contract construction, and Oxlint enforce their boundaries.
…bridge requests as Effects Expose the client-safe Effect authoring surface and typed RyotClientError, return Effects from asynchronous adapter capabilities, and replace manually managed request Promises with interruptible bridge completions. Preserve existing wire messages, pending limits, session closure, Blob uploads, and supported cancellation paths. Run React query, mutation, presentation, refresh, and upload work on Effect fibers while leaving Promise conversion at React or platform boundaries. Update SDK, UI SDK, compiler, and contract tests and documentation; retain client API, compiler, and bridge versions at 1 for this greenfield project.
…e and feature Layers Adapt kernel client queries, operations, uploads, storage, and plugin bridge handlers to the Effect-native SDK. Run incoming bridge requests in fibers and interrupt owned work on cancellation, document replacement, and session close while retaining the existing protocol and synchronous React startup. Move client service wiring into feature-owned Layers, migrate kernel renderer queries to Effect definitions, and update client and renderer tests for navigation, entity interest, interruption, and React framework boundaries.
…orkflow failures Preserve implementation failure channels through script, provider, operation, automation, and workflow definitions instead of erasing them to unknown. Keep SandboxHostError at host capability boundaries and map external YouTube client rejections into typed Effect failures. Update type and replay coverage plus SDK and plugin-kit guidance, while leaving the neutral plugin-kit Effect surface and sandbox security and replay protocol unchanged.
…yped Effects Migrate media, fitness, and fixture client queries, mutations, and entity presentation loaders to compose the new Effect-returning client capabilities directly. Move sequential client writes into mutation definitions and keep React rendering and actual framework callbacks at their boundaries. Replace first-party sandbox Promise orchestration and global Error failure channels with typed domain failures, decode external JSON through schemas, and retain SandboxHostError for host capabilities. Update plugin tests and media authoring guidance without changing provider or product behavior.
… Effect adapters Make CLI watch and manifest work use the Effect clock and schema-backed JSON; read plugin archives from async iterables through Effect Streams while preserving canonical ZIP bytes. Give Vite compiler and Testcontainers boundaries typed failures and keep native Promise conversion at the external tool interfaces. Add focused failure-mapping and deterministic-output coverage, update package guidance, and preserve separate client and sandbox compiler engines.
…lows through Effect Move browser-extension storage, message, metadata, and retry workflows into Effect programs with WXT callbacks as adapters. Convert website server helpers and React Router loaders and actions to Effect-based orchestration, running native Promises only where framework entrypoints require them. Keep rendering pure, remove unused extension cache methods, and preserve website and extension build behavior while retiring their temporary async lint exemptions in the final enforcement change.
…ect scopes Convert shared fake HTTP, request, provisioning, WebSocket, process, and client fixtures to scoped Effect resources instead of leaving Effect and re-entering through Promises. Keep API and browser tests on it.live with effect-playwright and use narrow inline exceptions at real browser-realm and third-party callback boundaries. Enable normal Effect lint for E2E with only the two untouched seed scripts exempted, document the boundary policy, and make the client-plugin lifecycle test scroll the active retained screen before clicking. Preserve worker counts, database pool limits, and sandbox settings.
…hout migration overrides Promote service requirement, Layer composition, generator, Effect function, Promise round-trip, and interruption diagnostics to errors. Replace the workflow source scanner with normal import/property restrictions, keep strict-effect-provide as guidance, and remove every temporary per-workspace warning downgrade. Record stable application Effect and Layer rules in root AGENTS.md and retire the temporary lint baseline document after its migration queue is empty. Keep the task plan text intact apart from its final newline.
…andbox smoke imports Replace the five relative cross-package imports in the production sandbox runtime smoke image with @ryot-app/kernel-backend subpath imports. The backend package already exports these source modules, so server assembly no longer reaches into its workspace directory by relative path.
…ature modules Replace boot-owned repository and service dependency registries with feature-owned Layers for content, auth, uploads, imports, integrations, plugins, sandbox, backup, and workflow operations. Keep explicit runtime and migration Layer identities and leave boot responsible for infrastructure, workflow definitions, and cross-feature composition. Extract entity mutation persistence into its own module so schema revalidation, lock ordering, writes, and trigger planning remain together, while the entity service owns transactions, deadlock retries, and post-commit dispatch. Share the entity snapshot and draft comparisons across the service and persistence boundary.
…olated database setup Read and split the generated baseline migration in one backend test helper, then replay its statements through each suite’s chosen SQL executor. Keep schema, transaction, and domain fixture decisions with their owning tests. Provide a scoped create/drop helper for tests that require a separate PostgreSQL database. Migrate the existing database-backed suites to these helpers without changing their seed data or transaction assertions. Verified 98 tests in 11 focused suites against PostgreSQL and passed backend typechecking.
…e in DatabaseSession Remove redundant mapDatabaseErrors calls around DatabaseSession.transaction across backend services, workflows, and the authentication adapter. The session already converts native Drizzle and SQL failures into DbError while preserving domain failures and invalid-session state, so these outer passes contributed no additional mapping. Leave direct executor query mapping and transaction retry boundaries intact. Backend checks and 166 focused tests across 13 suites passed with a real PostgreSQL connection.
… storage Provide OAuthStorage inside the canonical OAuthTokenService Layer so API transport, authentication, and plugin-catalog features no longer repeat the same dependency wiring. Preserve the separately injected token-service factory for tests and keep OAuthLauncher’s independent storage requirement explicit. Verified the kernel client typecheck and check task, plus 43 focused authentication, transport, plugin-event, and Layer composition tests.
…te domains The effecttsgo schema-number diagnostic flags Schema.Number and Schema.NumberFromString because they accept NaN, Infinity, and -Infinity. None of these values have a meaningful non-finite interpretation, so decode them with Schema.Finite and Schema.FiniteFromString instead: - typescript-compiler: diagnostic line, column, and length positions. - kernel-renderers: the refresh-generation counter in the entity browser, collection detail, and results table serialized query-input tuples. - fixture plugin: Pokemon number fields and the move power column. - ryotql and plugin-kit tests: count/score/value selections and operation outputs now use the finite variants, matching what production recipes should model. Decoded types stay `number`, so no caller changes are required. Verified with `bun turbo check`; the affected packages report 0 warnings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ve, schema form, fixture, and website
Clear the remaining effecttsgo and typescript lint warnings in these leaf
packages without changing behavior:
- Use typed schema decoders where the input already matches the schema's
Encoded type, so the compiler keeps checking the input instead of
discarding it through the `decodeUnknown*` variants:
- client-ui-sdk schema-form validation: URL, email, date, and datetime
checks now use `Schema.decodeResult`.
- plugin-archive: artifact, manifest, compiled-script metadata, and
client-artifact metadata decoding now use `Schema.decodeSync`.
- fixture plugin move and Pokemon search providers: search options now
use `Schema.decodeEffect`.
- fixture plugin: replace `Effect.all(xs.map(f), options)` with
`Effect.forEach(xs, f, options)` in both search providers; concurrency
limits are unchanged.
- plugin-archive: the compiled script `format` field decodes with
`Schema.Finite`, and `PluginArchivePackage.manifest` uses the
`PluginManifest` type directly instead of `typeof PluginManifest.Type`.
- plugin-archive test: the async iterator that models a transport failing
after its first chunk now awaits a rejected promise rather than throwing
from an `async` generator with no `await`.
- website: the same-origin `/api/config` response keeps its assertion with
a justified `no-unsafe-type-assertion` suppression, since it is produced
by the site's own `api.config` loader.
Verified with `bun turbo check` (0 warnings in each package) and the
plugin-archive, client-ui-sdk, and fixture-plugin test suites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mpiler platform Layers
The sandbox and client plugin compilers provided their own Bun file system
and Vite build services inside every exported function, hiding the
dependency from callers and tripping effecttsgo/strict-effect-provide in
library code. Following the rule that feature modules own their canonical
Layers and boot modules compose them, the compilers now declare their
requirements and callers provide the Layer at their entrypoint.
sandbox-compiler:
- `compileBuiltInSandboxEntry`, `compileSandboxPackageEntries`, and
`compilePluginSandboxEntries` now require `FileSystem | ViteBuildService`.
The public wrapper plus `*Internal` pairs collapse into single exports.
- Remove `compileBuiltInSandboxEntries`, which had no callers.
- Export the existing `sandboxCompilerPlatformLayer` through a new
`./platform` package entrypoint.
- The standalone compiler worker still provides the Layer itself, with a
justified suppression, because the worker process is its own entrypoint.
client-plugin-compiler:
- `compileClientPluginModule` and `buildClientRuntime` no longer provide the
two duplicated private `compilerLayer`s; the new `platform.ts` exports one
`clientPluginCompilerPlatformLayer` from the package index.
- `buildClientRuntime` is now a plain Effect value instead of a zero-argument
function returning one (effecttsgo/lazy-effect).
- UTF-8 decoding of client sources and emitted module/text files uses
`Effect.try` instead of `try`/`catch` inside `Effect.gen`, with the same
`RYOT_CLIENT_UTF8` diagnostics.
Callers:
- cli: the `import.meta.main` entrypoint provides `BunServices.layer` merged
with both compiler platform Layers (justified suppression). Its package.json
and manifest decoding use the typed `Schema.decodeEffect`.
- apps/server: `assemble.ts` and `dev.ts` entrypoints add
`clientPluginCompilerPlatformLayer` to their runtime Layer.
- kernel/backend: the plugin-load test support harness and the runner
integration tests add `sandboxCompilerPlatformLayer`.
- e2e: the compiled-package fixture provides both compiler Layers for now;
the following commit moves this into the e2e harness.
Tests:
- sandbox-compiler, client-plugin-compiler, cli, and the backend built-in
compiler test share their Layers through `it.layer` instead of piping
`Effect.provide` into each test. The cli watch test uses
`layer(..., { excludeTestServices: true })` because it needs the live
clock, and the client runtime test becomes an Effect test instead of
awaiting `Effect.runPromise` in an async Vitest body.
READMEs for both compilers document that callers provide the platform
Layer at their entrypoint.
Verified with `bun turbo check` and the sandbox-compiler,
client-plugin-compiler, cli, fixture-plugin, and fitness-plugin test suites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t for shared services E2E fixtures provided their own Layers: the contract client and `webRequest` provided `FetchHttpClient.layer`, the compiled-package fixture provided both compiler platform Layers, and 16 browser tests piped `Effect.provide(browserLayer)`. Each of those raised effecttsgo/strict-effect-provide, and they hid from the type checker which services a test actually needed. Every test is its own runtime entrypoint, so the harness now owns one Layer and fixtures only declare their requirements: - New `src/support/e2e-runtime.ts` merges `FetchHttpClient.layer`, `PlaywrightSpawner.layer(chromium)`, `sandboxCompilerPlatformLayer`, and `clientPluginCompilerPlatformLayer`, exposing `E2eServices`, `provideE2eServices`, and `runPromise`. This is the single justified strict-effect-provide suppression. The Playwright spawner Layer is lazy and only launches a browser inside `PlaywrightSpawner.withBrowser`, so API-only tests pay nothing for it. - `~/support/effect-test` exposes exactly what the suite uses: plain synchronous `it` and `it.live`, which supplies the per-test Scope and the shared services without TestClock. The TestClock-bearing variants are simply absent, which removes the previous unsafe type assertion that hid them. The harness also re-exports `runPromise` for Vitest hooks. - Hooks in 32 test files use the harness `runPromise` instead of `Effect.runPromise`; global setup and the seed scripts use `runPromise` from `e2e-runtime`. The seed script change is limited to that call swap. - `ContractSession.call`, `webRequest`, and `compilePluginPackage` now return Effects that require their services. `browserLayer` and every per-test browser provide are removed. - `collectRecipeItems` and `mapAuthError` are generic over requirements, and the OIDC fixture return types include `HttpClient`. - plugin-kit: `invokeOperationRecipe` threads the transport's requirements through instead of forcing `never`, so an HTTP-backed transport can be passed without providing a client inside it. AGENTS.md and README.md now describe the harness `runPromise` for hooks and say fixtures declare, rather than provide, shared services. Verified with `bun turbo check`; e2e goes from 44 warnings to 0 and all 38 check tasks pass. The plugin-kit test suite passes. The e2e suite itself was not run because it requires the full provisioned stack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d decoders Resolve every value-level Effect lint warning in @ryot-app/contract (180 warnings; only the single strict-effect-provide in src/client.ts remains, tracked separately with the other entrypoint provides). - effecttsgo/schema-number (72): numeric wire fields move from `Schema.Number` to `Schema.Finite`. This is an intentional tightening of the HTTP contract: decoding now rejects `NaN`, `Infinity`, and `-Infinity`. JSON cannot carry those values, so no well-formed client or server payload changes shape; the schemas simply state the finite domain they always had. The decoded TypeScript type is still `number`, so no consumer types change. - effecttsgo/prefer-typed-schema-decoder (107, 58 of them in manifest.test.ts): calls whose input is already assignable to the schema's Encoded type use `decode*` instead of `decodeUnknown*`, keeping compile-time checking of the input. The rule only fires where the input is already correctly typed, so tests that deliberately decode malformed input are unaffected. - effecttsgo/unnecessary-pipe-chain (1): merge the chained `.pipe` calls in `RequiredEntitySchemaSlug` into one. Every edit targeted the exact file:line:column reported by the linter. Verified with `bun turbo check` (all tasks pass, including every downstream typecheck) and the contract test suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IgnisDa
force-pushed
the
ultra-rewrite
branch
18 times, most recently
from
October 2, 2026 06:01
7f2e4f4 to
e04ce62
Compare
…flicts Merge origin/main through 0ade105, including queued import reports and unknown Plex show lookup. Retain the branch's current Plex setup wording and import-report formatting because the incoming behavior is already present.
Update the Docker runtime and sandbox compiler pins from 2.8.1 to 2.9.7. Replace the Linux amd64 and arm64 archive checksums with the official 2.9.7 release checksums. Verified with bun run check, non-e2e tests, and separate sandbox, filesystem, and async-flow e2e runs (13 tests passed) using Deno 2.9.7.
Move the script collector, its tests, and its canonical Layer into the garbage-collection module. Update plugin bootstrap and automation retention to import the collector from its new owner while preserving collection behavior and domain repository ownership.
Track workflow roots and terminal replies atomically with Effect mailbox writes. Collect successful trees after 24 hours and trees containing failures after seven days, while preserving suspended executions, active descendants, and pending messages. Fence admission with root-row locks and retain compact tombstones to prevent expired execution IDs, new descendants, and late replies from recreating collected state. Run bounded cleanup through Effect MessageStorage.clearAddress for workflow and durable-clock mailboxes, with persisted progress for retrying interrupted cleanup. Wire collection into the frequent cron, include the regenerated database baseline, and document the lifecycle. Add SQL retention, admission race, rollback, cleanup retry, and engine-restart recovery tests. Validate with bun run check, the non-e2e test suite, and the import-durability e2e test.
… conflicts Merge origin/main through ff97bb4, the V1 authentication escalation and cross-user authorization fix. Move the incoming OIDC frontend flow into crates/frontend with the branch's package scope and formatting, keep the branch's frontend scripts, and drop the incoming V1 integration tests and generated GraphQL client because the branch already removed both workspaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… grants PluginFrame captured its first document grant src for its whole lifetime, so a frame retained past the 15-minute grant TTL kept an expired URL as its iframe src. Any later reload of that element requested the expired grant and rendered the document-grant-not-found response. A changed grantId now closes the bridge and points the iframe at the renewed grant. Repeated preparations with the same grant and logical document replacements over the bridge still leave the live iframe untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…overy Port the password-recovery and admin-token hardening from main's V1 security fix to the TypeScript backend. Reset links expire after 30 minutes. Delivering a god-mode reset link now atomically tracks it per user, only while the request's capture ID still owns the email's reservation, and revokes the link it replaces before publishing; tracking or revocation failures abort delivery. A forged capture header on the public reset request changes nothing. Password reset and account reset revoke the tracked link, so a link issued before an account reset cannot set the recreated account's password. Password reset now also deletes the user's API keys and purges their cached copies, so keys created before recovery stop authenticating. Admin-token checks compare SHA-256 digests with timingSafeEqual instead of plain string equality. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Revisiting a retained composition reordered the LRU map, so React moved its frame wrapper and the iframe reloaded. Render frames sorted by key so surviving frames are never moved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace expiring document grant URLs with an authenticated POST /api/client-pages/document endpoint. The service resolves the composition through the same identity check as freshness, and answers a stale or foreign identity with ClientPageDocumentStale. The client renders the structured document into a srcdoc iframe sandboxed with allow-scripts, with a base pointing at the selected server and a no-referrer policy, so no capability URL is ever a frame's document URL. Document grants are removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@coderabbitai ignore
Closes #1827: added episode groups in the UI
Closes #1441: added spotify importer and integration
Closes #785: added anilist integration [WIP]
Closes #1806: added export to local storage
Closes #1819: added editable fitness targets