diff --git a/README.md b/README.md index b41749a16..4ec524aad 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ Contains the source code for the NativeScript's Android Runtime. [NativeScript]( - [Main Projects](#main-projects) - [Helper Projects](#helper-projects) +- [SBG vs Runtime Dex Generation](#sbg-vs-runtime-dex-generation) - [Architecture Diagram](#architecture-diagram) - [Build Prerequisites](#build-prerequisites) - [How to build](#how-to-build) @@ -29,9 +30,115 @@ The repo is structured in the following projects (ordered by dependencies): ## Helper Projects -* [**android-static-binding-generator**](android-static-binding-generator) - build tool that generates bindings based on the user's javascript code. +* [**android-static-binding-generator**](android-static-binding-generator) - build tool that generates bindings based on the user's javascript code. See [SBG vs Runtime Dex Generation](#sbg-vs-runtime-dex-generation) for the production vs HMR-dev split. * [**project-template**](build-artifacts/project-template-gradle) - this is an empty placeholder Android Application project, used by the [NativeScript CLI](https://github.com/NativeScript/nativescript-cli) when building an Android project. +## SBG vs Runtime Dex Generation + +The Android runtime turns every JavaScript `.extend('com.tns.Foo', Bar, { ... })` +(or `Bar.extend({ ... })`) call into a real Java subclass with a dispatching +proxy. There are **two** code paths that produce that subclass — a build-time +path and a runtime path — and which one runs depends on whether the +`.extend(...)` call site was visible to the Static Binding Generator (SBG) +when the APK was built. + +### Build-time path (production / classic CLI) + +[**android-static-binding-generator**](android-static-binding-generator) (SBG) +runs over the bundled JS once during `nativescript build android`: + +1. SBG parses every JS file the bundle includes and finds every + `.extend('com.tns.X', BaseClass, { ... })` call (and the modern + `BaseClass.extend({ ... })` shorthand). +2. For each call it asks + [**android-binding-generator**](test-app/runtime-binding-generator)'s + `ProxyGenerator`/`Dump` to emit a `.dex` for a Java class named + `com.tns.gen.` (or, for `@JavaProxy(...)`-style + explicit names, the user-chosen name). +3. The resulting dex files are packaged into the APK alongside the JS + bundle and listed in metadata so the runtime can find them via the + classloader. + +In production this means the Java class is already present and loadable +the very first time the JS `.extend(...)` call runs. The runtime never +generates a fresh dex. + +### Runtime path (HMR-dev / dynamic `.extend`) + +When SBG can't see the call at build time, the runtime generates the dex +on demand. This happens in two common shapes: + +* **HMR/Vite dev workflow** — the build-time bundle (`bundle.mjs`) is just + the HMR bootstrap; modules like + `@nativescript/core/ui/frame/fragment.android.ts` are fetched over HTTP + at runtime. SBG never sees those `.extend(...)` calls, so no pre-baked + dex exists. +* **`unnamed-extend` use cases** — `eval`-generated extends, or extends + whose first argument is computed at runtime, escape the SBG scan even + in production builds. + +The runtime path is wired through +[`com.tns.ClassResolver`](test-app/runtime/src/main/java/com/tns/ClassResolver.java) +→ [`com.tns.DexFactory`](test-app/runtime/src/main/java/com/tns/DexFactory.java): + +1. `ClassResolver.resolveClass` first tries `classStorageService.retrieveClass(name)`. + In production this hits the SBG-generated dex and we're done. +2. On `LookedUpClassNotFound` (typical for HMR), if a `baseClassName` is + present, `ClassResolver` falls back to `DexFactory.resolveClass(...)` + which runs the same `ProxyGenerator`/`Dump` pipeline SBG uses — only + it does it at runtime, writes the dex into the app's per-thumb cache + under `/.dex`, wraps it in a `.jar`, and loads it via + `DexClassLoader` (or `BaseDexClassLoader` injection when the + `injectIntoParentClassLoader` flag is on). +3. Generated proxy class names normalize JVM inner-class `$` to `_` + (both in `Dump`'s class signature and in `DexFactory`'s + `loadClass(...)` arg). The actual JVM lookup name is always + `com.tns.gen.` — never `com.tns.gen.$`. + +### Edge cases worth knowing about + +* **`$` vs `_` mismatches** — `Class.forName(baseClassName)` requires JVM + `$` inner-class syntax (`android.app.Application$ActivityLifecycleCallbacks`), + but `classLoader.loadClass(generatedName)` requires the `_`-normalized + sibling (`com.tns.gen.android.app.Application_ActivityLifecycleCallbacks`). + Both `DexFactory.resolveClass` and `DexFactory.findClass` apply the + normalization in the loadable-name path while preserving `$` for the + reflective base-class lookup. Removing either normalization + reintroduces the "Didn't find class + `com.tns.gen.android.app.Application$ActivityLifecycleCallbacks`" + ClassNotFoundException for runtime-generated proxies. +* **Cache invalidation** — the runtime dex cache is keyed on a per-build + `dexThumb` (the runtime regenerates it whenever the JS bundle changes). + Stale dex from a previous boot is purged in + `DexFactory.updateDexThumbAndPurgeCache`. Don't bypass the thumb — a + dex generated against an older base class can crash at method dispatch + time when the base API drifts. +* **Parent-classloader injection** — Android framework code that calls + `Class.forName(name)` searches the app's `PathClassLoader`, *not* an + isolated `DexClassLoader`. When a generated proxy needs to be visible + to framework reflection (e.g. `FragmentFactory`, `Activity` resolution + from `AndroidManifest.xml`), construct the `DexFactory` with + `injectIntoParentClassLoader=true` so its `injectDexIntoClassLoader` + helper splices the generated jar's `dexElements` onto the parent's + `pathList`. +* **Don't normalize synthetic module keys** — module registry keys like + `ns-vendor://...`, `optional:...`, `node:...`, `blob:...` are not + filesystem paths and must be preserved verbatim through invalidation + + reload. They are NOT the same kind of "synthetic name" as the + `com.tns.gen.<...>` class names above and don't go through this dex + pipeline. + +### What to look at when this breaks + +* `ClassResolver.java` — the fallback decision between + `classStorageService` and `DexFactory`. +* `DexFactory.java` — runtime dex generation, cache, and the + `$`→`_` normalization. +* `ProxyGenerator.java` + `Dump.java` (under `runtime-binding-generator`) + — what the generated class actually looks like in bytecode. +* `android-static-binding-generator` — what SBG sees (and crucially + what it *doesn't* see) at build time. + ## Architecture Diagram The NativeScript Android Runtime architecture can be summarized in the following diagram. diff --git a/test-app/app/src/main/assets/app/mainpage.js b/test-app/app/src/main/assets/app/mainpage.js index 8184e7553..fb03e6ba8 100644 --- a/test-app/app/src/main/assets/app/mainpage.js +++ b/test-app/app/src/main/assets/app/mainpage.js @@ -77,3 +77,6 @@ require('./tests/testQueueMicrotask'); require("./tests/testConcurrentAccess"); require("./tests/testESModules.mjs"); +require("./tests/testNsDevBoundary.mjs"); +require("./tests/testHttpCanonicalKey.mjs"); +require("./tests/testNodeBuiltinsAndOptionalModules.mjs"); diff --git a/test-app/app/src/main/assets/app/tests/esm/meta-no-hot.mjs b/test-app/app/src/main/assets/app/tests/esm/meta-no-hot.mjs new file mode 100644 index 000000000..2f7a26b88 --- /dev/null +++ b/test-app/app/src/main/assets/app/tests/esm/meta-no-hot.mjs @@ -0,0 +1,5 @@ +// Fixture for testNsDevBoundary: reports whether the runtime attached a +// native `import.meta.hot` (it must NOT — hot contexts are injected by the +// JS dev client via source rewrite, never by the runtime). +export const hasHot = typeof import.meta.hot !== "undefined"; +export const hotValue = import.meta.hot; diff --git a/test-app/app/src/main/assets/app/tests/testHttpCanonicalKey.mjs b/test-app/app/src/main/assets/app/tests/testHttpCanonicalKey.mjs new file mode 100644 index 000000000..33ee93e90 --- /dev/null +++ b/test-app/app/src/main/assets/app/tests/testHttpCanonicalKey.mjs @@ -0,0 +1,77 @@ +// HTTP canonical-key identity tests. +// +// Pins the behavior of the native CanonicalizeHttpUrlKey (the loader/registry +// key) via the debug-only __NS_DEV__.canonicalizeHttpUrlKey diagnostic. Pure +// string logic — no dev server required. +// +// Module identity IS the canonical URL: the dev server serves every module +// under one URL and freshness is handled by __NS_DEV__.invalidateModules +// (registry + prewarm-cache evict + fetch nonce), never by URL variation. +// There is deliberately no path-tag vocabulary (__ns_boot__/__ns_hmr__) +// to collapse, and no versioned-bridge-endpoint normalization. + +describe("HTTP canonical key", function () { + function canonFn() { + const dev = globalThis.__NS_DEV__; + return dev && typeof dev.canonicalizeHttpUrlKey === "function" + ? dev.canonicalizeHttpUrlKey + : null; + } + function canon(url) { + return canonFn()(url); + } + + it("is available in dev builds", function () { + if (!canonFn()) { + pending("__NS_DEV__.canonicalizeHttpUrlKey not available (release build?)"); + return; + } + expect(typeof canonFn()).toBe("function"); + }); + + it("drops the fragment and unwraps file://http wrappers", function () { + if (!canonFn()) { pending("release"); return; } + expect(canon("http://h/ns/m/foo.js#frag")).toBe("http://h/ns/m/foo.js"); + expect(canon("file://http://h/x.js")).toBe("http://h/x.js"); + }); + + it("does NOT collapse versioned endpoint paths or path tags", function () { + if (!canonFn()) { pending("release"); return; } + // Path is identity — no /ns/rt/ → /ns/rt collapse, no + // __ns_hmr__/__ns_boot__ tag folding. + expect(canon("http://h/ns/rt/42")).toBe("http://h/ns/rt/42"); + expect(canon("http://h/ns/rt/42/x.js")).toBe("http://h/ns/rt/42/x.js"); + expect(canon("http://h/ns/m/__ns_hmr__/v7/foo.js")) + .toBe("http://h/ns/m/__ns_hmr__/v7/foo.js"); + }); + + it("strips import/t/v markers and sorts remaining params on dev endpoints", function () { + if (!canonFn()) { pending("release"); return; } + expect(canon("http://h/ns/m/a?import=1&b=2&a=3")).toBe("http://h/ns/m/a?a=3&b=2"); + expect(canon("http://h/ns/m/a?b=2&a=1")).toBe("http://h/ns/m/a?a=1&b=2"); + expect(canon("http://h/ns/core?import=1")).toBe("http://h/ns/core"); + expect(canon("http://h/ns/m/a?t=123&v=abc&x=1")).toBe("http://h/ns/m/a?x=1"); + }); + + it("preserves the query verbatim on non-dev endpoints", function () { + if (!canonFn()) { pending("release"); return; } + // Public-internet module URLs: the query can be part of identity + // (auth, content versioning, routing) — only the fragment is dropped. + expect(canon("http://h/a?import=1&b=2&a=3")).toBe("http://h/a?import=1&b=2&a=3"); + expect(canon("https://cdn.example.com/pkg.js?token=x#frag")) + .toBe("https://cdn.example.com/pkg.js?token=x"); + }); + + it("preserves the t param on @ng/component endpoints", function () { + if (!canonFn()) { pending("release"); return; } + // Angular HMR component-update endpoint: `t` identifies a specific + // recompile and must remain a distinct registry entry. + expect(canon("http://h/ns/m/app/@ng/component?c=x&t=111")) + .toBe("http://h/ns/m/app/@ng/component?c=x&t=111"); + }); + + it("leaves a non-http(s) specifier unchanged", function () { + if (!canonFn()) { pending("release"); return; } + expect(canon("~/local/foo.js")).toBe("~/local/foo.js"); + }); +}); diff --git a/test-app/app/src/main/assets/app/tests/testNodeBuiltinsAndOptionalModules.mjs b/test-app/app/src/main/assets/app/tests/testNodeBuiltinsAndOptionalModules.mjs new file mode 100644 index 000000000..2af282e34 --- /dev/null +++ b/test-app/app/src/main/assets/app/tests/testNodeBuiltinsAndOptionalModules.mjs @@ -0,0 +1,128 @@ +// Tests the ESM resolver's synthetic-module paths: +// - node: built-in polyfills (in-memory ES modules) +// - bare-specifier optional-module placeholders +// - ns-vendor:// vendor-registry resolution via configureRuntime importMap +// - blob: URL module re-use across imports +// +// The Android resolver's Node-builtin polyfill set is broader than iOS's; +// only the explicitly-tested specifiers are asserted here. + +describe("Node built-in and optional module resolution", function () { + it("provides an in-memory polyfill for node:url", async function () { + const mod = await import("node:url"); + const modAgain = await import("node:url"); + + expect(mod).toBeDefined(); + expect(modAgain).toBe(mod); + expect(typeof mod.fileURLToPath).toBe("function"); + expect(typeof mod.pathToFileURL).toBe("function"); + + const p = mod.fileURLToPath("file:///foo/bar.txt"); + expect(p === "/foo/bar.txt" || p === "foo/bar.txt").toBe(true); + + const u = mod.pathToFileURL("/foo/bar.txt"); + expect(u instanceof URL).toBe(true); + expect(u.protocol).toBe("file:"); + }); + + it("creates an in-memory placeholder for likely-optional modules", async function () { + // Use a name that IsLikelyOptionalModule will treat as optional + // (no slashes, no extension, no scope prefix). + const mod = await import("__ns_optional_test_module__"); + const modAgain = await import("__ns_optional_test_module__"); + + expect(mod).toBeDefined(); + expect(modAgain).toBe(mod); + expect(typeof mod.default).toBe("object"); + + let threw = false; + try { + // eslint-disable-next-line no-unused-expressions + mod.default.someProperty; + } catch (e) { + threw = true; + } + expect(threw).toBe(true); + }); + + it("resolves import-map vendor modules through the explicit vendor registry", async function () { + const dev = globalThis.__NS_DEV__; + const configureRuntime = dev && dev.configureRuntime; + if (typeof configureRuntime !== "function") { + pending("__NS_DEV__.configureRuntime not available"); + return; + } + + const previousRegistry = globalThis.__nsVendorRegistry; + const vendorRegistry = new Map(); + globalThis.__nsVendorRegistry = vendorRegistry; + vendorRegistry.set("__ns_test_vendor__", { + default: { source: "vendor-default" }, + namedValue: 7, + makeValue() { + return "vendor-named"; + }, + }); + + try { + configureRuntime({ + importMap: { + imports: { + __ns_test_vendor__: "ns-vendor://__ns_test_vendor__", + }, + }, + }); + + const mod = await import("__ns_test_vendor__"); + const modAgain = await import("__ns_test_vendor__"); + + expect(mod).toBeDefined(); + expect(modAgain).toBe(mod); + expect(mod.default).toEqual({ source: "vendor-default" }); + expect(mod.namedValue).toBe(7); + expect(mod.makeValue()).toBe("vendor-named"); + } finally { + configureRuntime({ importMap: { imports: {} } }); + if (typeof previousRegistry === "undefined") { + delete globalThis.__nsVendorRegistry; + } else { + globalThis.__nsVendorRegistry = previousRegistry; + } + } + }); + + it("reuses blob URL modules across concurrent and repeated imports", async function () { + delete globalThis.__nsBlobEvalCount; + + const blobSource = [ + "globalThis.__nsBlobEvalCount = (globalThis.__nsBlobEvalCount || 0) + 1;", + "export const evalCount = globalThis.__nsBlobEvalCount;", + "export const kind = 'blob-module';", + "export default { evalCount, kind };", + ].join("\n"); + + const url = URL.createObjectURL(new Blob([blobSource], { type: "text/javascript" }), { + ext: ".mjs", + }); + + expect(typeof url).toBe("string"); + expect(url.indexOf("blob:nativescript/")).toBe(0); + + try { + const [first, second] = await Promise.all([import(url), import(url)]); + const third = await import(url); + + expect(first).toBeDefined(); + expect(second).toBe(first); + expect(third).toBe(first); + expect(first.evalCount).toBe(1); + expect(second.evalCount).toBe(1); + expect(third.evalCount).toBe(1); + expect(first.kind).toBe("blob-module"); + expect(globalThis.__nsBlobEvalCount).toBe(1); + } finally { + URL.revokeObjectURL(url); + delete globalThis.__nsBlobEvalCount; + } + }); +}); diff --git a/test-app/app/src/main/assets/app/tests/testNsDevBoundary.mjs b/test-app/app/src/main/assets/app/tests/testNsDevBoundary.mjs new file mode 100644 index 000000000..6850bc744 --- /dev/null +++ b/test-app/app/src/main/assets/app/tests/testNsDevBoundary.mjs @@ -0,0 +1,62 @@ +// Dev-loader boundary tests. +// +// Pins the mechanism-only native contract: the runtime exposes exactly the +// `__NS_DEV__` namespace (configureRuntime, invalidateModules, +// kickstartPrefetch, getLoadedModuleUrls, setDevBootComplete, +// terminateAllWorkers, and the debug-only canonicalizeHttpUrlKey), and +// nothing else. HMR *policy* — `import.meta.hot`, hot-data/accept/dispose +// registries, dev-session state, boot orchestration — lives in the JS dev +// client (@nativescript/vite), not in the runtime. + +describe("__NS_DEV__ dev-loader boundary", function () { + it("exposes the __NS_DEV__ namespace with the core primitives", function () { + const dev = globalThis.__NS_DEV__; + expect(dev).toBeDefined(); + expect(typeof dev.configureRuntime).toBe("function"); + expect(typeof dev.invalidateModules).toBe("function"); + expect(typeof dev.kickstartPrefetch).toBe("function"); + expect(typeof dev.getLoadedModuleUrls).toBe("function"); + expect(typeof dev.setDevBootComplete).toBe("function"); + // Main isolate: worker termination is installed here (and ONLY here — + // worker isolates must not receive it). + expect(typeof dev.terminateAllWorkers).toBe("function"); + }); + + it("keeps the dev surface confined to __NS_DEV__ (no flat __ns* globals)", function () { + // The contract is the single namespace object: no dev primitive is + // reachable as a flat global, so tooling can feature-detect exactly + // one thing and the global namespace stays unpolluted. + [ + "__nsConfigureRuntime", + "__nsConfigureDevRuntime", + "__nsInvalidateModules", + "__nsKickstartHmrPrefetch", + "__nsGetLoadedModuleUrls", + "__nsSetDevBootComplete", + "__nsTerminateAllWorkers", + "__nsCanonicalizeHttpUrlKey", + "__nsStartDevSession", + "__nsGetHotData", + "__nsRegisterHotAccept", + "__nsRegisterHotDispose", + ].forEach(function (name) { + expect(typeof globalThis[name]).toBe("undefined"); + }); + }); + + it("does not attach import.meta.hot natively", async function () { + const mod = await import("~/tests/esm/meta-no-hot.mjs"); + // Hot contexts are injected by the JS dev client via source rewrite; + // a module loaded outside a dev session must see no hot object. + expect(mod.hasHot).toBe(false); + expect(mod.hotValue).toBeUndefined(); + }); + + it("getLoadedModuleUrls returns an array of registry keys", function () { + const urls = globalThis.__NS_DEV__.getLoadedModuleUrls(); + expect(Array.isArray(urls)).toBe(true); + urls.forEach(function (u) { + expect(typeof u).toBe("string"); + }); + }); +}); diff --git a/test-app/runtime/src/main/cpp/CallbackHandlers.cpp b/test-app/runtime/src/main/cpp/CallbackHandlers.cpp index 5ce450fef..54506911a 100644 --- a/test-app/runtime/src/main/cpp/CallbackHandlers.cpp +++ b/test-app/runtime/src/main/cpp/CallbackHandlers.cpp @@ -1345,6 +1345,22 @@ CallbackHandlers::WorkerObjectTerminateCallback(const v8::FunctionCallbackInfo &args) { + // `__NS_DEV__.terminateAllWorkers()` — main-isolate-only dev helper. + // Tears down every worker parented by this isolate through the WorkerWrapper + // registry. TerminateChildren snapshots the registry under its lock, + // terminates and clears each worker, and lets each one cascade into its own + // nested workers, so a worker self-terminating in parallel can't invalidate + // the walk. Returns the number of direct (top-level) workers torn down so + // the HMR client can log it. + auto isolate = args.GetIsolate(); + HandleScope scope(isolate); + + int terminated = WorkerWrapper::TerminateChildren(isolate); + args.GetReturnValue().Set(terminated); +} + void CallbackHandlers::WorkerGlobalCloseCallback(const v8::FunctionCallbackInfo &args) { auto isolate = args.GetIsolate(); diff --git a/test-app/runtime/src/main/cpp/CallbackHandlers.h b/test-app/runtime/src/main/cpp/CallbackHandlers.h index eddcca93d..ef5550adf 100644 --- a/test-app/runtime/src/main/cpp/CallbackHandlers.h +++ b/test-app/runtime/src/main/cpp/CallbackHandlers.h @@ -148,6 +148,18 @@ namespace tns { */ static void WorkerGlobalCloseCallback(const v8::FunctionCallbackInfo &args); + /* + * `__NS_DEV__.terminateAllWorkers()`, installed on the main-thread + * isolate only (see HMRSupport's InitializeHmrDevGlobals — worker + * threads do NOT receive it, so a stuck worker can't take down its + * peers). Terminates every worker parented by the main isolate via + * WorkerWrapper::TerminateChildren, which snapshots the registry, + * terminates and clears each worker, and lets each one cascade into + * its own nested workers. Returns the number of direct (top-level) + * workers torn down so the HMR client can log it. + */ + static void TerminateAllWorkersCallback(const v8::FunctionCallbackInfo &args); + /* * Is called when an unhandled exception is thrown inside the worker * Will execute 'onerror' if one is provided inside the Worker Scope diff --git a/test-app/runtime/src/main/cpp/DevFlags.cpp b/test-app/runtime/src/main/cpp/DevFlags.cpp index 224601b10..c4cec187b 100644 --- a/test-app/runtime/src/main/cpp/DevFlags.cpp +++ b/test-app/runtime/src/main/cpp/DevFlags.cpp @@ -1,6 +1,7 @@ // DevFlags.cpp #include "DevFlags.h" #include "JEnv.h" +#include "NativeScriptAssert.h" #include #include #include @@ -8,21 +9,25 @@ namespace tns { -bool IsScriptLoadingLogEnabled() { - static std::atomic cached{-1}; // -1 unknown, 0 false, 1 true +// Cache the result of a parameterless `static boolean` method on +// `com.tns.Runtime`. `cached` and `initFlag` are caller-owned statics so +// every flag stays independently memoized; the helper does the JNI dance +// once and pins the result. Returns false (the safe default) if the +// class or method cannot be resolved. +static bool CachedBoolFlagFromJava(std::atomic& cached, + std::once_flag& initFlag, + const char* javaStaticBoolMethod) { int v = cached.load(std::memory_order_acquire); if (v != -1) { return v == 1; } - - static std::once_flag initFlag; - std::call_once(initFlag, []() { + std::call_once(initFlag, [&]() { bool enabled = false; try { JEnv env; jclass runtimeClass = env.FindClass("com/tns/Runtime"); if (runtimeClass != nullptr) { - jmethodID mid = env.GetStaticMethodID(runtimeClass, "getLogScriptLoadingEnabled", "()Z"); + jmethodID mid = env.GetStaticMethodID(runtimeClass, javaStaticBoolMethod, "()Z"); if (mid != nullptr) { jboolean res = env.CallStaticBooleanMethod(runtimeClass, mid); enabled = (res == JNI_TRUE); @@ -33,10 +38,33 @@ bool IsScriptLoadingLogEnabled() { } cached.store(enabled ? 1 : 0, std::memory_order_release); }); - return cached.load(std::memory_order_acquire) == 1; } +bool IsScriptLoadingLogEnabled() { + static std::atomic cached{-1}; + static std::once_flag initFlag; + return CachedBoolFlagFromJava(cached, initFlag, "getLogScriptLoadingEnabled"); +} + +// Default OFF because the volume is high (one line per fetch, hundreds per +// cold boot, hundreds per HMR refresh). Opt in via package.json: +// { "httpFetchUrlLog": true } +bool IsHttpFetchUrlLogEnabled() { + static std::atomic cached{-1}; + static std::once_flag initFlag; + bool enabled = CachedBoolFlagFromJava(cached, initFlag, "getHttpFetchUrlLogEnabled"); + + static std::once_flag bannerFlag; + std::call_once(bannerFlag, [enabled]() { + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-loader] fetch-url-log=%s", + enabled ? "enabled" : "disabled"); + } + }); + return enabled; +} + // Security config static std::once_flag s_securityConfigInitFlag; @@ -44,10 +72,25 @@ static bool s_allowRemoteModules = false; static std::vector s_remoteModuleAllowlist; static bool s_isDebuggable = false; -// Helper to check if a URL starts with a given prefix -static bool UrlStartsWith(const std::string& url, const std::string& prefix) { - if (prefix.size() > url.size()) return false; - return url.compare(0, prefix.size(), prefix) == 0; +// Returns true when `url` is covered by allowlist `entry`, matching only on +// URL component boundaries so lookalike-host and lookalike-port values are +// refused: an entry of "https://cdn.example.com" does NOT authorize +// "https://cdn.example.com.attacker.com/x.js" or +// "https://cdn.example.com:9999/x.js". The character after the matched entry +// text must be a path/query/fragment boundary ('/', '?', '#'), or the URL must +// end exactly at the entry, or the entry must already end in '/'. +// +// Deny-by-default: an entry without an explicit port does not match a URL +// that adds one. To allow a specific port, include it in the entry. +static bool RemoteUrlMatchesAllowlistEntry(const std::string& url, + const std::string& entry) { + if (entry.empty()) return false; + if (url.size() < entry.size()) return false; + if (url.compare(0, entry.size(), entry) != 0) return false; + if (url.size() == entry.size()) return true; // exact match + if (entry.back() == '/') return true; // entry ended at a boundary + const char next = url[entry.size()]; + return next == '/' || next == '?' || next == '#'; } void InitializeSecurityConfig() { @@ -110,6 +153,11 @@ bool IsRemoteModulesAllowed() { return s_allowRemoteModules || s_isDebuggable; } +bool IsDebuggable() { + InitializeSecurityConfig(); + return s_isDebuggable; +} + bool IsRemoteUrlAllowed(const std::string& url) { InitializeSecurityConfig(); @@ -128,9 +176,9 @@ bool IsRemoteUrlAllowed(const std::string& url) { return true; } - // Check if URL matches any allowlist prefix - for (const std::string& prefix : s_remoteModuleAllowlist) { - if (UrlStartsWith(url, prefix)) { + // Check if URL matches any allowlist entry on a component boundary + for (const std::string& entry : s_remoteModuleAllowlist) { + if (RemoteUrlMatchesAllowlistEntry(url, entry)) { return true; } } diff --git a/test-app/runtime/src/main/cpp/DevFlags.h b/test-app/runtime/src/main/cpp/DevFlags.h index db571d49f..f3f6dca14 100644 --- a/test-app/runtime/src/main/cpp/DevFlags.h +++ b/test-app/runtime/src/main/cpp/DevFlags.h @@ -9,6 +9,14 @@ namespace tns { // First call queries Java once; subsequent calls are atomic loads only. bool IsScriptLoadingLogEnabled(); +// HTTP module loader flags +// +// Returns true when one log line should be emitted per HTTP fetch URL. +// Default OFF because the volume is high (one line per fetch, hundreds per +// cold boot, hundreds per HMR refresh). Opt in via package.json: +// "httpFetchUrlLog": true|false +bool IsHttpFetchUrlLogEnabled(); + // Security config // "security.allowRemoteModules" from nativescript.config @@ -18,6 +26,12 @@ bool IsRemoteModulesAllowed(); // If no allowlist is configured but allowRemoteModules is true, all URLs are allowed. bool IsRemoteUrlAllowed(const std::string& url); +// Mirrors com.tns.Runtime.isDebuggable() (config.isDebuggable), cached once via +// InitializeSecurityConfig(). Use this to gate dev/HMR-only native surfaces so +// they never execute in a plain release build. Returns false (fail-safe) until +// the security config has been initialized or if the JNI lookup fails. +bool IsDebuggable(); + // Init security configuration void InitializeSecurityConfig(); diff --git a/test-app/runtime/src/main/cpp/HMRSupport.cpp b/test-app/runtime/src/main/cpp/HMRSupport.cpp index 16cac04d8..ce91f9927 100644 --- a/test-app/runtime/src/main/cpp/HMRSupport.cpp +++ b/test-app/runtime/src/main/cpp/HMRSupport.cpp @@ -1,193 +1,191 @@ // HMRSupport.cpp +// +// The native half of the NativeScript dev-loader contract. The runtime +// exposes *mechanism* only (sync HTTP module fetch, prewarm cache + +// list-mode kickstart, eviction plumbing, dev-boot-complete signal), +// consolidated under the single `__NS_DEV__` namespace object. All HMR +// *policy* — boot orchestration, `import.meta.hot`, hot-callback +// registries, full reload, CSS apply, WebSocket protocol — lives in the +// JS dev client (`@nativescript/vite`). #include "HMRSupport.h" + #include "ArgConverter.h" -#include "JEnv.h" +#include "CallbackHandlers.h" #include "DevFlags.h" +#include "JEnv.h" +#include "ModuleInternalCallbacks.h" #include "NativeScriptAssert.h" +#include "NativeScriptException.h" + #include +#include #include -#include -#include +#include +#include #include +#include +#include #include +#include +#include +#include +#include namespace tns { +// ────────────────────────────────────────────────────────────────────────── +// Local v8 string helper: thin convenience wrapper so call sites can read +// more compactly. +static inline v8::Local ToV8String(v8::Isolate* isolate, const char* str) { + if (str == nullptr) { + return v8::String::Empty(isolate); + } + return v8::String::NewFromUtf8(isolate, str, v8::NewStringType::kNormal).ToLocalChecked(); +} + static inline bool StartsWith(const std::string& s, const char* prefix) { size_t n = strlen(prefix); return s.size() >= n && s.compare(0, n, prefix) == 0; } -// Per-module hot data and callbacks. Keyed by canonical module path (file path or URL). -static std::unordered_map> g_hotData; -static std::unordered_map>> g_hotAccept; -static std::unordered_map>> g_hotDispose; +static inline bool EndsWith(const std::string& s, const char* suffix) { + size_t n = strlen(suffix); + return s.size() >= n && s.compare(s.size() - n, n, suffix) == 0; +} -v8::Local GetOrCreateHotData(v8::Isolate* isolate, const std::string& key) { - auto it = g_hotData.find(key); - if (it != g_hotData.end() && !it->second.IsEmpty()) { - return it->second.Get(isolate); +void MirrorGlobalOnGlobalThis(v8::Isolate* isolate, v8::Local context, + const char* name) { + std::string src = + "if (typeof globalThis !== 'undefined' && typeof globalThis." + + std::string(name) + + " === 'undefined') {" + " Object.defineProperty(globalThis, '" + std::string(name) + + "', { value: this." + std::string(name) + + ", writable: true, configurable: true, enumerable: false });" + "}"; + + v8::Local script; + if (v8::Script::Compile(context, ToV8String(isolate, src.c_str())) + .ToLocal(&script)) { + script->Run(context).FromMaybe(v8::Local()); } - v8::Local obj = v8::Object::New(isolate); - g_hotData[key].Reset(isolate, obj); - return obj; } -void RegisterHotAccept(v8::Isolate* isolate, const std::string& key, v8::Local cb) { - if (cb.IsEmpty()) return; - g_hotAccept[key].emplace_back(v8::Global(isolate, cb)); +static void SetBooleanGlobal(v8::Isolate* isolate, v8::Local context, + const char* key, bool value) { + context->Global() + ->Set(context, ToV8String(isolate, key), v8::Boolean::New(isolate, value)) + .FromMaybe(false); } -void RegisterHotDispose(v8::Isolate* isolate, const std::string& key, v8::Local cb) { - if (cb.IsEmpty()) return; - g_hotDispose[key].emplace_back(v8::Global(isolate, cb)); -} +// ───────────────────────────────────────────────────────────── +// Dev-boot completion flag +// +// Native-side mirror of `__NS_HMR_BOOT_COMPLETE__`. Read by the kickstart +// pump-wait so its gate is a single relaxed atomic load on the HMR-time +// hot path. The JS dev client flips this via +// `__NS_DEV__.setDevBootComplete(bool)` once the real app root view +// commits; boot orchestration itself is entirely userland. +static std::atomic g_devSessionBootComplete{false}; -std::vector> GetHotAcceptCallbacks(v8::Isolate* isolate, const std::string& key) { - std::vector> out; - auto it = g_hotAccept.find(key); - if (it != g_hotAccept.end()) { - for (auto& gfn : it->second) { - if (!gfn.IsEmpty()) out.push_back(gfn.Get(isolate)); - } - } - return out; +static inline bool IsDevSessionBootComplete() { + return g_devSessionBootComplete.load(std::memory_order_relaxed); } -std::vector> GetHotDisposeCallbacks(v8::Isolate* isolate, const std::string& key) { - std::vector> out; - auto it = g_hotDispose.find(key); - if (it != g_hotDispose.end()) { - for (auto& gfn : it->second) { - if (!gfn.IsEmpty()) out.push_back(gfn.Get(isolate)); - } +void SetDevBootComplete(v8::Isolate* isolate, v8::Local context, + bool value) { + SetBooleanGlobal(isolate, context, "__NS_HMR_BOOT_COMPLETE__", value); + g_devSessionBootComplete.store(value, std::memory_order_relaxed); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[dev-boot] __NS_HMR_BOOT_COMPLETE__=%s", value ? "true" : "false"); } - return out; } -void InitializeImportMetaHot(v8::Isolate* isolate, - v8::Local context, - v8::Local importMeta, - const std::string& modulePath) { - using v8::Function; - using v8::FunctionCallbackInfo; - using v8::Local; - using v8::Object; - using v8::String; - using v8::Value; - - v8::HandleScope scope(isolate); - - auto makeKeyData = [&](const std::string& key) -> Local { - return ArgConverter::ConvertToV8String(isolate, key); - }; - - auto acceptCb = [](const FunctionCallbackInfo& info) { - v8::Isolate* iso = info.GetIsolate(); - Local data = info.Data(); - std::string key; - if (!data.IsEmpty()) { - v8::String::Utf8Value s(iso, data); - key = *s ? *s : ""; - } - v8::Local cb; - if (info.Length() >= 1 && info[0]->IsFunction()) { - cb = info[0].As(); - } else if (info.Length() >= 2 && info[1]->IsFunction()) { - cb = info[1].As(); - } - if (!cb.IsEmpty()) { - RegisterHotAccept(iso, key, cb); - } - info.GetReturnValue().Set(v8::Undefined(iso)); - }; - - auto disposeCb = [](const FunctionCallbackInfo& info) { - v8::Isolate* iso = info.GetIsolate(); - Local data = info.Data(); - std::string key; - if (!data.IsEmpty()) { v8::String::Utf8Value s(iso, data); key = *s ? *s : ""; } - if (info.Length() >= 1 && info[0]->IsFunction()) { - RegisterHotDispose(iso, key, info[0].As()); - } - info.GetReturnValue().Set(v8::Undefined(iso)); - }; - - auto declineCb = [](const FunctionCallbackInfo& info) { - info.GetReturnValue().Set(v8::Undefined(info.GetIsolate())); - }; - - auto invalidateCb = [](const FunctionCallbackInfo& info) { - info.GetReturnValue().Set(v8::Undefined(info.GetIsolate())); - }; +// ───────────────────────────────────────────────────────────── +// HTTP loader helpers - Local hot = Object::New(isolate); - hot->CreateDataProperty(context, ArgConverter::ConvertToV8String(isolate, "data"), - GetOrCreateHotData(isolate, modulePath)).Check(); - hot->CreateDataProperty(context, ArgConverter::ConvertToV8String(isolate, "prune"), - v8::Boolean::New(isolate, false)).Check(); - hot->CreateDataProperty( - context, ArgConverter::ConvertToV8String(isolate, "accept"), - v8::Function::New(context, acceptCb, makeKeyData(modulePath)).ToLocalChecked()).Check(); - hot->CreateDataProperty( - context, ArgConverter::ConvertToV8String(isolate, "dispose"), - v8::Function::New(context, disposeCb, makeKeyData(modulePath)).ToLocalChecked()).Check(); - hot->CreateDataProperty( - context, ArgConverter::ConvertToV8String(isolate, "decline"), - v8::Function::New(context, declineCb, makeKeyData(modulePath)).ToLocalChecked()).Check(); - hot->CreateDataProperty( - context, ArgConverter::ConvertToV8String(isolate, "invalidate"), - v8::Function::New(context, invalidateCb, makeKeyData(modulePath)).ToLocalChecked()).Check(); - - importMeta->CreateDataProperty(context, ArgConverter::ConvertToV8String(isolate, "hot"), hot).Check(); -} - -// Drop fragments and normalize parameters for consistent registry keys. std::string CanonicalizeHttpUrlKey(const std::string& url) { - if (!(StartsWith(url, "http://") || StartsWith(url, "https://"))) { - return url; + // Some loaders wrap HTTP module URLs as file://http(s)://... + std::string normalizedUrl = url; + if (StartsWith(normalizedUrl, "file://http://") || StartsWith(normalizedUrl, "file://https://")) { + normalizedUrl = normalizedUrl.substr(strlen("file://")); } - // Remove fragment - size_t hashPos = url.find('#'); - std::string noHash = (hashPos == std::string::npos) ? url : url.substr(0, hashPos); + if (!(StartsWith(normalizedUrl, "http://") || StartsWith(normalizedUrl, "https://"))) { + return normalizedUrl; + } + // Drop fragment entirely + size_t hashPos = normalizedUrl.find('#'); + std::string noHash = (hashPos == std::string::npos) ? normalizedUrl : normalizedUrl.substr(0, hashPos); - // Split into origin+path and query - size_t qPos = noHash.find('?'); + // Locate path start and query start + size_t schemePos = noHash.find("://"); + if (schemePos == std::string::npos) { + // Unexpected shape; fall back to removing whole query + size_t q = noHash.find('?'); + return (q == std::string::npos) ? noHash : noHash.substr(0, q); + } + size_t pathStart = noHash.find('/', schemePos + 3); + if (pathStart == std::string::npos) { + // No path; nothing to normalize + return noHash; + } + size_t qPos = noHash.find('?', pathStart); std::string originAndPath = (qPos == std::string::npos) ? noHash : noHash.substr(0, qPos); std::string query = (qPos == std::string::npos) ? std::string() : noHash.substr(qPos + 1); - // Normalize bridge endpoints to keep a single realm across HMR updates: - // - /ns/rt/ -> /ns/rt - // - /ns/core/ -> /ns/core - size_t schemePos = originAndPath.find("://"); - if (schemePos != std::string::npos) { - size_t pathStart = originAndPath.find('/', schemePos + 3); - if (pathStart != std::string::npos) { - std::string pathOnly = originAndPath.substr(pathStart); - auto normalizeBridge = [&](const char* needle) { - size_t nlen = strlen(needle); - if (pathOnly.size() <= nlen) return false; - if (pathOnly.compare(0, nlen, needle) != 0) return false; - if (pathOnly.size() == nlen) return true; - if (pathOnly[nlen] != '/') return false; - size_t i = nlen + 1; - size_t j = i; - while (j < pathOnly.size() && isdigit((unsigned char)pathOnly[j])) j++; - // Only normalize exact version segment: /ns/*/ (no further segments) - if (j == i) return false; - if (j != pathOnly.size()) return false; - originAndPath = originAndPath.substr(0, pathStart) + std::string(needle); - return true; - }; - if (!normalizeBridge("/ns/rt")) { - normalizeBridge("/ns/core"); - } + // IMPORTANT: This function is used as an HTTP module registry/cache key. + // For general-purpose HTTP module loading (public internet), the query string + // can be part of the module's identity (auth, content versioning, routing, etc). + // Therefore we only apply query normalization (sorting/dropping) for known + // NativeScript dev endpoints where `t`/`v`/`import` are purely cache busters. + // + // The dev server serves every module under ONE canonical URL — module + // identity IS the URL string. Freshness after an HMR edit is handled by + // `__NS_DEV__.invalidateModules` (registry + prefetch-cache evict) plus the + // eviction-driven fetch nonce in `PerformHttpFetchOnceSync`, never by URL + // variation. There is deliberately no path-tag vocabulary to collapse here. + // + // Special cases that LOOK like dev endpoints but aren't normalized: + // + // `/@ng/component` (Angular HMR component-update endpoint) + // The `t` (timestamp) parameter is the WHOLE POINT of the URL — it + // identifies a specific recompile of the component's metadata after + // a `.html`/style edit. Stripping it would collapse every HMR fetch + // to the same cache key (the boot-time call uses `Date.now()` and + // each subsequent save uses a new `Date.now()`), and the second + // `__ns_import(...)` would hit V8's module cache, resolve the + // boot-time `_UpdateMetadata` default export, and call + // `ɵɵreplaceMetadata` with stale instructions. Result: server logs + // `(client) hmr update`, the listener fires, but the visual never + // changes because the runtime swapped the live view's metadata + // with the same metadata it already had. Treat the path as a + // non-dev endpoint and preserve the query verbatim so each + // timestamped fetch is a distinct registry entry. + // + // Apply the special-case check BEFORE the dev-endpoint short-circuit so + // it covers paths under `/ns/m//@ng/component` (the + // resolved URL Angular's compiler produces relative to the component's + // `import.meta.url`). + { + std::string pathOnly = originAndPath.substr(pathStart); + if (pathOnly.find("/@ng/component") != std::string::npos) { + // Preserve query as-is — `t` is the version discriminator. + return noHash; + } + const bool isDevEndpoint = + StartsWith(pathOnly, "/ns/") || + StartsWith(pathOnly, "/node_modules/.vite/") || + StartsWith(pathOnly, "/@id/") || + StartsWith(pathOnly, "/@fs/"); + if (!isDevEndpoint) { + // Preserve query as-is (fragment already removed). + return noHash; } } if (query.empty()) return originAndPath; - // Strip ?import markers and sort remaining query params for stability + // Keep all params except typical import markers or t/v cache busters; sort for stability. std::vector kept; size_t start = 0; while (start <= query.size()) { @@ -196,7 +194,8 @@ std::string CanonicalizeHttpUrlKey(const std::string& url) { if (!pair.empty()) { size_t eq = pair.find('='); std::string name = (eq == std::string::npos) ? pair : pair.substr(0, eq); - if (!(name == "import")) kept.push_back(pair); + // Drop import marker and common cache-busting stamps. + if (!(name == "import" || name == "t" || name == "v")) kept.push_back(pair); } if (amp == std::string::npos) break; start = amp + 1; @@ -211,6 +210,344 @@ std::string CanonicalizeHttpUrlKey(const std::string& url) { return rebuilt; } +// Resolve a relative/root-absolute import specifier against a parent URL +// using plain string manipulation. Only relative (`./`, `../`) and +// root-absolute (`/`) specifiers are resolved here; bare specifiers and +// already-absolute URLs fall through unchanged. Exposed via +// `HMRSupport.h` so `ModuleInternalCallbacks.cpp` can share this single +// resolver instead of carrying its own copy. +std::string ResolveImportSpecifierAgainstUrl(const std::string& specifier, + const std::string& parentUrl) { + if (specifier.empty()) return ""; + // Already absolute. + if (StartsWith(specifier, "http://") || StartsWith(specifier, "https://")) { + return specifier; + } + bool isRelative = StartsWith(specifier, "./") || StartsWith(specifier, "../"); + bool isRootAbs = !specifier.empty() && specifier[0] == '/'; + if (!isRelative && !isRootAbs) return ""; + + if (!(StartsWith(parentUrl, "http://") || StartsWith(parentUrl, "https://"))) { + return ""; + } + // Drop fragment + query from parent. + std::string base = parentUrl; + size_t hp = base.find('#'); if (hp != std::string::npos) base = base.substr(0, hp); + size_t qp = base.find('?'); if (qp != std::string::npos) base = base.substr(0, qp); + + size_t schemePos = base.find("://"); + if (schemePos == std::string::npos) return ""; + size_t pathStart = base.find('/', schemePos + 3); + std::string origin = (pathStart == std::string::npos) ? base : base.substr(0, pathStart); + std::string path = (pathStart == std::string::npos) ? std::string("/") : base.substr(pathStart); + + std::string specPath = specifier; + std::string suffix; + size_t specQ = specPath.find('?'); + size_t specH = specPath.find('#'); + size_t cut = std::string::npos; + if (specQ != std::string::npos && specH != std::string::npos) cut = std::min(specQ, specH); + else if (specQ != std::string::npos) cut = specQ; + else if (specH != std::string::npos) cut = specH; + if (cut != std::string::npos) { suffix = specPath.substr(cut); specPath = specPath.substr(0, cut); } + + std::string newPath; + if (isRootAbs) { + newPath = specPath; + } else { + size_t lastSlash = path.find_last_of('/'); + std::string baseDir = (lastSlash == std::string::npos) ? std::string("/") : path.substr(0, lastSlash + 1); + newPath = baseDir + specPath; + } + // Normalize `.` and `..` segments. + std::vector stack; + bool absolute = !newPath.empty() && newPath[0] == '/'; + size_t i = 0; + while (i <= newPath.size()) { + size_t j = newPath.find('/', i); + std::string seg = (j == std::string::npos) ? newPath.substr(i) : newPath.substr(i, j - i); + if (seg.empty() || seg == ".") { + // skip + } else if (seg == "..") { + if (!stack.empty()) stack.pop_back(); + } else { + stack.push_back(seg); + } + if (j == std::string::npos) break; + i = j + 1; + } + std::string norm = absolute ? "/" : std::string(); + for (size_t k = 0; k < stack.size(); ++k) { + if (k > 0) norm += "/"; + norm += stack[k]; + } + return origin + norm + suffix; +} + +// ───────────────────────────────────────────────────────────── +// Eviction-driven fetch cache-bust +// +// When the HMR client invalidates a module, the NEXT network fetch of +// that module must not be satisfiable by any HTTP cache layer between +// the runtime and the dev server (OS URL caches, proxies, a +// host-installed HttpResponseCache). `InvalidateModules` marks the +// canonical keys of the eviction set here; `PerformHttpFetchOnceSync` +// then appends a unique `__ns_dev_nonce` query parameter to the +// wire-level request for any marked URL, guaranteeing the cache sees +// a URL it has never stored. The nonce is transport-only — it never +// enters the module registry key (identity stays the canonical URL), +// and the server and the registry never see a varied URL. +static std::mutex g_bustNextFetchMutex; +static std::unordered_set g_bustNextFetchKeys; + +void MarkUrlsForCacheBust(const std::vector& urls) { + if (urls.empty()) return; + std::lock_guard lock(g_bustNextFetchMutex); + for (const auto& url : urls) { + if (url.empty()) continue; + if (!(StartsWith(url, "http://") || StartsWith(url, "https://"))) continue; + g_bustNextFetchKeys.insert(CanonicalizeHttpUrlKey(url)); + } +} + +// Peek (do not consume) — the fetch may be retried on transient failure +// and the retry must still carry a nonce. Cleared on fetch success. +static bool IsUrlMarkedForCacheBust(const std::string& url) { + std::lock_guard lock(g_bustNextFetchMutex); + if (g_bustNextFetchKeys.empty()) return false; + return g_bustNextFetchKeys.find(CanonicalizeHttpUrlKey(url)) != g_bustNextFetchKeys.end(); +} + +static void ClearCacheBustForUrl(const std::string& url) { + std::lock_guard lock(g_bustNextFetchMutex); + if (g_bustNextFetchKeys.empty()) return; + g_bustNextFetchKeys.erase(CanonicalizeHttpUrlKey(url)); +} + +static void ClearAllCacheBustMarks() { + std::lock_guard lock(g_bustNextFetchMutex); + g_bustNextFetchKeys.clear(); +} + +// ============================================================================ +// HTTP body cache + parallel kickstart prewarm +// ============================================================================ +// +// V8 only exposes a synchronous ResolveModuleCallback for static imports. +// Each call into HttpFetchText() blocks the JS thread on a synchronous +// network turn, which forces serial fetching from the JS thread's +// perspective. +// +// `__NS_DEV__.kickstartPrefetch(urls)` lets the JS dev client hand the +// runtime a server-computed module closure (cold-boot graph or HMR +// eviction set) to fetch in one parallel wave BEFORE V8 walks the import +// graph. Bodies land in `g_prefetchCache` keyed by full URL; the +// always-on cache read in `HttpFetchText` then serves V8's synchronous +// walk at memory speed. +// +// The runtime performs NO import scanning and NO speculative graph +// discovery of its own — the server owns the module graph and supplies +// explicit URL lists. +// +// Correctness invariants: +// 1. Cache reads consume (one-shot). A second HttpFetchText for the +// same URL after a cache hit triggers a fresh network fetch — this +// is the right behavior for HMR where re-fetching means we got a +// newer version of the module. +// 2. Every kickstart fetch goes through IsRemoteUrlAllowed() exactly +// the same way HttpFetchText does. The security gate is preserved. +// 3. Kickstart overwrites cache entries unconditionally — a body the +// client explicitly asked to re-fetch is authoritative by +// construction (the previous entry is stale). + +namespace { + +std::mutex g_prefetchMutex; +// Heap-allocated (leaky singleton) to prevent V8 crash during +// __cxa_finalize_ranges. See g_moduleRegistry comment in +// ModuleInternalCallbacks.cpp for full rationale. +auto* _g_prefetchCache = new std::unordered_map(); +auto& g_prefetchCache = *_g_prefetchCache; + +bool LooksLikeJsSourceUrl(const std::string& url) { + size_t qpos = url.find('?'); + std::string path = (qpos == std::string::npos) ? url : url.substr(0, qpos); + // Block clearly non-JS content; on cache hit V8 would attempt to compile + // CSS/images/etc. as ES modules and fail in confusing ways. + if (tns::EndsWith(path, ".css") || tns::EndsWith(path, ".scss") || + tns::EndsWith(path, ".sass") || tns::EndsWith(path, ".less")) return false; + if (tns::EndsWith(path, ".png") || tns::EndsWith(path, ".jpg") || + tns::EndsWith(path, ".jpeg") || tns::EndsWith(path, ".gif") || + tns::EndsWith(path, ".svg") || tns::EndsWith(path, ".webp") || + tns::EndsWith(path, ".ico")) return false; + if (tns::EndsWith(path, ".json")) return false; + if (tns::EndsWith(path, ".html") || tns::EndsWith(path, ".htm")) return false; + if (tns::EndsWith(path, ".woff") || tns::EndsWith(path, ".woff2") || + tns::EndsWith(path, ".ttf") || tns::EndsWith(path, ".otf") || + tns::EndsWith(path, ".eot")) return false; + if (tns::EndsWith(path, ".mp4") || tns::EndsWith(path, ".webm") || + tns::EndsWith(path, ".mp3") || tns::EndsWith(path, ".wav")) return false; + return true; +} + +// Pluggable host yield. Default: no-op. Embedders that want a JS-thread +// runloop pump during cold-boot fetches can install one via +// `RegisterHttpFetchYield` (e.g. ALooper_pollOnce(0)). +void NoopHttpFetchYield() {} +std::atomic g_httpFetchYield{&NoopHttpFetchYield}; + +inline void InvokeHttpFetchYield() { + auto cb = g_httpFetchYield.load(std::memory_order_acquire); + if (cb != nullptr) cb(); +} + +} // anonymous namespace + +void RegisterHttpFetchYield(void (*callback)()) { + g_httpFetchYield.store(callback, std::memory_order_release); +} + +void ClearHttpModulePrefetchCache() { + std::lock_guard lock(g_prefetchMutex); + g_prefetchCache.clear(); +} + +// Drop a specific URL set from `g_prefetchCache`. Used by +// `InvalidateModules` so an HMR eviction purges any stale HTTP body +// the previous kickstart wave left behind. See the doc comment in +// HMRSupport.h for the cache-poisoning case this fixes. +void EvictHttpModulePrefetchCacheUrls(const std::vector& urls) { + if (urls.empty()) return; + std::lock_guard lock(g_prefetchMutex); + size_t hits = 0; + for (const std::string& u : urls) { + auto it = g_prefetchCache.find(u); + if (it != g_prefetchCache.end()) { g_prefetchCache.erase(it); ++hits; } + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[prefetch][evict] urls=%lu hits=%lu remaining=%lu", + (unsigned long)urls.size(), (unsigned long)hits, + (unsigned long)g_prefetchCache.size()); + } +} + +// Thread-local capture of the most recent JNI-level fetch failure +// (e.g. `ConnectException: failed to connect to /10.0.2.2 (port 5173) +// after 15000ms: connect failed: ECONNREFUSED`). Callers that just +// got back `status=0` from `HttpFetchText` can pull this string and +// splice it into the JS error to give users actionable detail rather +// than a generic "Failed to fetch" line. +// +// Thread-local because each V8 isolate / worker has its own JS +// thread, and concurrent fetches would otherwise clobber each +// other's diagnostic. +static thread_local std::string g_lastHttpFetchErrorReason; + +static void RecordLastHttpFetchError(const char* stage, + const std::string& excClass, + const std::string& excMsg) { + // Format is grep-friendly and short enough to splice into a JS + // Error message without exploding the line length: + // stage=get-response-code class=java.net.ConnectException msg=... + g_lastHttpFetchErrorReason.assign("stage="); + g_lastHttpFetchErrorReason.append(stage ? stage : "?"); + g_lastHttpFetchErrorReason.append(" class="); + g_lastHttpFetchErrorReason.append(excClass); + g_lastHttpFetchErrorReason.append(" msg="); + g_lastHttpFetchErrorReason.append(excMsg); +} + +void ClearLastHttpFetchErrorReason() { + g_lastHttpFetchErrorReason.clear(); +} + +std::string TakeLastHttpFetchErrorReason() { + std::string out = std::move(g_lastHttpFetchErrorReason); + g_lastHttpFetchErrorReason.clear(); + return out; +} + +// Decide whether a captured fetch failure reason looks like the +// transient okhttp / socket-pool class of bug that one retry on a +// fresh connection reliably clears. +// +// The list is deliberately narrow. Hard failures like `ConnectException`, +// `UnknownHostException`, `MalformedURLException`, or the runtime's own +// security gate (`status=403`) are NOT retryable — retrying would only mask +// config bugs (wrong host/port, missing allowlist entry) behind extra latency. +// +// Patterns covered: +// * `unexpected end of stream` — server closed the socket mid-handshake +// (Http1xStream.readResponseHeaders). +// * `Connection reset` / `SocketException` / `EOFException` — same root +// cause, different surfacing depending on when the RST/FIN landed. +// * `Software caused connection abort` — Android/Linux variant, seen on +// emulators under load. +// * `Stream closed` / `StreamResetException` — HTTP/2 codepath (some reverse +// proxies upgrade the tunnel to h2). +static bool IsRetryableFetchReason(const std::string& reason) { + if (reason.find("unexpected end of stream") != std::string::npos) return true; + if (reason.find("Connection reset") != std::string::npos) return true; + if (reason.find("Software caused connection abort") != std::string::npos) return true; + if (reason.find("EOFException") != std::string::npos) return true; + if (reason.find("SocketException") != std::string::npos) return true; + if (reason.find("StreamResetException") != std::string::npos) return true; + if (reason.find("Stream closed") != std::string::npos) return true; + return false; +} + +// Raw JNI fetch — no cache lookup, no allowlist gate. Used by the +// kickstart threads (which already pre-filtered URLs) so the public +// `HttpFetchText` can keep its allowlist-and-cache logic in one place +// without recursing into itself. Returns true on success (2xx, +// non-empty body). +static bool PerformHttpFetchOnceSync(const std::string& url, + std::string& out, + std::string& contentType, + int& status); + +// If a Java exception is pending, drain it into `outClassName` / +// `outMessage` and clear it so subsequent JNI calls don't ABORT the +// process. Returns true when an exception was actually present. +// +// We grab both the simple class name (e.g. `ConnectException`) and +// the `toString()` payload because the latter often includes the +// underlying OS errno (`Connection refused`, `Network unreachable`, +// `failed to connect to /10.0.2.2 (port 5173)` etc.) — exactly the +// diagnostic the silent `status=0` symptom hides. +static bool DrainPendingJniException(JEnv& env, std::string& outClassName, std::string& outMessage) { + outClassName.clear(); + outMessage.clear(); + jthrowable th = env.ExceptionOccurred(); + if (!th) return false; + env.ExceptionClear(); + + jclass clsThrowable = env.GetObjectClass(th); + if (clsThrowable) { + jclass clsClass = env.FindClass("java/lang/Class"); + if (clsClass) { + jmethodID getName = env.GetMethodID(clsClass, "getName", "()Ljava/lang/String;"); + if (getName) { + jstring jName = static_cast(env.CallObjectMethod(clsThrowable, getName)); + env.ExceptionClear(); + if (jName) { + outClassName = ArgConverter::jstringToString(jName); + } + } + } + jmethodID toString = env.GetMethodID(clsThrowable, "toString", "()Ljava/lang/String;"); + if (toString) { + jstring jMsg = static_cast(env.CallObjectMethod(th, toString)); + env.ExceptionClear(); + if (jMsg) { + outMessage = ArgConverter::jstringToString(jMsg); + } + } + } + env.ExceptionClear(); + return true; +} + // Minimal HTTP fetch using java.net.* via JNI. Returns true on success (2xx) and non-empty body. // Security: This is the single point of enforcement for remote module loading. // In debug mode, all URLs are allowed. In production, checks security.allowRemoteModules @@ -219,19 +556,178 @@ bool HttpFetchText(const std::string& url, std::string& out, std::string& conten out.clear(); contentType.clear(); status = 0; - + // Start each fetch with a clean diagnostic slot so a successful + // fetch can't leave a stale reason from a previous failure. + ClearLastHttpFetchErrorReason(); + // Security gate: check if remote module loading is allowed before any HTTP fetch. if (!IsRemoteUrlAllowed(url)) { - status = 403; // Forbidden + status = 403; if (IsScriptLoadingLogEnabled()) { DEBUG_WRITE("[http-esm][security][blocked] %s", url.c_str()); } return false; } - + + // Cache-read fast path. The JS dev client populates `g_prefetchCache` + // via `__NS_DEV__.kickstartPrefetch(urls)` right before importing (cold + // boot) or re-importing (HMR); by the time V8's synchronous walk asks + // for a module, the body is already here and the walk runs at memory + // speed instead of network speed. + // + // Cache reads are one-shot; consuming the entry guarantees that a + // re-fetch (e.g. after HMR) goes back to the network for fresh source. + { + std::string cached; + bool cacheHit = false; + { + std::lock_guard lock(g_prefetchMutex); + auto it = g_prefetchCache.find(url); + if (it != g_prefetchCache.end()) { + cached = std::move(it->second); + g_prefetchCache.erase(it); + cacheHit = true; + } + } + if (cacheHit) { + out = std::move(cached); + contentType = "application/javascript"; + status = 200; + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-loader][prefetch][hit] %s (%lu bytes)", url.c_str(), (unsigned long)out.size()); + } + // Yield to the host between back-to-back cache hits so any + // installed heartbeat/runloop pump gets a turn. + InvokeHttpFetchYield(); + return true; + } + } + + // Slow path: synchronous fetch with bounded retry on transient okhttp-class + // failures. Android's stock HttpURLConnection (backed by okhttp) periodically + // half-recycles sockets when the server's keep-alive timeout fires before the + // next request lands; the next use of that socket throws + // `IOException: unexpected end of stream` from + // `Http1xStream.readResponseHeaders`. It manifests as random per-request + // failures across modules on cold boot, and a fresh connection on the next + // attempt succeeds. + // + // `Connection: close` prevents okhttp from pooling our own connection but + // doesn't help when the server already poisoned the pool from an earlier + // in-flight fetch. The retry covers both cases, and the system-wide + // `http.keepAlive=false` set inside `PerformHttpFetchOnceSync` keeps okhttp + // from pooling in the first place. + constexpr int kMaxAttempts = 3; + for (int attempt = 1; attempt <= kMaxAttempts; ++attempt) { + // Clear the slot at the start of each attempt so a previous + // attempt's reason can't leak into a later one. + ClearLastHttpFetchErrorReason(); + if (PerformHttpFetchOnceSync(url, out, contentType, status)) { + if (attempt > 1 && IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][retry-ok] url=%s attempt=%d", url.c_str(), attempt); + } + // Yield to the host after the sync fetch block so any installed + // pump can repaint before V8 calls us again. + InvokeHttpFetchYield(); + return true; + } + std::string reason = TakeLastHttpFetchErrorReason(); + if (attempt >= kMaxAttempts || !IsRetryableFetchReason(reason)) { + // Re-stash so the caller sees the same reason we just consumed. + if (!reason.empty()) { + RecordLastHttpFetchError("final-attempt", "captured", reason); + } + return false; + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][retry] url=%s attempt=%d/%d reason=%s", + url.c_str(), attempt, kMaxAttempts, reason.c_str()); + } + // Short linear backoff. A stale pooled socket only needs one + // tick to clear; longer waits would just add cold-boot latency + // on what's typically dozens of static imports. + std::this_thread::sleep_for(std::chrono::milliseconds(25 * attempt)); + } + return false; +} + +// True raw HTTP fetch path. Kept separate from HttpFetchText so the +// kickstart (which already filtered URLs and intends to populate the +// cache) doesn't re-check the cache itself. +static bool PerformHttpFetchOnceSync(const std::string& url, + std::string& out, + std::string& contentType, + int& status) { + out.clear(); + contentType.clear(); + status = 0; + // Entry trace gated behind `logScriptLoading`. Cold boot fires this dozens of + // times per session, so it stays off by default; enable `logScriptLoading` + // when triaging the HTTP-ESM path. + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][enter] url=%s", url.c_str()); + } + + // Eviction-driven cache-bust: if this URL's canonical key was marked + // by `InvalidateModules` (via `MarkUrlsForCacheBust`), append a + // unique nonce query parameter so any HTTP cache layer sees a URL it + // has never stored and must go to origin. The dev server ignores + // unknown query params on module routes, so the response body is + // unchanged. First-touch fetches don't need busting — nothing has + // cached them yet — so unmarked URLs go out verbatim (some Vite + // virtual routes require exact-match URLs and 404 on unknown query + // params). + std::string fetchUrl = url; + const bool bustRequested = IsUrlMarkedForCacheBust(url); + if (bustRequested) { + static std::atomic s_fetchSeq{0}; + const uint64_t seq = s_fetchSeq.fetch_add(1, std::memory_order_relaxed); + const uint64_t nowMs = (uint64_t)std::chrono::duration_cast( + std::chrono::system_clock::now().time_since_epoch()).count(); + fetchUrl += (url.find('?') == std::string::npos) ? '?' : '&'; + fetchUrl += "__ns_dev_nonce="; + fetchUrl += std::to_string(nowMs); + fetchUrl += "-"; + fetchUrl += std::to_string(seq); + } + try { JEnv env; + // One-time process-wide kill switch for okhttp's connection + // pool. Android's stock HttpURLConnection (which okhttp backs) + // pools sockets across requests, and when Vite's keep-alive + // timeout fires before our next request we end up reusing a + // dead socket and hitting + // `IOException: unexpected end of stream`. Setting + // `http.keepAlive=false` forces a fresh TCP connection per + // fetch, which sidesteps the pool entirely. + // + // We also set `Connection: close` on the per-request headers + // below as a belt-and-suspenders signal — the property covers + // the pool, the header covers the wire. Doing this once via + // an atomic guard keeps the cost out of the hot path on + // repeat fetches (this is a synchronous V8 module-loader + // hot path that can fire dozens of times per cold boot). + static std::atomic sKeepAliveDisabled{false}; + if (!sKeepAliveDisabled.exchange(true)) { + jclass clsSystem = env.FindClass("java/lang/System"); + if (clsSystem) { + jmethodID setProperty = env.GetStaticMethodID( + clsSystem, "setProperty", + "(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;"); + if (setProperty) { + jstring jKey = env.NewStringUTF("http.keepAlive"); + jstring jVal = env.NewStringUTF("false"); + env.CallStaticObjectMethod(clsSystem, setProperty, jKey, jVal); + // Don't care about the previous value or about exceptions + // here — `System.setProperty` only throws SecurityException + // under a SecurityManager and Android apps don't install one. + env.ExceptionClear(); + } + } + } + // Allow network operations on the current thread (dev-only HMR path) // Some Android environments enforce StrictMode which throws NetworkOnMainThreadException // when performing network I/O on the main thread. Since this fetch runs on the JS/V8 thread @@ -261,10 +757,39 @@ bool HttpFetchText(const std::string& url, std::string& out, std::string& conten if (!clsURL) return false; jmethodID urlCtor = env.GetMethodID(clsURL, "", "(Ljava/lang/String;)V"); jmethodID openConnection = env.GetMethodID(clsURL, "openConnection", "()Ljava/net/URLConnection;"); - jstring jUrlStr = env.NewStringUTF(url.c_str()); + jstring jUrlStr = env.NewStringUTF(fetchUrl.c_str()); jobject urlObj = env.NewObject(clsURL, urlCtor, jUrlStr); - jobject conn = env.CallObjectMethod(urlObj, openConnection); + // `URL` ctor throws MalformedURLException on bad input. Drain it + // so we can blame the right thing in logs rather than the silent + // path below. + { + std::string excClass, excMsg; + if (DrainPendingJniException(env, excClass, excMsg)) { + RecordLastHttpFetchError("url-ctor", excClass, excMsg); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][exception] stage=url-ctor url=%s class=%s msg=%s", + url.c_str(), excClass.c_str(), excMsg.c_str()); + } + return false; + } + } + + jobject conn = env.CallObjectMethod(urlObj, openConnection); + // `URL.openConnection()` can throw IOException for unsupported + // protocols or proxy lookup failures. Capture the message before + // it gets eaten by the bare `return false`. + { + std::string excClass, excMsg; + if (DrainPendingJniException(env, excClass, excMsg)) { + RecordLastHttpFetchError("open-connection", excClass, excMsg); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][exception] stage=open-connection url=%s class=%s msg=%s", + url.c_str(), excClass.c_str(), excMsg.c_str()); + } + return false; + } + } if (!conn) return false; jclass clsConn = env.GetObjectClass(conn); @@ -290,6 +815,19 @@ bool HttpFetchText(const std::string& url, std::string& out, std::string& conten jmethodID getErrorStream = isHttp ? env.GetMethodID(clsHttp, "getErrorStream", "()Ljava/io/InputStream;") : nullptr; if (isHttp && getResponseCode) { status = env.CallIntMethod(conn, getResponseCode); + // `getResponseCode()` is the call that actually performs the TCP + // connect — so this is where ConnectException / SocketTimeout / + // UnknownHost surface. Drain the exception here so it doesn't return + // as a bare `status=0`. + std::string excClass, excMsg; + if (DrainPendingJniException(env, excClass, excMsg)) { + RecordLastHttpFetchError("get-response-code", excClass, excMsg); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][exception] stage=get-response-code url=%s class=%s msg=%s", + url.c_str(), excClass.c_str(), excMsg.c_str()); + } + return false; + } } // Read InputStream (prefer error stream on HTTP error codes) @@ -301,6 +839,20 @@ bool HttpFetchText(const std::string& url, std::string& out, std::string& conten if (!inStream) { inStream = env.CallObjectMethod(conn, getInputStream); } + // `getInputStream()` is the second place a connect failure surfaces + // (when the previous `getResponseCode` path didn't trigger it, + // e.g. for non-HTTP URLConnection subclasses). + { + std::string excClass, excMsg; + if (DrainPendingJniException(env, excClass, excMsg)) { + RecordLastHttpFetchError("get-input-stream", excClass, excMsg); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][exception] stage=get-input-stream url=%s class=%s msg=%s", + url.c_str(), excClass.c_str(), excMsg.c_str()); + } + return false; + } + } if (!inStream) return false; jclass clsIS = env.GetObjectClass(inStream); @@ -344,10 +896,600 @@ bool HttpFetchText(const std::string& url, std::string& out, std::string& conten } if (status == 0) status = 200; // assume OK if not HTTP - return status >= 200 && status < 300 && !out.empty(); + bool ok = status >= 200 && status < 300 && !out.empty(); + // A fresh body arrived from origin — the bust request (if any) has + // been satisfied. Clear the mark so steady-state re-fetches of the + // same URL don't keep paying the nonce (and stay exact-match for + // routes that require it). + if (ok && bustRequested) { + ClearCacheBustForUrl(url); + } + return ok; + } catch (NativeScriptException& nse) { + // `JEnv::CheckForJavaException()` converts any pending Java + // exception into a `NativeScriptException` and rethrows on the + // C++ side. Because JEnv has already called `ExceptionClear`, + // `DrainPendingJniException` at the JNI call sites above sees + // nothing — the only place the original Java message survives + // is on this exception object. So we record it here too, + // covering both the wrapped JNI calls (`env.GetMethodID`, + // `env.CallVoidMethod`, etc.) and the raw `m_env->` calls in + // the same try block. + std::string what = nse.what() ? nse.what() : ""; + if (what.empty()) { + what = nse.GetErrorMessage(); + } + RecordLastHttpFetchError("native-script-exception", "tns::NativeScriptException", what); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][exception] stage=native-script-exception url=%s msg=%s", + url.c_str(), what.c_str()); + } + return false; + } catch (std::exception& ex) { + std::string what = ex.what() ? ex.what() : ""; + RecordLastHttpFetchError("std-exception", "std::exception", what); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][exception] stage=std-exception url=%s msg=%s", + url.c_str(), what.c_str()); + } + return false; } catch (...) { + RecordLastHttpFetchError("unknown-cpp-exception", "", ""); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][fetch][exception] stage=unknown-cpp-exception url=%s", + url.c_str()); + } + return false; + } +} + +// ───────────────────────────────────────────────────────────── +// List-mode kickstart prewarm. +// +// The dev server owns the module graph: it computes the inverse-dep +// closure for HMR updates (`evictPaths`) and can crawl the entry graph +// for cold boot. The client hands that explicit URL list to +// `__NS_DEV__.kickstartPrefetch(urls)`, which fetches every entry in one +// parallel wave into `g_prefetchCache` before V8 starts its serial +// synchronous walk. Concurrency is bounded by a counting semaphore +// implemented with mutex + condition variable (NDKs do not yet ship the +// C++20 `std::counting_semaphore`). + +namespace { + +class CountingSemaphore { + public: + explicit CountingSemaphore(int initial) : count_(initial) {} + void Acquire() { + std::unique_lock lk(m_); + cv_.wait(lk, [this]{ return count_ > 0; }); + --count_; + } + void Release() { + { + std::lock_guard lk(m_); + ++count_; + } + cv_.notify_one(); + } + + private: + std::mutex m_; + std::condition_variable cv_; + int count_; +}; + +struct KickstartContext { + std::mutex mutex; + std::unordered_set visited; + std::atomic fetchedCount{0}; + std::atomic bytes{0}; + std::unique_ptr concurrency; + + // Outstanding-work counter: each scheduled fetch increments + decrements a + // counter under a mutex, and the wait loop blocks on `cv.wait_for` for + // transitions to zero. + std::mutex pendingMutex; + std::condition_variable pendingCv; + int pending = 0; + + void EnterPending() { + std::lock_guard lk(pendingMutex); + ++pending; + } + void LeavePending() { + { + std::lock_guard lk(pendingMutex); + if (pending > 0) --pending; + } + pendingCv.notify_all(); + } + // Wait up to `sliceMs` for `pending == 0`. Returns true if drained. + bool WaitDrainSlice(int sliceMs) { + std::unique_lock lk(pendingMutex); + return pendingCv.wait_for(lk, std::chrono::milliseconds(sliceMs), + [this]{ return pending == 0; }); + } +}; + +void KickstartScheduleUrls(std::shared_ptr ctx, + std::vector urls) { + for (const std::string& urlRef : urls) { + if (urlRef.empty()) continue; + if (!StartsWith(urlRef, "http://") && !StartsWith(urlRef, "https://")) continue; + if (!LooksLikeJsSourceUrl(urlRef)) continue; + if (!IsRemoteUrlAllowed(urlRef)) continue; + + bool fresh; + { + std::lock_guard lock(ctx->mutex); + fresh = ctx->visited.insert(urlRef).second; + } + if (!fresh) continue; + + // No "already cached" short-circuit here — the caller has explicitly + // told us "fetch these URLs fresh". Any body sitting in + // `g_prefetchCache` for one of them is a leftover from a previous + // wave that V8 didn't consume; honoring it would feed V8 a stale + // body on the next walk — the "1 cycle behind" symptom for `.ts` + // edits with many transitive importers. (`InvalidateModules` + // pre-clears the cache for the eviction set, so this is + // defense-in-depth — but the kickstart may also be invoked + // manually for diagnostics, and we want it to be correct in + // isolation.) + + ctx->EnterPending(); + std::string urlCopy = urlRef; + auto ctxCopy = ctx; + std::thread([ctxCopy, urlCopy]() { + ctxCopy->concurrency->Acquire(); + std::string body, contentType; + int status = 0; + bool ok = PerformHttpFetchOnceSync(urlCopy, body, contentType, status); + if (ok && status >= 200 && status < 300 && !body.empty()) { + const size_t bodySize = body.size(); + // Overwrite unconditionally — the fresh body we just fetched is + // by definition the authoritative copy; any older cache entry is + // stale by construction (the caller has just told us so). + { + std::lock_guard lock(g_prefetchMutex); + g_prefetchCache[urlCopy] = std::move(body); + } + ctxCopy->fetchedCount.fetch_add(1, std::memory_order_relaxed); + ctxCopy->bytes.fetch_add(bodySize, std::memory_order_relaxed); + } + ctxCopy->concurrency->Release(); + ctxCopy->LeavePending(); + }).detach(); + } +} + +} // anonymous namespace + +bool KickstartHmrPrefetchUrlsSync(const std::vector& urls, + int maxConcurrent, + double timeoutSeconds, + size_t* outFetchedCount, + uint64_t* outElapsedMs) { + if (urls.empty()) return false; + // Drop empty / non-allowlisted URLs up front. We still want a + // truthy result even if some entries get filtered, because partial + // success is strictly better than the no-kickstart baseline. + std::vector filtered; + filtered.reserve(urls.size()); + for (const auto& u : urls) { + if (u.empty()) continue; + if (!IsRemoteUrlAllowed(u)) continue; + filtered.push_back(u); + } + if (filtered.empty()) return false; + + if (maxConcurrent <= 0) maxConcurrent = 16; + if (timeoutSeconds <= 0.0) timeoutSeconds = 10.0; + + const auto start = std::chrono::steady_clock::now(); + + // Diagnostic seed — we record the first URL purely so the log line + // has a recognizable anchor when the user is correlating with their + // server-side `[hmr-ws][update] file=...` line. + const std::string diagSeed = filtered.front(); + const size_t requestedCount = filtered.size(); + + auto ctx = std::make_shared(); + ctx->concurrency = std::make_unique(maxConcurrent); + + KickstartScheduleUrls(ctx, std::move(filtered)); + + // Wait loop. Uses a slice-based timeout so the host runloop (via the + // pluggable yield hook) gets a chance to drain between slices — this + // matters most during cold boot, before the dev client has called + // `__NS_DEV__.setDevBootComplete(true)`. 50ms is short enough to feel + // responsive and long enough to avoid spinning. + const int sliceMs = 50; + const auto deadline = start + std::chrono::milliseconds(static_cast(timeoutSeconds * 1000.0)); + bool drained = false; + while (true) { + drained = ctx->WaitDrainSlice(sliceMs); + if (drained) break; + if (std::chrono::steady_clock::now() >= deadline) break; + if (!IsDevSessionBootComplete()) { + InvokeHttpFetchYield(); + } + } + + const auto end = std::chrono::steady_clock::now(); + const uint64_t elapsedMs = + std::chrono::duration_cast(end - start).count(); + const size_t fetched = ctx->fetchedCount.load(std::memory_order_relaxed); + const size_t bytes = ctx->bytes.load(std::memory_order_relaxed); + + if (outFetchedCount) *outFetchedCount = fetched; + if (outElapsedMs) *outElapsedMs = elapsedMs; + + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[hmr-kickstart][list] first=%s urls=%lu fetched=%lu bytes=%lu ms=%llu status=%s concurrency=%d", + diagSeed.c_str(), + (unsigned long)requestedCount, + (unsigned long)fetched, + (unsigned long)bytes, + (unsigned long long)elapsedMs, + drained ? "drained" : "timeout", + maxConcurrent); + } + + return drained; +} + +void CleanupHMRGlobals() { + // Drop any kickstart-prewarmed module sources. These are plain + // std::string buffers (no v8::Global), but flushing them on teardown + // prevents stale source from leaking into a re-launched runtime in + // the same process. + ClearHttpModulePrefetchCache(); + ClearAllCacheBustMarks(); + // Reset the boot-complete flag so a re-launched runtime in the same + // process starts in "cold boot" mode again (yield pump armed). + g_devSessionBootComplete.store(false, std::memory_order_relaxed); +} + +// ───────────────────────────────────────────────────────────── +// Dev-loader JS-callable globals +// +// The runtime's dev surface is deliberately small: it exposes +// *mechanism* only (resolution config, registry eviction, parallel +// prewarm, registry introspection, boot-complete signal). All HMR +// *policy* — boot orchestration, `import.meta.hot`, full reload, CSS +// apply, WebSocket protocol — lives in the JS dev client +// (`@nativescript/vite`). + +namespace { + +// Sets the function name on the v8 Function for nicer stack traces and +// attaches it as a method of the `__NS_DEV__` namespace object. +void InstallDevFunction(v8::Isolate* isolate, v8::Local context, + v8::Local target, const char* name, + v8::FunctionCallback callback) { + v8::Local fnTpl = + v8::FunctionTemplate::New(isolate, callback); + v8::Local fn = fnTpl->GetFunction(context).ToLocalChecked(); + fn->SetName(ToV8String(isolate, name)); + target->CreateDataProperty(context, ToV8String(isolate, name), fn) + .Check(); +} + +// Parse an import-map value (a JSON string OR a JS object of shape +// `{ imports: { "": "", ... } }`) into flat (key → URL) entries using +// V8's own JSON/object model. Returns true if it found an `imports` object +// (even if empty); false if the value is unusable. Only flat string key→URL +// mappings are honored; non-string import values are skipped. +bool ReadImportMapEntries(v8::Isolate* isolate, + v8::Local context, + v8::Local importMapValue, + std::vector>* out) { + v8::Local mapVal = importMapValue; + if (mapVal->IsString()) { + v8::Local parsed; + if (!v8::JSON::Parse(context, mapVal.As()).ToLocal(&parsed)) { + return false; + } + mapVal = parsed; + } + if (!mapVal->IsObject()) return false; + v8::Local mapObj = mapVal.As(); + + v8::Local importsVal; + if (!mapObj->Get(context, ToV8String(isolate, "imports")).ToLocal(&importsVal) || + !importsVal->IsObject()) { return false; } + v8::Local imports = importsVal.As(); + v8::Local keys; + if (!imports->GetOwnPropertyNames(context).ToLocal(&keys)) return false; + + for (uint32_t i = 0; i < keys->Length(); ++i) { + v8::Local keyVal; + if (!keys->Get(context, i).ToLocal(&keyVal)) continue; + v8::Local valVal; + if (!imports->Get(context, keyVal).ToLocal(&valVal) || !valVal->IsString()) continue; + v8::String::Utf8Value keyUtf8(isolate, keyVal); + v8::String::Utf8Value valUtf8(isolate, valVal); + if (*keyUtf8 && *valUtf8) { + out->emplace_back(std::string(*keyUtf8), std::string(*valUtf8)); + } + } + return true; +} + +// `__NS_DEV__.configureRuntime(config)` — apply the dev client's resolver +// configuration: the bare-specifier import map and the volatile URL +// patterns. Bare-specifier resolution happens inside V8's synchronous +// `ResolveModuleCallback` — an embedder host callback JS cannot install +// or intercept — which is why this must be native. +void ConfigureDevRuntimeCallback(const v8::FunctionCallbackInfo& info) { + v8::Isolate* isolate = info.GetIsolate(); + v8::HandleScope scope(isolate); + v8::Local ctx = isolate->GetCurrentContext(); + bool logScriptLoading = tns::IsScriptLoadingLogEnabled(); + + if (info.Length() < 1 || !info[0]->IsObject()) { + if (logScriptLoading) { + DEBUG_WRITE("[__NS_DEV__.configureRuntime] expected config object argument"); + } + return; + } + v8::Local config = info[0].As(); + + // importMap: accept either a JSON string or an object with `{imports:{}}`. + // The dev server's runtime-config endpoint serializes as an object; + // older entry paths pass a serialized JSON string. Accept both shapes. + v8::Local importMapVal; + if (config->Get(ctx, ToV8String(isolate, "importMap")).ToLocal(&importMapVal) && + !importMapVal->IsUndefined() && !importMapVal->IsNull()) { + std::vector> importEntries; + if (ReadImportMapEntries(isolate, ctx, importMapVal, &importEntries) && + !importEntries.empty()) { + SetImportMapEntries(importEntries); + if (logScriptLoading) { + DEBUG_WRITE("[__NS_DEV__.configureRuntime] import map set (%zu entries)", importEntries.size()); + } + } + } + + // volatilePatterns: list of URL substrings that should always re-fetch. + v8::Local vpVal; + if (config->Get(ctx, ToV8String(isolate, "volatilePatterns")).ToLocal(&vpVal) && vpVal->IsArray()) { + v8::Local arr = vpVal.As(); + std::vector patterns; + patterns.reserve(arr->Length()); + for (uint32_t i = 0; i < arr->Length(); i++) { + v8::Local elem; + if (arr->Get(ctx, i).ToLocal(&elem) && elem->IsString()) { + v8::String::Utf8Value utf8(isolate, elem); + if (*utf8) patterns.emplace_back(*utf8); + } + } + if (!patterns.empty()) { + SetVolatilePatterns(patterns); + if (logScriptLoading) { + DEBUG_WRITE("[__NS_DEV__.configureRuntime] %zu volatile patterns set", patterns.size()); + } + } + } +} + +void InvalidateModulesCallback(const v8::FunctionCallbackInfo& info) { + v8::Isolate* isolate = info.GetIsolate(); + v8::HandleScope scope(isolate); + v8::Local ctx = isolate->GetCurrentContext(); + + if (info.Length() < 1 || !info[0]->IsArray()) { + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[__NS_DEV__.invalidateModules] expected array of URL strings"); + } + return; + } + v8::Local urlsArray = info[0].As(); + std::vector urls; + urls.reserve(urlsArray->Length()); + for (uint32_t i = 0; i < urlsArray->Length(); i++) { + v8::Local v; + if (!urlsArray->Get(ctx, i).ToLocal(&v) || !v->IsString()) continue; + v8::String::Utf8Value utf8(isolate, v); + if (*utf8) urls.emplace_back(*utf8); + } + // Observability: surface every URL the runtime is asked to drop so we + // can correlate "asked to evict X" against "actually had X loaded as + // Y" when canonicalization differs. Verbose-gated since per-event + // chatter is only useful while debugging an eviction mismatch. + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[ns-hmr][android-invalidate] called urls.count=%zu", urls.size()); + size_t shown = 0; + for (const auto& u : urls) { + if (shown >= 32) break; + DEBUG_WRITE("[ns-hmr][android-invalidate] url[%zu]=%s", shown, u.c_str()); + ++shown; + } + if (urls.size() > shown) { + DEBUG_WRITE("[ns-hmr][android-invalidate] (hidden %zu more URL(s))", urls.size() - shown); + } + } + InvalidateModules(urls); +} + +// `__NS_DEV__.kickstartPrefetch(urls, options?)` lets the HMR client tell +// the runtime "the next (re-)import will walk this module set — please +// pre-fill the loader cache with every listed body before V8 starts +// walking". The list is always server-computed (the dev server owns the +// module graph: eviction closures for HMR, entry-graph crawls for cold +// boot); the runtime performs no graph discovery of its own. A single +// string argument is accepted as a one-element list. +// +// Returns `{ ok, fetched, ms }` so JS can log the result. On failure +// callers should fall back to V8's normal synchronous walk. +void KickstartHmrPrefetchCallback(const v8::FunctionCallbackInfo& info) { + v8::Isolate* isolate = info.GetIsolate(); + v8::HandleScope scope(isolate); + v8::Local ctx = isolate->GetCurrentContext(); + + auto buildResult = [&](bool ok, size_t fetched, uint64_t elapsedMs) { + v8::Local result = v8::Object::New(isolate); + (void)result->Set(ctx, ToV8String(isolate, "ok"), v8::Boolean::New(isolate, ok)); + (void)result->Set(ctx, ToV8String(isolate, "fetched"), + v8::Integer::NewFromUnsigned(isolate, (uint32_t)fetched)); + (void)result->Set(ctx, ToV8String(isolate, "ms"), + v8::Number::New(isolate, (double)elapsedMs)); + info.GetReturnValue().Set(result); + }; + + if (info.Length() < 1 || (!info[0]->IsString() && !info[0]->IsArray())) { + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[__NS_DEV__.kickstartPrefetch] expected (urls: string[], options?) or (url: string, options?)"); + } + buildResult(false, 0, 0); + return; + } + + int maxConcurrent = 16; + double timeoutSeconds = 10.0; + if (info.Length() >= 2 && info[1]->IsObject()) { + v8::Local options = info[1].As(); + v8::Local mcVal; + if (options->Get(ctx, ToV8String(isolate, "maxConcurrent")).ToLocal(&mcVal) && + mcVal->IsNumber()) { + double mc = mcVal->NumberValue(ctx).FromMaybe(16.0); + if (mc >= 1.0 && mc <= 64.0) maxConcurrent = (int)mc; + } + v8::Local toVal; + if (options->Get(ctx, ToV8String(isolate, "timeoutMs")).ToLocal(&toVal) && + toVal->IsNumber()) { + double ms = toVal->NumberValue(ctx).FromMaybe(10000.0); + if (ms >= 100.0 && ms <= 60000.0) timeoutSeconds = ms / 1000.0; + } + } + + std::vector urls; + if (info[0]->IsArray()) { + v8::Local arr = info[0].As(); + const uint32_t len = arr->Length(); + urls.reserve(len); + for (uint32_t i = 0; i < len; i++) { + v8::Local elem; + if (!arr->Get(ctx, i).ToLocal(&elem)) continue; + if (!elem->IsString()) continue; + v8::String::Utf8Value u8(isolate, elem); + if (!*u8) continue; + std::string s(*u8); + if (s.empty()) continue; + urls.push_back(std::move(s)); + } + } else { + v8::String::Utf8Value u8(isolate, info[0]); + if (*u8) { + std::string s(*u8); + if (!s.empty()) urls.push_back(std::move(s)); + } + } + + if (urls.empty()) { + buildResult(false, 0, 0); + return; + } + + size_t fetched = 0; + uint64_t elapsedMs = 0; + bool ok = KickstartHmrPrefetchUrlsSync(urls, maxConcurrent, timeoutSeconds, + &fetched, &elapsedMs); + buildResult(ok, fetched, elapsedMs); +} + +void GetLoadedModuleUrlsCallback(const v8::FunctionCallbackInfo& info) { + v8::Isolate* isolate = info.GetIsolate(); + v8::HandleScope scope(isolate); + v8::Local ctx = isolate->GetCurrentContext(); + std::vector urls = GetLoadedModuleUrls(); + v8::Local result = v8::Array::New(isolate, static_cast(urls.size())); + for (uint32_t i = 0; i < urls.size(); i++) { + (void)result->Set(ctx, i, ToV8String(isolate, urls[i].c_str())); + } + info.GetReturnValue().Set(result); +} + +// `__NS_DEV__.setDevBootComplete(value?: boolean)` — the JS dev client calls +// this (with `true`, or no argument) once the real app root view has +// committed. It flips both the JS-visible `__NS_HMR_BOOT_COMPLETE__` +// global and the native atomic that disarms the cold-boot yield cadence +// in the kickstart pump-wait. The client may also pass `false` before +// a full JS-realm reload to re-arm the boot-time behaviors. +void SetDevBootCompleteCallback(const v8::FunctionCallbackInfo& info) { + v8::Isolate* isolate = info.GetIsolate(); + v8::HandleScope scope(isolate); + v8::Local ctx = isolate->GetCurrentContext(); + + bool value = true; + if (info.Length() >= 1 && !info[0]->IsUndefined() && !info[0]->IsNull()) { + value = info[0]->BooleanValue(isolate); + } + + tns::SetDevBootComplete(isolate, ctx, value); +} + +// Debug-only diagnostic: expose CanonicalizeHttpUrlKey to JS so the test +// harness can pin its identity behavior. Not part of the @nativescript/vite +// client API; release builds omit it. +void CanonicalizeHttpUrlKeyCallback(const v8::FunctionCallbackInfo& info) { + v8::Isolate* isolate = info.GetIsolate(); + v8::HandleScope scope(isolate); + if (info.Length() < 1 || !info[0]->IsString()) { + info.GetReturnValue().SetEmptyString(); + return; + } + v8::String::Utf8Value u(isolate, info[0]); + std::string key = CanonicalizeHttpUrlKey(*u ? std::string(*u) : std::string()); + info.GetReturnValue().Set(ToV8String(isolate, key.c_str())); +} + +} // anonymous namespace + +void InitializeHmrDevGlobals(v8::Isolate* isolate, v8::Local context, + bool isWorker) { + // The dev host API lives here: `__NS_DEV__`. + // + // Installed in EVERY build, release included. That is deliberate: the + // security boundary sits at the network layer, not the namespace — + // every HTTP fetch the runtime can make passes through + // `IsRemoteUrlAllowed()` (DevFlags.cpp), which in release builds denies + // everything unless the app config explicitly opts in via + // `security.allowRemoteModules`. Remote module loading is a supported + // release feature behind that opt-in, and an opted-in app needs + // `configureRuntime` / `invalidateModules` / `kickstartPrefetch` to + // operate it. For default-config release apps the members are inert. + v8::Local dev = v8::Object::New(isolate); + + InstallDevFunction(isolate, context, dev, "configureRuntime", ConfigureDevRuntimeCallback); + InstallDevFunction(isolate, context, dev, "invalidateModules", InvalidateModulesCallback); + InstallDevFunction(isolate, context, dev, "kickstartPrefetch", KickstartHmrPrefetchCallback); + InstallDevFunction(isolate, context, dev, "getLoadedModuleUrls", GetLoadedModuleUrlsCallback); + InstallDevFunction(isolate, context, dev, "setDevBootComplete", SetDevBootCompleteCallback); + + // Main-isolate only: terminating workers from inside a worker would let + // a stuck worker take down its peers (see CallbackHandlers.h). + if (!isWorker) { + InstallDevFunction(isolate, context, dev, "terminateAllWorkers", + CallbackHandlers::TerminateAllWorkersCallback); + } + + if (IsDebuggable()) { + // Debug-only diagnostic: expose the HTTP canonical-key function to JS so + // the test harness can pin its identity behavior across cache-busters + // and dev-endpoint query normalization. + InstallDevFunction(isolate, context, dev, "canonicalizeHttpUrlKey", + CanonicalizeHttpUrlKeyCallback); + } + + context->Global() + ->Set(context, ToV8String(isolate, "__NS_DEV__"), dev) + .FromMaybe(false); + MirrorGlobalOnGlobalThis(isolate, context, "__NS_DEV__"); } } // namespace tns diff --git a/test-app/runtime/src/main/cpp/HMRSupport.h b/test-app/runtime/src/main/cpp/HMRSupport.h index f08e7fa09..d769e75e6 100644 --- a/test-app/runtime/src/main/cpp/HMRSupport.h +++ b/test-app/runtime/src/main/cpp/HMRSupport.h @@ -3,23 +3,187 @@ #include #include -#include + +// Forward declare v8 types to keep this header lightweight and avoid +// requiring V8 headers at include sites. +namespace v8 { +class Isolate; +template class Local; +class Object; +class Function; +class Context; +class Value; +} namespace tns { -// import.meta.hot support -v8::Local GetOrCreateHotData(v8::Isolate* isolate, const std::string& key); -void RegisterHotAccept(v8::Isolate* isolate, const std::string& key, v8::Local cb); -void RegisterHotDispose(v8::Isolate* isolate, const std::string& key, v8::Local cb); -std::vector> GetHotAcceptCallbacks(v8::Isolate* isolate, const std::string& key); -std::vector> GetHotDisposeCallbacks(v8::Isolate* isolate, const std::string& key); -void InitializeImportMetaHot(v8::Isolate* isolate, - v8::Local context, - v8::Local importMeta, - const std::string& modulePath); - -// Dev HTTP loader helpers +// HMRSupport: the native half of the NativeScript dev-loader contract. +// +// The runtime deliberately exposes *mechanism* only: +// - the synchronous HTTP text fetch backing the HTTP ESM loader +// (V8's ResolveModuleCallback is synchronous, so the fetch must be +// native), +// - a body prewarm cache + list-mode kickstart so a server-computed +// module closure can be fetched in one parallel wave before V8's +// serial synchronous walk, +// - eviction plumbing (prefetch-cache evict + an eviction-driven +// fetch nonce that defeats any HTTP cache layer between the +// runtime and the dev server), +// - the dev-boot-complete signal that disarms cold-boot-only +// behaviors (host yield pump, kickstart pump-wait). +// +// Everything else — boot orchestration, `import.meta.hot`, hot-callback +// registries, full reload, CSS apply, WebSocket protocol — is HMR +// *policy* and lives in the JS dev client (`@nativescript/vite`). + +// ───────────────────────────────────────────────────────────── +// HTTP loader helpers (used by dev/HMR and general-purpose HTTP module loading) +// +// Normalize an HTTP(S) URL into a stable module registry/cache key. +// - Always strips URL fragments. +// - For NativeScript dev endpoints, drops known cache busters (t/v/import) +// and sorts remaining query params for stability. +// - For non-dev/public URLs, preserves the full query string as part of the +// cache key. +// Module identity IS the (canonical) URL — the dev server serves every +// module under exactly one URL and never varies it for freshness. std::string CanonicalizeHttpUrlKey(const std::string& url); + +// Resolve a relative/root-absolute import specifier against a parent URL +// using plain string manipulation. Only relative (`./`, `../`) and +// root-absolute (`/`) specifiers are resolved here; bare specifiers +// return the empty string, and already-absolute http(s) URLs are +// returned unchanged. Returns the empty string when `parentUrl` is not +// http(s). +std::string ResolveImportSpecifierAgainstUrl(const std::string& specifier, + const std::string& parentUrl); + +// Minimal text fetch for HTTP ESM loader. Returns true on 2xx with non-empty body. +// - out: response body +// - contentType: Content-Type header if present +// - status: HTTP status code +// +// On a fast path, returns from the in-memory kickstart-prewarm cache +// without touching the network (destructive one-shot read). On the slow +// path, performs a synchronous JNI fetch with a bounded retry on +// transient socket-pool failures. bool HttpFetchText(const std::string& url, std::string& out, std::string& contentType, int& status); +// Return the most recent low-level fetch error reason for the calling +// thread, or an empty string if the last fetch succeeded (or no fetch +// has run on this thread yet). +// +// Format is grep-friendly and intended for splicing into the JS-side +// error message that `ModuleInternalCallbacks` throws when +// `HttpFetchText` returns `status=0`: +// +// stage=get-response-code class=java.net.ConnectException msg=... +// +// The slot is thread-local because each isolate has its own JS thread; +// concurrent fetches on different threads cannot clobber each other. +// "Take" semantics — the slot is cleared on read so a stale reason +// can never leak into a later, successful fetch. +std::string TakeLastHttpFetchErrorReason(); + +// Drop all entries in the prewarm cache. Safe to call from any thread. +// Used by Runtime teardown and by HMR cache-poison scenarios where the +// dev server has indicated a graph version bump. +void ClearHttpModulePrefetchCache(); + +// Register a "yield" callback that `HttpFetchText` should invoke around its +// synchronous network turn so the caller can pump its own runloop (e.g. the +// JS-thread runloop so a placeholder UI can repaint during cold-boot). +// +// Default: a no-op. Android's main NativeScript isolate runs JS on the UI +// thread, so there is no separate JS-thread runloop to pump here; a host +// that drives its own loop can install a real pump (e.g. one calling +// ALooper_pollOnce(0)) via this hook. +// +// Pass `nullptr` to disable any yielding (used by hosts that drive their own +// run loop or by tests that want bit-for-bit deterministic fetch timing). +// Safe to call from any thread; reads use acquire/release ordering. +void RegisterHttpFetchYield(void (*callback)()); + +// Drop a specific URL set from the prewarm cache. Safe to call from any +// thread; missing keys are silently ignored. Used by `InvalidateModules` +// so that an HMR eviction also purges any stale HTTP body a previous +// kickstart wave left behind. Without this, the kickstart's cache plus +// `HttpFetchText`'s destructive-read fast path would happily serve V8 a +// stale body from the prior save — visible to the user as a 1-cycle lag +// between save and visual update. +void EvictHttpModulePrefetchCacheUrls(const std::vector& urls); + +// Mark a URL set (canonicalized internally) so that the NEXT network +// fetch of each URL carries a unique `__ns_dev_nonce` query parameter, +// guaranteeing no HTTP cache layer between the runtime and the dev +// server (OS URL caches, proxies, a host-installed HttpResponseCache) +// can satisfy the request. Called by `InvalidateModules` for the +// eviction set; marks are consumed when a fresh body arrives. +// The nonce is transport-only and never affects module identity. +void MarkUrlsForCacheBust(const std::vector& urls); + +// List-mode kickstart prewarm. Fetches ONLY the explicit URL list it +// was given (no body scanning, no graph recursion — the dev server owns +// the module graph and supplies closures: `evictPaths` for HMR, an +// entry-graph crawl for cold boot). Fetches run in parallel (up to +// `maxConcurrent`), each body landing in the prewarm cache that +// `HttpFetchText` reads. Blocks the calling thread until the wave +// drains or `timeoutSeconds` elapses. +// +// By feeding the precomputed list we turn N sequential +// `LoadHttpModuleForUrl` calls (the importer chain during V8's +// ResolveModuleCallback walk) into a single parallel wave that +// completes before V8 starts walking. +// +// Cleared/blocked URLs are filtered up front; partial success is +// reported as success (the V8 walk falls back to per-module +// HttpFetchText for anything we couldn't pre-fill). +// +// `outFetchedCount` (optional) receives the number of distinct URLs +// fetched. `outElapsedMs` (optional) receives wall-clock time. +bool KickstartHmrPrefetchUrlsSync(const std::vector& urls, + int maxConcurrent, + double timeoutSeconds, + size_t* outFetchedCount, + uint64_t* outElapsedMs); + +// Flip the dev-boot-complete signal: sets the JS-visible +// `__NS_HMR_BOOT_COMPLETE__` global and the native atomic that gates the +// cold-boot-only behaviors (kickstart pump-wait yield cadence). Exposed +// to JS as `__NS_DEV__.setDevBootComplete(value?: boolean)`. +void SetDevBootComplete(v8::Isolate* isolate, v8::Local context, + bool value); + +// Clear process-wide dev-loader state (prewarm cache, cache-bust marks, +// boot-complete flag). MUST be called during Runtime teardown before +// isolate disposal — and only for the MAIN isolate (worker teardown must +// not wipe shared state the main isolate still uses). +void CleanupHMRGlobals(); + +// Mirror a globally-installed value onto `globalThis.` so +// `globalThis.` lookups resolve when the runtime installs the +// canonical value on the realm's global object. +void MirrorGlobalOnGlobalThis(v8::Isolate* isolate, v8::Local context, + const char* name); + +// ───────────────────────────────────────────────────────────── +// Dev host namespace installer +// +// Installs the single `__NS_DEV__` namespace object that carries every +// JS-callable dev primitive that any tooling can depend on. +// Idempotent per realm; safe to call from any place that has a fresh +// context + isolate scope. Installed on the realm's global object AND +// mirrored on globalThis. +// +// `__NS_DEV__` members: +// - configureRuntime(config) (import map + volatile patterns) +// - invalidateModules(urls) (registry + cache eviction) +// - kickstartPrefetch(urls, opts?) (parallel HTTP prewarm, list mode) +// - getLoadedModuleUrls() (registry introspection) +// - setDevBootComplete(value?) (boot-complete signal) +// - terminateAllWorkers() (main isolate only; see CallbackHandlers.h) +// - canonicalizeHttpUrlKey(url) (debug builds only; test diagnostic) +void InitializeHmrDevGlobals(v8::Isolate* isolate, v8::Local context, + bool isWorker); + } // namespace tns diff --git a/test-app/runtime/src/main/cpp/MetadataNode.cpp b/test-app/runtime/src/main/cpp/MetadataNode.cpp index 7d3335fb2..2974f4b94 100644 --- a/test-app/runtime/src/main/cpp/MetadataNode.cpp +++ b/test-app/runtime/src/main/cpp/MetadataNode.cpp @@ -1741,8 +1741,6 @@ bool MetadataNode::GetExtendLocation(v8::Isolate* isolate, string& extendLocatio } string srcFileName = ArgConverter::ConvertToString(scriptName); - // trim 'file://' to normalize path to always begin with "/data/" - srcFileName = Util::ReplaceAll(srcFileName, "file://", ""); string fullPathToFile; if (srcFileName == "") { @@ -1754,11 +1752,60 @@ bool MetadataNode::GetExtendLocation(v8::Isolate* isolate, string& extendLocatio // preceding the underscore (_) fullPathToFile = "script"; } else { - string hardcodedPathToSkip = Constants::APP_ROOT_FOLDER_PATH; + // ── Normalize srcFileName down to a path-like string ────────── + // srcFileName is not always `file:///.js`: + // HTTP ESM loading (HMR dev workflow) passes a full URL like + // `http://127.0.0.1:5173/ns/core/...` with no `.js` suffix and + // no app-root prefix, so naive scheme/app-root/`.js` stripping + // can yield an empty `fullPathToFile` and crash downstream on + // an empty token list. + // + // The logic below is shape-aware: + // 1. Strip a leading URL scheme + authority (`file://`, + // `http://host:port`, `https://host:port`). + // 2. Strip a leading `APP_ROOT_FOLDER_PATH` if present. + // 3. Strip the trailing `.js` / `.mjs` extension if present + // (HMR URLs typically omit it). + // 4. Tokenize on `/`, `.`, `-`, ` ` and take the last + // non-empty token, falling back to `"script"` so the + // metadata generator always has a stable class name. + string normalized = srcFileName; + + auto stripPrefix = [](string& s, const string& prefix) { + if (s.size() >= prefix.size() && + s.compare(0, prefix.size(), prefix) == 0) { + s.erase(0, prefix.size()); + } + }; + + stripPrefix(normalized, "file://"); + if (normalized.rfind("http://", 0) == 0 || + normalized.rfind("https://", 0) == 0) { + size_t schemeEnd = normalized.find("://"); + size_t pathStart = normalized.find('/', schemeEnd + 3); + if (pathStart == string::npos) { + normalized.clear(); + } else { + normalized.erase(0, pathStart + 1); + } + } - int startIndex = hardcodedPathToSkip.length(); - int strToTakeLen = (srcFileName.length() - startIndex - 3); // 3 refers to .js at the end of file name - fullPathToFile = srcFileName.substr(startIndex, strToTakeLen); + const string& appRoot = Constants::APP_ROOT_FOLDER_PATH; + if (!appRoot.empty()) { + stripPrefix(normalized, appRoot); + } + + auto endsWith = [](const string& s, const string& suffix) { + return s.size() >= suffix.size() && + s.compare(s.size() - suffix.size(), suffix.size(), suffix) == 0; + }; + if (endsWith(normalized, ".mjs")) { + normalized.resize(normalized.size() - 4); + } else if (endsWith(normalized, ".js")) { + normalized.resize(normalized.size() - 3); + } + + fullPathToFile = normalized; std::replace(fullPathToFile.begin(), fullPathToFile.end(), '/', '_'); std::replace(fullPathToFile.begin(), fullPathToFile.end(), '.', '_'); @@ -1766,10 +1813,21 @@ bool MetadataNode::GetExtendLocation(v8::Isolate* isolate, string& extendLocatio std::replace(fullPathToFile.begin(), fullPathToFile.end(), ' ', '_'); std::vector pathParts; - Util::SplitString(fullPathToFile, "_", pathParts); - std::string lastPathPart = pathParts.back(); + // An unconditional `pathParts.back()` SEGVs when + // `fullPathToFile` is empty. Walk backwards for the last + // non-empty token; if none, use a sentinel. + std::string lastPathPart; + for (auto it = pathParts.rbegin(); it != pathParts.rend(); ++it) { + if (!it->empty()) { + lastPathPart = *it; + break; + } + } + if (lastPathPart.empty()) { + lastPathPart = "script"; + } fullPathToFile = lastPathPart; } diff --git a/test-app/runtime/src/main/cpp/ModuleInternal.cpp b/test-app/runtime/src/main/cpp/ModuleInternal.cpp index bd41f8c2a..dc2172506 100644 --- a/test-app/runtime/src/main/cpp/ModuleInternal.cpp +++ b/test-app/runtime/src/main/cpp/ModuleInternal.cpp @@ -19,6 +19,7 @@ #include "CallbackHandlers.h" #include "ManualInstrumentation.h" #include "Runtime.h" +#include "DevFlags.h" #include #include #include @@ -26,13 +27,15 @@ #include #include #include +#include using namespace v8; using namespace std; using namespace tns; -// Global module registry for ES modules: maps absolute file paths → compiled Module handles -std::unordered_map> g_moduleRegistry; +// Per-isolate ES module registry definition lives in ModuleInternalCallbacks.cpp +// (thread_local + leaky-singleton accessor). Declared via ModuleInternalCallbacks.h +// so every call site below shares the same per-thread map. // Helper function to check if a module name looks like an optional external module bool ModuleInternal::IsLikelyOptionalModule(const std::string& moduleName) { @@ -226,10 +229,121 @@ void ModuleInternal::RequireNativeCallback(const v8::FunctionCallbackInfo context, const string& path) { TNSPERF(); auto isolate = m_isolate; + + // HTTP(S) URL fast path. Android's `require()` only resolves file-system + // paths (it delegates to Java's `Module.resolvePath`), so + // `globalThis.require('http://...')` would fail the Java resolve and leave a + // pending V8 exception that the discarded `require->Call(...)` result hides, + // making the dev session report success without evaluating anything. Detect + // HTTP / ES-module specifiers here and route them through + // `tns::LoadHttpModuleForUrl` (fetch + compile + register), then instantiate + // and evaluate against the same `ResolveModuleCallback` the rest of the ESM + // loader uses, so static imports share the registry the dynamic-import + // callback populates. + if (path.rfind("http://", 0) == 0 || path.rfind("https://", 0) == 0) { + bool logEnabled = tns::IsScriptLoadingLogEnabled(); + if (logEnabled) { + DEBUG_WRITE("[run-module][http-esm][begin] %s", path.c_str()); + } + + std::string errMsg; + auto maybeMod = tns::LoadHttpModuleForUrl(isolate, context, path, &errMsg); + Local module; + if (!maybeMod.ToLocal(&module)) { + if (logEnabled) { + DEBUG_WRITE("[run-module][http-esm][load-fail] %s reason=%s", + path.c_str(), errMsg.c_str()); + } + throw NativeScriptException(string("Cannot load HTTP module ") + path + + (errMsg.empty() ? "" : (": " + errMsg))); + } + + if (module->GetStatus() == Module::kUninstantiated) { + TryCatch tcLink(isolate); + bool linked = module->InstantiateModule(context, &ResolveModuleCallback).FromMaybe(false); + if (!linked) { + if (logEnabled) { + DEBUG_WRITE("[run-module][http-esm][instantiate-fail] %s", path.c_str()); + } + if (tcLink.HasCaught()) { + throw NativeScriptException(tcLink, "Cannot instantiate HTTP module " + path); + } + throw NativeScriptException(string("Cannot instantiate HTTP module ") + path); + } + } + + if (module->GetStatus() != Module::kEvaluated) { + TryCatch tcEval(isolate); + Local evalResult; + if (!module->Evaluate(context).ToLocal(&evalResult)) { + if (logEnabled) { + DEBUG_WRITE("[run-module][http-esm][evaluate-fail] %s", path.c_str()); + } + if (tcEval.HasCaught()) { + throw NativeScriptException(tcEval, "Cannot evaluate HTTP module " + path); + } + throw NativeScriptException(string("Cannot evaluate HTTP module ") + path); + } + + // Drain top-level-await the same way `LoadESModule` does so a + // module returning a pending Promise is fully settled before we + // return — otherwise the caller would advance while evaluation is + // still in flight. + if (!evalResult.IsEmpty() && evalResult->IsPromise()) { + Local promise = evalResult.As(); + const int maxAttempts = 100; + int attempts = 0; + while (attempts < maxAttempts) { + isolate->PerformMicrotaskCheckpoint(); + Promise::PromiseState state = promise->State(); + if (state != Promise::kPending) { + if (state == Promise::kRejected) { + Local reason = promise->Result(); + std::string reasonStr; + if (!reason.IsEmpty()) { + v8::Local reasonV8; + if (reason->ToString(context).ToLocal(&reasonV8)) { + reasonStr = ArgConverter::ConvertToString(reasonV8); + } + } + DEBUG_WRITE("[run-module][http-esm][evaluate-rejected] %s reason=%s", + path.c_str(), + reasonStr.empty() ? "" : reasonStr.c_str()); + isolate->ThrowException(reason); + throw NativeScriptException( + string("HTTP module evaluation promise rejected: ") + path + + (reasonStr.empty() ? "" : (" — " + reasonStr))); + } + break; + } + attempts++; + usleep(100); + } + } + } + + if (logEnabled) { + DEBUG_WRITE("[run-module][http-esm][ok] %s", path.c_str()); + } + return; + } + auto globalObject = context->Global(); auto require = globalObject->Get(context, ArgConverter::ConvertToV8String(isolate, "require")).ToLocalChecked().As(); Local args[] = { ArgConverter::ConvertToV8String(isolate, path) }; - require->Call(context, globalObject, 1, args); + + // Surface JS exceptions thrown by `require` instead of leaving them as + // pending V8 exceptions: a discarded `require->Call(...)` result lets the + // C++ caller see success while the JS side still carries the exception, + // which the next entry-point then mis-attributes. + TryCatch tc(isolate); + Local callResult; + if (!require->Call(context, globalObject, 1, args).ToLocal(&callResult)) { + if (tc.HasCaught()) { + throw NativeScriptException(tc, "require() failed for module " + path); + } + throw NativeScriptException(string("require() failed for module ") + path); + } } void ModuleInternal::LoadWorker(Local context, const string& path) { @@ -602,8 +716,23 @@ Local ModuleInternal::LoadESModule(Isolate* isolate, const std::string& p if (state != Promise::kPending) { if (state == Promise::kRejected) { Local reason = promise->Result(); + // Best-effort extract a human-readable reason so the + // wrapped C++ exception names the actual cause instead + // of just "Module evaluation promise rejected: ". + std::string reasonStr; + if (!reason.IsEmpty()) { + v8::Local reasonV8; + if (reason->ToString(context).ToLocal(&reasonV8)) { + reasonStr = ArgConverter::ConvertToString(reasonV8); + } + } + DEBUG_WRITE("[esm][evaluate-rejected] %s reason=%s", + path.c_str(), + reasonStr.empty() ? "" : reasonStr.c_str()); isolate->ThrowException(reason); - throw NativeScriptException(string("Module evaluation promise rejected: ") + path); + throw NativeScriptException( + string("Module evaluation promise rejected: ") + path + + (reasonStr.empty() ? "" : (" — " + reasonStr))); } break; } diff --git a/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp b/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp index 73d04a557..a5083fb8d 100644 --- a/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp +++ b/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp @@ -1,4 +1,5 @@ #include "ModuleInternal.h" +#include "ModuleInternalCallbacks.h" #include "ArgConverter.h" #include "NativeScriptException.h" #include "NativeScriptAssert.h" @@ -9,7 +10,11 @@ #include #include #include +#include +#include #include +#include +#include #include "HMRSupport.h" #include "DevFlags.h" #include "JEnv.h" @@ -18,8 +23,107 @@ using namespace v8; using namespace std; using namespace tns; -// External global module registry declared in ModuleInternal.cpp -extern std::unordered_map> g_moduleRegistry; +// ──────────────────────────────────────────────────────────────────────────── +// Forward declarations for helpers defined below their first use. Every helper +// called from ResolveModuleCallback / ImportModuleDynamicallyCallback is +// declared here so definition order within the translation unit doesn't matter. +namespace tns { +static inline bool StartsWith(const std::string& s, const char* prefix); +static inline bool EndsWith(const std::string& s, const char* suffix); +static std::string CanonicalizeRegistryKey(const std::string& key); +static std::string NormalizeViteSpecifier(const std::string& specifier); +static std::string LookupImportMap(const std::string& specifier); +static bool HasUrlScheme(const std::string& spec); +static bool IsSyntheticNamespaceKey(const std::string& key); +static bool IsVolatileUrl(const std::string& url); +static v8::MaybeLocal CompileModuleForResolveRegisterOnly( + v8::Isolate* isolate, v8::Local context, + const std::string& source, const std::string& registryKey); +static v8::MaybeLocal ResolveFromVendorRegistry( + v8::Isolate* isolate, v8::Local context, + const std::string& vendorId); +} // namespace tns + +// ──────────────────────────────────────────────────────────────────────────── +// Per-isolate ES module registry. +// +// `g_moduleRegistry` is `thread_local`: each NS isolate (the main JS thread +// plus each Worker thread) gets its own per-thread map. `v8::Global` +// handles are isolate-bound; sharing one map across isolates lets thread A +// fetch a handle thread B created, and V8 will fail the identity check during +// `InstantiateModule` (the dependency module belongs to a different isolate +// than the module it's being linked into). We use the reference + leaky +// singleton pattern so that every call site continues to write +// `g_moduleRegistry[key] = …` without churning ~100+ call sites to use +// accessor functions, AND so that we don't run a destructor on the underlying +// map when the worker thread tears down (which would call +// `v8::Global::Reset()` on handles whose isolate may already be gone). +// +// On each thread's first use of `g_moduleRegistry`, the initializer below +// runs once per thread to bind the reference to that thread's map. +namespace { +using ModuleHandleMap = std::unordered_map>; + +ModuleHandleMap& MakePerIsolateModuleRegistry() { + thread_local auto* p = new ModuleHandleMap(); + return *p; +} +} // namespace + +namespace tns { +thread_local std::unordered_map>& g_moduleRegistry = + MakePerIsolateModuleRegistry(); +} // namespace tns + +// ──────────────────────────────────────────────────────────────────────────── +// Per-process module-resolution state (import map + vendor cache + fallbacks). +// +// These are leaky singletons for the same reason as the HMR registries in +// HMRSupport.cpp: their destructors would call `v8::Global::Reset()` +// during `__cxa_finalize_ranges`, after the owning isolate is already gone, +// crashing the process on app exit. Heap-allocate + leak (`new` once, never +// `delete`) so the OS reclaims memory on process exit and we never run a +// destructor that touches V8. +// +// Vendor cache is `thread_local` because vendor SyntheticModules are +// isolate-bound. Reusing one across isolates breaks the linker's +// export-table check. The other registries are process-wide configuration +// (no v8::Global) and intentionally shared across isolates. + +namespace tns { + +static std::mutex g_importMapMutex; +static auto* _g_importMap = new std::unordered_map(); +static auto& g_importMap = *_g_importMap; + +static std::mutex g_volatilePatternsMutex; +static auto* _g_volatilePatterns = new std::vector(); +static auto& g_volatilePatterns = *_g_volatilePatterns; + +namespace { +using ModuleHandleMap = std::unordered_map>; + +ModuleHandleMap& MakePerIsolateVendorModuleCache() { + thread_local auto* p = new ModuleHandleMap(); + return *p; +} +} // namespace + +static thread_local std::unordered_map>& g_vendorModuleCache = + MakePerIsolateVendorModuleCache(); + +// Set of canonical keys currently being resolved (loop-breaker for cyclic +// HTTP imports). thread_local because dynamic-import waits are isolate-bound. +namespace { +std::unordered_set& MakePerIsolateInFlightSet() { + thread_local auto* p = new std::unordered_set(); + return *p; +} +} // namespace +static thread_local std::unordered_set& g_modulesInFlight = + MakePerIsolateInFlightSet(); + +} // namespace tns // Forward declaration used by logging helper std::string GetApplicationPath(); @@ -55,7 +159,7 @@ static void LogHttpCompileDiagnostics(v8::Isolate* isolate, String::Utf8Value l8(isolate, maybeLine.ToLocalChecked()); if (*l8) srcLineStr = *l8; } - // Heuristics similar to iOS for quick triage + // Classify the failure for quick triage. if (msgStr.find("Unexpected identifier") != std::string::npos || msgStr.find("Unexpected token") != std::string::npos) { if (msgStr.find("export") != std::string::npos && @@ -124,70 +228,6 @@ static std::string NormalizeDotSegments(const std::string& path) { return norm; } -// Helper: resolve relative or root-absolute spec against an HTTP(S) referrer URL. -// Returns empty string if resolution is not possible. -static std::string ResolveHttpRelative(const std::string& referrerUrl, const std::string& spec) { - if (referrerUrl.empty()) { - return std::string(); - } - auto startsWith = [](const std::string& s, const char* pre) -> bool { - size_t n = strlen(pre); - return s.size() >= n && s.compare(0, n, pre) == 0; - }; - if (!(startsWith(referrerUrl, "http://") || startsWith(referrerUrl, "https://"))) { - return std::string(); - } - // Normalize referrer: drop fragment and query - std::string base = referrerUrl; - size_t hashPos = base.find('#'); - if (hashPos != std::string::npos) base = base.substr(0, hashPos); - size_t qPos = base.find('?'); - if (qPos != std::string::npos) base = base.substr(0, qPos); - - // Extract origin and path - size_t schemePos = base.find("://"); - if (schemePos == std::string::npos) { - return std::string(); - } - size_t pathStart = base.find('/', schemePos + 3); - std::string origin = (pathStart == std::string::npos) ? base : base.substr(0, pathStart); - std::string path = (pathStart == std::string::npos) ? std::string("/") : base.substr(pathStart); - - // Separate query/fragment from spec - std::string specPath = spec; - std::string specSuffix; - size_t specQ = specPath.find('?'); - size_t specH = specPath.find('#'); - size_t cut = std::string::npos; - if (specQ != std::string::npos && specH != std::string::npos) { - cut = std::min(specQ, specH); - } else if (specQ != std::string::npos) { - cut = specQ; - } else if (specH != std::string::npos) { - cut = specH; - } - if (cut != std::string::npos) { - specSuffix = specPath.substr(cut); - specPath = specPath.substr(0, cut); - } - - // Build new path - std::string newPath; - if (!specPath.empty() && specPath[0] == '/') { - // Root-absolute relative to origin - newPath = specPath; - } else { - // Relative to directory of referrer path - size_t lastSlash = path.find_last_of('/'); - std::string baseDir = (lastSlash == std::string::npos) ? std::string("/") : path.substr(0, lastSlash + 1); - newPath = baseDir + specPath; - } - - // Normalize "." and ".." segments - std::string normPath = NormalizeDotSegments(newPath); - return origin + normPath + specSuffix; -} - // Helper: resolve a relative "./" or "../" specifier against a file:// referrer // URL, returning an absolute file:// URL. Returns empty if not applicable. static std::string ResolveFileRelative(const std::string& referrerUrl, const std::string& spec) { @@ -210,6 +250,632 @@ static std::string ResolveFileRelative(const std::string& referrerUrl, const std return filePrefix + NormalizeDotSegments(baseDir + spec); } +// Resolution of relative / root-absolute import specifiers against an http(s) +// referrer lives in `HMRSupport.cpp` (`ResolveImportSpecifierAgainstUrl`); call +// sites below invoke `tns::ResolveImportSpecifierAgainstUrl(spec, referrer)`. + +// ──────────────────────────────────────────────────────────────────────────── +// Module-resolution helpers (ESM resolver hardening). Helpers referenced before +// their definition are forward-declared at the top of this file. +namespace tns { + +static inline bool StartsWith(const std::string& s, const char* prefix) { + size_t n = strlen(prefix); + return s.size() >= n && s.compare(0, n, prefix) == 0; +} + +static inline bool EndsWith(const std::string& s, const char* suffix) { + size_t n = strlen(suffix); + return s.size() >= n && s.compare(s.size() - n, n, suffix) == 0; +} + +// Synthetic-namespace keys (ns-vendor://, optional:, node:, blob:) are NOT +// filesystem paths. Treat them as opaque identifiers — never collapse, +// percent-decode, or path-normalize them — so they keep their exact registry +// identity through invalidation and reload. +static bool IsSyntheticNamespaceKey(const std::string& key) { + return StartsWith(key, "ns-vendor://") || StartsWith(key, "optional:") || + StartsWith(key, "node:") || StartsWith(key, "blob:"); +} + +// Returns true for any specifier with a leading URL scheme of the form +// `:` where the scheme contains no `/` (filesystem paths and bare +// specifiers stay false). Used by InitializeImportMetaObject to decide +// whether `import.meta.url` should preserve the specifier verbatim vs. +// wrap it in `file://`. +static bool HasUrlScheme(const std::string& spec) { + size_t schemePos = spec.find(':'); + if (schemePos == std::string::npos || schemePos == 0) return false; + size_t slashPos = spec.find('/'); + if (slashPos != std::string::npos && slashPos < schemePos) return false; + // Scheme chars must be alphanumeric / `+` / `-` / `.` per RFC 3986. + for (size_t i = 0; i < schemePos; ++i) { + char c = spec[i]; + if (!std::isalnum(static_cast(c)) && c != '+' && c != '-' && c != '.') { + return false; + } + } + return true; +} + +// Matches `url` against the `volatilePatterns` configured by Vite via +// `__NS_DEV__.configureRuntime({ volatilePatterns: [...] })` (substring match). +// Android's HTTP loader does not consult this: it enforces volatility +// structurally — a consume-once prefetch read plus eviction on HMR +// invalidation (see HttpFetchText in HMRSupport.cpp). It is available for a +// read-gated cache policy, which the iOS loader uses. +static bool IsVolatileUrl(const std::string& url) { + std::lock_guard lock(g_volatilePatternsMutex); + for (const auto& pat : g_volatilePatterns) { + if (url.find(pat) != std::string::npos) return true; + } + return false; +} + +// Canonicalize a raw module key into a stable registry key. +// +// Rules: +// - HTTP/HTTPS keys go through `CanonicalizeHttpUrlKey` (drop fragment, +// normalize bridge endpoints, sort query, strip `?import`). +// - `file://http(s)://...` keys unwrap the outer scheme, then HTTP-canon. +// - `blob:` keys are preserved verbatim (they're opaque random IDs). +// - `ns-vendor://`, `optional:`, `node:` (and any other custom scheme +// where the scheme appears before the first slash) are preserved +// verbatim — these are NOT filesystem paths. +// - Plain filesystem paths fall through unchanged (the Android resolver +// resolves them lazily via candidate scans). +static std::string CanonicalizeRegistryKey(const std::string& key) { + if (key.empty()) { + return key; + } + + if (StartsWith(key, "http://") || StartsWith(key, "https://") || + StartsWith(key, "file://http://") || StartsWith(key, "file://https://")) { + return CanonicalizeHttpUrlKey(key); + } + if (StartsWith(key, "blob:")) { + return key; + } + if (IsSyntheticNamespaceKey(key)) { + return key; + } + // Any other custom scheme, or a plain filesystem path: preserve verbatim. + // The Android resolver resolves filesystem paths lazily via candidate scans. + return key; +} + +// Compile a module body for "register only" mode used by the HTTP loader +// and vendor wrapper. Caller owns instantiation + evaluation; we just +// produce the v8::Module and seat it in `g_moduleRegistry` under +// `registryKey` so resolver callbacks can find it during link. +static v8::MaybeLocal CompileModuleForResolveRegisterOnly( + v8::Isolate* isolate, v8::Local context, + const std::string& source, const std::string& registryKey) { + v8::EscapableHandleScope scope(isolate); + v8::Local sourceText = ArgConverter::ConvertToV8String(isolate, source); + v8::Local urlString = ArgConverter::ConvertToV8String(isolate, registryKey); + v8::ScriptOrigin origin(isolate, urlString, 0, 0, false, -1, v8::Local(), false, false, + true /* is_module */); + v8::ScriptCompiler::Source src(sourceText, origin); + + v8::Local mod; + { + v8::TryCatch tc(isolate); + if (!v8::ScriptCompiler::CompileModule(isolate, &src).ToLocal(&mod)) { + if (IsScriptLoadingLogEnabled()) { + v8::Local ex = tc.Exception(); + v8::String::Utf8Value m(isolate, ex); + DEBUG_WRITE("[http-esm][compile-register][fail] key=%s msg=%s", + registryKey.c_str(), *m ? *m : "(unknown)"); + } + return v8::MaybeLocal(); + } + } + g_moduleRegistry[registryKey].Reset(isolate, mod); + return scope.Escape(mod); +} + +// Normalize a Vite-rewritten specifier into the canonical import-map key. +// Handles two common Vite dev-server rewrite patterns: +// 1. Prebundled deps: "/node_modules/.vite/deps/solid-js.js?v=abc" → "solid-js" +// "/node_modules/.vite/deps/@tanstack_solid-router.js" → "@tanstack/solid-router" +// 2. Explicit node_modules paths: +// "/node_modules/@angular/core/fesm2022/core.mjs" → "@angular/core/fesm2022/core.mjs" +// "/node_modules/tslib/tslib.es6.mjs" → "tslib" +// +// For explicit node_modules paths we preserve non-main-entry subpaths so the +// import map's trailing-slash HTTP prefixes can keep complex package build +// outputs on HTTP. Only bare package roots and simple root-level main entries +// collapse back to the package id for vendor/exact import-map resolution. +static std::string NormalizeViteSpecifier(const std::string& specifier) { + // Pattern 1: Vite prebundled deps — /node_modules/.vite/deps/.js + { + const std::string viteDepsPrefix = "/node_modules/.vite/deps/"; + const std::string viteDepsPrefix2 = "node_modules/.vite/deps/"; + std::string prefix; + if (specifier.compare(0, viteDepsPrefix.size(), viteDepsPrefix) == 0) + prefix = viteDepsPrefix; + else if (specifier.compare(0, viteDepsPrefix2.size(), viteDepsPrefix2) == 0) + prefix = viteDepsPrefix2; + + if (!prefix.empty()) { + std::string id = specifier.substr(prefix.size()); + auto qpos = id.find('?'); + if (qpos != std::string::npos) id = id.substr(0, qpos); + auto dotpos = id.rfind('.'); + if (dotpos != std::string::npos) id = id.substr(0, dotpos); + if (!id.empty() && id[0] == '@') { + auto upos = id.find('_'); + if (upos != std::string::npos) { + id = id.substr(0, upos) + "/" + id.substr(upos + 1); + auto upos2 = id.find('_', upos + 1); + if (upos2 != std::string::npos) { + id = id.substr(0, upos2); + } + } + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map][normalize] vite-deps: %s -> %s", specifier.c_str(), id.c_str()); + } + return id; + } + } + + // Pattern 2: Resolved node_modules path — /node_modules//... + { + const std::string nmPrefix = "/node_modules/"; + const std::string nmPrefix2 = "node_modules/"; + std::string sub; + if (specifier.compare(0, nmPrefix.size(), nmPrefix) == 0) + sub = specifier.substr(nmPrefix.size()); + else if (specifier.compare(0, nmPrefix2.size(), nmPrefix2) == 0) + sub = specifier.substr(nmPrefix2.size()); + + if (!sub.empty() && sub[0] != '.') { + if (sub.compare(0, 6, ".vite/") == 0) return ""; + + std::string subNoQuery = sub; + std::string querySuffix; + auto subQueryPos = sub.find('?'); + if (subQueryPos != std::string::npos) { + subNoQuery = sub.substr(0, subQueryPos); + querySuffix = sub.substr(subQueryPos); + } + + std::string pkgName; + if (subNoQuery[0] == '@') { + auto slash1 = subNoQuery.find('/'); + if (slash1 != std::string::npos) { + auto slash2 = subNoQuery.find('/', slash1 + 1); + pkgName = (slash2 != std::string::npos) ? subNoQuery.substr(0, slash2) : subNoQuery; + } + } else { + auto slash = subNoQuery.find('/'); + pkgName = (slash != std::string::npos) ? subNoQuery.substr(0, slash) : subNoQuery; + } + if (!pkgName.empty()) { + std::string normalized = pkgName; + std::string remainder; + if (subNoQuery.size() > pkgName.size()) { + remainder = subNoQuery.substr(pkgName.size()); + if (!remainder.empty() && remainder[0] == '/') { + remainder.erase(0, 1); + } + } + + if (!remainder.empty()) { + bool preserveSubpath = remainder.find('/') != std::string::npos; + + if (!preserveSubpath) { + const std::string pkgBaseName = pkgName.substr(pkgName.find_last_of('/') + 1); + std::string withoutExt = remainder; + auto dot = withoutExt.rfind('.'); + if (dot != std::string::npos) { + withoutExt = withoutExt.substr(0, dot); + } + std::string withoutPlatform = withoutExt; + for (const char* suffix : {".ios", ".android", ".visionos"}) { + if (EndsWith(withoutPlatform, suffix)) { + withoutPlatform = withoutPlatform.substr(0, withoutPlatform.size() - strlen(suffix)); + break; + } + } + const bool isRootLevelMainEntry = withoutPlatform == "index" || + withoutPlatform == pkgBaseName || + withoutPlatform.rfind(pkgBaseName + ".", 0) == 0; + preserveSubpath = !isRootLevelMainEntry; + } + + if (preserveSubpath) { + normalized = pkgName + "/" + remainder + querySuffix; + } + } + + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map][normalize] node_modules: %s -> %s", specifier.c_str(), normalized.c_str()); + } + return normalized; + } + } + } + + return ""; +} + +// Look up a specifier in the import map. Supports both exact matches and +// prefix matches (trailing-slash entries like "solid-js/" that map subpaths). +// Returns the mapped URL or empty string if no match. +static std::string LookupImportMap(const std::string& specifier) { + std::lock_guard lock(g_importMapMutex); + auto it = g_importMap.find(specifier); + if (it != g_importMap.end()) { + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map] exact: %s -> %s", specifier.c_str(), it->second.c_str()); + } + return it->second; + } + std::string bestKey; + std::string bestValue; + for (const auto& kv : g_importMap) { + const std::string& key = kv.first; + if (key.empty() || key.back() != '/') continue; + if (specifier.size() > key.size() && specifier.compare(0, key.size(), key) == 0) { + if (key.size() > bestKey.size()) { + bestKey = key; + bestValue = kv.second; + } + } + } + if (!bestKey.empty()) { + std::string remainder = specifier.substr(bestKey.size()); + std::string resolved = bestValue + remainder; + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map] prefix: %s -> %s (via %s)", specifier.c_str(), resolved.c_str(), bestKey.c_str()); + } + return resolved; + } + return ""; +} + +// Escape `s` as a single-quoted JS string literal. Returns the literal +// including the surrounding quotes so call sites can splice it directly +// into a generated source string (e.g. `"foo(" + JsStringLiteral(id) + ")"`). +// Handles backslash, single quote, the JS line terminators (\n, \r, +// U+2028, U+2029), and other ASCII control characters via `\xNN`. +static std::string JsStringLiteral(const std::string& s) { + std::string out; + out.reserve(s.size() + 2); + out.push_back('\''); + for (size_t i = 0; i < s.size(); ) { + unsigned char c = static_cast(s[i]); + if (c == '\\') { out += "\\\\"; ++i; continue; } + if (c == '\'') { out += "\\'"; ++i; continue; } + if (c == '\n') { out += "\\n"; ++i; continue; } + if (c == '\r') { out += "\\r"; ++i; continue; } + if (c == 0xE2 && i + 2 < s.size() && + static_cast(s[i + 1]) == 0x80 && + (static_cast(s[i + 2]) == 0xA8 || + static_cast(s[i + 2]) == 0xA9)) { + out += (static_cast(s[i + 2]) == 0xA8) ? "\\u2028" : "\\u2029"; + i += 3; + continue; + } + if (c < 0x20) { + char buf[7]; + std::snprintf(buf, sizeof(buf), "\\x%02X", c); + out += buf; + ++i; + continue; + } + out.push_back(static_cast(c)); + ++i; + } + out.push_back('\''); + return out; +} + +// Helper: returns true if `name` is a valid JS identifier that can appear in +// `export const = ...` without quoting. Conservative check — rejects +// anything that could cause a parse error in the generated ESM wrapper. +static bool IsValidJSIdentifier(const std::string& name) { + if (name.empty()) return false; + char first = name[0]; + if (!((first >= 'a' && first <= 'z') || (first >= 'A' && first <= 'Z') || + first == '_' || first == '$')) + return false; + for (size_t i = 1; i < name.size(); i++) { + char c = name[i]; + if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || + (c >= '0' && c <= '9') || c == '_' || c == '$')) + return false; + } + return true; +} + +// Create an ESM wrapper that re-exports all named exports from the vendor +// registry. The vendor bootstrap (JS side) populates +// globalThis.__nsVendorRegistry with pre-bundled module namespace objects +// (via `import * as`). This function enumerates the actual property names +// of the vendor module and generates explicit `export const X = __mod['X'];` +// statements so V8's ESM resolution finds every named export. +static v8::MaybeLocal ResolveFromVendorRegistry(v8::Isolate* isolate, + v8::Local context, + const std::string& vendorId) { + auto cached = g_vendorModuleCache.find(vendorId); + if (cached != g_vendorModuleCache.end()) { + v8::Local mod = cached->second.Get(isolate); + if (!mod.IsEmpty() && mod->GetStatus() != v8::Module::kErrored) { + return mod; + } + cached->second.Reset(); + g_vendorModuleCache.erase(cached); + } + + std::vector exportNames; + + v8::TryCatch tc(isolate); + do { + v8::Local global = context->Global(); + + v8::Local regVal; + if (!global->Get(context, ArgConverter::ConvertToV8String(isolate, "__nsVendorRegistry")).ToLocal(®Val) || + regVal->IsNullOrUndefined()) { + break; + } + v8::Local registry = regVal.As(); + + v8::Local getFnVal; + if (!registry->Get(context, ArgConverter::ConvertToV8String(isolate, "get")).ToLocal(&getFnVal) || + !getFnVal->IsFunction()) { + break; + } + v8::Local getArgs[] = { ArgConverter::ConvertToV8String(isolate, vendorId) }; + v8::Local modVal; + if (!getFnVal.As()->Call(context, registry, 1, getArgs).ToLocal(&modVal) || + modVal->IsNullOrUndefined()) { + break; + } + + v8::Local modObj = modVal.As(); + v8::Local keys; + if (!modObj->GetOwnPropertyNames(context).ToLocal(&keys)) { + break; + } + + for (uint32_t i = 0; i < keys->Length(); i++) { + v8::Local key; + if (!keys->Get(context, i).ToLocal(&key) || !key->IsString()) continue; + v8::String::Utf8Value keyUtf8(isolate, key); + if (!*keyUtf8) continue; + std::string name(*keyUtf8); + if (name != "default" && IsValidJSIdentifier(name)) { + exportNames.push_back(name); + } + } + } while (false); + + if (tc.HasCaught()) { + tc.Reset(); + } + + std::string moduleKey = "ns-vendor://" + vendorId; + // Two failure modes are distinguished so the runtime error names the + // class of problem: registry not yet populated (wrapper evaluated + // before `installVendorBootstrap()` ran) vs. specifier absent from a + // populated registry (vendor bundle does not ship this entry). + // `vendorId` is escaped through `JsStringLiteral` so any character is + // safe to embed inside the generated JS source. + const std::string idLiteral = JsStringLiteral(vendorId); + std::string src = + "const __reg = globalThis.__nsVendorRegistry;\n" + "if (!__reg || __reg.size === 0) {\n" + " throw new Error('ns-vendor wrapper ' + " + idLiteral + + " + ' evaluated before __nsVendorRegistry was populated');\n" + "}\n" + "const __mod = __reg.get(" + idLiteral + ");\n" + "if (!__mod) {\n" + " throw new Error('ns-vendor specifier ' + " + idLiteral + + " + ' not in __nsVendorRegistry (' + __reg.size + ' entries)');\n" + "}\n" + "export default __mod.default !== undefined ? __mod.default : __mod;\n"; + + for (const auto& name : exportNames) { + src += "export const " + name + " = __mod[" + JsStringLiteral(name) + "];\n"; + } + + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map][vendor] generating wrapper for ns-vendor://%s with %lu named exports", + vendorId.c_str(), (unsigned long)exportNames.size()); + } + + v8::MaybeLocal m = CompileModuleForResolveRegisterOnly(isolate, context, src, moduleKey); + if (!m.IsEmpty()) { + v8::Local mod; + if (m.ToLocal(&mod)) { + g_vendorModuleCache[vendorId].Reset(isolate, mod); + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map][vendor] resolved ns-vendor://%s", vendorId.c_str()); + } + } + } + return m; +} + +// Public: replace the import map with parsed (key → URL) entries. The dev server's +// import-map JSON is parsed by V8 in the caller (HMRSupport.cpp); only the flat +// `imports` table reaches here. +void SetImportMapEntries(const std::vector>& entries) { + std::lock_guard lock(g_importMapMutex); + g_importMap.clear(); + for (const auto& kv : entries) { + if (!kv.first.empty()) { + g_importMap[kv.first] = kv.second; + } + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map] loaded %lu entries", (unsigned long)g_importMap.size()); + } +} + +void SetVolatilePatterns(const std::vector& patterns) { + std::lock_guard lock(g_volatilePatternsMutex); + g_volatilePatterns = patterns; + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map] volatile patterns: %lu", (unsigned long)g_volatilePatterns.size()); + } +} + +void CleanupImportMapGlobals() { + { + std::lock_guard lock(g_importMapMutex); + g_importMap.clear(); + } + { + std::lock_guard lock(g_volatilePatternsMutex); + g_volatilePatterns.clear(); + } + for (auto& kv : g_vendorModuleCache) { kv.second.Reset(); } + g_vendorModuleCache.clear(); + g_modulesInFlight.clear(); +} + +std::vector GetLoadedModuleUrls() { + std::vector urls; + urls.reserve(g_moduleRegistry.size()); + for (const auto& kv : g_moduleRegistry) { + if (!kv.first.empty()) urls.push_back(kv.first); + } + return urls; +} + +void RemoveModuleFromRegistry(const std::string& canonicalKey) { + const std::string registryKey = CanonicalizeRegistryKey(canonicalKey); + // Defensive: never wipe a sentinel key. + if (registryKey == "@" || + registryKey.find("__invalid_at__.mjs") != std::string::npos) { + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[resolver][guard] ignore remove for sentinel %s", registryKey.c_str()); + } + return; + } + + auto it = g_moduleRegistry.find(registryKey); + if (it != g_moduleRegistry.end()) { + bool isHttpKey = StartsWith(registryKey, "http://") || StartsWith(registryKey, "https://"); + if (IsScriptLoadingLogEnabled() && !isHttpKey) { + DEBUG_WRITE("[resolver] removing stale module %s", registryKey.c_str()); + } + it->second.Reset(); + g_moduleRegistry.erase(it); + } +} + +size_t InvalidateModules(const std::vector& keys) { + size_t removed = 0; + std::vector urlsToEvict; + urlsToEvict.reserve(keys.size()); + for (const auto& raw : keys) { + if (raw.empty()) continue; + const std::string registryKey = CanonicalizeRegistryKey(raw); + auto it = g_moduleRegistry.find(registryKey); + if (it != g_moduleRegistry.end()) { + it->second.Reset(); + g_moduleRegistry.erase(it); + ++removed; + } + if (StartsWith(registryKey, "http://") || StartsWith(registryKey, "https://")) { + urlsToEvict.push_back(registryKey); + } + } + if (!urlsToEvict.empty()) { + // Second layer: drop stale HTTP bodies from the kickstart prewarm + // cache for every URL we just invalidated. Without this, the next + // `HttpFetchText` for an evicted URL would happily return a stale + // body a previous kickstart wave left in the cache, and V8 would + // compile that stale source — producing the "1 cycle behind" lag + // for edits with many transitive importers. + EvictHttpModulePrefetchCacheUrls(urlsToEvict); + + // Third layer: any HTTP cache between the runtime and the dev server + // (OS URL cache, proxy, host-installed HttpResponseCache) is outside + // the runtime's direct control. Mark every invalidated key so the + // NEXT network fetch of that URL carries a unique `__ns_dev_nonce` + // query param — the cache sees a URL it has never stored and must go + // to origin. The nonce is transport-only; module identity stays the + // canonical URL. + MarkUrlsForCacheBust(urlsToEvict); + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[resolver][invalidate] requested=%lu removed=%lu", + (unsigned long)keys.size(), (unsigned long)removed); + } + return removed; +} + +v8::MaybeLocal LoadHttpModuleForUrl(v8::Isolate* isolate, + v8::Local context, + const std::string& url, + std::string* errorMessage) { + if (url.empty()) { + if (errorMessage) *errorMessage = "[http-esm][load] empty URL"; + return v8::MaybeLocal(); + } + const std::string registryKey = CanonicalizeRegistryKey(url); + + auto it = g_moduleRegistry.find(registryKey); + if (it != g_moduleRegistry.end()) { + return it->second.Get(isolate); + } + + // Loop-breaker: if we're already fetching this URL inside this isolate + // (cyclic HTTP import), don't recurse. The caller's outer fetch will + // populate the registry; on the second pass our cache lookup above will + // succeed. + if (g_modulesInFlight.count(registryKey) > 0) { + if (errorMessage) *errorMessage = "[http-esm][load] cyclic-inflight " + registryKey; + return v8::MaybeLocal(); + } + g_modulesInFlight.insert(registryKey); + + std::string body; + std::string contentType; + int status = 0; + bool ok = HttpFetchText(url, body, contentType, status) && !body.empty(); + g_modulesInFlight.erase(registryKey); + + if (!ok) { + if (errorMessage) { + *errorMessage = std::string("[http-esm][load] fetch-fail ") + url + + " status=" + std::to_string(status); + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][load][fetch-fail] request=%s key=%s status=%d", + url.c_str(), registryKey.c_str(), status); + } + return v8::MaybeLocal(); + } + + v8::MaybeLocal loaded = + CompileModuleForResolveRegisterOnly(isolate, context, body, registryKey); + if (loaded.IsEmpty()) { + if (errorMessage) { + *errorMessage = std::string("[http-esm][load] compile-fail ") + url; + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][load][compile-fail] request=%s key=%s bytes=%zu", + url.c_str(), registryKey.c_str(), body.size()); + } + return v8::MaybeLocal(); + } + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[http-esm][load][ok] request=%s key=%s type=%s bytes=%zu", + url.c_str(), registryKey.c_str(), contentType.c_str(), body.size()); + } + return loaded; +} + +} // namespace tns + // Import meta callback to support import.meta.url and import.meta.dirname void InitializeImportMetaObject(Local context, Local module, Local meta) { Isolate* isolate = context->GetIsolate(); @@ -241,11 +907,21 @@ void InitializeImportMetaObject(Local context, Local module, Lo DEBUG_WRITE("InitializeImportMetaObject: Registry size: %zu", g_moduleRegistry.size()); } - // Convert to URL for import.meta.url; keep http(s) untouched, file paths with file:// + // Convert to URL for import.meta.url: + // - http(s) keys keep the URL verbatim + // - Synthetic-namespace keys (`node:`, `blob:`, `ns-vendor://`, + // `optional:`) MUST preserve their identity — wrapping them in + // `file://` would make `import.meta.url` decode them as filesystem + // paths in user code, which is wrong (they're not files). + // - Any other URL-schemed specifier (`:` before any `/`) + // also passes through verbatim (`tns::HasUrlScheme` check). + // - File-system paths get the standard `file://` wrap. std::string moduleUrl; if (!modulePath.empty()) { if (modulePath.rfind("http://", 0) == 0 || modulePath.rfind("https://", 0) == 0) { moduleUrl = modulePath; + } else if (tns::HasUrlScheme(modulePath)) { + moduleUrl = modulePath; } else { moduleUrl = "file://" + modulePath; } @@ -263,15 +939,22 @@ void InitializeImportMetaObject(Local context, Local module, Lo // Set import.meta.url property meta->CreateDataProperty(context, ArgConverter::ConvertToV8String(isolate, "url"), url).Check(); - // Add import.meta.dirname support (extract directory) + // Add import.meta.dirname support (extract directory). + // + // For synthetic-namespace keys (node:, blob:, ns-vendor://, optional:) + // the concept of a "directory" doesn't apply — the module isn't a file + // on disk. dirname falls back to the full URL so user code that joins + // paths still produces a recognizable key (and `node:url` / + // `blob:abc-def` etc. don't accidentally read as filesystem prefixes). std::string dirname; if (!modulePath.empty()) { if (modulePath.rfind("http://", 0) == 0 || modulePath.rfind("https://", 0) == 0) { - // For URLs, compute dirname by trimming after last '/' size_t q = modulePath.find('?'); std::string noQuery = (q == std::string::npos) ? modulePath : modulePath.substr(0, q); size_t lastSlash = noQuery.find_last_of('/'); dirname = (lastSlash == std::string::npos) ? modulePath : noQuery.substr(0, lastSlash); + } else if (tns::HasUrlScheme(modulePath)) { + dirname = modulePath; // synthetic — no real directory } else { size_t lastSlash = modulePath.find_last_of("/\\"); if (lastSlash != std::string::npos) { @@ -289,8 +972,12 @@ void InitializeImportMetaObject(Local context, Local module, Lo // Set import.meta.dirname property meta->CreateDataProperty(context, ArgConverter::ConvertToV8String(isolate, "dirname"), dirnameStr).Check(); - // Attach import.meta.hot for HMR - tns::InitializeImportMetaHot(isolate, context, meta, modulePath); + // NOTE: the runtime deliberately does NOT attach `import.meta.hot`. + // Hot contexts are HMR *policy* and are injected by the JS dev client + // (`@nativescript/vite` rewrites served module source to + // `import.meta.hot = __NS_HOT_REGISTRY__.createHotContext(id)`). + // Modules loaded outside a dev session see no hot object at all — + // standard HMR code always guards with `if (import.meta.hot)`. } // Helper function to check if a file exists and is a regular file @@ -344,13 +1031,115 @@ v8::MaybeLocal ResolveModuleCallback(v8::Local context, DEBUG_WRITE("ResolveModuleCallback: Resolving '%s'", spec.c_str()); } - // Normalize malformed http:/ and https:/ prefixes + // ── ESM resolver hardening ───────────────────────────────────────────── + // Heal common bundler-rewrite anomalies BEFORE any registry lookup so + // identity-mismatched keys never enter `g_moduleRegistry`. + // + // 1. A lone "@" (bundler dropped the package id) → rewrite to a + // sentinel string that downstream resolvers ignore. + // 2. "@/" (root-absolute alias the dev server didn't expand) + // → strip the prefix so the path lookup operates on "/". + // 3. Malformed `http:/` (one slash, often from Vite stripping + // the second slash through string ops) → re-insert. + if (spec == "@") { + // Sentinel — never matches a real module. Synthesize a clear + // failure so the user's error trace anchors to the bad import. + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[resolver][guard] bare-@ specifier; returning empty"); + } + return v8::MaybeLocal(); + } + if (spec.size() >= 2 && spec[0] == '@' && spec[1] == '/') { + std::string rewritten = spec.substr(1); // "@/foo" → "/foo" + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[resolver][guard] rewrite @/ -> / for spec=%s -> %s", + spec.c_str(), rewritten.c_str()); + } + spec = rewritten; + } if (spec.rfind("http:/", 0) == 0 && spec.rfind("http://", 0) != 0) { spec.insert(5, "/"); } else if (spec.rfind("https:/", 0) == 0 && spec.rfind("https://", 0) != 0) { spec.insert(6, "/"); } + // ── Import-map lookup (bare specifiers only) ────────────────────────── + // Bare specifiers (no `.`, `/`, scheme) are mapped through the dev + // server's import map before anything else. The map can resolve to: + // - `ns-vendor://` → SyntheticModule via vendor registry + // - `http(s)://...` → HTTP loader path below + // - `` → falls through to filesystem candidate scan + // Vite-rewritten `/node_modules/...` specifiers are normalized via + // `NormalizeViteSpecifier` first so the import-map key matches. + if (!spec.empty() && spec[0] != '.' && spec[0] != '/' && + spec.find("://") == std::string::npos && !tns::HasUrlScheme(spec)) { + std::string lookupKey = spec; + std::string vNorm = tns::NormalizeViteSpecifier(spec); + if (!vNorm.empty()) lookupKey = vNorm; + std::string mapped = tns::LookupImportMap(lookupKey); + if (!mapped.empty()) { + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map][hit] %s -> %s", spec.c_str(), mapped.c_str()); + } + spec = mapped; + } + } else if (!spec.empty() && spec[0] == '/' && + spec.find("://") == std::string::npos && + spec.find("/node_modules/") != std::string::npos) { + // Root-absolute node_modules path — try import-map after normalization. + std::string vNorm = tns::NormalizeViteSpecifier(spec); + if (!vNorm.empty()) { + std::string mapped = tns::LookupImportMap(vNorm); + if (!mapped.empty()) { + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[import-map][hit-root-abs] %s -> %s (via %s)", + spec.c_str(), mapped.c_str(), vNorm.c_str()); + } + spec = mapped; + } + } + } + + // ── Synthetic-namespace identity preservation ───────────────────────── + // ns-vendor:// / optional: / node: / blob: are NOT filesystem paths. + // Resolve them via dedicated paths instead of falling through to the + // candidate-scan logic below (which would try to stat them as files). + if (spec.rfind("ns-vendor://", 0) == 0) { + std::string vendorId = spec.substr(strlen("ns-vendor://")); + v8::Local vendorMod; + if (tns::ResolveFromVendorRegistry(isolate, context, vendorId).ToLocal(&vendorMod)) { + return v8::MaybeLocal(vendorMod); + } + // Vendor registry doesn't have it — throw a clear "not found" so the + // import rejects with a useful message instead of a stat-as-file miss. + std::string msg = "Vendor module not found: " + spec; + isolate->ThrowException(v8::Exception::Error( + ArgConverter::ConvertToV8String(isolate, msg))); + return v8::MaybeLocal(); + } + if (spec.rfind("blob:", 0) == 0) { + // Blob URLs are issued by URL.createObjectURL — the corresponding + // module should already be in g_moduleRegistry under the exact blob + // key (loader wrote it on creation). Look up verbatim; no + // normalization (the random ID is the identity). + auto it = g_moduleRegistry.find(spec); + if (it != g_moduleRegistry.end()) { + return v8::MaybeLocal(it->second.Get(isolate)); + } + std::string msg = "Blob module not found: " + spec; + isolate->ThrowException(v8::Exception::Error( + ArgConverter::ConvertToV8String(isolate, msg))); + return v8::MaybeLocal(); + } + if (spec.rfind("optional:", 0) == 0) { + // Optional-module sentinel — return empty so V8 throws the standard + // ESM resolve failure (the caller is responsible for swallowing). + if (tns::IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[resolver][optional] not found: %s", spec.c_str()); + } + return v8::MaybeLocal(); + } + // Attempt to resolve relative or root-absolute specifiers against an HTTP referrer URL std::string referrerPath; for (auto& kv : g_moduleRegistry) { @@ -367,7 +1156,7 @@ v8::MaybeLocal ResolveModuleCallback(v8::Local context, }; if (!startsWithHttp(spec) && (specIsRelative || specIsRootAbs)) { if (!referrerPath.empty() && startsWithHttp(referrerPath)) { - std::string resolved = ResolveHttpRelative(referrerPath, spec); + std::string resolved = tns::ResolveImportSpecifierAgainstUrl(spec, referrerPath); if (!resolved.empty()) { if (IsScriptLoadingLogEnabled()) { DEBUG_WRITE("ResolveModuleCallback: HTTP-relative resolved '%s' + '%s' -> '%s'", @@ -387,7 +1176,7 @@ v8::MaybeLocal ResolveModuleCallback(v8::Local context, if (!origin.empty() && (origin.rfind("http://", 0) == 0 || origin.rfind("https://", 0) == 0)) { std::string refBase = origin; if (refBase.back() != '/') refBase += '/'; - std::string resolved = ResolveHttpRelative(refBase, spec); + std::string resolved = tns::ResolveImportSpecifierAgainstUrl(spec, refBase); if (!resolved.empty()) { if (IsScriptLoadingLogEnabled()) { DEBUG_WRITE("[http-esm][http-origin][fallback] origin=%s spec=%s -> %s", refBase.c_str(), spec.c_str(), resolved.c_str()); @@ -417,10 +1206,19 @@ v8::MaybeLocal ResolveModuleCallback(v8::Local context, std::string body, ct; int status = 0; if (!tns::HttpFetchText(spec, body, ct, status)) { + // Pull any JNI-captured reason BEFORE composing the error so + // the user sees `connect failed: ECONNREFUSED` (or whatever) + // alongside the bare `status=0` instead of having to dig + // through logcat. + std::string reason = tns::TakeLastHttpFetchErrorReason(); if (IsScriptLoadingLogEnabled()) { - DEBUG_WRITE("[http-esm][fetch][fail] url=%s status=%d", spec.c_str(), status); + DEBUG_WRITE("[http-esm][fetch][fail] url=%s status=%d reason=%s", + spec.c_str(), status, reason.c_str()); } std::string msg = std::string("Failed to fetch ") + spec + ", status=" + std::to_string(status); + if (!reason.empty()) { + msg += ", " + reason; + } isolate->ThrowException(v8::Exception::Error(ArgConverter::ConvertToV8String(isolate, msg))); return v8::MaybeLocal(); } @@ -892,7 +1690,7 @@ v8::MaybeLocal ImportModuleDynamicallyCallback( referrerUrl = *r8 ? *r8 : ""; } if ((specIsRelative || specIsRootAbs) && isHttpLike(referrerUrl)) { - std::string resolved = ResolveHttpRelative(referrerUrl, spec); + std::string resolved = tns::ResolveImportSpecifierAgainstUrl(spec, referrerUrl); if (!resolved.empty()) { if (IsScriptLoadingLogEnabled()) { DEBUG_WRITE("[http-esm][dyn][http-rel] base=%s spec=%s -> %s", referrerUrl.c_str(), spec.c_str(), resolved.c_str()); @@ -903,6 +1701,46 @@ v8::MaybeLocal ImportModuleDynamicallyCallback( } } + // Blob URL dynamic import — synthetic, must preserve identity (no + // canonicalization). The HTTP/file machinery below would mis-handle a + // `blob:` key as a fetch target. Caller already registered the module + // under the exact blob key when URL.createObjectURL was called. + if (spec.rfind("blob:", 0) == 0) { + auto it = g_moduleRegistry.find(spec); + if (it == g_moduleRegistry.end()) { + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("[dyn-import][blob][miss] %s", spec.c_str()); + } + resolver->Reject(context, + v8::Exception::Error(ArgConverter::ConvertToV8String( + isolate, std::string("Blob module not found: ") + spec))).Check(); + return scope.Escape(resolver->GetPromise()); + } + v8::Local blobMod = it->second.Get(isolate); + if (blobMod->GetStatus() == v8::Module::kUninstantiated) { + if (!blobMod->InstantiateModule(context, &ResolveModuleCallback).FromMaybe(false)) { + resolver->Reject(context, + v8::Exception::Error(ArgConverter::ConvertToV8String( + isolate, "Blob module instantiate failed"))).Check(); + return scope.Escape(resolver->GetPromise()); + } + } + if (blobMod->GetStatus() != v8::Module::kEvaluated) { + if (blobMod->Evaluate(context).IsEmpty()) { + resolver->Reject(context, + v8::Exception::Error(ArgConverter::ConvertToV8String( + isolate, "Blob module evaluation failed"))).Check(); + return scope.Escape(resolver->GetPromise()); + } + } + if (blobMod->GetStatus() == v8::Module::kErrored) { + resolver->Reject(context, blobMod->GetException()).Check(); + return scope.Escape(resolver->GetPromise()); + } + resolver->Resolve(context, blobMod->GetModuleNamespace()).Check(); + return scope.Escape(resolver->GetPromise()); + } + // Handle HTTP(S) dynamic import directly // Security: HttpFetchText gates remote module access centrally. if (!spec.empty() && isHttpLike(spec)) { @@ -915,15 +1753,21 @@ v8::MaybeLocal ImportModuleDynamicallyCallback( if (it != g_moduleRegistry.end()) { mod = it->second.Get(isolate); if (IsScriptLoadingLogEnabled()) { - DEBUG_WRITE("[http-esm][dyn][cache] hit %s", canonical.c_str()); + DEBUG_WRITE("[http-esm][dyn][http-cache hit] %s", canonical.c_str()); } } else { std::string body, ct; int status = 0; if (!tns::HttpFetchText(spec, body, ct, status)) { + std::string reason = tns::TakeLastHttpFetchErrorReason(); if (IsScriptLoadingLogEnabled()) { - DEBUG_WRITE("[http-esm][dyn][fetch][fail] url=%s status=%d", spec.c_str(), status); + DEBUG_WRITE("[http-esm][dyn][fetch][fail] url=%s status=%d reason=%s", + spec.c_str(), status, reason.c_str()); } - resolver->Reject(context, v8::Exception::Error(ArgConverter::ConvertToV8String(isolate, std::string("Failed to fetch ")+spec))).Check(); + std::string rejMsg = std::string("Failed to fetch ") + spec + ", status=" + std::to_string(status); + if (!reason.empty()) { + rejMsg += ", " + reason; + } + resolver->Reject(context, v8::Exception::Error(ArgConverter::ConvertToV8String(isolate, rejMsg))).Check(); return scope.Escape(resolver->GetPromise()); } if (IsScriptLoadingLogEnabled()) { @@ -958,6 +1802,22 @@ v8::MaybeLocal ImportModuleDynamicallyCallback( return scope.Escape(resolver->GetPromise()); } } + // With top-level-await enabled, Evaluate() returns a promise instead of + // an empty MaybeLocal on throw; the module's errored state must be read + // from its status. Propagate the real exception so import() rejects + // (and the dev client can surface it) instead of resolving a + // half-evaluated namespace. + if (mod->GetStatus() == v8::Module::kErrored) { + v8::Local exception = mod->GetException(); + if (IsScriptLoadingLogEnabled()) { + v8::String::Utf8Value exc8(isolate, exception); + DEBUG_WRITE("[http-esm][dyn][eval][errored] %s: %s", canonical.c_str(), + *exc8 ? *exc8 : "(no message)"); + } + g_moduleRegistry.erase(canonical); + resolver->Reject(context, exception).Check(); + return scope.Escape(resolver->GetPromise()); + } resolver->Resolve(context, mod->GetModuleNamespace()).Check(); return scope.Escape(resolver->GetPromise()); } @@ -1025,6 +1885,16 @@ v8::MaybeLocal ImportModuleDynamicallyCallback( } } + // Top-level-await semantics: a throwing module leaves Evaluate() with a + // (rejected) promise, so the errored state must be read from status. + if (module->GetStatus() == v8::Module::kErrored) { + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("ImportModuleDynamicallyCallback: Evaluation errored for '%s'", spec.c_str()); + } + resolver->Reject(context, module->GetException()).Check(); + return scope.Escape(resolver->GetPromise()); + } + resolver->Resolve(context, module->GetModuleNamespace()).Check(); if (IsScriptLoadingLogEnabled()) { DEBUG_WRITE("ImportModuleDynamicallyCallback: Successfully resolved '%s'", spec.c_str()); diff --git a/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.h b/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.h index 908c30ba7..1b1f75299 100644 --- a/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.h +++ b/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.h @@ -3,6 +3,68 @@ #include "v8.h" +#include +#include +#include + +namespace tns { + +// Per-isolate ES module registry. `thread_local`: each NS isolate (main thread + +// each Worker thread) gets its own per-thread map, because v8::Global +// handles are isolate-bound — sharing one map across isolates would let one +// thread try to read another thread's handle and trip V8's identity-check on +// instantiation. See the long-form comment above the definition in +// ModuleInternalCallbacks.cpp for the cross-isolate-handle bug this prevents. +extern thread_local std::unordered_map>& g_moduleRegistry; + +// Import-map and volatile-pattern configuration. +// +// `SetImportMapEntries` populates the process-wide bare-specifier → URL map +// used by `ResolveModuleCallback`; the dev server's import-map JSON is parsed +// at the V8 layer by the callers, which pass the flat entries here. +// `SetVolatilePatterns` accepts a list of URL substrings that should always +// re-fetch (never serve from the kickstart prewarm cache). Both are applied +// via `__NS_DEV__.configureRuntime` at session start and again at every HMR +// graph version bump. + +// Set the process-wide import map from the given flat (bare-specifier → URL) +// entries. The callers hold the parsed import-map object and extract its +// `imports` table at the V8 layer; this module stores the resulting entries. +void SetImportMapEntries(const std::vector>& entries); +void SetVolatilePatterns(const std::vector& patterns); + +// Drop all per-process import-map / vendor / in-flight state. +// Called from `Runtime::~Runtime()` on the MAIN isolate only — workers +// share the process-wide registries but the main isolate owns their +// lifetime, so wiping them on a worker would race with the next +// re-launched main isolate. +void CleanupImportMapGlobals(); + +// Snapshot the keys currently registered in `g_moduleRegistry` (file paths + +// canonical HTTP URLs). Used by `CollectSessionModuleUrls` to enumerate +// modules the dev session needs to invalidate. +std::vector GetLoadedModuleUrls(); + +// Evict the given keys (canonical registry keys) from `g_moduleRegistry`. +// No-op if the key is missing. Used by `__NS_DEV__.invalidateModules` and +// by the JS dev client's HMR cycle to drop stale modules before +// re-importing. +void RemoveModuleFromRegistry(const std::string& canonicalKey); + +// Drop a list of keys + their HTTP cache entries in one pass. Returns the +// number of registry entries removed. +size_t InvalidateModules(const std::vector& keys); + +// Fetch + compile + register path used when an HTTP(S) URL is loaded as a +// module entry point (see ModuleInternal). The caller is responsible for +// instantiation + evaluation on the JS thread. +v8::MaybeLocal LoadHttpModuleForUrl(v8::Isolate* isolate, + v8::Local context, + const std::string& url, + std::string* errorMessage); + +} // namespace tns + // Module resolution callback for ES modules v8::MaybeLocal ResolveModuleCallback(v8::Local context, v8::Local specifier, diff --git a/test-app/runtime/src/main/cpp/Runtime.cpp b/test-app/runtime/src/main/cpp/Runtime.cpp index df0fb6e60..256280592 100644 --- a/test-app/runtime/src/main/cpp/Runtime.cpp +++ b/test-app/runtime/src/main/cpp/Runtime.cpp @@ -3,9 +3,12 @@ #include #include #include +#include +#include #include #include +#include #include #include #include @@ -28,6 +31,7 @@ #include "SimpleAllocator.h" #include "SimpleProfiler.h" #include "URLImpl.h" +#include "HMRSupport.h" #include "URLPatternImpl.h" #include "URLSearchParamsImpl.h" #include "Util.h" @@ -52,22 +56,122 @@ using namespace tns; bool tns::LogEnabled = true; SimpleAllocator g_allocator; -void SIG_handler(int sigNumber) { - stringstream msg; - msg << "JNI Exception occurred ("; +namespace { +struct SigHandlerBacktraceState { + void** current; + void** end; +}; + +// libunwind callback that walks the C++ stack frame-by-frame. Captures the +// PC for each frame into the passed array up to capacity. Signal-handler +// safe (does not allocate, does not call into libc beyond the unwinder). +_Unwind_Reason_Code SigHandlerUnwindCallback(struct _Unwind_Context* ctx, void* arg) { + auto* state = static_cast(arg); + uintptr_t pc = _Unwind_GetIP(ctx); + if (pc) { + if (state->current == state->end) { + return _URC_END_OF_STACK; + } + *state->current++ = reinterpret_cast(pc); + } + return _URC_NO_REASON; +} + +// Format one backtrace frame to logcat. We use dladdr to resolve the shared +// object and symbol; if the symbol is mangled we attempt to demangle. +// Output looks like: +// #07 pc 0x00000000004a8b0c libNativeScript.so (tns::Runtime::Init+12) +// Matching the format Android's stock crash dump uses so it's familiar. +void LogBacktraceFrame(int idx, void* pc) { + Dl_info info{}; + const char* libName = "?"; + const char* symName = "?"; + uintptr_t relPc = reinterpret_cast(pc); + uintptr_t symOff = 0; + char* demangled = nullptr; + + if (dladdr(pc, &info) && info.dli_fname) { + libName = info.dli_fname; + if (info.dli_fbase) { + relPc = relPc - reinterpret_cast(info.dli_fbase); + } + if (info.dli_sname) { + symName = info.dli_sname; + symOff = reinterpret_cast(pc) - reinterpret_cast(info.dli_saddr); + int status = 0; + demangled = abi::__cxa_demangle(info.dli_sname, nullptr, nullptr, &status); + if (status == 0 && demangled) { + symName = demangled; + } + } + } + + __android_log_print(ANDROID_LOG_FATAL, "TNS.Native", + " #%02d pc 0x%016lx %s (%s+%lu)", idx, + static_cast(relPc), libName, symName, + static_cast(symOff)); + if (demangled) free(demangled); +} +} // namespace + +void SIG_handler(int sigNumber, siginfo_t* sigInfo, void* /*ucontext*/) { + // Reset all fatal signal handlers to default IMMEDIATELY. If the body of + // this handler itself crashes (e.g. malloc is broken because the original + // SEGV was in the allocator), the second signal will kill the process + // cleanly and let Android write a proper tombstone, instead of looping + // forever between handler and signal. + ::signal(SIGABRT, SIG_DFL); + ::signal(SIGSEGV, SIG_DFL); + ::signal(SIGBUS, SIG_DFL); + ::signal(SIGFPE, SIG_DFL); + ::signal(SIGILL, SIG_DFL); + + // Async-signal-handler caveats: we intentionally avoid std::stringstream + // and other allocating code on the signal stack. __android_log_print and + // dladdr are not strictly signal-safe but are reliable on Android in + // practice and give us the diagnostic we need. + const char* sigName = "UNKNOWN"; switch (sigNumber) { - case SIGABRT: - msg << "SIGABRT"; - break; - case SIGSEGV: - msg << "SIGSEGV"; - break; - default: - // Shouldn't happen, but for completeness - msg << "Signal #" << sigNumber; - break; - } - msg << ").\n=======\nCheck the 'adb logcat' for additional information about " + case SIGABRT: sigName = "SIGABRT"; break; + case SIGSEGV: sigName = "SIGSEGV"; break; + case SIGBUS: sigName = "SIGBUS"; break; + case SIGFPE: sigName = "SIGFPE"; break; + case SIGILL: sigName = "SIGILL"; break; + default: sigName = "Signal"; break; + } + + // ── Header (must precede backtrace so it isn't lost when the handler + // throws and unwinds the stack). ────────────────────────────────── + __android_log_print(ANDROID_LOG_FATAL, "TNS.Native", + "=== Native crash caught by NativeScript runtime ==="); + __android_log_print(ANDROID_LOG_FATAL, "TNS.Native", + "signal %d (%s), code %d, fault addr %p, tid %d", + sigNumber, sigName, + sigInfo ? sigInfo->si_code : -1, + sigInfo ? sigInfo->si_addr : nullptr, + static_cast(gettid())); + + // ── C++ backtrace via _Unwind_Backtrace. Capped at 64 frames so we + // never run the signal stack out. ──────────────────────────────── + constexpr size_t kMaxFrames = 64; + void* frames[kMaxFrames] = {}; + SigHandlerBacktraceState state = {frames, frames + kMaxFrames}; + _Unwind_Backtrace(&SigHandlerUnwindCallback, &state); + const int frameCount = static_cast(state.current - frames); + __android_log_print(ANDROID_LOG_FATAL, "TNS.Native", + "backtrace (%d frames):", frameCount); + for (int i = 0; i < frameCount; ++i) { + LogBacktraceFrame(i, frames[i]); + } + __android_log_print(ANDROID_LOG_FATAL, "TNS.Native", + "=== end native crash ==="); + + // Throw so the JS-side error pipeline still reports the crash. + // Note: throwing from a signal handler is technically UB, but the runtime + // has relied on it for years and works under libgcc/libunwind+itanium-abi. + stringstream msg; + msg << "JNI Exception occurred (" << sigName + << ").\n=======\nCheck the 'adb logcat' for additional information about " "the error.\n=======\n"; throw NativeScriptException(msg.str()); } @@ -106,10 +210,34 @@ void Runtime::Init(JavaVM* vm, void* reserved) { // handle SIGABRT/SIGSEGV only on API level > 20 as the handling is not so // efficient in older versions if (m_androidVersion > 20) { + // Install an alternate signal stack BEFORE registering the handler so + // that we can still produce a meaningful backtrace even when the crash + // was caused by a stack overflow on the main JS thread (the original + // stack is full and would deadlock the handler otherwise). + // NDK r23+ defines SIGSTKSZ via sysconf so it isn't constexpr — use a + // fixed 64 KiB which comfortably covers Android's MINSIGSTKSZ. + constexpr size_t kAltStackSize = 64 * 1024; + static thread_local char altStackBuf[kAltStackSize]; + stack_t altStack{}; + altStack.ss_sp = altStackBuf; + altStack.ss_size = kAltStackSize; + altStack.ss_flags = 0; + sigaltstack(&altStack, nullptr); + struct sigaction action; - action.sa_handler = SIG_handler; + memset(&action, 0, sizeof(action)); + sigemptyset(&action.sa_mask); + // SA_SIGINFO enables the 3-arg handler so we get siginfo_t (fault addr + // and si_code). SA_ONSTACK lets the handler run on an alternate stack + // — important so we still produce a useful backtrace if the original + // crash was a stack overflow. + action.sa_flags = SA_SIGINFO | SA_ONSTACK; + action.sa_sigaction = SIG_handler; sigaction(SIGABRT, &action, NULL); sigaction(SIGSEGV, &action, NULL); + sigaction(SIGBUS, &action, NULL); + sigaction(SIGFPE, &action, NULL); + sigaction(SIGILL, &action, NULL); } // Set terminate handler for uncaught exceptions std::set_terminate(LogAndAbortUncaught); @@ -271,6 +399,13 @@ Runtime::~Runtime() { delete this->m_loopTimer; CallbackHandlers::RemoveIsolateEntries(m_isolate); if (m_isMainThread) { + // Clear process-wide dev-loader state (prewarm cache, cache-bust marks, + // boot flag, import map, vendor registry). Main isolate only: workers + // share these registries but the main isolate owns their lifetime — + // wiping them on worker teardown would race with the live main isolate. + tns::CleanupHMRGlobals(); + tns::CleanupImportMapGlobals(); + if (m_mainLooper_fd[0] != -1) { ALooper_removeFd(m_mainLooper, m_mainLooper_fd[0]); } @@ -768,6 +903,10 @@ Isolate* Runtime::PrepareV8Runtime(const string& filesPath, globalTemplate->Set(ArgConverter::ConvertToV8String(isolate, "Worker"), workerFuncTemplate); + + // Worker termination for dev tooling is exposed as + // `__NS_DEV__.terminateAllWorkers()`, installed by + // InitializeHmrDevGlobals below for the main isolate only. } /* @@ -785,79 +924,23 @@ Isolate* Runtime::PrepareV8Runtime(const string& filesPath, Local context = Context::New(isolate, nullptr, globalTemplate); - auto blob_methods = R"js( - const BLOB_STORE = new Map(); - URL.createObjectURL = function (object, options = null) { - try { - if (object instanceof Blob || object instanceof File) { - const id = java.util.UUID.randomUUID().toString(); - const ret = `blob:nativescript/${id}`; - BLOB_STORE.set(ret, { - blob: object, - type: object?.type, - ext: options?.ext, - }); - return ret; - } - } catch (error) { - return null; - } - return null; - }; - URL.revokeObjectURL = function (url) { - BLOB_STORE.delete(url); - }; - const InternalAccessor = class {}; - InternalAccessor.getData = function (url) { - return BLOB_STORE.get(url); - }; - URL.InternalAccessor = InternalAccessor; - Object.defineProperty(URL.prototype, 'searchParams', { - get() { - if (this._searchParams == null) { - this._searchParams = new URLSearchParams(this.search); - Object.defineProperty(this._searchParams, '_url', { - enumerable: false, - writable: false, - value: this, - }); - this._searchParams._append = this._searchParams.append; - this._searchParams.append = function (name, value) { - this._append(name, value); - this._url.search = this.toString(); - }; - this._searchParams._delete = this._searchParams.delete; - this._searchParams.delete = function (name) { - this._delete(name); - this._url.search = this.toString(); - }; - this._searchParams._set = this._searchParams.set; - this._searchParams.set = function (name, value) { - this._set(name, value); - this._url.search = this.toString(); - }; - this._searchParams._sort = this._searchParams.sort; - this._searchParams.sort = function () { - this._sort(); - this._url.search = this.toString(); - }; - } - return this._searchParams; - }, - }); - )js"; - auto global = context->Global(); v8::Context::Scope contextScope{context}; - v8::Local script; - v8::Script::Compile(context, - ArgConverter::ConvertToV8String(isolate, blob_methods)) - .ToLocal(&script); - - v8::Local out; - script->Run(context).ToLocal(&out); + // Install URL.createObjectURL / URL.revokeObjectURL / blob registry / + // URL.searchParams accessor. The script literal lives in `URLImpl` so + // it can be shared between runtimes. + URLImpl::InstallBlobMethods(context); + + // Install the `__NS_DEV__` dev-loader namespace on EVERY isolate (main + // and worker) in EVERY build. The runtime's dev surface is mechanism + // only — the security boundary sits at the network layer + // (`DevFlags::IsRemoteUrlAllowed`), not at namespace installation; in a + // default-config release app the members are inert because every fetch + // they could trigger is denied. Workers get the same surface minus + // `terminateAllWorkers` (main-isolate only, see CallbackHandlers.h). + tns::InitializeHmrDevGlobals(isolate, context, /*isWorker=*/!m_isMainThread); m_objectManager->Init(isolate); m_module.Init(isolate, callingDir); diff --git a/test-app/runtime/src/main/cpp/URLImpl.cpp b/test-app/runtime/src/main/cpp/URLImpl.cpp index a2167d692..8efaefe80 100644 --- a/test-app/runtime/src/main/cpp/URLImpl.cpp +++ b/test-app/runtime/src/main/cpp/URLImpl.cpp @@ -9,6 +9,95 @@ using namespace ada; URLImpl::URLImpl(url_aggregator url) : url_(url) {} +// Install URL.createObjectURL / URL.revokeObjectURL / URL.InternalAccessor + +// URL.prototype.searchParams accessor onto the realm's URL constructor. +// +// Blob IDs use `java.util.UUID.randomUUID().toString()` (lower-case +// RFC-4122 v4), producing `blob:nativescript/` keys. +void URLImpl::InstallBlobMethods(v8::Local context) { + v8::Isolate* isolate = context->GetIsolate(); + // URL.createObjectURL/revokeObjectURL and blob URL registry + // Blob URLs have the format: blob:/ + // We use blob:nativescript/ as NativeScript's origin identifier + auto blob_methods = R"js( + const BLOB_STORE = new Map(); + URL.createObjectURL = function (object, options = null) { + try { + if (object instanceof Blob || object instanceof File) { + const id = java.util.UUID.randomUUID().toString(); + const ret = `blob:nativescript/${id}`; + BLOB_STORE.set(ret, { + blob: object, + type: object?.type, + ext: options?.ext, + }); + return ret; + } + } catch (error) { + return null; + } + return null; + }; + URL.revokeObjectURL = function (url) { + BLOB_STORE.delete(url); + }; + const InternalAccessor = class {}; + InternalAccessor.getData = function (url) { + return BLOB_STORE.get(url); + }; + // Get the text content directly from a blob URL (for HMR) + InternalAccessor.getText = async function (url) { + const data = BLOB_STORE.get(url); + if (!data || !data.blob) return null; + return await data.blob.text(); + }; + URL.InternalAccessor = InternalAccessor; + Object.defineProperty(URL.prototype, 'searchParams', { + get() { + if (this._searchParams == null) { + this._searchParams = new URLSearchParams(this.search); + Object.defineProperty(this._searchParams, '_url', { + enumerable: false, + writable: false, + value: this, + }); + this._searchParams._append = this._searchParams.append; + this._searchParams.append = function (name, value) { + this._append(name, value); + this._url.search = this.toString(); + }; + this._searchParams._delete = this._searchParams.delete; + this._searchParams.delete = function (name) { + this._delete(name); + this._url.search = this.toString(); + }; + this._searchParams._set = this._searchParams.set; + this._searchParams.set = function (name, value) { + this._set(name, value); + this._url.search = this.toString(); + }; + this._searchParams._sort = this._searchParams.sort; + this._searchParams.sort = function () { + this._sort(); + this._url.search = this.toString(); + }; + } + return this._searchParams; + }, + }); + )js"; + + v8::Local script; + auto compiled = v8::Script::Compile( + context, ArgConverter::ConvertToV8String(isolate, blob_methods)) + .ToLocal(&script); + + if (compiled) { + v8::Local outVal; + (void)script->Run(context).ToLocal(&outVal); + } +} + URLImpl *URLImpl::GetPointer(v8::Local object) { auto ptr = object->GetAlignedPointerFromInternalField(0); if (ptr == nullptr) { diff --git a/test-app/runtime/src/main/cpp/URLImpl.h b/test-app/runtime/src/main/cpp/URLImpl.h index e36002ec2..56bcfee99 100644 --- a/test-app/runtime/src/main/cpp/URLImpl.h +++ b/test-app/runtime/src/main/cpp/URLImpl.h @@ -21,6 +21,15 @@ namespace tns { static v8::Local GetCtor(v8::Isolate *isolate); + // Compiles and runs the JS polyfill that installs + // `URL.createObjectURL` / `URL.revokeObjectURL`, the in-process blob + // registry (`URL.InternalAccessor`) used by the HMR loader, and the + // `URL.prototype.searchParams` accessor. Must be called once per + // realm AFTER `URL` and `URLSearchParams` constructors are installed. + // The script literal lives here so every runtime (main and worker) + // shares one copy. + static void InstallBlobMethods(v8::Local context); + static void Ctor(const v8::FunctionCallbackInfo &args); diff --git a/test-app/runtime/src/main/cpp/Version.h b/test-app/runtime/src/main/cpp/Version.h index 17348a68c..2fb2e3c09 100644 --- a/test-app/runtime/src/main/cpp/Version.h +++ b/test-app/runtime/src/main/cpp/Version.h @@ -1,2 +1,2 @@ -#define NATIVE_SCRIPT_RUNTIME_VERSION "0.0.0.0" -#define NATIVE_SCRIPT_RUNTIME_COMMIT_SHA "RUNTIME_COMMIT_SHA_PLACEHOLDER" \ No newline at end of file +#define NATIVE_SCRIPT_RUNTIME_VERSION "9.1.0-alpha.7" +#define NATIVE_SCRIPT_RUNTIME_COMMIT_SHA "no commit sha was provided by build.gradle build" \ No newline at end of file diff --git a/test-app/runtime/src/main/cpp/WorkerWrapper.cpp b/test-app/runtime/src/main/cpp/WorkerWrapper.cpp index d23c3d30e..011e2dadf 100644 --- a/test-app/runtime/src/main/cpp/WorkerWrapper.cpp +++ b/test-app/runtime/src/main/cpp/WorkerWrapper.cpp @@ -387,8 +387,7 @@ void WorkerWrapper::BackgroundLooper(std::shared_ptr self) { } } - // Deliver messages that were posted before the worker was ready - // (replaces the old Java Handshake + pendingWorkerMessages). + // Deliver messages that were posted before the worker was ready. DrainPendingTasks(); if (!isTerminating_ && !isClosing_) { @@ -437,8 +436,8 @@ void WorkerWrapper::BackgroundLooper(std::shared_ptr self) { if (runtime_ != nullptr) { try { // Java-side detach (GcListener.unsubscribe + runtimeCache.remove) - // must happen before the isolate is disposed, preserving the old - // WorkerThreadHandler -> TerminateWorkerCallback ordering. + // must happen before the isolate is disposed so Java never holds + // a runtime whose isolate is already gone. JEnv env; env.CallStaticVoidMethod(RUNTIME_CLASS, DETACH_WORKER_RUNTIME_METHOD_ID, runtimeId); } catch (NativeScriptException& ex) { @@ -529,7 +528,7 @@ void WorkerWrapper::ClearWorkerOnParent(int workerId) { } } -void WorkerWrapper::TerminateChildren(Isolate* parentIsolate) { +int WorkerWrapper::TerminateChildren(Isolate* parentIsolate) { std::vector> children; { std::lock_guard lock(registryMutex_); @@ -547,6 +546,8 @@ void WorkerWrapper::TerminateChildren(Isolate* parentIsolate) { child->Terminate(); ClearWorkerOnParent(child->workerId_); } + + return static_cast(children.size()); } WorkerWrapper* WorkerWrapper::FromIsolate(Isolate* isolate) { diff --git a/test-app/runtime/src/main/cpp/WorkerWrapper.h b/test-app/runtime/src/main/cpp/WorkerWrapper.h index 464145d2f..0e0aa182d 100644 --- a/test-app/runtime/src/main/cpp/WorkerWrapper.h +++ b/test-app/runtime/src/main/cpp/WorkerWrapper.h @@ -94,9 +94,8 @@ class WorkerWrapper : public std::enable_shared_from_this { int lineno); /* - * Registry of live workers, keyed by workerId. Replaces the old - * CallbackHandlers::id2WorkerMap. Guarded by a mutex because the worker - * shutdown path posts cleanup from the worker thread. + * Registry of live workers, keyed by workerId. Guarded by a mutex + * because the worker shutdown path posts cleanup from the worker thread. */ static int NextWorkerId(); static std::shared_ptr GetById(int workerId); @@ -113,8 +112,9 @@ class WorkerWrapper : public std::enable_shared_from_this { * Must run on the parent's thread, before the parent isolate is disposed * (the children's Worker object persistents live in that isolate). * Cascades: each child terminates its own children during shutdown. + * Returns the number of direct child workers that were torn down. */ - static void TerminateChildren(v8::Isolate* parentIsolate); + static int TerminateChildren(v8::Isolate* parentIsolate); /* * Resolves the wrapper of a worker isolate via its isolate data slot. diff --git a/test-app/runtime/src/main/java/com/tns/AppConfig.java b/test-app/runtime/src/main/java/com/tns/AppConfig.java index c163cbd54..ee5446c50 100644 --- a/test-app/runtime/src/main/java/com/tns/AppConfig.java +++ b/test-app/runtime/src/main/java/com/tns/AppConfig.java @@ -24,7 +24,8 @@ protected enum KnownKeys { DiscardUncaughtJsExceptions("discardUncaughtJsExceptions", false), EnableLineBreakpoins("enableLineBreakpoints", false), EnableMultithreadedJavascript("enableMultithreadedJavascript", false), - LogScriptLoading("logScriptLoading", false); + LogScriptLoading("logScriptLoading", false), + HttpFetchUrlLog("httpFetchUrlLog", false); private final String name; private final Object defaultValue; @@ -86,6 +87,9 @@ public AppConfig(File appDir) { if (rootObject.has(KnownKeys.LogScriptLoading.getName())) { values[KnownKeys.LogScriptLoading.ordinal()] = rootObject.getBoolean(KnownKeys.LogScriptLoading.getName()); } + if (rootObject.has(KnownKeys.HttpFetchUrlLog.getName())) { + values[KnownKeys.HttpFetchUrlLog.ordinal()] = rootObject.getBoolean(KnownKeys.HttpFetchUrlLog.getName()); + } if (rootObject.has(KnownKeys.DiscardUncaughtJsExceptions.getName())) { values[KnownKeys.DiscardUncaughtJsExceptions.ordinal()] = rootObject.getBoolean(KnownKeys.DiscardUncaughtJsExceptions.getName()); } @@ -206,6 +210,11 @@ public boolean getLogScriptLoading() { return (v instanceof Boolean) ? ((Boolean)v).booleanValue() : false; } + public boolean getHttpFetchUrlLog() { + Object v = values[KnownKeys.HttpFetchUrlLog.ordinal()]; + return (v instanceof Boolean) ? ((Boolean)v).booleanValue() : false; + } + // Security conf /** diff --git a/test-app/runtime/src/main/java/com/tns/ClassResolver.java b/test-app/runtime/src/main/java/com/tns/ClassResolver.java index 0dba4c64a..058671a64 100644 --- a/test-app/runtime/src/main/java/com/tns/ClassResolver.java +++ b/test-app/runtime/src/main/java/com/tns/ClassResolver.java @@ -1,6 +1,7 @@ package com.tns; import com.tns.system.classes.loading.ClassStorageService; +import com.tns.system.classes.loading.LookedUpClassNotFound; import java.io.IOException; @@ -26,7 +27,35 @@ Class resolveClass(String baseClassName, String fullClassName, DexFactory dex } if (clazz == null) { - clazz = classStorageService.retrieveClass(className); + try { + clazz = classStorageService.retrieveClass(className); + } catch (LookedUpClassNotFound notFound) { + // HMR / dev fallback. The Static Binding Generator pre-generates + // a Java stub for every `.extend('com.tns.Name', { ... })` call + // it sees at build time, and production bakes those stubs into + // the APK dex. In Vite HMR mode the bundle is just the boot + // loader and modules are fetched over HTTP at runtime, so the + // SBG never sees those `.extend(...)` calls and the class is + // missing from the dex. + // + // With a baseClassName, JS is extending a known Java type, so + // route through `DexFactory.resolveClass` (the same path as + // `com.tns.gen.*` bindings) to generate + load a dex at runtime. + // Lookups without a baseClassName are plain `Class.forName`-style + // requests and must still fail loudly. + // + // Gated on `isDebuggable()`: runtime dex generation and + // parent-classloader injection are only needed in dev. In + // release every stub is baked in, so a miss is a genuinely + // missing class that should fail rather than silently mutate + // the app classloader. + if (!canonicalBaseClassName.isEmpty() && com.tns.Runtime.isDebuggable()) { + clazz = dexFactory.resolveClass(canonicalBaseClassName, name, className, methodOverrides, implementedInterfaces, isInterface); + } + if (clazz == null) { + throw notFound; + } + } } return clazz; diff --git a/test-app/runtime/src/main/java/com/tns/DexFactory.java b/test-app/runtime/src/main/java/com/tns/DexFactory.java index 56b37462e..1fdf572af 100644 --- a/test-app/runtime/src/main/java/com/tns/DexFactory.java +++ b/test-app/runtime/src/main/java/com/tns/DexFactory.java @@ -35,6 +35,26 @@ public class DexFactory { private static final String COM_TNS_GEN_PREFIX = "com.tns.gen."; + // Generated proxy classes always live under names where the original + // `$` separators in inner-class qualifiers have been replaced with + // `_`. The proxy generator writes the dex with `_`, so any load via + // `classLoader.loadClass(...)` must use the same form. This helper + // is the single canonical entry point for that normalization so + // every load site agrees. + // + // Scoped strictly to the `com.tns.gen.` prefix; unrelated + // inner-class lookups (e.g. `java.util.HashMap$Entry`) are returned + // unchanged so JVM inner-class syntax keeps working. + private static String normalizeProxyClassName(String name) { + if (name == null) { + return null; + } + if (!name.startsWith(COM_TNS_GEN_PREFIX) || name.indexOf('$') < 0) { + return name; + } + return name.replace('$', '_'); + } + private final Logger logger; private final File dexDir; private final File odexDir; @@ -121,9 +141,19 @@ public Class resolveClass(String baseClassName, String name, String className String desiredDexClassName = this.getClassToProxyName(fullClassName); // when interfaces are extended as classes, we still want to preserve - // just the interface name without the extra file, line, column information + // just the interface name without the extra file, line, column information. + // + // The loadable proxy class name uses the `_`-normalized form: the proxy + // is written into the dex with `_`, while `classToProxy` keeps `$` so + // `Class.forName(classToProxy)` (in `generateDex`) can resolve the base + // type via JVM inner-class syntax. Without normalizing here, a + // `$`-containing base (e.g. unnamed + // `.extend(android.app.Application.ActivityLifecycleCallbacks, ...)`) + // fails `loadClass` with ClassNotFoundException even though the dex is + // present. Surfaces in HMR / dev; production uses an SBG-generated + // `@JavaProxy(...)` sibling and bypasses this path. if (!baseClassName.isEmpty() && isInterface) { - fullClassName = COM_TNS_GEN_PREFIX + classToProxy; + fullClassName = normalizeProxyClassName(COM_TNS_GEN_PREFIX + classToProxy); } File dexFile = this.getDexFile(desiredDexClassName); @@ -204,6 +234,29 @@ public Class findClass(String className) throws ClassNotFoundException { return existingClass; } + // Generated proxy classes live under `_`-normalized names (see + // `normalizeProxyClassName`). If JNI hands us a `$`-containing + // `com.tns.gen.*` lookup (a metadata dispatch that passed a `$`-bearing + // base name straight through), try the normalized sibling — both the + // `injectedDexClasses` cache and `loadClass` — before falling back to + // the raw form. Scoped to the `com.tns.gen.` prefix so unrelated + // inner-class lookups (e.g. `java.util.HashMap$Entry`) still resolve via + // JVM inner-class syntax at the tail of this method. + String normalizedName = normalizeProxyClassName(canonicalName); + if (!normalizedName.equals(canonicalName)) { + existingClass = this.injectedDexClasses.get(normalizedName); + if (existingClass != null) { + return existingClass; + } + try { + return classLoader.loadClass(normalizedName); + } catch (ClassNotFoundException ignored) { + // fall through to the raw canonical lookup so the original + // failure (not a noise-from-the-fallback failure) is what + // propagates to the caller. + } + } + return classLoader.loadClass(canonicalName); } diff --git a/test-app/runtime/src/main/java/com/tns/Runtime.java b/test-app/runtime/src/main/java/com/tns/Runtime.java index bb8f3db19..77237b917 100644 --- a/test-app/runtime/src/main/java/com/tns/Runtime.java +++ b/test-app/runtime/src/main/java/com/tns/Runtime.java @@ -291,6 +291,20 @@ public static boolean getLogScriptLoadingEnabled() { } return false; } + + // Expose httpFetchUrlLog flag for native code without re-reading package.json. + // Default OFF (per-fetch log volume is high). Opt in via package.json + // "httpFetchUrlLog": true to diagnose HTTP module loader behavior. + public static boolean getHttpFetchUrlLogEnabled() { + Runtime runtime = com.tns.Runtime.getCurrentRuntime(); + if (runtime != null && runtime.config != null && runtime.config.appConfig != null) { + return runtime.config.appConfig.getHttpFetchUrlLog(); + } + if (staticConfiguration != null && staticConfiguration.appConfig != null) { + return staticConfiguration.appConfig.getHttpFetchUrlLog(); + } + return false; + } // Security config @@ -336,9 +350,9 @@ public static boolean isRemoteUrlAllowed(String url) { return true; } - // Check if URL matches any allowlist prefix + // Check if URL matches any allowlist entry on a component boundary for (String prefix : allowlist) { - if (url != null && prefix != null && url.startsWith(prefix)) { + if (url != null && prefix != null && remoteUrlMatchesAllowlistEntry(url, prefix)) { return true; } } @@ -346,6 +360,29 @@ public static boolean isRemoteUrlAllowed(String url) { return false; } + // Returns true when `url` is covered by allowlist `entry`, matching only on + // URL component boundaries (deny-by-default). Mirrors the native + // RemoteUrlMatchesAllowlistEntry in DevFlags.cpp. + private static boolean remoteUrlMatchesAllowlistEntry(String url, String entry) { + if (url == null || entry == null || entry.isEmpty()) { + return false; + } + if (url.length() < entry.length()) { + return false; + } + if (!url.startsWith(entry)) { + return false; + } + if (url.length() == entry.length()) { + return true; // exact match + } + if (entry.charAt(entry.length() - 1) == '/') { + return true; // entry ended at a boundary + } + char next = url.charAt(entry.length()); + return next == '/' || next == '?' || next == '#'; + } + /** * Returns the remote module allowlist as a String array for JNI. */