feat: ESM resolver hardening, http loader, dev-mode globals

README.md

@@ -7,6 +7,7 @@ Contains the source code for the NativeScript's Android Runtime. [NativeScript](
 
 - [Main Projects](#main-projects)
 - [Helper Projects](#helper-projects)
+- [SBG vs Runtime Dex Generation](#sbg-vs-runtime-dex-generation)
 - [Architecture Diagram](#architecture-diagram)
 - [Build Prerequisites](#build-prerequisites)
 - [How to build](#how-to-build)
@@ -29,9 +30,115 @@ The repo is structured in the following projects (ordered by dependencies):
 
 ## Helper Projects
 
-* [**android-static-binding-generator**](android-static-binding-generator) - build tool that generates bindings based on the user's javascript code.
+* [**android-static-binding-generator**](android-static-binding-generator) - build tool that generates bindings based on the user's javascript code. See [SBG vs Runtime Dex Generation](#sbg-vs-runtime-dex-generation) for the production vs HMR-dev split.
 * [**project-template**](build-artifacts/project-template-gradle) - this is an empty placeholder Android Application project, used by the [NativeScript CLI](https://github.com/NativeScript/nativescript-cli) when building an Android project.
 
+## SBG vs Runtime Dex Generation
+
+The Android runtime turns every JavaScript `.extend('com.tns.Foo', Bar, { ... })`
+(or `Bar.extend({ ... })`) call into a real Java subclass with a dispatching
+proxy. There are **two** code paths that produce that subclass — a build-time
+path and a runtime path — and which one runs depends on whether the
+`.extend(...)` call site was visible to the Static Binding Generator (SBG)
+when the APK was built.
+
+### Build-time path (production / classic CLI)
+
+[**android-static-binding-generator**](android-static-binding-generator) (SBG)
+runs over the bundled JS once during `nativescript build android`:
+
+1. SBG parses every JS file the bundle includes and finds every
+   `.extend('com.tns.X', BaseClass, { ... })` call (and the modern
+   `BaseClass.extend({ ... })` shorthand).
+2. For each call it asks
+   [**android-binding-generator**](test-app/runtime-binding-generator)'s
+   `ProxyGenerator`/`Dump` to emit a `.dex` for a Java class named
+   `com.tns.gen.<BaseFqn-with-$-as-_>` (or, for `@JavaProxy(...)`-style
+   explicit names, the user-chosen name).
+3. The resulting dex files are packaged into the APK alongside the JS
+   bundle and listed in metadata so the runtime can find them via the
+   classloader.
+
+In production this means the Java class is already present and loadable
+the very first time the JS `.extend(...)` call runs. The runtime never
+generates a fresh dex.
+
+### Runtime path (HMR-dev / dynamic `.extend`)
+
+When SBG can't see the call at build time, the runtime generates the dex
+on demand. This happens in two common shapes:
+
+* **HMR/Vite dev workflow** — the build-time bundle (`bundle.mjs`) is just
+  the HMR bootstrap; modules like
+  `@nativescript/core/ui/frame/fragment.android.ts` are fetched over HTTP
+  at runtime. SBG never sees those `.extend(...)` calls, so no pre-baked
+  dex exists.
+* **`unnamed-extend` use cases** — `eval`-generated extends, or extends
+  whose first argument is computed at runtime, escape the SBG scan even
+  in production builds.
+
+The runtime path is wired through
+[`com.tns.ClassResolver`](test-app/runtime/src/main/java/com/tns/ClassResolver.java)
+→ [`com.tns.DexFactory`](test-app/runtime/src/main/java/com/tns/DexFactory.java):
+
+1. `ClassResolver.resolveClass` first tries `classStorageService.retrieveClass(name)`.
+   In production this hits the SBG-generated dex and we're done.
+2. On `LookedUpClassNotFound` (typical for HMR), if a `baseClassName` is
+   present, `ClassResolver` falls back to `DexFactory.resolveClass(...)`
+   which runs the same `ProxyGenerator`/`Dump` pipeline SBG uses — only
+   it does it at runtime, writes the dex into the app's per-thumb cache
+   under `<dexDir>/<name>.dex`, wraps it in a `.jar`, and loads it via
+   `DexClassLoader` (or `BaseDexClassLoader` injection when the
+   `injectIntoParentClassLoader` flag is on).
+3. Generated proxy class names normalize JVM inner-class `$` to `_`
+   (both in `Dump`'s class signature and in `DexFactory`'s
+   `loadClass(...)` arg). The actual JVM lookup name is always
+   `com.tns.gen.<base-with-$-as-_>` — never `com.tns.gen.<base>$<nested>`.
+
+### Edge cases worth knowing about
+
+* **`$` vs `_` mismatches** — `Class.forName(baseClassName)` requires JVM
+  `$` inner-class syntax (`android.app.Application$ActivityLifecycleCallbacks`),
+  but `classLoader.loadClass(generatedName)` requires the `_`-normalized
+  sibling (`com.tns.gen.android.app.Application_ActivityLifecycleCallbacks`).
+  Both `DexFactory.resolveClass` and `DexFactory.findClass` apply the
+  normalization in the loadable-name path while preserving `$` for the
+  reflective base-class lookup. Removing either normalization
+  reintroduces the "Didn't find class
+  `com.tns.gen.android.app.Application$ActivityLifecycleCallbacks`"
+  ClassNotFoundException for runtime-generated proxies.
+* **Cache invalidation** — the runtime dex cache is keyed on a per-build
+  `dexThumb` (the runtime regenerates it whenever the JS bundle changes).
+  Stale dex from a previous boot is purged in
+  `DexFactory.updateDexThumbAndPurgeCache`. Don't bypass the thumb — a
+  dex generated against an older base class can crash at method dispatch
+  time when the base API drifts.
+* **Parent-classloader injection** — Android framework code that calls
+  `Class.forName(name)` searches the app's `PathClassLoader`, *not* an
+  isolated `DexClassLoader`. When a generated proxy needs to be visible
+  to framework reflection (e.g. `FragmentFactory`, `Activity` resolution
+  from `AndroidManifest.xml`), construct the `DexFactory` with
+  `injectIntoParentClassLoader=true` so its `injectDexIntoClassLoader`
+  helper splices the generated jar's `dexElements` onto the parent's
+  `pathList`.
+* **Don't normalize synthetic module keys** — module registry keys like
+  `ns-vendor://...`, `optional:...`, `node:...`, `blob:...` are not
+  filesystem paths and must be preserved verbatim through invalidation
+  + reload. They are NOT the same kind of "synthetic name" as the
+  `com.tns.gen.<...>` class names above and don't go through this dex
+  pipeline.
+
+### What to look at when this breaks
+
+* `ClassResolver.java` — the fallback decision between
+  `classStorageService` and `DexFactory`.
+* `DexFactory.java` — runtime dex generation, cache, and the
+  `$`→`_` normalization.
+* `ProxyGenerator.java` + `Dump.java` (under `runtime-binding-generator`)
+  — what the generated class actually looks like in bytecode.
+* `android-static-binding-generator` — what SBG sees (and crucially
+  what it *doesn't* see) at build time.
+
 ## Architecture Diagram
 The NativeScript Android Runtime architecture can be summarized in the following diagram. 
 

test-app/app/src/main/assets/app/mainpage.js

@@ -77,3 +77,6 @@ require('./tests/testQueueMicrotask');
 require("./tests/testConcurrentAccess");
 
 require("./tests/testESModules.mjs");
+require("./tests/testNsDevBoundary.mjs");
+require("./tests/testHttpCanonicalKey.mjs");
+require("./tests/testNodeBuiltinsAndOptionalModules.mjs");

test-app/app/src/main/assets/app/tests/esm/meta-no-hot.mjs

@@ -0,0 +1,5 @@
+// Fixture for testNsDevBoundary: reports whether the runtime attached a
+// native `import.meta.hot` (it must NOT — hot contexts are injected by the
+// JS dev client via source rewrite, never by the runtime).
+export const hasHot = typeof import.meta.hot !== "undefined";
+export const hotValue = import.meta.hot;

test-app/app/src/main/assets/app/tests/testHttpCanonicalKey.mjs

@@ -0,0 +1,77 @@
+// HTTP canonical-key identity tests.
+//
+// Pins the behavior of the native CanonicalizeHttpUrlKey (the loader/registry
+// key) via the debug-only __NS_DEV__.canonicalizeHttpUrlKey diagnostic. Pure
+// string logic — no dev server required.
+//
+// Module identity IS the canonical URL: the dev server serves every module
+// under one URL and freshness is handled by __NS_DEV__.invalidateModules
+// (registry + prewarm-cache evict + fetch nonce), never by URL variation.
+// There is deliberately no path-tag vocabulary (__ns_boot__/__ns_hmr__)
+// to collapse, and no versioned-bridge-endpoint normalization.
+
+describe("HTTP canonical key", function () {
+    function canonFn() {
+        const dev = globalThis.__NS_DEV__;
+        return dev && typeof dev.canonicalizeHttpUrlKey === "function"
+            ? dev.canonicalizeHttpUrlKey
+            : null;
+    }
+    function canon(url) {
+        return canonFn()(url);
+    }
+
+    it("is available in dev builds", function () {
+        if (!canonFn()) {
+            pending("__NS_DEV__.canonicalizeHttpUrlKey not available (release build?)");
+            return;
+        }
+        expect(typeof canonFn()).toBe("function");
+    });
+
+    it("drops the fragment and unwraps file://http wrappers", function () {
+        if (!canonFn()) { pending("release"); return; }
+        expect(canon("http://h/ns/m/foo.js#frag")).toBe("http://h/ns/m/foo.js");
+        expect(canon("file://http://h/x.js")).toBe("http://h/x.js");
+    });
+
+    it("does NOT collapse versioned endpoint paths or path tags", function () {
+        if (!canonFn()) { pending("release"); return; }
+        // Path is identity — no /ns/rt/<v> → /ns/rt collapse, no
+        // __ns_hmr__/__ns_boot__ tag folding.
+        expect(canon("http://h/ns/rt/42")).toBe("http://h/ns/rt/42");
+        expect(canon("http://h/ns/rt/42/x.js")).toBe("http://h/ns/rt/42/x.js");
+        expect(canon("http://h/ns/m/__ns_hmr__/v7/foo.js"))
+            .toBe("http://h/ns/m/__ns_hmr__/v7/foo.js");
+    });
+
+    it("strips import/t/v markers and sorts remaining params on dev endpoints", function () {
+        if (!canonFn()) { pending("release"); return; }
+        expect(canon("http://h/ns/m/a?import=1&b=2&a=3")).toBe("http://h/ns/m/a?a=3&b=2");
+        expect(canon("http://h/ns/m/a?b=2&a=1")).toBe("http://h/ns/m/a?a=1&b=2");
+        expect(canon("http://h/ns/core?import=1")).toBe("http://h/ns/core");
+        expect(canon("http://h/ns/m/a?t=123&v=abc&x=1")).toBe("http://h/ns/m/a?x=1");
+    });
+
+    it("preserves the query verbatim on non-dev endpoints", function () {
+        if (!canonFn()) { pending("release"); return; }
+        // Public-internet module URLs: the query can be part of identity
+        // (auth, content versioning, routing) — only the fragment is dropped.
+        expect(canon("http://h/a?import=1&b=2&a=3")).toBe("http://h/a?import=1&b=2&a=3");
+        expect(canon("https://cdn.example.com/pkg.js?token=x#frag"))
+            .toBe("https://cdn.example.com/pkg.js?token=x");
+    });
+
+    it("preserves the t param on @ng/component endpoints", function () {
+        if (!canonFn()) { pending("release"); return; }
+        // Angular HMR component-update endpoint: `t` identifies a specific
+        // recompile and must remain a distinct registry entry.
+        expect(canon("http://h/ns/m/app/@ng/component?c=x&t=111"))
+            .toBe("http://h/ns/m/app/@ng/component?c=x&t=111");
+    });
+
+    it("leaves a non-http(s) specifier unchanged", function () {
+        if (!canonFn()) { pending("release"); return; }
+        expect(canon("~/local/foo.js")).toBe("~/local/foo.js");
+    });
+});

test-app/app/src/main/assets/app/tests/testNodeBuiltinsAndOptionalModules.mjs

@@ -0,0 +1,128 @@
+// Tests the ESM resolver's synthetic-module paths:
+// - node: built-in polyfills (in-memory ES modules)
+// - bare-specifier optional-module placeholders
+// - ns-vendor:// vendor-registry resolution via configureRuntime importMap
+// - blob: URL module re-use across imports
+//
+// The Android resolver's Node-builtin polyfill set is broader than iOS's;
+// only the explicitly-tested specifiers are asserted here.
+
+describe("Node built-in and optional module resolution", function () {
+  it("provides an in-memory polyfill for node:url", async function () {
+    const mod = await import("node:url");
+    const modAgain = await import("node:url");
+
+    expect(mod).toBeDefined();
+    expect(modAgain).toBe(mod);
+    expect(typeof mod.fileURLToPath).toBe("function");
+    expect(typeof mod.pathToFileURL).toBe("function");
+
+    const p = mod.fileURLToPath("file:///foo/bar.txt");
+    expect(p === "/foo/bar.txt" || p === "foo/bar.txt").toBe(true);
+
+    const u = mod.pathToFileURL("/foo/bar.txt");
+    expect(u instanceof URL).toBe(true);
+    expect(u.protocol).toBe("file:");
+  });
+
+  it("creates an in-memory placeholder for likely-optional modules", async function () {
+    // Use a name that IsLikelyOptionalModule will treat as optional
+    // (no slashes, no extension, no scope prefix).
+    const mod = await import("__ns_optional_test_module__");
+    const modAgain = await import("__ns_optional_test_module__");
+
+    expect(mod).toBeDefined();
+    expect(modAgain).toBe(mod);
+    expect(typeof mod.default).toBe("object");
+
+    let threw = false;
+    try {
+      // eslint-disable-next-line no-unused-expressions
+      mod.default.someProperty;
+    } catch (e) {
+      threw = true;
+    }
+    expect(threw).toBe(true);
+  });
+
+  it("resolves import-map vendor modules through the explicit vendor registry", async function () {
+    const dev = globalThis.__NS_DEV__;
+    const configureRuntime = dev && dev.configureRuntime;
+    if (typeof configureRuntime !== "function") {
+      pending("__NS_DEV__.configureRuntime not available");
+      return;
+    }
+
+    const previousRegistry = globalThis.__nsVendorRegistry;
+    const vendorRegistry = new Map();
+    globalThis.__nsVendorRegistry = vendorRegistry;
+    vendorRegistry.set("__ns_test_vendor__", {
+      default: { source: "vendor-default" },
+      namedValue: 7,
+      makeValue() {
+        return "vendor-named";
+      },
+    });
+
+    try {
+      configureRuntime({
+        importMap: {
+          imports: {
+            __ns_test_vendor__: "ns-vendor://__ns_test_vendor__",
+          },
+        },
+      });
+
+      const mod = await import("__ns_test_vendor__");
+      const modAgain = await import("__ns_test_vendor__");
+
+      expect(mod).toBeDefined();
+      expect(modAgain).toBe(mod);
+      expect(mod.default).toEqual({ source: "vendor-default" });
+      expect(mod.namedValue).toBe(7);
+      expect(mod.makeValue()).toBe("vendor-named");
+    } finally {
+      configureRuntime({ importMap: { imports: {} } });
+      if (typeof previousRegistry === "undefined") {
+        delete globalThis.__nsVendorRegistry;
+      } else {
+        globalThis.__nsVendorRegistry = previousRegistry;
+      }
+    }
+  });
+
+  it("reuses blob URL modules across concurrent and repeated imports", async function () {
+    delete globalThis.__nsBlobEvalCount;
+
+    const blobSource = [
+      "globalThis.__nsBlobEvalCount = (globalThis.__nsBlobEvalCount || 0) + 1;",
+      "export const evalCount = globalThis.__nsBlobEvalCount;",
+      "export const kind = 'blob-module';",
+      "export default { evalCount, kind };",
+    ].join("\n");
+
+    const url = URL.createObjectURL(new Blob([blobSource], { type: "text/javascript" }), {
+      ext: ".mjs",
+    });
+
+    expect(typeof url).toBe("string");
+    expect(url.indexOf("blob:nativescript/")).toBe(0);
+
+    try {
+      const [first, second] = await Promise.all([import(url), import(url)]);
+      const third = await import(url);
+
+      expect(first).toBeDefined();
+      expect(second).toBe(first);
+      expect(third).toBe(first);
+      expect(first.evalCount).toBe(1);
+      expect(second.evalCount).toBe(1);
+      expect(third.evalCount).toBe(1);
+      expect(first.kind).toBe("blob-module");
+      expect(globalThis.__nsBlobEvalCount).toBe(1);
+    } finally {
+      URL.revokeObjectURL(url);
+      delete globalThis.__nsBlobEvalCount;
+    }
+  });
+});