From c54334a0ecdd695b9e1d44dd4f1c76328dd9bb6a Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Mon, 6 Jul 2026 21:47:04 +0000 Subject: [PATCH 1/4] Document Flash local (non-pip) module bundling for endpoints Documents runpod/flash PR #352: automatic bundling of local Python modules that endpoints import. Adds an 'Import local modules' section to create-endpoints (supported import forms, transitive resolution, 8 MiB live cap, parent-directory constraint), a force-include note to the build CLI reference, and troubleshooting entries for the new resolution and payload-size errors. --- flash/cli/build.mdx | 6 ++++++ flash/create-endpoints.mdx | 16 ++++++++++++++++ flash/troubleshooting.mdx | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+) diff --git a/flash/cli/build.mdx b/flash/cli/build.mdx index 53b368844..9f3412c2f 100644 --- a/flash/cli/build.mdx +++ b/flash/cli/build.mdx @@ -76,6 +76,12 @@ Flash also respects your `.gitignore` file and excludes any files matching those If you use other environment file variants like `.env.dev` or `.env.staging`, add them to your `.gitignore` to exclude them from deployment artifacts. +### Local modules are force-included + +Local (non-pip) modules that your endpoints import are always bundled, even when an ignore pattern would otherwise exclude them. During the build, Flash resolves the import closure of each `@Endpoint` file and force-includes the files those imports depend on. For example, if an endpoint imports a sibling `test_helpers.py`, Flash bundles it even though the `test_*.py` ignore rule would normally drop it. + +If an `@Endpoint` file imports a local module that Flash can't resolve (such as a broken relative import or a file outside the project root), the build fails with a clear error instead of shipping a broken artifact. A file that fails resolution but doesn't define an endpoint is skipped with a warning. For details on how local imports are resolved and bundled, see [Import local modules](/flash/create-endpoints#import-local-modules). + ## Build artifacts After running `flash build`: diff --git a/flash/create-endpoints.mdx b/flash/create-endpoints.mdx index b52ff4963..77b56bb11 100644 --- a/flash/create-endpoints.mdx +++ b/flash/create-endpoints.mdx @@ -242,6 +242,22 @@ async def process_video(video_data): return {"processed": True} ``` +## Import local modules + +Your endpoint can import local (non-pip) Python modules that live alongside it in your project, such as a sibling `utils.py` file or a `helpers/` package. Flash detects these imports, follows them transitively, and ships the module source to the worker for you, so an import like `import utils` or `from helpers import load` works remotely with no extra configuration. + +Flash resolves local imports whether they appear at the top of the file or inside the function body, and it supports absolute imports (`import utils`), relative imports (`from . import helpers`), and dynamic imports with a literal name (`importlib.import_module("plugin")`). It also pulls in the `__init__.py` files for any packages you import. Flash can't resolve dynamic imports whose module name is computed at runtime, so it emits a warning and leaves those for you to make available on the worker. + +Flash bundles only local project files. Imports that resolve to the standard library or an installed pip package are left to the worker image, so continue to declare pip packages through the `dependencies` parameter. + +### Live execution size limit + +When you run an endpoint live (calling an `@Endpoint` function directly, or during `flash dev`), Flash ships the resolved module source inline with the request. The combined source is capped at 8 MiB. If your local dependencies exceed this limit, deploy the app with `flash deploy` instead, which bundles local modules into the build artifact rather than the request payload. + +### Keep endpoints at or above their local dependencies + +On the live execution path, Flash treats a module imported by absolute name from a parent directory as external and doesn't ship it, which causes a `ModuleNotFoundError` on the worker. For example, this happens with `import shared` when `shared.py` sits above your endpoint file. To avoid it, place your endpoint at or above its local dependencies, or use `flash deploy`, which resolves imports against the whole project directory. + ## Parallel execution Endpoint functions are async. Use Python's `asyncio` to run multiple operations concurrently: diff --git a/flash/troubleshooting.mdx b/flash/troubleshooting.mdx index a1c3c7f97..800d2487e 100644 --- a/flash/troubleshooting.mdx +++ b/flash/troubleshooting.mdx @@ -244,6 +244,23 @@ You have three options: Python 3.12 is recommended for best performance with no cold-start overhead. Python 3.10, 3.11, and 3.13 incur additional cold-start overhead on GPU workers because an alternative Python interpreter must be installed. +### Local module could not be resolved + +**Error:** +``` +endpoint.py: relative import (level=1, module='helpers') could not be resolved to a local file under /path/to/project +``` + +**Cause:** An `@Endpoint` file imports a local (non-pip) module that Flash can't resolve to a bundle-able file, such as a broken relative import, a missing package submodule, or a file outside the project root. Flash fails the build instead of shipping an artifact that would break on the worker. + +**Solution:** + +1. Fix the import so it points to a file that exists under your project root. +2. Move the imported module under the project directory if it lives outside it. +3. Confirm that package imports have an `__init__.py` file. + +For how Flash resolves and bundles local imports, see [Import local modules](/flash/create-endpoints#import-local-modules). + ## Deployment errors ### Tarball too large @@ -464,6 +481,25 @@ Payload size X MB exceeds limit of 10.0 MB 3. **Split large requests**: Break large datasets into smaller chunks and process them in multiple requests. +### Local module payload too large + +**Error:** +``` +Inline module payload is X bytes, over the 8388608-byte live-serverless cap. +Use `flash deploy` for endpoints with large local dependencies. +``` + +**Cause:** On the live execution path (calling an `@Endpoint` function directly or during `flash dev`), Flash ships your local module source inline with the request, and the combined source exceeds the 8 MiB cap. + +**Solution:** Deploy the app with `flash deploy`, which bundles local modules into the build artifact instead of the request payload: + +```bash +flash deploy + +# If using uv: +uv run flash deploy +``` + ### Deserialization timeout **Error:** From a58527358aaa99cc6e5a7d61fa0021c76711cbb3 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Mon, 20 Jul 2026 23:44:03 +0000 Subject: [PATCH 2/4] Correct and refine Flash local-module bundling docs Rewrite the build page's local-module section to match merged behavior (the build validates against the ignore filter and fails loudly on ignore-excluded or unresolvable endpoint imports; it does not force-include ignored files), clarify transitive pip dependencies, add the ignore-excluded LocalModuleResolutionError variant to troubleshooting, and tighten cross-links. Ref: runpod/flash#352 (SLS-360) --- flash/cli/build.mdx | 15 ++++++++++++--- flash/create-endpoints.mdx | 12 +++++++----- flash/troubleshooting.mdx | 17 ++++++++++++++++- 3 files changed, 35 insertions(+), 9 deletions(-) diff --git a/flash/cli/build.mdx b/flash/cli/build.mdx index 9f3412c2f..fd9972dde 100644 --- a/flash/cli/build.mdx +++ b/flash/cli/build.mdx @@ -76,11 +76,20 @@ Flash also respects your `.gitignore` file and excludes any files matching those If you use other environment file variants like `.env.dev` or `.env.staging`, add them to your `.gitignore` to exclude them from deployment artifacts. -### Local modules are force-included +### Local modules and the ignore filter -Local (non-pip) modules that your endpoints import are always bundled, even when an ignore pattern would otherwise exclude them. During the build, Flash resolves the import closure of each `@Endpoint` file and force-includes the files those imports depend on. For example, if an endpoint imports a sibling `test_helpers.py`, Flash bundles it even though the `test_*.py` ignore rule would normally drop it. +Local (non-pip) modules that your endpoints import are bundled automatically, provided they pass the ignore filter: your `.gitignore` plus the built-in patterns listed above. During the build, Flash resolves the transitive local-import closure of every shipped Python file and checks it against the files the ignore filter already selected. -If an `@Endpoint` file imports a local module that Flash can't resolve (such as a broken relative import or a file outside the project root), the build fails with a clear error instead of shipping a broken artifact. A file that fails resolution but doesn't define an endpoint is skipped with a warning. For details on how local imports are resolved and bundled, see [Import local modules](/flash/create-endpoints#import-local-modules). +If shipped code imports a local module that an ignore rule excludes (for example, a `test_*.py` sibling or a file under `tests/`), the build fails with a `LocalModuleResolutionError` instead of silently overriding your ignore rules or shipping a broken artifact. Flash names each excluded file and the file that imports it: + +``` +Shipped code imports local modules that the build ignore rules (.gitignore or built-in defaults) exclude: + utils/helpers.py (imported by endpoint.py) + +Shipping them would silently override a deliberate exclusion, and omitting them would break the worker with ModuleNotFoundError. Remove the matching ignore pattern or stop importing these modules from shipped code. +``` + +If an `@Endpoint` file has a local import Flash can't resolve at all (a broken relative import, or a file outside your project root), the build also fails with a clear error. A file that fails resolution but doesn't define an endpoint is skipped with a warning and the build continues. For details on how local imports are resolved and bundled, see [Import local modules](/flash/create-endpoints#import-local-modules). ## Build artifacts diff --git a/flash/create-endpoints.mdx b/flash/create-endpoints.mdx index 2c91ca7cd..3234e2f38 100644 --- a/flash/create-endpoints.mdx +++ b/flash/create-endpoints.mdx @@ -206,7 +206,7 @@ dependencies=["transformers==4.36.0", "torch>=2.0.0", "numpy<2.0"] ### Import packages inside the function body -You must import packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker. +You must import pip/installed packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker. This rule applies to installed packages only; local project modules can be imported at the top of the file because Flash ships their source (see [Import local modules](#import-local-modules)). **Correct:** imports inside the function. ```python @@ -246,15 +246,17 @@ async def process_video(video_data): Your endpoint can import local (non-pip) Python modules that live alongside it in your project, such as a sibling `utils.py` file or a `helpers/` package. Flash detects these imports, follows them transitively, and ships the module source to the worker for you, so an import like `import utils` or `from helpers import load` works remotely with no extra configuration. -Flash resolves local imports whether they appear at the top of the file or inside the function body, and it supports absolute imports (`import utils`), relative imports (`from . import helpers`), and dynamic imports with a literal name (`importlib.import_module("plugin")`). It also pulls in the `__init__.py` files for any packages you import. Flash can't resolve dynamic imports whose module name is computed at runtime, so it emits a warning and leaves those for you to make available on the worker. +Flash resolves local imports whether they appear at the top of the file or inside the function body, and it supports absolute imports (`import utils`), relative imports (`from . import helpers`), and dynamic imports with a literal name (`importlib.import_module("plugin")`). It also pulls in the `__init__.py` files for any packages you import. Flash can't resolve dynamic imports whose module name is computed at runtime, so it emits a warning, and you're responsible for making those modules available on the worker. -Flash bundles only local project files. Imports that resolve to the standard library or an installed pip package are left to the worker image, so continue to declare pip packages through the `dependencies` parameter. +Flash bundles only local project files. Standard library modules are already present in the worker image, and pip packages must still be declared through the `dependencies` parameter. This applies transitively: if a bundled local module imports a pip package at its top level, that package must still be declared in the `dependencies` of any endpoint that uses the module. + +On `flash build` and `flash deploy`, local modules are bundled when they pass the ignore filter, and importing a local module that an ignore rule excludes (or one Flash can't resolve) fails the build. See [`flash build` - Local modules and the ignore filter](/flash/cli/build#local-modules-and-the-ignore-filter) for details. ### Live execution size limit -When you run an endpoint live (calling an `@Endpoint` function directly, or during `flash dev`), Flash ships the resolved module source inline with the request. The combined source is capped at 8 MiB. If your local dependencies exceed this limit, deploy the app with `flash deploy` instead, which bundles local modules into the build artifact rather than the request payload. +When you run an endpoint live (calling an `@Endpoint` function directly, or during `flash dev`), Flash ships the resolved module source inline with the request. The combined source is capped at 8 MiB. If your local dependencies exceed this limit, deploy the app with `flash deploy` instead, which bundles local modules into the build artifact rather than the request payload. See [Local module payload too large](/flash/troubleshooting#local-module-payload-too-large) for the corresponding error. -### Keep endpoints at or above their local dependencies +### Local modules in a parent directory On the live execution path, Flash treats a module imported by absolute name from a parent directory as external and doesn't ship it, which causes a `ModuleNotFoundError` on the worker. For example, this happens with `import shared` when `shared.py` sits above your endpoint file. To avoid it, place your endpoint at or above its local dependencies, or use `flash deploy`, which resolves imports against the whole project directory. diff --git a/flash/troubleshooting.mdx b/flash/troubleshooting.mdx index eb5af586f..638d6ebb1 100644 --- a/flash/troubleshooting.mdx +++ b/flash/troubleshooting.mdx @@ -251,14 +251,27 @@ Python 3.12 is recommended for best performance with no cold-start overhead. Pyt endpoint.py: relative import (level=1, module='helpers') could not be resolved to a local file under /path/to/project ``` -**Cause:** An `@Endpoint` file imports a local (non-pip) module that Flash can't resolve to a bundle-able file, such as a broken relative import, a missing package submodule, or a file outside the project root. Flash fails the build instead of shipping an artifact that would break on the worker. +Or, for an ignore-excluded import: + +``` +Shipped code imports local modules that the build ignore rules (.gitignore or built-in defaults) exclude: + utils/helpers.py (imported by endpoint.py) + +Shipping them would silently override a deliberate exclusion, and omitting them would break the worker with ModuleNotFoundError. Remove the matching ignore pattern or stop importing these modules from shipped code. +``` + +**Cause:** This `LocalModuleResolutionError` has two variants: an `@Endpoint` file imports a local module that Flash can't resolve, or shipped code imports a local module that an ignore pattern excludes. **Solution:** +For an unresolvable import: + 1. Fix the import so it points to a file that exists under your project root. 2. Move the imported module under the project directory if it lives outside it. 3. Confirm that package imports have an `__init__.py` file. +For an ignore-excluded import, remove the matching ignore pattern, or stop importing that module from your shipped code. + For how Flash resolves and bundles local imports, see [Import local modules](/flash/create-endpoints#import-local-modules). ## Deployment errors @@ -500,6 +513,8 @@ flash deploy uv run flash deploy ``` +See [Live execution size limit](/flash/create-endpoints#live-execution-size-limit) for details on how local module source is shipped. + ### Deserialization timeout **Error:** From e8b677472d0ac1f39239ebd243c977e38b3a96aa Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Tue, 28 Jul 2026 08:11:34 +0000 Subject: [PATCH 3/4] Add release-notes entry for Flash local module bundling (v1.19.0) --- release-notes.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release-notes.mdx b/release-notes.mdx index ebb7c240a..192e4a1dc 100644 --- a/release-notes.mdx +++ b/release-notes.mdx @@ -13,6 +13,10 @@ rss: true +**July 28, 2026** + +

New Release [Automatic local module bundling for Flash endpoints](/flash/create-endpoints#import-local-modules)

Flash now automatically bundles the local Python modules your endpoints import — sibling files and packages that aren't installed via pip — and ships their source to the worker. Imports like `import utils` or `from helpers import x` now work remotely with no extra configuration. + **July 17, 2026**

New Release [Scale Instant Clusters - BETA](/instant-clusters/scale-clusters)

You can now add pods to a running Instant Cluster to increase GPU capacity without recreating it. New pods join the cluster's private network automatically, matching the existing GPU type, template, and network storage. Currently in beta for eligible accounts. From f4d34ce47990cc8d08345cc20617b22dd62ea8f0 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" Date: Fri, 7 Aug 2026 14:46:59 +0000 Subject: [PATCH 4/4] docs: remove inter-word dash connectors from draft prose Address PR review feedback (runpod/docs#775): replace the em-dash parenthetical connector in the July 28 release-notes entry with parentheses, and drop the 'flash build - ' connector prefix from the create-endpoints link text. Preserves compound modifiers, error-message strings, and code identifiers. --- flash/create-endpoints.mdx | 2 +- release-notes.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/flash/create-endpoints.mdx b/flash/create-endpoints.mdx index 3234e2f38..b2ce31eae 100644 --- a/flash/create-endpoints.mdx +++ b/flash/create-endpoints.mdx @@ -250,7 +250,7 @@ Flash resolves local imports whether they appear at the top of the file or insid Flash bundles only local project files. Standard library modules are already present in the worker image, and pip packages must still be declared through the `dependencies` parameter. This applies transitively: if a bundled local module imports a pip package at its top level, that package must still be declared in the `dependencies` of any endpoint that uses the module. -On `flash build` and `flash deploy`, local modules are bundled when they pass the ignore filter, and importing a local module that an ignore rule excludes (or one Flash can't resolve) fails the build. See [`flash build` - Local modules and the ignore filter](/flash/cli/build#local-modules-and-the-ignore-filter) for details. +On `flash build` and `flash deploy`, local modules are bundled when they pass the ignore filter, and importing a local module that an ignore rule excludes (or one Flash can't resolve) fails the build. See [Local modules and the ignore filter](/flash/cli/build#local-modules-and-the-ignore-filter) for details. ### Live execution size limit diff --git a/release-notes.mdx b/release-notes.mdx index 2acf88891..8818a6347 100644 --- a/release-notes.mdx +++ b/release-notes.mdx @@ -15,7 +15,7 @@ rss: true **July 28, 2026** -

New Release [Automatic local module bundling for Flash endpoints](/flash/create-endpoints#import-local-modules)

Flash now automatically bundles the local Python modules your endpoints import — sibling files and packages that aren't installed via pip — and ships their source to the worker. Imports like `import utils` or `from helpers import x` now work remotely with no extra configuration. +

New Release [Automatic local module bundling for Flash endpoints](/flash/create-endpoints#import-local-modules)

Flash now automatically bundles the local Python modules your endpoints import (sibling files and packages that aren't installed via pip) and ships their source to the worker. Imports like `import utils` or `from helpers import x` now work remotely with no extra configuration. **July 23, 2026**