Skip to content

feat(bundles): submit a stack together with the files it reads - #70

Merged
martypitt merged 1 commit into
mainfrom
feat/stack-bundles
Sep 22, 2026
Merged

martypitt merged 1 commit into
mainfrom
feat/stack-bundles

Conversation

@martypitt

@martypitt martypitt commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

The problem

A stack script that reads a file — s3 { bucket("tickets") { file(path) } }, or API Gateway's upcoming restApiFromFile(...) — could only read it from the Nebula server's own filesystem. The project that defines the stack lives somewhere else, so the path pointed at a machine where the file does not exist. Stacks that read files were only usable if someone had put the file on the server by hand.

What this adds

A stack can now be submitted as a bundle: the script plus a map of relative path → content. Nebula unpacks the resources into a directory belonging to that submission, and the stack reads them through resources inside stack { }.

stack {
   val sales = resources.readText("data/sales.csv")

   s3 {
      bucket("tickets") {
         file("data/sales.csv")       // resolved through the bundle
      }
   }
}

API

Additive throughout — nothing existing changed shape.

  • POST /stacks/bundle and PUT /stacks/bundle/{id} sit alongside the script-only routes.

  • The /stream/stacks message gained a bundles field next to stacks:

    { "stacks":  { "films": "stack {}" } }
    { "bundles": { "films": { "script": "stack {}", "resources": { "data/sales.csv": { "content": "...", "encoding": "TEXT" } } } } }

    Both default to empty and normalise to the same type, so an Orbital that predates bundles keeps working. A bundle with no resources behaves exactly as a script-only submission does — no directory is created.

Resources are text by default, with a BASE64 flag for binary content. Orbital only sends text, because a project's files reach it through the schema pipeline as UTF-8 strings.

Safety

Every resource lookup is confined to the bundle directory:

  • absolute paths rejected (including Windows-style ones on a unix host),
  • .. segments rejected,
  • the resolved path is toRealPath()'d — following symlinks — and re-checked to still be inside the bundle.

The same check runs at unpack time against the entry names, so a ../../etc/cron.d/evil entry is refused before anything is written (zip-slip), and a rejected bundle leaves nothing on disk. Size limits — 10MB per file, 50MB per bundle, 500 files — are checked against the decoded bytes before any file is created.

This confines resource access. A Nebula script is arbitrary Kotlin and can always open a file directly; the guarantee is that a bundle-relative path resolves inside the bundle. That is what protects against a malformed or hostile resource name, which is the part that comes from outside.

Lifecycle

StackRunner now dedupes on the script and the resource content, so resubmitting an unchanged script with an edited CSV restarts the stack rather than being skipped as a duplicate. An unpacked bundle is deleted when the stack is replaced or removed, and when a submission is rejected. Stopping a stack keeps its bundle, since a stopped stack can be started again.

Tests

61 passing across nebula-dsl (41) and nebula-runtime (20, 1 pre-existing skip):

  • StackResourcesTest — the containment rules, including a symlink planted inside a bundle that points at a file outside it, and a symlinked directory.
  • StackBundleUnpackerTest — hostile entry names, partial bundles leaving nothing behind, size limits, fingerprints.
  • StackBundleExecutionTest — a bundle compiled through the real scripting host, running, and serving a file that shipped with it. This is the one that fails if the bundle is unpacked but never reaches the script.
  • StreamStacksBundleTest — the same over the websocket Orbital actually uses.
  • UpdateStackRequestCompatibilityTest — both message shapes deserialise and normalise identically.

Existing nebula-runtime and non-container nebula-dsl tests pass unchanged. S3ExecutorTest has one failure on my machine from Docker network-pool exhaustion (all predefined address pools have been fully subnetted), unrelated to this change — the other two in that spec pass.

Docs

docs/pages/bundles.mdx (new), plus §8 of NEBULA-AGENT.md, the S3 provider page, and CLAUDE.md. The @orbital/nebula convention in NEBULA-AGENT.md widens from orbital/nebula/*.nebula.kts to orbital/nebula/**, so the files beside a script reach Orbital in the first place.

Companion change

Orbital side: feat/nebula-stack-bundles — https://gitlab.com/vyne/vyne/-/merge_requests/2231.

🤖 Generated with Claude Code

https://claude.ai/code/session_011c7t8dqC37eQzTt9o44x9v

A stack script that needs an OpenAPI spec or a CSV could only read them
from the Nebula server's own filesystem, which is a different machine
from the project that defines the stack. Stacks that read files were
therefore only usable when someone had put the file on the server by
hand.

A stack can now be submitted as a bundle: the script plus a map of
relative path -> content. Nebula unpacks the resources into a directory
belonging to that submission, and the stack reads them through
`resources` inside `stack { }`. Providers that read a file (`s3 {
file(path) }`, and API Gateway's `restApiFromFile` when it lands)
resolve a relative path through the bundle; an absolute path keeps its
existing meaning, so stacks written before bundles are unaffected.

Additive throughout:

- `POST /stacks/bundle` and `PUT /stacks/bundle/{id}` sit alongside the
  script-only routes, which are unchanged.
- The `/stream/stacks` message gained a `bundles` field next to
  `stacks`. Both default to empty and normalise to the same thing, so an
  Orbital that predates bundles keeps working, and a bundle with no
  resources behaves exactly as a script-only submission does.

Every resource lookup is confined to the bundle directory: absolute
paths and `..` segments are rejected, and the resolved path is turned
into a real path (following symlinks) and re-checked to still be inside
the bundle. The same check runs at unpack time, so a `../../etc/cron.d`
entry is refused before anything is written. Size limits (10MB per file,
50MB per bundle, 500 files) are checked against the decoded bytes before
any file is created, and a rejected submission leaves nothing on disk.

Resources are text by default, with a BASE64 encoding flag for binary
content. Orbital only sends text, because the files reach it through the
schema pipeline, which carries additional sources as text.

StackRunner now dedupes on the script *and* the resource content, so
editing a CSV next to an unchanged script restarts the stack rather than
being skipped as a duplicate. A stack's unpacked bundle is deleted when
it is replaced or removed; stopping a stack keeps it, since a stopped
stack can be started again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011c7t8dqC37eQzTt9o44x9v
@netlify

netlify Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for orbital-nebula ready!

Name Link
🔨 Latest commit 0ab698b
🔍 Latest deploy log https://app.netlify.com/projects/orbital-nebula/deploys/6ab22a57e9dcd10007d96360
😎 Deploy Preview https://deploy-preview-70--orbital-nebula.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@martypitt
martypitt merged commit 738deb9 into main Sep 22, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant