Relocatable _build for fast git worktrees (parallel coding agents): move a build instead of recompiling

Coding agents increasingly work on a Phoenix app in parallel, each in its own git worktree (Claude Code --worktree and subagents with worktree isolation, Conductor, worktrunk, git-cow). On copy-on-write filesystems (APFS clonefile, btrfs/XFS reflinks) a new worktree can get _build and deps from the main checkout for free, in milliseconds. Since Elixir v1.15 Mix then recompiles the whole project anyway, because the manifest's cwd is part of the compiler cache key (elixir-lang/elixir#12462): every .beam names its source by absolute path, and __DIR__ is absolute too. elixir-lang/elixir#13241 discussed this for CI caches; --no-check-cwd (v1.19.2) skips the check but leaves the stale paths in place (stack traces, ex_doc links, __DIR__).

I measured what it takes to move a build correctly instead, on a Phoenix 1.8 app (67 modules plus a path dependency, Elixir 1.20.4, OTP 28):

Full recompile after the move: 3.8 s, the dominant cost of creating a worktree.
Rewriting the old root to the new one instead: 0.14 s, after which Mix compiles nothing except the embed_templates modules (below). The paths live in a small, known set of places:
- the compile manifest: cwd and the local deps config
- .beam chunks: CInf (:source), Dbgi (:file), Attr (e.g. @external_resource), LitT (compile-time literals such as __DIR__)
- the .app file (compile-time config computed from __DIR__)
After that, module_info(:compile)[:source], stack traces, Tidewave and LiveDebugger all point into the new checkout. It is ~100 lines of :beam_lib.
The remaining recompiles come from embed_templates: __mix_recompile__?/0 compares an md5 of the templates' absolute paths (__mix_recompile__?/0 hashes absolute template paths, so a moved or cloned project recompiles every embed_templates module phoenix_template#15, with a two-line fix).

End to end, a new worktree with its own database (PostgreSQL 18 CREATE DATABASE ... STRATEGY FILE_COPY with file_copy_method = clone) and its own mix phx.server is serving in ~2.8 s instead of ~35 s.

Ideas, in order of ambition (most of it lives in Mix, but Phoenix apps are where this hurts most):

mix compile --relocate-from OLD_ROOT, or relocation done automatically when only the cwd changed: Mix rewrites the paths above instead of recompiling. Everything it needs is already in the manifest.
Exact relocation: the compiler knows when it expands __DIR__ / __ENV__.file; recording that per module would let Mix recompile only the modules that captured the old root in a way it cannot rewrite.
Longer term: store source paths relative to the project root in .beam files (as :relative_paths already does for quoted nodes and diagnostics) and resolve them at runtime.
Phoenix side: path-independent recompile checks (phoenix_template#15), and perhaps generator defaults that avoid baking __DIR__-derived absolute paths into compile-time config.

This would make worktree-per-agent setups start in a couple of seconds instead of paying a full compile per task, and would also help CI caches and container builds that move _build between paths. Happy to work on PRs for whichever of these you'd accept.

I think there's not much to do on the Phoenix side after phoenixframework/phoenix_template#16?

@onnimonni

Ideas, in order of ambition (most of it lives in Mix, but Phoenix apps are where this hurts most):
4. Phoenix side: path-independent recompile checks (phoenix_template#15), and perhaps generator defaults that avoid baking __DIR__-derived absolute paths into compile-time config.

I think there's not much to do on the Phoenix side after phoenixframework/phoenix_template#16?

I think there are only two things left, :esbuild and :tailwind configs in config/config.exs.

They depend on __DIR__ for :cd and :env/NODE_PATH.

Details

$ git grep -nA9 'config :tailwind' -- installer
installer/templates/phx_single/config/config.exs.eex:60:config :tailwind,
installer/templates/phx_single/config/config.exs.eex-61-  version: "4.3.3",
installer/templates/phx_single/config/config.exs.eex-62-  <%= @app_name %>: [
installer/templates/phx_single/config/config.exs.eex-63-    args: ~w(
installer/templates/phx_single/config/config.exs.eex-64-      --input=assets/css/app.css
installer/templates/phx_single/config/config.exs.eex-65-      --output=priv/static/assets/css/app.css
installer/templates/phx_single/config/config.exs.eex-66-    ),
installer/templates/phx_single/config/config.exs.eex-67-    cd: Path.expand("..<%= if @in_umbrella, do: "/apps/#{@app_name}" %>", __DIR__),
installer/templates/phx_single/config/config.exs.eex-68-    env: %{"NODE_PATH" => [Path.expand("../deps", __DIR__), Mix.Project.build_path()]}
installer/templates/phx_single/config/config.exs.eex-69-  ]<% end %>
--
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex:39:config :tailwind,
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-40-  version: "4.3.3",
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-41-  <%= @web_app_name %>: [
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-42-    args: ~w(
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-43-      --input=assets/css/app.css
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-44-      --output=priv/static/assets/css/app.css
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-45-    ),
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-46-    cd: Path.expand("../apps/<%= @web_app_name %>", __DIR__),
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-47-    env: %{"NODE_PATH" => [Path.expand("../deps", __DIR__), Mix.Project.build_path()]}
installer/templates/phx_umbrella/apps/app_name_web/config/config.exs.eex-48-  ]<% end %>

Both :cd and :env are passed to System.cmd -> Port.open

cd -> C's chdir: either absolute or relative path would work. Need to check the cwd guarantees when the System.cmd calls are done.
Details
env -> NODE_PATH is supposed to contain absolute paths. But we may be able to move away from it, e.g. using symlinks in node_modules -- need to check the Windows story.
Details
- https://nodejs.org/docs/latest/api/cli.html#node_pathpath
- https://nodejs.org/docs/latest/api/modules.html#loading-from-the-global-folders
- NODE_PATH deprecation (TC voted negative) nodejs/node#1627
- an aside, NODE_PATH is not used to resolve ES module imports -- https://nodejs.org/docs/latest/api/esm.html#no-node_path
The paths we add to NODE_PATH:
- deps: this is to make phoenix, phoenix_live_view, phoenix_html, and daisyui available to JS/CSS assets without requiring npm and node_modules. They get there via mix deps.get.
- Mix.Project.build_path(): for colocated assets generated into _build/<env>/phoenix-colocated/<app_name>.

@onnimonni would you like to investigate this further?

@rhcarvalho

Follow-up after looking into the remaining Phoenix-side pieces. I tested a relocated copy (APFS clone of the project including _build, compiled with check_cwd: false).

Still broken after phoenix_template#16, now with PRs:

Store embed_templates root relative to cwd for __mix_recompile__? phoenix_template#17: __mix_recompile__? still globs the absolute root captured at compile time. A template added in the copy was silently never compiled (function_exported?(MyAppWeb.Layouts, :extra, 1) == false) until something else touched the module. The fix stores the root relative to cwd. The new test fails on main.
Store component call file relative to cwd phoenix_live_view#4489: component call warnings (missing/unknown attrs, emitted from the caller's @after_verify) printed the original checkout's absolute paths. Example: └─ /original/checkout/lib/my_app_web/live/foo_live/form.ex:24. An editor or agent working in the worktree then edits the wrong copy.
Store ~p warning file relative to cwd #6889: same issue for ~p route warnings (warn_location).

@rhcarvalho, on :esbuild / :tailwind __DIR__ configs: I don't think they need changes for this use case. config/config.exs is evaluated on every Mix invocation, so in the relocated copy Application.get_env(:tailwind, :my_app)[:cd] and NODE_PATH already point at the new directory. Nothing from them ends up in the .app file or in compiled modules. They would only matter if the evaluated config were persisted, e.g. in a release built in one place and run from another.

Remaining absolute paths in a relocated build:

CInf source in every module: IEx open, ex_doc.
LiveView debug_heex_annotations caller data ({module, fun, file, line} literals in every LiveView, used for editor jump links).
Path-capturing literals from user code (__DIR__).

I'm prototyping idea 1/2 in Mix:

On a cwd change, rewrite the old root in the CInf/Dbgi/Docs chunks.
Recompile only the modules whose literals, atoms, attributes or line table mention the old root.

I'll post numbers when it works.

Follow-up with numbers for ideas 1/2: relocation in Mix, plus making every code-level path relative.

Libraries: no compiled code captures the project root anymore. Besides the three PRs above, these PRs remove the remaining absolute paths from compiled code:

Store component call file relative to cwd phoenix_live_view#4489: now also covers the file in the component caller tuple used for stack traces, and collocated templates' @external_resource.
Store embed_templates root relative to cwd for __mix_recompile__? phoenix_template#17: now also stores @external_resource relative.
Store query expression files relative to the current directory elixir-ecto/ecto#4811: query expressions stored the caller's absolute file for Ecto.QueryError, at 28 places in the query builders.

With all of them, a scan of the .beam chunks of a generated app (phx.new, phx.gen.auth, 40 phx.gen.live resources, 195 modules) finds the absolute root only in compile metadata: CInf :source and Dbgi.

Mix relocation prototype (onnimonni/elixir@relocate-build, one commit on Elixir main, ~135 lines in Mix.Compilers.Elixir plus 2 tests):

When the cwd changes, Mix no longer recompiles everything. It rewrites the old root in the chunks that only describe the source (CInf, Dbgi, Docs).
Modules whose other chunks mention the old root (literals, atoms, attributes, line table) captured it, e.g. through __DIR__, and only those are recompiled.
--no-relocate opts out.
Afterwards module_info(:compile)[:source] points into the new checkout.

New worktree (APFS clone of the project including _build), 5 interleaved rounds:

	time	modules recompiled	paths afterwards
today (cwd check → full recompile)	1.17s	195	correct
relocation	0.42s	0	correct
`--no-check-cwd`	0.34s	0	metadata points at the old checkout

Without the library PRs, 131/195 modules capture the root (LiveView component calls), and relocation only saves ~5%.

I didn't try making :source/debug info relative (idea 3). It runs into what was discussed in elixir-lang/elixir#14744: dependencies compile with their own cwd, escripts/archives need absolute paths, and cover/IEx open/ex_doc read :source. Relocation fixes the reason for the cwd check (elixir-lang/elixir#12462, stale :source) for ~80ms. If that direction sounds acceptable, I'll open an Elixir issue with the patch.

@onnimonni

Status:

Two open PRs in LiveView
No changes required in :esbuild and :tailwind configs in the mix phx.new installer templates
Other ideas are related to Elixir upstream

If that direction sounds acceptable, I'll open an Elixir issue with the patch.

If you think getting input from other folks in the community would be beneficial to set direction, I'd suggest following https://github.com/elixir-lang/elixir#proposing-new-features and starting a discussion in the mailing list. For small and obvious wins, I think focused PRs are likely to be welcomed by the Elixir team.

I'll close this as the part related to Phoenix seems to be complete. Thanks @onnimonni!

Navigation Menu

Repository navigation

Relocatable _build for fast git worktrees (parallel coding agents): move a build instead of recompiling #6855

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Relocatable _build for fast git worktrees (parallel coding agents): move a build instead of recompiling #6855

Description

Activity

SteffenDE commented on Oct 1, 2026

rhcarvalho commented on Oct 14, 2026

onnimonni commented on Oct 31, 2026

onnimonni commented on Nov 1, 2026

rhcarvalho commented on Nov 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions