Skip to content

feat(plugin): provider functions can declare targets & provider_state - #165

Merged
raphaelvigee merged 1 commit into
masterfrom
feat/buildfile-plugin-declarations
Oct 9, 2026
Merged

raphaelvigee merged 1 commit into
masterfrom
feat/buildfile-plugin-declarations

Conversation

@raphaelvigee

@raphaelvigee raphaelvigee commented Jul 24, 2026 •

Copy link
Copy Markdown
Member

Why

A tool author who wants heph users to "pave" their tool shouldn't have to write a provider or a driver. Most such plugins are rules: they expand to a target(driver = "exec", …). BUILD files are already full Starlark, and providers already expose functions as heph.<provider>.<fn>(…). The missing piece: a provider function could return a value but could not declare a target.

What

  • ProviderFn::call returns an FnOutcome (crates/plugin/src/provider.rs): the value substituted at the call site plus any target() / provider_state() the call declared.
  • The buildfile provider merges declarations into the calling package (ProviderNativeFn::invoke) through the same sinks the builtins use, with call-site provenance stamped on each declared target.
  • Declarations cross the plugin ABI, so a cdylib provider function declares targets exactly as an in-process one does — and a function in one plugin can build its rule out of another provider's rule.

A declared target cannot carry what a hand-written one cannot

  • Same checks as target() through a shared validate_target_decl: non-empty name, valid labels. Keys target() consumes itself (name, driver, labels, transitive, approval) are rejected as driver config.
  • transitive is the untyped map target(transitive = …) takes, parsed by the same sandbox_from: relative addresses resolve against the calling package, every dep is hashed, ids are deterministic. A typed Sandbox would have let a plugin declare an unhashed transitive dep — a stale cache hit for every consumer.
  • Ambiguity is refused, not resolved. Declarations are sorted (targets by name, states by provider) because package order reaches downstream def hashes. One call declaring a name twice, or two states for one provider, is an error: state is read last-wins, so the winner would be whichever the function happened to yield first, and the package would be configured differently between runs of the same tree.
  • Everything is validated before anything is merged.

The ABI extension

CallFunctionResponse gained declared_targets / declared_states; both call requests gained accepts_declarations, which the caller sets to say it carries them back. ABI_SEMVER 0.14.0 → 0.15.0 — minor: additive wire fields, frozen surface untouched (scripts/abi-check.sh clean), but a callee must reach the version before declaring.

Skew turns on the flag, not the version. An older caller cannot send it, so it decodes as false — "cannot carry" — and a declaring function then fails the call. Otherwise prost would skip the fields, the caller would build a package the author never wrote, and hash it as if intended. All four pairings fail closed; ABI_VERSIONING.md documents the pattern, including that the flag's zero value must be the refusal.

The one hop nothing can police

A function relaying another provider function's outcome must pass its declarations on. FnOutcome's fields are private so the value comes out through absorb, which keeps them — but that is containment, not a guarantee, and the docs say so rather than overclaiming. Worth knowing the asymmetry: losing a relayed target fails loudly ("target not found"); losing a relayed state is silent, and the package caches as if the state had never been written.

BREAKING: Rust source only

ProviderFn::call returns Result<FnOutcome> instead of Result<Value>, and an FnOutcome is built with From<Value> + declare_target / declare_state rather than a struct literal. Accepted pre-1.0: every implementer is in-tree and updated here. Already-built cdylibs keep working — they just cannot declare.

Example

gen = heph.codegen.rule(name = "gen_a", srcs = glob("*.proto"))
target(name = "use", driver = "exec", deps = [gen])

Tests

  • crates/e2e — the joint: a declaration made behind the ABI (make_dyn_provider + StableRemoteProvider, the path a loaded cdylib takes) becomes a target the engine resolves and builds. Verified load-bearing: with accepts_declarations flipped to false it fails.
  • plugin-sdk (--features stabby) — every declaration field survives the seam; the refusal when the caller cannot carry them, driven at the guest serve path; declarations crossing the reverse (plugin→host) direction; and the plugin→provider→plugin relay chain, the case this exists for.
  • plugin-stabby — the host direction at the wire: a declaring host function answers a plugin that carries declarations, and errors for one that does not.
  • plugin-abi — lossless round trip compared structurally with several keys per map; and an absent accepts_declarations, encoded with shadow pre-0.15.0 schemas, reads as a refusal and is answered with an error.
  • plugin-buildfile — declared target, state and returned address land in the package beside hand-written targets; declared transitive equals the hand-written equivalent's; ordering; and loud failures for empty name, invalid label, reserved config key, duplicate name, duplicate state, state without a provider.
  • lint, feature-gated clippy and the ABI guard are clean.

Not in bin-e2e: the declarations ride inside the existing prost payload over unchanged vtable slots, so get_stabbied's type report is byte-identical and dlopen has nothing new to reject. A separate process would buy separate compilation and nothing else.

Follow-ups

  • A concrete authoring surface on top of this.
  • A package-wide policy for duplicate target names, which target() doesn't reject either. This PR only rejects duplicates within one declaring call.

🤖 Generated with Claude Code

@raphaelvigee
raphaelvigee force-pushed the feat/buildfile-plugin-declarations branch 2 times, most recently from 510c371 to 931225a Compare August 1, 2026 21:55
@raphaelvigee
raphaelvigee force-pushed the feat/buildfile-plugin-declarations branch from 931225a to 809b6eb Compare September 19, 2026 16:29
@raphaelvigee
raphaelvigee force-pushed the feat/buildfile-plugin-declarations branch from 809b6eb to f34d1d6 Compare October 9, 2026 09:26
A provider function (surfaced as `heph.<provider>.<fn>` in BUILD files)
could previously only return a value. It can now also declare `target()`
and `provider_state()` via a new `FnOutcome` return bundle, which the
buildfile provider merges into the calling package as if hand-written.

This is the "build-file plugin" primitive: a wrapper (e.g. a codegen rule
around the `exec` driver) stands up targets for its caller, instead of
the author shipping a provider or driver of their own.

A declared target cannot carry what a hand-written one cannot:
- same checks as `target()` (name, label grammar) via `validate_target_decl`;
  keys `target()` consumes itself are rejected as driver config;
- `transitive` is the untyped map `target(transitive=…)` takes, parsed by
  the same `sandbox_from` — package-relative addresses, every dep hashed,
  deterministic ids. A typed `Sandbox` would have let a function declare an
  unhashed transitive dep: a stale cache hit for every consumer;
- declarations are sorted (targets by name, duplicates rejected; states by
  provider) before merging, since package order reaches downstream def
  hashes and a function iterating a `HashMap` must not flap cache keys;
- everything is validated before anything is merged.

Declarations cross the plugin ABI, so a cdylib provider function declares
targets exactly as an in-process one does, and a function in one plugin
can build its rule out of another provider's rule.
`CallFunctionResponse` gained `declared_targets`/`declared_states` and
both call requests gained `accepts_declarations`, which the *caller* sets
to say it carries them back. ABI_SEMVER 0.14.0 -> 0.15.0 (minor: additive
wire fields, frozen surface untouched, but a callee must reach it before
declaring). Skew turns on the flag rather than the version: an older
caller cannot send it, false means "cannot carry", and a declaring
function then fails the call. Prost would otherwise skip the fields and
the caller would build a package the author did not write, and hash it as
if intended.

One call may not declare the same target name twice, nor two states for one
provider: state is read last-wins, so the winner would be whichever the
function happened to yield first — a package configured differently between
runs of one tree, each hashing as if intended.

`FnOutcome`'s fields are private. The one hop no guard can police is a
function relaying *another* provider function's outcome: its declarations
reach the package only if passed on, so the value comes out through
`absorb` (which keeps them) rather than a field read that would discard
them unremarked.

BREAKING (Rust source, plugin author surface): `ProviderFn::call` returns
`anyhow::Result<FnOutcome>` instead of `anyhow::Result<Value>`, and
`FnOutcome` is built with `From<Value>` + `declare_target`/`declare_state`
rather than a struct literal. Accepted pre-1.0: every implementer is
in-tree and updated here. Already-built cdylibs are unaffected by the
source change, and a 0.14.0 cdylib keeps working — it just cannot declare.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@raphaelvigee
raphaelvigee force-pushed the feat/buildfile-plugin-declarations branch from f34d1d6 to c99be55 Compare October 9, 2026 10:10
@raphaelvigee
raphaelvigee merged commit 407d305 into master Oct 9, 2026
23 checks passed
@raphaelvigee
raphaelvigee deleted the feat/buildfile-plugin-declarations branch October 9, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant