Skip to content

python-sdk: Rust query engine M1 — algebra ADR, cldk-query-core, PyO3 adapter, fluent facade, dual backends #280

Description

@rahlk

PROBLEM

python-sdk has an approved fluent-query design (#212) but no engine. This issue delivers milestone 1 of the Rust query engine: a deferred, explainable, backend-neutral query core incubated in-repo as a Python-independent crate, surfaced through a fluent Python facade.

SCOPE BOUNDARY

No schema-v2 change; no analyzer work. Consumes schema 2.0.0 as-is. Cross-service vocabulary (services, gRPC/proto boundaries) is Epic E — no service()/rpc()/across_services() verbs here. Dataflow slicing stays on the existing cldk.graph engine for M1 (replacement is the follow-on child). Packaging/release-matrix work is the finishing child, not this one (local builds via maturin develop are enough here).

GOALS (the contract, as a checklist)

  1. Algebra ADR locked first: LogicalPlan vocabulary designed fresh from the 07-09 primitives + RFC sketch; recorded before core code lands.
  2. Root-level crates/ Cargo workspace (workspace Cargo.toml at repo root; polars/ruff idiom): cldk-query-core (no PyO3 anywhere in it) + cldk-python (bindings compiled to the private submodule cldk._native; maturin driven from the root pyproject via tool.maturin.manifest-path).
  3. Core reads schema-2.0.0 analysis.json via serde CPG models; identity is can:// ids end to end.
  4. Fluent chains build plans without executing; strict terminator boundary (.collect(), .uris(), .count(), .explain()); iterating a Query raises with a hint.
  5. .explain() renders initial + optimized plans, backend lowering, and any Opaque split points.
  6. Optimizer performs at least: filter pushdown, adjacent-traversal collapse, dead-branch elimination.
  7. Backends: in-memory (serde substrate) and Neo4j (core lowers to Cypher, executes over Bolt via neo4rs); identical results on both.
  8. Opaque(fn) plan-split semantics: Rust prefix → URIs → Python lambda → re-enter Rust.
  9. PlanEnvelope wire format (semver string): serialized plans round-trip and reconstruct without Python fluent objects.
  10. M1 gate: the two Odoo PoE audit queries (public_unauth_sudo, inapp_sudo_no_gate) run end-to-end on both backends and .explain() reproduces the manual audit evidence.

CAVEATS AND KNOWN RISKS

  • Algebra redesign is on the critical path — nothing in the core builds until the ADR is locked; timebox it.
  • neo4rs in-core widens the pre-extraction API surface; Bolt auth/session semantics must not leak into plan types.
  • The plan-split re-entry (goal 8) is the hardest boundary piece — a lambda between two Rust stages must not force full materialization of intermediate objects, only URIs.
  • Python-version support: PyO3 abi3 target must cover the SDK's supported range (3.11+).

DEFINITION OF DONE

  • M1 gate exact: both PoE queries return the hand-verified entity sets from the audit, not merely non-empty results; in-memory and Neo4j results byte-identical after URI sort.
  • cargo test green in cldk-query-core with no Python interpreter available.
  • Plan serialization round-trip test: chain → envelope → JSON → envelope → identical plan hash.
  • Facade conformance: every fluent verb maps to a registry entry; illegal verbs raise the registry diagnostic.

Part of #279

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions