Skip to content

feat!: [codex] support polymorphic handles and configs - #917

Draft
voltjia wants to merge 1 commit into
masterfrom
codex/polymorphic-handle-config
Draft

feat!: [codex] support polymorphic handles and configs#917
voltjia wants to merge 1 commit into
masterfrom
codex/polymorphic-handle-config

Conversation

@voltjia

@voltjia voltjia commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Make Config and Handle polymorphic through virtual Clone() contracts and final-class CRTP helpers.
  • Give each operator an immutable owned Config clone while borrowing the per-call Handle through an RAII scope, so cache hits do not clone or allocate.
  • Add derived-config identity to cache keys, support config-aware operator constructors, and add a compile/link/run C++ smoke test for cloning, caching, lifetimes, and handle restoration.

Motivation

Operator-specific options currently have to be added to the shared src/config.h, and assigning a derived Config or Handle to OperatorBase slices its dynamic state. This change provides an extensible foundation so an operator can own its configuration type without expanding the shared base for every feature.

This PR is deliberately independent of #800 and contains no Triton/JIT changes.

Type of Change

  • feat - new extensibility API
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - focused C++ API coverage
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

The shared dispatch and public C++ types are used by every backend.

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Validated from commit f9a26ee6c95dc408d2cf9897d972149947c95420 in accelerator-dev/nvidia:latest, using the CPU backend and InfiniRT master 96e0613421f88e90959656764639c8a3282cb318:

cmake --build /workspace/build --target infiniops infinirt -j 16
# passed

cmake --install /workspace/build
# passed

python -m pip install . --no-build-isolation --no-deps \
  --target /workspace/python-env \
  --config-settings=cmake.define.INFINI_OPS_SMOKE_BUILD=ON \
  --config-settings=cmake.define.INFINI_RT_ROOT=/workspace/install \
  --config-settings=cmake.define.WITH_CPU=ON
# wheel built and installed

python -m pytest \
  tests/test_cpp_api.py::test_cpp_polymorphic_config_and_handle_smoke -q
# 1 passed in 1.81s

clang-format 21.1.8 --dry-run --Werror <changed C++ headers>
ruff 0.15.22 format --check tests/test_cpp_api.py
# passed

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
CPU yes core build and focused smoke passed focused C++ API test passed
NVIDIA yes pending CI shared dispatch header; backend not run locally
Iluvatar yes pending CI shared dispatch header; backend not available locally
MetaX yes pending CI shared dispatch header; backend not available locally
Cambricon yes pending CI shared dispatch header; backend not available locally
Moore yes pending CI shared dispatch header; backend not available locally
Ascend yes pending CI shared dispatch header; backend not available locally
Additional C++ API test output
python -m pytest tests/test_cpp_api.py -q
4 passed, 3 failed

The three failures occur before test execution because the generated public
operator instantiation header exposes existing deprecated Swiglu and
CausalSoftmax declarations while the smoke compiler uses -Werror. None of the
failing declarations are changed by this PR.

Benchmark / Performance Impact

N/A - this is not a performance PR.

A cache hit does not call Clone() and does not allocate. Relative to the current key construction, it adds dynamic-type hashing, one virtual cache_fingerprint() call, and hash combines. A cache miss or explicit Make() adds one allocation/copy for the owned Config. Normal invocation borrows the Handle; only the explicit owning set_handle() API clones it.

Notes for Reviewers

  • This is ABI-breaking: Config and Handle gain virtual tables, and OperatorBase changes layout and copy/move semantics. Downstream C++ consumers and plugins must rebuild.
  • Concrete subclasses should inherit CloneableConfig<T> or CloneableHandle<T> and be final. Clone validation rejects dynamic-type or observable-base-state loss.
  • Config::CacheFingerprint() must be stable and include every derived field that affects construction or invocation. Dynamic type and implementation index are included separately.
  • Operator configuration is initialized once. A config-aware implementation constructor receives the same owned clone later exposed through config(), preventing stale construction references.
  • Call() appends config identity even when a custom CacheKeyBuilder omits it; specialized sampling builders use the shared helper as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant