Add: kernel callable residency cache and generation guards - #2190
Draft
YunjiQin wants to merge 3 commits into
Draft
Add: kernel callable residency cache and generation guards#2190YunjiQin wants to merge 3 commits into
YunjiQin wants to merge 3 commits into
Conversation
Kernel mode is simpler's second execution identity: instead of owning
the device, a context borrows the caller's already-current device and
stream to enqueue one bounded asynchronous operator per launch, so a
PyPTO program is capturable by ACLGraph as an ordinary node. This
change freezes the public surface that identity hangs off and gives the
context a write-once identity the guards can key on. It creates no
resources. The program path gains one call — simpler_init latches
PROGRAM — and no behavior: latching a fresh context always succeeds, is
idempotent, and nothing on that path reads the latch.
- runtime_c_api.h declares the lifecycle entries
simpler_kernel_mode_{supported,init,prepare_callable,launch} and adds
the host-band code PTO_RUNTIME_ERR_INVALID_STATE. The existing
finalize_device stays the fifth lifecycle entry, and a kernel context
now reaches it. Kernel-mode capacity is
a mode invariant rather than a gated state: config is context-static,
so each pooled arena region is committed at most once, and
setup_static_arena reports a grow or release request on a committed
region under kernel mode as an internal invariant break; capacity
intent travels in CallConfig.runtime_env like everywhere else.
- Execution identity is a write-once property of the context rather
than a state that evolves. ExecutionModeLatch (platform/include/host/
execution_mode_latch.h) replaces the four-state claim: the first init
entry to run latches the mode, and it never changes — not on finalize,
not on error. simpler_init latches PROGRAM before touching any
process or runner state, so the program/kernel mutual exclusion is
enforced on every program init instead of resting on a separate
declaration call. There is no unlatch, which the latch documents as a
consequence: a handle from a failed kernel init can never be recycled
into a program context. SimplerExecutionMode now has one definition
(task_interface/execution_mode.h) that both the wire header and the
latch consume, so the host-side identity and the value that travels to
the AICPU can no longer disagree.
- device_id_ records which device a context is on, not a claim on it —
ownership is what the latch carries. attach_current_thread splits
accordingly: bind_current_thread does the per-thread rtSetDevice and
nothing else; attach_current_thread is the program-mode adopt
(bind plus the one-shot op-execute watchdog and identity write) and
refuses on a kernel latch; adopt_borrowed_device records the device a
kernel context runs on without binding the thread and without
configure_aicore_op_timeout, whose aclrtSetOpExecuteTimeOutV2 would
rewrite the watchdog for every other user of a borrowed card. It does
resolve the timeout config, because the stream and scheduler timeouts
derived from it are read on both identities. DeviceRunner::finalize()
is the one caller that runs under both identities and skips the bind
on a kernel latch, so the kernel close path reaches its no-reset
branch instead of being turned away by a device bind it never needed.
- ensure_acl_ready(), force_reset_device(), and finalize()'s rt-layer
device reset refuse on a kernel-mode context (a2a3 + a5): the ACL
lifecycle belongs to the caller, and every call site of the five ACL
lifecycle APIs falls into three enumerable classes (below the
ensure_acl_ready guard, inside force_reset_device behind its own
guard, or gated on acl_ready_ which only the guarded path sets), with
finalize's rt-layer reset intercepted by its own kernel-mode branch —
so poison recovery can never reset the device out from under the
host process.
- kernel_invocation_header.h pins the envelope every kernel launch
ships to the AICPU (mode / callable / generation / payload length /
int32_t arg counts). Both sides of the wire come from one
build_runtimes.py build, so the struct carries no version or size
negotiation and the POD/standard-layout guards are its only
compile-time checks. generation is the occupancy counter of the
residency slot callable_id resolves to - a property of the slot, not
of the callable in it, so a generation carried by the callable could
not detect slot reuse - with zero reserved for "not recorded".
ChipCallable's sig_count includes the scalar entries and its
scalar_count reads 0 both for a scalar-free orchestration and for an
artifact built before the field existed, so a consumer derives the
effective scalar count - the field when nonzero, otherwise the
signature's SCALAR entries, the split count_callable_tensor_args
already computes - and compares tensor_count against sig_count minus
it. Subtracting the field directly would count an unrecorded
callable's scalars as tensors.
- Kernel-entry argument validation is shared by all eight host-runtime
components through kernel_entry_validation.h (one copy of the
null/range/image-size/alignment checks; a binary pointer and its size
must be present or absent together, and a callable image must be
aligned for ChipCallable so its CALLABLE_CHILD_ALIGN-relative storage_
lands aligned too), so a stub and a real implementation accept and
reject exactly the same arguments.
- KernelExecutionState and ExecutionModeClaimState carry the kernel
context phase machine (New/Collecting/ReadyEnqueued/Poisoned/
Closing/Closed with sticky, retriable Closing and separate
runtime-error and teardown-error slots) and the two restricted
operation vocabularies; synchronize, allocation, capture queries,
and model attachment stay unrepresentable in those tables, and a
launch implementation is obligated to route through them. Every
kernel-mode guard reads the identity through ExecutionModeLatch::
is_kernel() rather than comparing an enumerator at the call site, so
the test lives in one place instead of eight.
- ChipWorker dlsyms the four new symbols from every runtime, so a
component missing one fails at load, and clears them alongside the
other resolved pointers on all three teardown paths so none is left
dangling into the library DlHandleGuard dlcloses.
test_host_runtime_abi.py
asserts the export across all eight components, and table-driven UTs
cover the phase machine (including failed-rollback landing in
Closing with the create error reported and the cleanup error
latched), the shared argument validation, and the wire layout.
- ChipCallable additionally records scalar_count as a cached
derivation of the signature's SCALAR entries: make_callable rejects
a nonzero count that disagrees with the signature, while 0 also
means "not recorded" (legacy blobs read 0). The field occupies four
bytes of historical header tail padding, so every historical offset,
sizeof, and the kernel-cache ABI token are unchanged;
ChipCallable.build gains a trailing scalar_count=0 keyword and a
read-only property.
Two facts a reader should not have to re-derive. The latch refusal returns
PTO_RUNTIME_ERR_INVALID_STATE (-1003) rather than PTO_RUNTIME_ERR_INTERNAL
(-1000) on purpose: conftest.py scrapes "simpler_init failed with code <N>"
and treats -1000 as a poisoned card, so an identity conflict must not look
like one. And kernel_execution_state.cpp stays compiled into all four host
runtimes even though grepping KernelExecutionState now finds only its own
header and .cpp — it is the persistent-state change's foundation, not an
orphaned translation unit.
Every kernel-mode branch this adds is provably dead in this commit: no
production site latches KERNEL (`git grep 'latch(SIMPLER_MODE_KERNEL)' src`
is empty) because both simpler_kernel_mode_init stubs return before any latch
call, so is_kernel() is false on every context and the program path takes the
same branch it took before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Allocate the device Runtime image, register table and KernelArgs once per context, and own the dedicated AICPU stream, hidden AICore stream and context events until explicit close. Mount init and callable preparation on the write-once execution-mode latch. Preserve device identity and runtime ownership on failed teardown, reject repeated initialization before executor mutation, and leave the caller device binding and ACL lifecycle untouched. Cover lifecycle call order, creation rollback, callable preparation, unclosed destruction, retryable close, and all six ACL/reset interceptors. Exercise the real fatal-device finalizer using an injected drain result without poisoning or resetting a physical device. Launch remains a stub.
Manage kernel callable preparation by content and return context-local ID/generation handles. Keep up to 64 immutable residents in a bounded 512 MiB arena, reuse identical images, and reject capacity or stale handle errors without evicting existing residents. Add a separate AICPU invocation entry that validates the current device residency before calling a payload consumer. Keep unsupported payload execution explicit until the binder and runtime consumers are connected. Cover admission boundaries, rollback, stale handles and device-entry validation with unit tests, and document the call chain in Chinese.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kernel-mode preparation now owns callable residency: callers pass canonical content and receive a context-local
SimplerCallableHandle {callable_id, generation}. Identical content reuses the same handle without another upload or registration. Launch rejects missing or stale handles before dispatch.The cache admits at most 64 unique callable images into a 512 MiB arena, with classified capacity errors and rollback that preserves existing residents. There is no eviction or slot reuse. A separate
simpler_aicpu_kernel_execentry validates the current device residency before entering the payload consumer; its tests compile the production entry and cover stale snapshots after a slot generation change.Dependencies and scope
48b120b1; the dependency commits are intentionally separate from the K10a commit.UnsupportedPayload, and kernel support remains disabled. Device-entry validation is covered by CPU UT; real-device ACLGraph replay and its CANN error propagation are not yet validated.docs/zh-cn/kernel-callable-residency.md.Validation