diff --git a/cuda_core/AGENTS.md b/cuda_core/AGENTS.md index ff91284b5dd..14b43638f1d 100644 --- a/cuda_core/AGENTS.md +++ b/cuda_core/AGENTS.md @@ -63,6 +63,44 @@ This file describes `cuda_core`, the high-level Pythonic CUDA subpackage in the - Prefer explicit error propagation over silent fallback paths. - If you change public behavior, update tests and docs under `docs/source/`. +## Concurrency and free-threading + +`cuda.core` ships free-threaded (no-GIL) wheels and builds with +`freethreading_compatible=True`. The user-facing policy lives in +`docs/source/concurrency.rst`; the invariants below are for contributors. Reviewers +and agents should flag violations. + +- **Reads are safe; mutation is the boundary**: concurrent reads of an object are + supported, but concurrent mutation of the same public object (e.g., building one + graph from two threads, or `close()` racing another call) is the caller's + responsibility -- do not add locks to make it safe. Prefer immutable designs to + keep the mutable, thread-unsafe surface small. Protecting library-internal state + *is* in scope. +- **Distinct objects can still collide via shared driver state**: operating on + separate objects is not automatically safe when they share driver or context + state (e.g., changing peer device access while another thread touches affected + memory). Synchronizing these cases is the caller's responsibility; do not try to + lock around them internally. +- **Protect internal cached/module-level state**: guard lazily-populated + `cdef object` caches and module-level state so concurrent access cannot corrupt + interpreter state (CPython reference counts -- a strictly free-threading hazard). + Established patterns are `@cython.critical_section` on accessors (#2215), an + atomic initialization flag (#2216), and `dict.setdefault` for identity caches + (#2217). Guard state only on objects that are legitimately shared between threads; + objects that are not meant to be shared (e.g., the thread-local `Device`) do not + need such guards (see #2321). Reference-count integrity is guaranteed; cache + value-identity/idempotency is not. +- **Entry points assume the GIL is held**: the helpers in `_cpp/resource_handles.*` + are called from Cython with the GIL held and do not re-acquire it. Driver and + destructor callbacks run at arbitrary times, so they take the GIL (`with gil`) + and probe for interpreter shutdown before touching Python objects. +- **Lock ordering -- release the GIL before entering the driver**: any CUDA work + reachable from a host callback or a retained object's `__del__` must release the + GIL before calling the driver, to avoid GIL/driver-lock deadlocks (see the + `_py_host_trampoline` path and numba-cuda#321). Objects retained into a graph + (kernel arguments, memcpy/memset operands, `dst_owner`/`src_owner`, and + host-callback closures) inherit this contract. + ## API design guidelines These are some API design guidelines we try to follow when adding new APIs to diff --git a/cuda_core/docs/source/concurrency.rst b/cuda_core/docs/source/concurrency.rst new file mode 100644 index 00000000000..51a1baa1be9 --- /dev/null +++ b/cuda_core/docs/source/concurrency.rst @@ -0,0 +1,28 @@ +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. SPDX-License-Identifier: Apache-2.0 + +.. currentmodule:: cuda.core + +Concurrency and Thread Safety +============================= + +``cuda.core`` allows concurrent reads of its objects from multiple host +threads, but concurrent *mutation* of the same object is **not supported** -- +for example, adding nodes to the same graph, or closing a resource while +another thread is using it. Whenever an object is shared across threads and at +least one of them may mutate it, the application is responsible for providing +external synchronization. + +The library does protect the integrity of its own internal state (such as +cached attributes and reference counting), so that concurrent reads of the same +object, or any kind of concurrent use of *distinct* objects, cannot corrupt the +interpreter. This is an integrity guarantee only: the ordering and outcome of +concurrent operations on a shared object are otherwise undefined. + +Additional limitations apply because ``cuda.core`` inherits the concurrency +constraints of the underlying CUDA driver. Distinct ``cuda.core`` objects can +share driver or context state, so operating on separate objects is not always +safe. For example, modifying peer device access from one thread while another +thread accesses device memory affected by that change is unsafe, even though the +two threads use different objects. The application is responsible for +synchronizing operations that concurrently read and modify shared driver state. diff --git a/cuda_core/docs/source/index.rst b/cuda_core/docs/source/index.rst index 7e4e2ffd4ec..f059924b33d 100644 --- a/cuda_core/docs/source/index.rst +++ b/cuda_core/docs/source/index.rst @@ -1,4 +1,4 @@ -.. SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. .. SPDX-License-Identifier: Apache-2.0 ``cuda.core``: Pythonic access to CUDA core functionality @@ -14,6 +14,7 @@ Welcome to the documentation for ``cuda.core``. install examples interoperability + concurrency api api_nvml environment_variables