[Relax][Frontend][ONNX] Support dynamic index for Gather on shape#19967
Closed
hamzaqureshi5 wants to merge 297 commits into
Closed
[Relax][Frontend][ONNX] Support dynamic index for Gather on shape#19967hamzaqureshi5 wants to merge 297 commits into
hamzaqureshi5 wants to merge 297 commits into
Conversation
…elete the util (apache#19612) ## Summary `ApplyPassToFunction` is a general-purpose wrapper that runs a pass on only the functions in an IRModule whose name matches a regex. Its sole in-tree production callers are `DecomposeOpsForInference` / `DecomposeOpsForTraining` in `src/relax/transform/decompose_ops.cc`, and both callers always supply a literal function name (never a regex pattern). Inlining the logic as a file-local helper simplifies the module-level context and removes an abstraction that exists only to support one use case. - Inline the helper as `ApplyDecomposeToFunction` (exact-name match, not regex) in `src/relax/transform/decompose_ops.cc` - Delete `src/ir/apply_pass_to_function.cc`, its `transform.h` declaration, and the Python wrapper in `python/tvm/ir/transform.py` - Remove two DCE tests (`test_compatibility_with_apply_pass_to_function`, `test_well_formed_output_with_restricted_scope`) that tested the utility's plumbing rather than DCE behavior
…ression, and rename Simplify to StmtSimplify (apache#19604) ## Summary This PR cleans up technical debt in the TIR simplification machinery via two commits: **Commit 1: Phase out ControlFlowGraph and NarrowPredicateExpression** - Remove `ControlFlowGraph` (~2360 lines) from `src/tirx/analysis/` — used only in non-default config paths that are no longer maintained - Remove `NarrowPredicateExpression` from `src/arith/` — sole non-test caller was `ControlFlowGraph` - Remove gated config fields `propagate_knowns_to_prove_conditional` and `propagate_knowns_to_simplify_expressions` from `SimplifyConfig` - Remove `use_dataflow_analysis` from `RemoveNoOpConfig` - Delete the associated test files and test cases that tested the now-removed paths - ~3800 lines deleted **Commit 2: Rename Simplify → StmtSimplify** - Rename `src/tirx/transform/simplify.{h,cc}` → `stmt_simplify.{h,cc}` - Rename C++ identifiers: `Simplify` → `StmtSimplify`, `SimplifyConfig` → `StmtSimplifyConfig` - Rename FFI keys: `"tirx.Simplify"` → `"tirx.StmtSimplify"`, `"tirx.transform.Simplify"` → `"tirx.transform.StmtSimplify"` - Update Python wrappers and all call sites (~40 files) - Clarifies that this pass operates on statements (distinct from expression-level `arith::Analyzer::Simplify()`) ## Test plan - [x] `tests/python/tirx-transform/test_tir_transform_simplify.py` — 52 tests pass - [x] `tests/python/tirx-transform/test_tir_transform_remove_no_op.py` — 18 pass, 5 xfail - [x] `tests/python/arith/` — full arith test suite passes - [x] `tests/python/tirx-transform/` — full suite: 315 passed, 8 xfailed, 1 xpassed (pre-existing vectorize failure unrelated to this change) - [x] `pre-commit run --all-files` — all hooks pass
…ssConfig (apache#19614) ## Summary Now that the ffi container machinery (Array, Optional, Map, Variant) accepts bare int64_t and bool, the Integer/Bool ObjectRef wrappers add no value in attribute fields, pass-config options, function-attr flags, and OpAttrMap registries — every reader paid an extra .IntValue() / ->value unbox per access for no information gain. This PR is the first stage of phasing out class Integer and class Bool: migrate the bulk of those sites at the field-declaration and call-site level. A follow-up will rewrite the remaining IR-position `Integer(N)` / `Bool(b)` constructors to `IntImm(...)` / `const_true()` / `const_false()` and delete the two classes entirely. - Relax Attrs fields and their container forms (`Array<Integer>` / `Optional<Array<Integer>>` / `Optional<Integer>` / `Optional<Bool>`) migrated to bare `int64_t` / `bool` (manipulate.h, nn.h, op.h, statistical.h, script/builder/frame.h, target/virtual_device.h, distributed/global_info.h, relax/expr.h). - OpAttrMap registry (`set_attr<Bool>("FPurity", Bool(true))` ↔ `GetAttrMap<Bool>("FPurity")`) migrated to `bool` across ~38 files. - PassContext config registrations + `GetConfig<Bool>` / `GetConfig<Integer>` readers, and function-attr `GetAttr<Bool>` / `GetAttr<Integer>` readers (~42 files), all migrated; `HasNonzeroAttr` in `ir/attrs.h` dropped its `.IntValue()` unbox. - Schedule decision arrays (SampleCategorical candidates, perfect-tile factors, autobind thread_extents, multi-level-tiling levels) migrated to `Array<int64_t>` / `Optional<int64_t>` — this is a virtual-signature change on `ConcreteScheduleNode::SampleCategorical` and related methods, acceptable per the phase-out intent. - `Variant<Bool, Array<String>>` for `LiftTransformParams.shared_transform` migrated to `Variant<bool, Array<String>>`.
…che#19617) In continuation of apache#19594 **(DSO modularization)**, fixes for```tvm_rpc``` backend pickups --- ### Issue * Errors during remote ```tvm_rpc``` metaschedule sessions: ``` AttributeError: Unable to find function "tvm.contrib.random.random_fill_for_measure" on the remote RPC server. Please make sure 'USE_RANDOM' is turned ON in the config.cmake on the RPC server. ``` This error is due to missing ```libtvm_runtime_extra.so``` (home of contrib modules, e.g *random*) from ```tvm_rpc```. ---- ### Fixes * Before: ``` # readelf -a /usr/bin/tvm_rpc | grep NEED [ 7] .gnu.version_r VERNEED 0000000000403a88 00003a88 0x0000000000000001 (NEEDED) Shared library: [libtvm_runtime.so] 0x0000000000000001 (NEEDED) Shared library: [libtvm_ffi.so] 0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] ``` * After: ``` # readelf -a /usr/bin/tvm_rpc | grep NEED [ 7] .gnu.version_r VERNEED 0000000000404ed0 00004ed0 0x0000000000000001 (NEEDED) Shared library: [libtvm_runtime_extra.so] 0x0000000000000001 (NEEDED) Shared library: [libtvm_runtime_cuda.so] 0x0000000000000001 (NEEDED) Shared library: [libtvm_runtime_opencl.so] 0x0000000000000001 (NEEDED) Shared library: [libtvm_runtime.so] 0x0000000000000001 (NEEDED) Shared library: [libtvm_ffi.so] 0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6] 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] ``` Thank you !
… / phase out AttrFieldInfo (apache#19615) ## Summary Follow-up to apache#19607 that continues trimming `attrs.h` and adjacent files. The six commits land independently and each builds clean. - Phase out `OpNode::arguments` and `AttrFieldInfo` — the field stored metadata that no Python tooling, test, or C++ caller (beyond internal sanity checks) read; removing it deletes `AttrFieldInfo` plus ~335 chained `.add_argument(...)` calls. The remaining 12 internal consumers now read `op->num_inputs` and report indexed inputs (`input[i]`). - Drop the (unused) virtual destructor on `BaseAttrsNode` (ffi::Object uses a captured-typed deleter, no virtual dispatch needed) and inline the trivial 3-line `DictAttrs(Map)` constructor into the header. - Rename `BaseAttrsNode` → `AttrsNode`; the `Base` prefix existed only to distinguish from the `AttrsNodeReflAdapter` shim that apache#19607 removed. The `"ir.Attrs"` FFI registry key is unchanged. - Promote `DictAttrs` to NOTNULLABLE (`TVM_FFI_DEFINE_OBJECT_REF_METHODS_NOTNULLABLE` + COW macro). The no-arg `DictAttrs()` constructor already created an empty backing, so every existing call site already produced a defined object; ~15 defensive `attrs.defined()` checks (and a defensive Python `None` fallback in `Function`) are now redundant. - Inline the `WithAttr(DictAttrs, ...)` / `WithAttrs(DictAttrs, ...)` free-function overloads into the TFunc-template wrappers — those overloads had no external callers (no TVM_DLL, no Python binding). - Rename `AttrsWithDefaultValues<T>` → `PassConfigWithDefaults<T>` and move from `attrs.h` to `transform.h`; all 9 consumers are pass-config classes registered via `TVM_REGISTER_PASS_CONFIG_OPTION`. `attrs.h` shrinks from 363 → 262 lines.
…KernelLaunch together (apache#19605) ## Summary These three passes are logically a single host/device split step; having intermediaries between them obscures the model and blocks folding them into one pass. This PR moves each intermediary to the position its actual ordering constraint allows, so that `AnnotateDeviceRegions`, `SplitHostDevice`, and `LowerDeviceKernelLaunch` run consecutively in every pipeline. ## Rationale - `MergeSharedMemoryAllocations` moves **before** `AnnotateDeviceRegions` (the only legal position: `LowerDeviceKernelLaunch` requires at most one dyn-shmem allocation per kernel, so Merge cannot move past Lower). - `MakePackedAPI` moves **after** `LowerDeviceKernelLaunch` (Lower's `kCallingConv = kDeviceKernelLaunch` flag causes `MakePackedAPI` to correctly skip device kernels; the host body's lowered `tvm_call_packed` is transparent to `MakePackedAPI`'s subroutine rewriter). - `FP8StorageLegalize` / `BF16StorageLegalize` move **after** `MakePackedAPI` (their `buffer_map.size()==0` ICHECK requires `MakePackedAPI` to have cleared the map). Prereq for Phase 2: collapsing the three consecutive passes into a single `tirx.transform.SplitHostDevice` with three commented regions. ## Test plan - [x] tests/python/tirx-transform/ target-pass unit tests (25 pass) - [x] tests/python/s_tir/transform/test_merge_dynamic_shared_memory_allocations.py (5 pass) - [x] tests/python/tirx-transform/test_tir_transform_fp8_legalize.py / test_tir_transform_bf16_legalize.py (13 pass) - [x] tests/python/codegen/test_target_codegen_c_host.py / test_target_codegen_device.py (6 pass including test_subroutine_call — verifies Risk apache#2) - [x] pre-commit run --all-files clean - [ ] CI: lint / Windows / MacOS
…rs (apache#19616) ## Summary This PR adds Relax TFLite frontend support for the TFLite builtin control-flow / multi-subgraph operator family from apache#19519 item F: `CALL`, `IF`, `WHILE`, and `CALL_ONCE`. It builds on the multi-subgraph import infrastructure merged in PR apache#19587. The frontend already accepts TFLite models with extra subgraphs while converting only `Subgraphs(0)` into the Relax `main` function. This PR uses those extra subgraphs as callable or control-flow regions for the TFLite control-flow operators. The supported subset is intentionally pure tensor and guard-first: - `CALL` lowers a referenced TFLite subgraph to a private Relax function and emits a direct call. - `IF` lowers the then/else subgraphs to private Relax functions and emits a private wrapper function containing Relax `If`. - `WHILE` lowers the cond/body subgraphs to private Relax functions and emits a recursive private Relax function for the loop. - `CALL_ONCE` supports the empty-init no-op subset and explicitly rejects non-empty or resource-like init patterns. This PR does not model resource variable side effects. Those cases remain explicitly guarded instead of being imported with incorrect pure functional semantics. ## Design ### Shared Subgraph Lowering The frontend now keeps shared conversion state across the main graph and referenced subgraphs: - `lowered_subgraphs` - `lowered_if_functions` - `lowered_while_functions` - `lowering_stack` - `module_builder` Referenced pure tensor subgraphs are lowered through a recursive `OperatorConverter` using an isolated `ExprTable`, so subgraph tensor bindings cannot overwrite bindings from the main graph. Lowered subgraphs are cached by subgraph index and reused when the same region is referenced more than once. Generated private functions are registered through the shared parent `module_builder`, so nested cases such as `main CALL -> subgraph A -> CALL subgraph B` keep all private functions in the final IRModule. Recursive ordinary `CALL` subgraphs are guarded with `OpNotImplemented`. `WHILE` uses a dedicated recursive wrapper function instead, because recursion is part of the intended Relax representation for the loop itself. ### Boundary Validation The control-flow converters validate subgraph boundaries before lowering: - referenced subgraph indices must be valid - op input/output arity must match the referenced subgraph interface - branch and loop tensor shape/dtype metadata must match the surrounding op - `IF` and `WHILE` conditions must be scalar bool tensors - `WHILE` loop-carried input/output tensors must have matching metadata The shared `_check_subgraph_interface` helper is used by `CALL`, `IF`, and `WHILE` to keep arity and metadata checks consistent across the control-flow operators. `_require_scalar_bool_tensor` accepts both frontend `TensorWrapper` objects and raw TFLite tensors so caller and referenced-subgraph condition checks use the same path. These checks keep the first implementation conservative and make unsupported cases fail with targeted `OpNotImplemented` diagnostics. ### Tuple Outputs TFLite `CALL`, `IF`, and `WHILE` may produce multiple output tensors. The frontend maps those cases to Relax tuple returns: ```text single output -> tensor expression multi output -> Tuple(...) op outputs -> TupleGetItem(...) ``` This keeps the single-output IR simple while covering multi-output calls, multi-output branches, and multi-variable loop state. ## Operator Support | Operator | TFLite options | Relax lowering | Supported subset | |---|---|---|---| | `CALL` | `CallOptions.Subgraph()` | private Relax function call | pure tensor subgraphs, single or multiple outputs | | `IF` | `IfOptions.ThenSubgraphIndex()`, `ElseSubgraphIndex()` | private wrapper function containing Relax `If` | scalar bool condition, matching branch I/O metadata | | `WHILE` | `WhileOptions.CondSubgraphIndex()`, `BodySubgraphIndex()` | recursive private Relax function | scalar bool cond output, tensor loop-carried state | | `CALL_ONCE` | `CallOnceOptions.InitSubgraphIndex()` | no-op for empty init subgraph | empty init subgraph only | ## Not Included - Full `CALL_ONCE` resource/variable initialization semantics. - Resource, variant, hashtable, or variable tensor support. - TensorFlow-generated `tf.cond` / `tf.while_loop` smoke tests. - Dynamic-shape loop-state refinements beyond the current static metadata checks. ## Tests The tests manually build minimal TFLite flatbuffers and compare the imported Relax IR with `tvm.ir.assert_structural_equal`. Unsupported-boundary tests use `pytest.raises`. | Test | Coverage | |---|---| | `test_call_subgraph` | basic `CALL` to a pure tensor subgraph | | `test_call_subgraph_multi_output` | `CALL` tuple return and output binding | | `test_call_subgraph_nested_call` | nested `CALL` private function registration | | `test_call_subgraph_invalid_index_unsupported` | invalid `CALL` subgraph index | | `test_call_subgraph_io_mismatch_unsupported` | `CALL` arity mismatch | | `test_call_subgraph_output_metadata_mismatch_unsupported` | `CALL` output metadata guard | | `test_if_subgraphs` | basic `IF` branch selection | | `test_if_subgraphs_multi_output` | `IF` tuple branch returns | | `test_if_subgraphs_non_bool_condition_unsupported` | `IF` condition dtype guard | | `test_if_subgraphs_invalid_index_unsupported` | invalid then/else subgraph index | | `test_if_subgraphs_output_count_mismatch_unsupported` | branch output count guard | | `test_if_subgraphs_input_metadata_mismatch_unsupported` | branch input metadata guard | | `test_if_subgraphs_output_metadata_mismatch_unsupported` | branch output metadata guard | | `test_while_subgraphs` | basic recursive `WHILE` lowering | | `test_while_subgraphs_repeated_cond_body_pair` | shared cond/body loop function cache | | `test_while_subgraphs_two_loop_vars` | multi-variable loop state tuple path | | `test_while_subgraphs_non_bool_condition_unsupported` | `WHILE` cond output dtype guard | | `test_while_subgraphs_invalid_index_unsupported` | invalid cond/body subgraph index | | `test_while_subgraphs_zero_loop_vars_unsupported` | zero-loop-var guard | | `test_while_subgraphs_loop_state_metadata_mismatch_unsupported` | loop state metadata guard | | `test_while_subgraphs_output_count_mismatch_unsupported` | body output count guard | | `test_while_subgraphs_input_metadata_mismatch_unsupported` | cond/body input metadata guard | | `test_while_subgraphs_output_metadata_mismatch_unsupported` | cond/body output metadata guard | | `test_call_once_empty_init_subgraph` | empty `CALL_ONCE` no-op subset | | `test_call_once_non_empty_init_subgraph_unsupported` | non-empty init subgraph guard | | `test_call_once_inputs_outputs_unsupported` | `CALL_ONCE` op I/O guard | | `test_call_once_init_subgraph_io_unsupported` | init subgraph I/O guard | | `test_call_once_invalid_index_unsupported` | invalid init subgraph index | Local validation: ```bash python -m ruff format --check \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m ruff check \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m pytest \ tests/python/relax/test_frontend_tflite.py \ -k "call_subgraph or if_subgraphs or while_subgraphs or call_once" -q python -m pytest \ tests/python/relax/test_frontend_tflite.py -q ``` Result: ```text ruff format --check: 2 files already formatted ruff check: All checks passed 28 passed, 434 deselected 462 passed ``` ## References - Issue apache#19519 item F: TFLite control-flow / multi-subgraph operators - PR apache#19587: StableHLO region-based ops and multi-subgraph model support
…pache#19601) ## Summary This PR adds Relax TFLite frontend support for `UNIDIRECTIONAL_SEQUENCE_RNN` (BuiltinOperator 35), claimed in [apache#19519](apache#19519) Group A. The op executes a simple RNN cell over a time sequence. The converter unrolls the time steps at graph-construction time using Relax primitives. Cell equation: ``` h_t = fused_activation(x_t @ W.T + h_{t-1} @ Wr.T + b) ``` ## Changes - **Handler**: `convert_unidirectional_sequence_rnn` registered in `convert_map` (alphabetical, U-region after `UNPACK`) - **Inputs** (5): `input [batch, time, input_size]`, `input_weights [num_units, input_size]`, `recurrent_weights [num_units, num_units]`, `bias [num_units]`, `hidden_state [batch, num_units]` (variable, zero-initialised) - **Output**: `[batch, time, num_units]` (always batch-major) - **time_major=True**: input is transposed to batch-major before unrolling - **Activations**: NONE, RELU, RELU6, TANH, SIGMOID (via `convert_fused_activation_function`) - **Quantized**: raises `OpNotImplemented` (not yet supported) ## Testing Modern TF/Keras (2.x, Keras 3) no longer emits `UNIDIRECTIONAL_SEQUENCE_RNN`; `SimpleRNN` with `unroll=False` lowers to `WHILE`+TensorList ops, and `unroll=True` expands to elementwise ops. Tests therefore follow the same flatbuffer-construction pattern used by the StableHLO op PRs (apache#19536, apache#19587). Three tests added to `tests/python/relax/test_frontend_tflite.py`: - `test_unidirectional_sequence_rnn_none_activation` — `tvm.ir.assert_structural_equal` with identity weights / zero bias, NONE activation, time=1 - `test_unidirectional_sequence_rnn_relu_activation` — shape check, random weights, RELU activation, time=3 - `test_unidirectional_sequence_rnn_time_major` — shape check, `time_major=True` input layout ```bash python -m pytest tests/python/relax/test_frontend_tflite.py -k unidirectional_sequence_rnn -v ``` All 3 tests pass. pre-commit (ASF header, ruff check, ruff format) all pass. ## References - Issue [apache#19519](apache#19519) Group A: Sequence / recurrent model operators Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This PR renames `tirx::CallNode::annotations` to `attrs`, matching the existing Relax `CallNode::attrs` convention. Previously, TIRX Call metadata was stored in a `Map<String, Any>` field named `annotations`. This PR makes it a first-class `Attrs` field instead, so call-level metadata follows the same representation and naming style as Relax calls.
## Summary
`tvm::runtime::regex_match` was a thin C++ wrapper that bounced through
a
global `ffi::Function` back into Python's `re.match`. It was introduced
solely to avoid pulling `<regex>` into TVM (libstdc++ dual-ABI conflict
with
pre-cxx11 pytorch wheels). The only C++ caller is the DNNL JSON runtime,
where
every pattern reduces to substring containment — `re.match` anchors at
the
start only, so `.*X.*` is equivalent to `s.find(X) != npos`.
- Remove `src/runtime/regex.{h,cc}` and the Python
`tvm.runtime.regex_match`
global registration.
- Add file-local `contains` / `contains_any` helpers in
`dnnl_json_runtime.cc`
and inline `std::string::find` at the 15 call sites.
- Drop the dead `regex.h` include from
`src/relax/transform/update_param_struct_info.cc`.
No CMakeLists.txt change needed — `src/runtime/*.cc` is picked up by
glob.
`USE_DNNL` is OFF in the ci_gpu container, so DNNL-specific runtime
tests
are not exercised locally. The DNNL translation unit compiles cleanly
with
the inlined helpers, and the full TVM build (636 targets) passes.
## Summary The [Refactor] Phase out microTVM commit (apache#17554) removed the bulk of microTVM, but a few crumbs were left behind. This PR removes all of them: - `src/runtime/meta_data.h`: old snake_case header with a dmlc-based `FunctionInfo` struct. Replaced long ago by `src/runtime/metadata.h` (camelCase, `ffi::ObjectRef`-based). No file in the tree includes `meta_data.h`. - `src/runtime/crt/common/crt_runtime_api.c`: the sole remaining file in the `src/runtime/crt/` subtree. Includes `<tvm/runtime/crt/*>` headers that no longer exist; not picked up by any `RUNTIME_SRCS` glob; uncompilable in the current tree. - `cmake/utils/CRTConfig.cmake`: defines `generate_crt_config()`, which has no callers and references a missing `crt_config.h.template`. - `docs/conf.py`: sphinx-gallery exclusion for a tutorial file (`micro_mlperftiny.py`) that no longer exists; simplified the regex. No code changes elsewhere — these are all isolated leaves with zero callers or includers across `*.cc *.h *.c *.py *.cmake CMakeLists.txt`.
…eader-only (apache#19621) ## Summary The NVTXScopedRange utility is a thin RAII wrapper over nvtxRangePush/Pop with a no-op fallback when NVTX is not enabled. The two function bodies and the conditional include of `<nvtx3/nvToolsExt.h>` fit naturally inline in the header, eliminating the separate translation unit and its `TVM_RUNTIME_DLL` export annotations. - Move `include/tvm/runtime/nvtx.h` to `include/tvm/support/cuda/nvtx.h` under namespace `tvm::support`; delete `src/runtime/nvtx.cc`. - Inline the constructor/destructor; gate the real-vs-stub split with `TVM_NVTX_ENABLED` in the header. - Switch the CMake gate from a per-file `COMPILE_DEFINITIONS` on `nvtx.cc` to a global `add_compile_definitions(TVM_NVTX_ENABLED=1)` when `USE_CUDA AND USE_NVTX`, so every TU that includes the header agrees on the definition. - Update the three call-site files (`vm.cc`, `paged_kv_cache.cc`, `attn_utils.h`) to the new include path and qualify `NVTXScopedRange` as `support::NVTXScopedRange`.
… to tvm.support (apache#19624) ## Summary Lifts 10 host-toolchain / CLI / process / utility modules from `python/tvm/contrib/` to a new `python/tvm/support/` package, and deletes two dead contrib shims. `tvm.support` is the home for Python helpers that integrate TVM with external CLIs and host-side tools — compilers, archivers, subprocess pools, and build-info queries. These are load-bearing internal pieces that TVM's compile/link/run paths depend on. `tvm.contrib` is reserved for optional vendor SDK integrations and experimental features. The distinction is documented in the `tvm.support` package docstring. Moved (one commit each): - `tvm.contrib.cc` → `tvm.support.cc` - `tvm.contrib.nvcc` → `tvm.support.nvcc` - `tvm.contrib.rocm` → `tvm.support.rocm` - `tvm.contrib.ndk` → `tvm.support.ndk` - `tvm.contrib.xcode` → `tvm.support.xcode` - `tvm.contrib.clang` → `tvm.support.clang` - `tvm.contrib.emcc` → `tvm.support.emcc` - `tvm.contrib.popen_pool` → `tvm.support.popen_pool` - `tvm.contrib.utils` → `tvm.support.utils` - `tvm.contrib.tar` → `tvm.support.tar` Deleted: - `tvm.contrib.spirv` — single `optimize()` wrapping `spirv-opt`; zero importers. - `tvm.contrib.rpc` — self-deprecation shim with "removed in 0.5" banner; honoring it. Package conversion: - `python/tvm/support.py` → `python/tvm/support/__init__.py` with inclusion-rule docstring. - `libinfo()` extracted into `python/tvm/support/libinfo.py`. - `FrontendTestModule` dropped (audit confirmed zero callers outside its own definition). ## Compatibility Hard break — no `tvm.contrib.<mod>` re-export shims. All callers updated in this PR. C++-side FFI registry keys (`tvm.contrib.nvcc.*`, etc.) are unchanged — only the Python module path moves. Renaming the FFI keys is a separate follow-up.
…ut tvm/ir/repr.h (apache#19627) ## Summary Two-commit PR: 1. Bump `3rdparty/tvm-ffi` from `3c35034` to `98d0029` and migrate all 21 in-tree `SEqHashDef()` call sites to `SEqHashDefRecursive()` (the conservative variant matching the prior default behavior). Six let-style sites carry `TODO(tqchen)` comments indicating they should flip to `SEqHashDefNonRecursive` after the new tvm-ffi ships on pypi. 2. Phase out `include/tvm/ir/repr.h`. The bumped tvm-ffi now provides ostream `operator<<` for `Any`/`ObjectRef`/`Variant`/`Optional` directly in `tvm/ffi/extra/dataclass.h`, making the in-tree thin wrapper redundant. Rewrite 8 includers, rename `src/ir/repr.cc` → `src/ir/access_path_repr.cc` (preserves `node.AsRepr` + AccessPath/AccessStep `__ffi_repr__` registrations; drops zero-caller `tvm::Dump()`), delete the header. Also fixes a Python-level import regression in `python/tvm/ir/attrs.py` caused by the bump: tvm_ffi 0.1.12.dev changes the field-registration guard from `not hasattr(cls, name)` to `name not in cls.__dict__`, which breaks `DictAttrs` because `DictAttrsNode` registers a reflection field named `"__dict__"` — Python forbids installing a class descriptor with that name via `setattr`. Fix: define `__dict__` as an explicit Python property on `DictAttrs` so the auto-installation is skipped. ## TODO follow-ups After the new tvm-ffi releases on pypi, flip the 6 `SEqHashDefRecursive()` sites that carry `TODO(tqchen)` comments to `SEqHashDefNonRecursive()`. Locations are enumerated in the commit body of commit 1. ## Test plan - [x] Full ninja build clean (638/638). - [x] 118/118 cpptest pass. - [x] `import tvm; tvm.cuda(0).exist` returns True. - [x] `tests/python/all-platform-minimal-test`: 37 passed, 105 skipped. - [x] `tests/python/relax/test_struct_info.py`: 9 passed. - [x] `git grep -nE 'SEqHashDef\(|"tvm/ir/repr\.h"'` is empty. - [x] `pre-commit run --all-files` clean.
…he#19625) ## Summary `ReplaceGlobalVars` was a public IR-layer API with only one in-tree C++ caller (`relax::AttachGlobalSymbol`). The mechanism used a NodeFunctor vtable populated at static-init time by per-dialect `.cc` files in relax and tirx, which made the IR layer logically depend on its dialects even though the include graph did not show it. Move the dispatch logic into the consumer as file-local mutators and a private helper. Delete the public header, the IR-layer driver, both per-dialect dispatch registrations, the `IRModule.replace_global_vars` python method, and its dedicated test file. The behavior is still covered by `tests/python/relax/test_transform_attach_global_symbol.py` and by the pipelines that include the `AttachGlobalSymbol` pass.
…pache#19619) Fixes apache#18915 Vulkan codegen previously generated sequentially constistant OpControlBarrier SPIR-V instructions, which is invalid for Vulkan, where we would expect AcquireRelease.
…ead_map, texture, minrpc, disco, contrib (apache#19628) ## Background The TVM runtime has been growing organically. Several headers and directories live at the top level of `src/runtime/` despite only being consumed by a single backend subsystem. This PR applies the **locality principle**: code that has exactly one consumer moves to live next to that consumer. ## Changes ### Move 1: `thread_map.h` → `src/runtime/vulkan/` `ThreadMap` is only used by Vulkan device API headers. Moving it under `src/runtime/vulkan/` reflects this exclusive ownership. ### Move 2: `texture.h` → `src/runtime/opencl/` Texture storage utilities are OpenCL/Adreno-specific. Moving the header under `src/runtime/opencl/` makes ownership clear. ### Move 3: `minrpc/` → `src/runtime/rpc/minrpc/` The minrpc mini-RPC implementation belongs logically under the existing `src/runtime/rpc/` subtree. All consumers already live under rpc/ or reference it as a child of rpc/. ### Move 4: Introduce `src/runtime/extra/` boundary `disco/` and `contrib/` are the sole source directories for `libtvm_runtime_extra`. Grouping them under `src/runtime/extra/` makes the `libtvm_runtime_extra` build boundary visible in the filesystem, matching the modular runtime split introduced in apache#19444. - `src/runtime/disco/` → `src/runtime/extra/disco/` - `src/runtime/contrib/` → `src/runtime/extra/contrib/` - Public `include/tvm/runtime/disco/` is unchanged. ### Drive-by fixes - `apps/android_rpc/…/tvm_runtime.h`: Drop stale `minrpc_logger.cc` include (file no longer exists) and fix stale `tvm-ffi/src/ffi/extra/testing.cc` path to `tvm-ffi/src/ffi/testing/testing.cc`. ## Test Plan - [x] Full build (`ninja -j$(nproc)`) — succeeds - [x] `./cpptest` — 118 tests passed - [x] Python smoke: `tvm.__version__` + `tvm.cuda(0).exist` — pass - [x] `tests/python/all-platform-minimal-test` — 37 passed, 105 skipped - [x] `tests/python/runtime/test_runtime_rpc.py` — 2 passed, 21 skipped - [x] `tests/python/runtime/test_rpc_base.py` — 2 passed - [x] `pre-commit run --all-files` — all hooks pass
…he#19630) ## Summary `derived_object` was duplicated byte-for-byte across `python/tvm/runtime/support.py` and `python/tvm/s_tir/meta_schedule/utils.py`. The function is not a runtime feature and is used outside meta_schedule (tvm.relax, tvm.tirx), so neither location was the right home. Move the single canonical definition into a new `python/tvm/ir/utils.py`. `tvm.ir` loads before both `tvm.tirx` and `tvm.s_tir`, so eager top-level imports work from every consumer without load-order workarounds. Rewrite all 25 caller imports. Keep the better-typed `cls: type[T] -> type[T]` signature from the runtime-side copy. After this change `runtime/support.py` is empty and is removed; `meta_schedule/__init__.py` drops its now-dead re-export. No alias shims are left behind — callers update imports directly.
Remove tvm-lint from tvm-bot
…way dep, migrate dialect config to extra_config (apache#19631) ## Background The `tvm::ir` layer previously had a reverse dependency on `tvm::script`, injected via the `TVM_OBJECT_ENABLE_SCRIPT_PRINTER()` macro that added a `Script()` member method to IR node types (IRModule, PrimExpr, Buffer, PrimFunc, Stmt). This violated the intended one-way dependency: `script` should depend on `ir`, never the other way around. Additionally, `PrinterConfigNode` accumulated dialect-specific fields (`tir_prefix`, `tir_import_module`, `tirx_prefix`, `relax_prefix`) that created leakage between the generic printer infrastructure and dialect internals. ## Changes This PR restores the clean dependency direction and encapsulates dialect config properly, in 5 commits: 1. **Lift TVMScript entry point into `script/printer/printer.h`**: New header `include/tvm/script/printer/printer.h` introduces: - `tvm::Script()` free function replacing `TVMScriptPrinter::Script()` static method - `TVMScriptPrinter` class with vtable (`NodeFunctor<std::string(...)>`) - `TVM_REGISTER_SCRIPT_AS_REPR` macro for registering per-type repr callbacks 2. **Drop `TVM_OBJECT_ENABLE_SCRIPT_PRINTER` macro**: Remove the macro from all IR headers (`ir/expr.h`, `ir/module.h`, `tirx/buffer.h`, `tirx/function.h`, `tirx/stmt.h`), eliminating the reverse `ir` → `script` dependency. All call sites of `.Script()` member methods updated to use `tvm::Script()`. 3. **Move dialect-specific `PrinterConfig` fields to `extra_config`**: Remove `tir_prefix`, `tir_import_module`, `tirx_prefix`, `relax_prefix` from `PrinterConfigNode`. Dialect internals now read their config via `GetExtraConfig<T>(key, fallback)` with dotted keys (e.g., `"tirx.prefix"`). `buffer_dtype` is kept as a top-level field alongside `int_dtype`/`float_dtype` since it is a shared scalar-literal default, not a dialect-specific knob. 4. **Python: drop dialect kwargs, expose `extra_config`**: Update `PrinterConfig`, `Scriptable.script()`, `Scriptable.show()`, `Scriptable._relax_script()`, and `BasePyModule.script()` to use `extra_config: dict | None = None` instead of individual dialect kwargs. The tirx auto-switch logic is preserved. 5. **Fix transitive include breakage**: Explicitly add direct includes for `config.h` and `node_functor.h` where headers previously relied on transitive paths through `expr.h`/`module.h`. ## Testing - C++ unit tests: 118/118 pass - TVMScript printer tests: 771 passed, 1 skipped, 1 xfailed - TIR namespace tests (`tests/python/tirx/test_printer_tir_namespaces.py`): 13/13 pass - Relax AST printer tests: 24/24 pass - Minimal platform tests: 37/37 pass - Pre-commit (ASF headers, ruff, clang-format): all clean
…r proves over scalable vectors (apache#19638) ## Summary Phase out `src/arith/scalable_expression.{h,cc}`. The arith layer no longer attempts to prove anything about scalable vectors — proofs that depended on `Target::Current()` are removed. Scalable vectors remain a first-class concept; arith just doesn't reason about their lengths. ## Use-site summary Only 16 call sites total across 7 symbols (9 live, 7 proof-related). | Symbol | Live callers (kept) | Proof callers (deleted) | New home | |---|---|---|---| | `ExtractVscaleFactor` | 4 × `arith/rewrite_simplify.cc` + 2 × `tirx/ir/expr.cc` | — | file-local in each | | `IsVScaleCall` | 1 × `tirx/op/op.cc` + 1 × `tirx/transform/vectorize_loop.cc` | — | inline at use sites | | `ContainsVscaleCall` | 4 × `arith/rewrite_simplify.cc` + 1 × `s_tir/schedule/ir_comparator.cc` | — | inline at use sites | | `TargetHasVLA` | 2 × `tirx/transform/vectorize_loop.cc` | analyzer.cc + const_int_bound.cc | local in vectorize_loop.cc | | `GetVScaleValues` | 1 × `target/llvm/codegen_aarch64.cc` | analyzer.cc + const_int_bound.cc | inlined at codegen_aarch64 | | `CanProveVscaleExpressionFromKnownValues` | — | analyzer.cc | DELETE | | `SubstituteVScaleWithKnownValue` | — | internal only | DELETE | ## Changes (6 commits) 1. Move `ExtractVscaleFactor` to file-local anonymous-namespace helpers in `rewrite_simplify.cc` and `tirx/ir/expr.cc`. Function is small; per-file duplication is cleaner than a shared header. 2. Inline `IsVScaleCall` / `ContainsVscaleCall` / `TargetHasVLA` at call sites (1-3 line predicates, anonymous-namespace per consumer `.cc`). 3. Drop the scalable-vector proof scaffolding from `arith/analyzer.cc` (substitution-proof loop) and `arith/const_int_bound.cc` (vscale branch). `vscale()` calls fall back to `Everything()` — no special bound narrowing. 4. Delete `scalable_expression.{h,cc}`. Inline the `GetVScaleValues` body at `codegen_aarch64.cc` (computes `max_val = vector_width / 8` floor-rounded to a power of two for the LLVM `vscale_range` attribute). 5. Mark `pytest.mark.xfail` on 19 tests that relied on the deleted substitution-proof loop. 6. `pre-commit` line-length cleanup. ## Compatibility / intentional regression This is a hard break for any consumer of the deleted symbols. They were already in a private header (`src/arith/scalable_expression.h`, not under `include/`). 19 tests that proved vscale-bearing inequalities on SVE / RVV are xfailed. The proofs were target-dependent and the new policy is that arith does not attempt them.
## Summary Add Relax TFLite frontend support for the builtin `REDUCE_WINDOW` operator. This covers the ordinary TFLite op only, not `STABLEHLO_REDUCE_WINDOW`. The converter parses `ReduceWindowOptions` from `BuiltinOptions2`, validates the static window attributes, and lowers supported reduce functions through `topi.sliding_window` plus Relax reductions. Supported modes: - `ADD` - `MUL` - `MINIMUM` - `MAXIMUM` - `ALL` - `ANY` Empty output shapes are handled directly with `relax.op.zeros`. Quantized `REDUCE_WINDOW`, dynamic window attributes, and unsupported reduce functions remain rejected with explicit errors. ## Testing - `python -m py_compile python/tvm/relax/frontend/tflite/tflite_frontend.py tests/python/relax/test_frontend_tflite.py` - `python -m pytest tests/python/relax/test_frontend_tflite.py -k reduce_window -q -p no:tvm.testing.plugin` - `python -m pytest tests/python/relax/test_frontend_tflite.py -k "reduce_window or reduction_ops" -q -p no:tvm.testing.plugin` - `conda run -n test python -m ruff check python/tvm/relax/frontend/tflite/tflite_frontend.py tests/python/relax/test_frontend_tflite.py` ## Related Related to apache#19519.
## Summary Add Relax TFLite frontend support for `RNN` (BuiltinOperator 23), claimed in [apache#19519](apache#19519) Group A. Single-step RNN cell: ``` h = fused_activation(x @ W.T + h @ Wr.T + b) ``` ## Changes - **Handler**: `convert_rnn` registered in `convert_map` (alphabetical, after `RANGE`) - **Inputs** (5): `input [batch, input_size]`, `input_weights [num_units, input_size]`, `recurrent_weights [num_units, num_units]`, `bias [num_units]`, `hidden_state [batch, num_units]` (variable, zero-initialised) - **Output**: `[batch, num_units]` - **Activations**: all fused activations via `convert_fused_activation_function` - **Quantized**: raises `OpNotImplemented` ## Testing Two tests added to `tests/python/relax/test_frontend_tflite.py`: - `test_rnn_none_activation` — `tvm.ir.assert_structural_equal` with identity weights, NONE activation - `test_rnn_relu_activation` — shape check, random weights, RELU activation ```bash python -m pytest tests/python/relax/test_frontend_tflite.py -k rnn -v ``` ## References - Issue [apache#19519](apache#19519) Group A: Sequence / recurrent model operators
apache#19636) ## Background `class Integer : public IntImm` and `class Bool : public IntImm` were thin wrappers sharing `IntImmNode` with no separate node class and no FFI registration. They existed to provide implicit int→Integer constructors and a `.IntValue()` / `operator bool()` accessor, but the same functionality is available directly through `IntImm`. ## What this PR does Migrates all call sites away from `Integer` / `Bool` and then deletes the class definitions. The changes are split into four commits, each independently buildable: **Commit 1 – [REFACTOR][TIR]** Replace IR-position `Integer(N)` / `Bool(b)` constructors with `IntImm(DataType::Int(32), N)` / `IntImm(DataType::Bool(), b)` across ~62 source files (arith, relax analysis, s_tir schedule state, transform passes, codegen). **Commit 2 – [REFACTOR][SCHEDULE]** Migrate `Schedule` and `MetaSchedule` trace-boxing code: `Integer(N)` attrs in `TracedSchedule` → `IntImm(DataType::Int(32), N)`; `ffi::Array<Integer>` schedule-rule parameters → `int64_t`; `Bool(b)` attrs → `IntImm(DataType::Bool(), b)`. **Commit 3 – [REFACTOR][TOPI]** Migrate topi container signatures (`ffi::Array<Integer>` → `ffi::Array<int64_t>`) and update all internal usages (`.IntValue()` → plain int64_t, `.defined()` → removed, `->value` → direct indexing). Also handles stray `Integer` / `Bool` variables in clml codegen, make_packed_api, infer_layout_utils, and relax distributed code. **Commit 4 – [REFACTOR][IR]** Delete `class Bool`, `class Integer`, `TypeTraits<Bool>`, and `TypeTraits<Integer>` from `include/tvm/ir/expr.h`. ## Canonical replacements | Old | New | |-----|-----| | `Integer(N)` | `IntImm(DataType::Int(32), N)` | | `Bool(b)` | `IntImm(DataType::Bool(), b)` | | `x.IntValue()` | `x->value` | | `x` as bool | `x->value != 0` | | `ffi::Array<Integer>` | `ffi::Array<int64_t>` | ## Testing - All 118 C++ unit tests pass (`./cpptest`) - `tests/python/s_tir/` — 1251 passed (14 pre-existing failures unrelated to this change, all in TIR transform tests with annotation-mismatch errors) - `tests/python/relax/` — passes (excluding pre-existing torch/torchvision import failures in frontend tests)
## Summary Add LSTM (coupled input-forget) and SVDF single-step converters to the TFLite frontend. Both are float32-only; quantized variants are not supported yet. From apache#19519. ## Changes - **LSTM**: FULL kernel type, coupled input-forget gate only. Peephole, projection, and layer norm are not supported - **SVDF**: Standard SVDF with feature projection + time filtering + bias + fused activation - Both converters validate unsupported modes (quantized, non-coupled LSTM) with clear error messages ## Testing - `test_lstm_none_activation` — verifies LSTM converter produces correct IR shapes (batch, input_size) → (batch, num_units) with 3 params (input, h_state, c_state) - `test_svdf_none_activation` — verifies SVDF converter produces correct IR shapes (batch, input_size) → (batch, num_filters) with 2 params (input, state) ```bash python -m pytest tests/python/relax/test_frontend_tflite.py -k "lstm or svdf" -v ``` ## References - TFLite LSTM spec: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/lstm.cc - TFLite SVDF spec: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/svdf.cc
…htable Import Support (apache#19639) ## Summary This PR adds incremental Relax TFLite frontend support for the resource variable initialization subset: - `VAR_HANDLE` - `ASSIGN_VARIABLE` - `READ_VARIABLE` It builds on the TFLite control-flow / multi-subgraph support from apache#19616, especially `CALL_ONCE`. TFLite commonly represents initialization through a `CALL_ONCE` init subgraph, then uses resource handles from the main subgraph to read initialized variables. This PR supports that constrained initialization pattern without introducing general mutable runtime state into Relax. The PR also adds explicit frontend guards for the TFLite builtin hashtable operators: - `HASHTABLE` - `HASHTABLE_IMPORT` - `HASHTABLE_FIND` - `HASHTABLE_SIZE` These operators are intentionally left unsupported for now. TFLite builtin hashtable kernels are not generic tensor maps: their runtime implementations cover the `int64 -> string` and `string -> int64` table variants, and correct import requires proper `TensorType.STRING` support. Rejecting the operators is safer than lowering a synthetic numeric table semantics that TFLite does not actually implement. ## Design ### Shared Initialization State The frontend now keeps resource initialization data in shared conversion state: - `conversion_state["resource_values"]` - `conversion_state["in_call_once_init"]` This state is shared by the main graph converter and the `CALL_ONCE` init subgraph converter. Each converter instance still keeps its own local `self.resource_handles` map, keyed by TFLite tensor name. Resource variables use `container + shared_name` from `VarHandleOptions` when present, falling back to the handle tensor name. This keeps tensor-name bindings scoped to each subgraph while allowing init subgraphs and the main graph to agree on the same logical resource. ### CALL_ONCE Init Subgraphs `CALL_ONCE` now accepts a non-empty init subgraph when all operators are in the supported initialization subset: - `VAR_HANDLE` - `ASSIGN_VARIABLE` The init subgraph still must have no inputs and no outputs. The converter first checks every operator against the allowlist, then converts the init subgraph with a fresh `ExprTable` and shared conversion state. The init subconverter deliberately shares the parent `BlockBuilder`. This is safe for the current subset because all supported init operators update importer state and return `None`; they do not emit Relax bindings. A comment documents that this should be revisited if future `CALL_ONCE` init operators emit Relax expressions. ### Resource Variables `VAR_HANDLE` is declarative. It registers the output resource tensor in the current converter's local `resource_handles` map and returns `None`. `ASSIGN_VARIABLE` is accepted only while converting a supported `CALL_ONCE` init subgraph. It resolves the resource handle through the init converter's local handle map and stores the assigned tensor expression in shared `conversion_state["resource_values"]`. `READ_VARIABLE` resolves the main graph resource handle and returns the initialized expression from shared state. If the resource has not been initialized by a supported `CALL_ONCE` path, the frontend raises `OpNotImplemented`. This supports the common static-initialization inference pattern while avoiding incorrect lowering for runtime mutation. ### Hashtable Operators `HASHTABLE` registers the table handle and validates the dtype pair against TFLite kernel constraints (`int64/string` or `string/int64`). `HASHTABLE_IMPORT` in a supported `CALL_ONCE` init subgraph captures static metadata (table size, key/value dtypes) but does not store actual string data, because Relax does not yet support `TensorType.STRING`. `HASHTABLE_SIZE` returns a scalar Relax constant for statically imported tables. `HASHTABLE_FIND` is rejected with `OpNotImplemented` because Relax cannot represent TFLite string tensors or the runtime lookup semantics. ## Operator Support | Operator | TFLite options | Relax lowering | Supported subset | |---|---|---|---| | `VAR_HANDLE` | `VarHandleOptions` | handle registration only | main graph and supported `CALL_ONCE` init subgraphs | | `ASSIGN_VARIABLE` | `AssignVariableOptions` | store initialized Relax expression in shared importer state | supported `CALL_ONCE` init subgraphs only | | `READ_VARIABLE` | `ReadVariableOptions` | return initialized Relax expression | resource must have supported static initialization | | `HASHTABLE` | `HashtableOptions` | handle registration + dtype validation | validates `int64/string` or `string/int64` pair, rejects other combinations | | `HASHTABLE_IMPORT` | `HashtableImportOptions` | store static metadata (size, key/value dtype) | `CALL_ONCE` init subgraphs only, constant key/value shape validation | | `HASHTABLE_FIND` | `HashtableFindOptions` | unsupported guard | requires future `TensorType.STRING` support in Relax | | `HASHTABLE_SIZE` | `HashtableSizeOptions` | scalar Relax constant | returns `[size]` int64 for statically imported tables | ## Safety Checks - `ASSIGN_VARIABLE` outside `CALL_ONCE` initialization raises `OpNotImplemented`. - `READ_VARIABLE` without supported initialization raises `OpNotImplemented`. - `CALL_ONCE` init subgraphs with inputs or outputs remain unsupported. - `CALL_ONCE` init subgraphs containing operators outside the resource-variable initialization allowlist remain unsupported. - TFLite builtin hashtable operators raise `OpNotImplemented` until the frontend can model their real int64/string table semantics. ## Not Included - Runtime `ASSIGN_VARIABLE` mutation in the main graph. - Runtime resource-state threading through Relax function parameters and returns. - Cross-subgraph resource handle aliasing beyond the static `container/shared_name` matching pattern. - Multiple runtime writes with ordering semantics. - TFLite builtin hashtable lowering. - `TensorType.STRING` import support. ## Tests The tests manually build minimal TFLite flatbuffers and compare imported Relax IR with `tvm.ir.assert_structural_equal`. Unsupported patterns use `pytest.raises`. | Test | Coverage | |---|---| | `test_resource_variable_call_once_init_read` | `CALL_ONCE` init subgraph with `VAR_HANDLE + ASSIGN_VARIABLE`, then main graph `READ_VARIABLE` | | `test_assign_variable_main_subgraph_unsupported` | runtime/main graph `ASSIGN_VARIABLE` guard | | `test_read_variable_uninitialized_unsupported` | `READ_VARIABLE` without supported initialization guard | | `test_hashtable_call_once_import_find_unsupported` | hashtable init/find path remains unsupported | | `test_hashtable_call_once_import_size_unsupported` | hashtable init/size path remains unsupported | | `test_hashtable_import_main_subgraph_unsupported` | main graph `HASHTABLE_IMPORT` remains unsupported | | `test_hashtable_size_uninitialized_unsupported` | uninitialized `HASHTABLE_SIZE` remains unsupported | Local validation: ```bash python -m py_compile \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m ruff format --check \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m ruff check \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m pytest \ tests/python/relax/test_frontend_tflite.py \ -k "resource_variable or read_variable_uninitialized or hashtable" -q python -m pytest \ tests/python/relax/test_frontend_tflite.py -q ``` Result: ```text py_compile: passed ruff format --check: files already formatted ruff check: All checks passed targeted resource/hashtable tests: 6 passed full test_frontend_tflite.py: 472 passed ```
test_transform_lower_tirx.py imports and calls Simplify, but the pass is named StmtSimplify (the only simplify pass exported from tvm.tirx.transform). The stale name makes the module fail to import at collection time. Use StmtSimplify so the test collects and runs.
…ache#19634) ## Summary Add three TFLite sequence recurrent operators to the Relax frontend, all with coupled input-forget gate (FULL kernel) and float32-only support. - UNIDIRECTIONAL_SEQUENCE_LSTM - BIDIRECTIONAL_SEQUENCE_RNN - BIDIRECTIONAL_SEQUENCE_LSTM From apache#19519. ## Changes - **UNIDIRECTIONAL_SEQUENCE_LSTM**: same layout as single-step LSTM, unrolls over time and stacks per-step hidden states. Supports time_major, cell_clip, proj_clip, and fused activation. - **BIDIRECTIONAL_SEQUENCE_RNN**: separate fw/bw RNN cells, backward scans in reverse. Supports merge_outputs (concat fw + bw) and split outputs via Tuple. - **BIDIRECTIONAL_SEQUENCE_LSTM**: 48-input operator with fw/bw LSTM cells sharing the same input tensor. States at indices 35-38. - All converters propagate final states to exp_tab for multi-step correctness. - Peephole, projection, layer norm, and aux input are not supported (raise OpNotImplemented). ## Testing - `test_unidirectional_sequence_lstm_none_activation` — output shape [batch, time, num_units] - `test_bidirectional_sequence_rnn_none_activation` — merge_outputs=True, shape [batch, time, 2*num_units] - `test_bidirectional_sequence_lstm_none_activation` — merge_outputs=True, shape [batch, time, 2*num_units] ```bash python -m pytest tests/python/relax/test_frontend_tflite.py -k "sequence_lstm or sequence_rnn" -v ```
## Summary This PR adds Relax TFLite frontend support for the TFLite builtin `STABLEHLO_WHILE` operator. `STABLEHLO_WHILE` uses StableHLO `BuiltinOptions2` to reference its condition and body region subgraphs. Its loop semantics otherwise match the existing TFLite `WHILE` importer path: loop-carried tensors are passed to the cond/body subgraphs, the cond subgraph returns a scalar bool, and the body subgraph returns the updated loop state. ## Design ### Shared While Lowering The native TFLite `WHILE` converter is refactored through a shared `_convert_while_like` helper. Native `WHILE` and `STABLEHLO_WHILE` now share the same validation and lowering path after their options are parsed: - native `WHILE` reads `WhileOptions` from `BuiltinOptions` - `STABLEHLO_WHILE` reads `StablehloWhileOptions` from `BuiltinOptions2` Both paths lower the referenced cond/body subgraphs to private Relax functions and emit a recursive private Relax function for the loop. ### Boundary Validation `STABLEHLO_WHILE` reuses the same guard-first checks as native `WHILE`: - loop input count must match op output count - cond subgraph input metadata must match loop-carried tensors - cond subgraph must have exactly one output - cond output must be a scalar bool tensor - body subgraph input and output metadata must match loop-carried tensors - referenced cond/body subgraph indices must be valid non-main subgraphs The recursive loop-function cache key now includes the generated function prefix. This prevents native `WHILE` and `STABLEHLO_WHILE` from accidentally sharing a cached loop wrapper if they reference the same cond/body subgraph indices. ## Operator Support | Operator | TFLite options | Relax lowering | Supported subset | |---|---|---|---| | `STABLEHLO_WHILE` | `StablehloWhileOptions.CondSubgraphIndex()`, `BodySubgraphIndex()` from `BuiltinOptions2` | recursive private Relax function | tensor loop-carried state, scalar bool cond output, matching cond/body interfaces | ## Tests The tests manually build a minimal StableHLO while TFLite flatbuffer and compare the imported Relax IR with `tvm.ir.assert_structural_equal`. Unsupported patterns use `pytest.raises`. | Test | Coverage | |---|---| | `test_stablehlo_while` | basic `STABLEHLO_WHILE` recursive private function lowering | | `test_stablehlo_while_non_bool_condition_unsupported` | cond output scalar bool guard | | `test_stablehlo_while_invalid_index_unsupported` | invalid cond/body subgraph index guard | | `test_stablehlo_while_output_count_mismatch_unsupported` | body output arity guard | | `test_stablehlo_while_input_metadata_mismatch_unsupported` | cond subgraph input metadata guard | | `test_stablehlo_while_output_metadata_mismatch_unsupported` | body subgraph output metadata guard | Local validation: ```bash python -m py_compile \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m ruff check \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m pytest \ tests/python/relax/test_frontend_tflite.py \ -k stablehlo_while -q python -m pytest \ tests/python/relax/test_frontend_tflite.py \ -k stablehlo -q ``` Result: ```text py_compile: passed ruff check: All checks passed stablehlo_while tests: 6 passed stablehlo tests: 84 passed ``` ## References - Issue apache#19519 item I: remaining StableHLO operators in TFLite - PR apache#19587: StableHLO region-based ops and multi-subgraph model support - PR apache#19616: TFLite control-flow / multi-subgraph support
…on (apache#19643) This PR will fix apache#19592. LayerNorm could produce NaN on large-value, small-variance inputs due to catastrophic cancellation in var = E[x^2] - E[x]^2. Switch to a numerically stable two-pass formulation: - pass1 computes mean via sum(x) / N - pass2 computes variance via sum((x - mean)^2) / N
…19916) Structural diagnostics can identify a field below an object that TVMScript renders without exposing that field. An underline alone then points at the nearest visible parent and hides the full internal location. This change keeps Script string-returning while making the diagnostic context self-contained. When the requested path is <root>.dtype, Script now returns this string by default: ```text Access path: <root>.dtype Note: The underlined object is the nearest visible parent of this path. T.int32 ^^^^^^^ ``` render_invisible_path_info defaults to true. Calls without target paths are unchanged, and callers can set it to false to retain the legacy underline-only string. The implementation reuses the printer span-selection logic to capture the deepest visible path and assembles the minimal access-path/note/script block in C++. Pass-error enrichment uses the same Script path. Production Python remains unchanged; focused Python tests assert the complete strings for default-on, explicit-false, hidden, exact-visible, unavailable-visible, pass-error, and structural-equality cases.
…e#19904) FlashInfer 0.6.3 changes the paged-attention plan/run ABI: the prefill/decode plans take new arguments (e.g. window_left, fixed_split_size, disable_split_kv; the decode plan now dispatches dtype through empty q/kv tensors), the runs add enable_pdl and drop the explicit stream, and the kernels consume separate key/value paged caches read through tensor strides rather than one combined tensor. This updates the runtime attention backend (paged MHA, ragged, decode and MLA) to the new signatures and to the Array<int64_t> plan-info representation. FlashInfer 0.6.3 reads tensors from `data` directly and does not honor the DLPack `byte_offset` field. mlc's auxiliary index tensors (qo_indptr, kv_indptr, page_indptr, page_indices, length_info) are views packed into a shared workspace and so carry a non-zero byte_offset; passed as-is the kernels read the wrong addresses (e.g. a ragged prefill processed only the first query row). Three zero-copy DLPack view helpers address this: `ZeroByteOffsetView` folds byte_offset into the data pointer, `PagedKVCacheView` exposes the combined (num_pages, 2, ...) page tensor as separate strided key/value caches, and `SliceLastDimView` slices the last dimension for MLA. This also completes the MLA FlashInfer path, which previously shipped only the test and module generator. The MLA run splits the query into nope/pe parts and the paged cache into ckv/kpe parts, and the ragged self-attention is given its own uncompressed head dims and per-query kv head count via a 5-element backend spec, since they differ from the compressed MLA cache. The MHA and MLA FlashInfer KV-cache tests are re-enabled as regression coverage, guarded on FlashInfer availability (inline-RoPE is skipped as unsupported by FlashInfer).
## Related Issue closes apache#19689 ## Why The Relax AffineGrid op only handled 2D (4D theta/grid); 5D 3D inputs from ONNX failed. ## How - Generalize struct-info inference to 2D/3D via spatial = size_sinfo->ndim. - Branch TOPI affine_grid compute on 2D vs 3D. - Add the 3D permute path in the frontend and a test_affine_grid_3d case. --------- Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
## Why The bool→int8 backing-array conversion was duplicated inline across `FlattenBuffer` and `LowerTIRxCleanup`. ## How - Add `LowerBoolBuffer` pass that rewrites `bool` buffers to `int8` and inserts load/store casts. - Remove the duplicated bool handling from `FlattenBuffer` and `LowerTIRxCleanup`. - Run it after FlattenBuffer (before VectorizeLoop) in every pipeline, with structural and build-and-run tests. --------- Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
This pr adds a onnx helper that cleaned by apache#19880 and fixes ci error
## Summary - Make `PrimExpr` a typed C++ view over `Expr` values whose `ExprNode::ty` is `PrimType`, instead of using a separate runtime node class as the proof of primitive-ness. - Use the shared `ir::Call` node for Relax, TIRX, and primitive-valued calls, while keeping primitive-only APIs explicit at their semantic boundaries. - Keep Python on the general `Expr` surface for primitive-typed values so `isinstance` behavior does not imply a nominal primitive-expression subclass. ## Design Rationale The main advantage of this change is that common expression nodes such as `Call` can be unified without specializing each one to `PrimType`. A single `ir::Call` can represent a Relax tensor call, a Relax scalar call, or a primitive-valued intrinsic call; the result type stored in `ExprNode::ty` determines whether that particular value can be viewed as `PrimExpr`. This keeps the IR node hierarchy focused on expression structure rather than result-type categories. Nodes that are intrinsically primitive, such as integer and floating-point literals or TIRX primitive operators, still have strongly typed C++ APIs and data structures. General nodes whose result type may vary, such as `Call`, remain general `Expr` nodes and are narrowed to `PrimExpr` only where primitive-only semantics are required. The PR also keeps the compatibility surface practical: C++ primitive-only APIs continue to accept `PrimExpr`, Python exposes a compatibility predicate for checking the primitive typed category, and visitors/printers use one natural `Call` path rather than duplicating Relax and primitive call handling. Missing expression types are represented explicitly with `Type::Missing()` so constructors can leave type inference to later analysis without relying on nullable `Type` values.
## Rationale
`SizeVar` encodes nonnegativity in runtime subtype identity, which is
fragile under cloning and remapping. Symbolic integer values should use
one `Var` representation, with nonnegative facts recorded in the
analyzer at the use sites that establish them.
## Changes
- Remove `SizeVar` from the C++, Python, TE, TVMScript, FFI, visitor,
and serialization surfaces, and migrate callers to `Var`.
- Preserve the existing Relax constraint ownership model and use
`MarkGlobalNonNegValue` as the canonical path for global nonnegative
facts.
- Preserve `T.handle()` as the normal opaque-handle form. An optional
dtype constructs a typed pointer, with `T.handle("void")` reserved for
an explicit pointer-to-void.
Expression unification gives TIRX a shared `Expr` surface, but its visitor and mutator APIs still expose primitive-only signatures. That mismatch prevents general expressions from flowing through the existing traversal structure and leaves statement traversal with overlapping customization hooks. This refactor generalizes the existing `ExprFunctor`, `ExprVisitor`, and `ExprMutator` signatures in place to accept and return `Expr`. Statement visitors and mutators expose a single virtual `VisitExpr(const Expr&)` hook, while primitive statement reconstruction uses a non-virtual checked `VisitPrimExpr` helper so invalid narrowing fails at the boundary. Public pre-order and post-order traversal entry points accept general `Expr` roots. The existing specialization, vtable, dispatch registration, and class structure remain intact; the change adds no parallel functor, fallback dispatcher, or alternate implementation path.
Replace MakeConst with direct IntImm construction at call sites whose static contracts guarantee scalar integer constants. The repository-wide audit converts 80 call sites across 32 files. Generic construction remains where runtime dtype, vector behavior, unsigned range, overload resolution, or invalid-input diagnostics require it. Eleven initially proposed conversions were reverted after compilation and focused tests exposed false positives. Validation: - LLVM-enabled compiler and runtime library build - Focused arithmetic, TIR, S-TIR, TOPI, TE, LLVM-codegen, reflection, and printing suites - Changed-file pre-commit and git diff checks
…9933) ## Rationale After `PrimExpr` and `Expr` share one typed expression hierarchy, expression types are part of semantic identity. Structurally identical syntax with different types compare and hash differently, while source spans remain diagnostic metadata. ## Invariant `ExprNode::ty` participates in structural equality and hashing by default. `GlobalVar` and Relax variables retain their symbol identity rules. `tirx.PrimFunc` compares and hashes authoritative source fields while excluding its derived type cache until all transformation paths maintain that cache eagerly. Nested symbolic-shape rendering is isolated from outer diagnostic configuration so diagnostic context cannot become script-token content. ## Changes - include expression types in generic structural equality and hashing - preserve GlobalVar and Relax variable identity plus definition-safe SeqExpr traversal - compare and hash PrimFunc from authoritative fields while excluding its stale derived type cache - normalize narrow Relax construction and expected-fixture types exposed by stricter identity - isolate nested symbolic-shape token rendering from outer printer configuration
Generated LLVM modules currently cache imported packed functions through an unsynchronized null check and plain store. Concurrent first calls can therefore race the lookup and cache update. This change: - routes generated C and LLVM lookups directly through TVMFFIEnvModLookupFromImports and removes the superseded TVMBackendGetFuncFromEnv wrapper; - gives each LLVM cached function a readable internal hidden initializer and publishes its module-owned result through TVMFFIHandleInitOnce; - removes the unused TVMBackendRunOnce export and dead LLVM static-init producer while preserving the live static-handle path. Validated with the existing LLVM, C-host, common codegen, static-init, and C++ test suites.
…pache#19941) The Python unittest CI step previously looped over each collected test directory and invoked `run_pytest` once per directory. Splitting related tests across many pytest invocations fragments Jenkins failure reports. This change collects the ordered test directory list into a `PYTEST_TARGETS` array first, then passes the full target set to a single `run_pytest` invocation. Target ordering, pytest flags (`--reruns=3`, `-n=1`), `PYTEST_ADDOPTS` environment setup, and per-test failure behavior are unchanged; the platform-minimal run remains a separate invocation with its own JUnit XML. This keeps related failures within one pytest report for clearer CI output.
Add tvm.testing.run_with_gpu_lock backed by the existing tvm_ffi.utils.FileLock. Migrate live local GPU tests to acquire the machine-local lock around device execution, synchronization, host transfer, and checks while leaving target construction and compilation outside the critical section. Replace the custom xdist scheduler with standard xdist_group placement for the order-dependent test family. RPC tests retain dynamic port allocation and per-test process isolation rather than gaining a broad category lock.
…e#19940) Replace the nested `DocToPythonScript` invocation in Relax dependent-shape printing with an expression-string Doc rendered during the active `PythonDocPrinter` traversal. This keeps recursive IR-to-doc conversion inside the active docsifier, preserves naming, precedence, escaping, source paths, and printer configuration, and avoids a nested top-level renderer or a new public rendering entry point. Expression-string escaping is streamed directly into the final output so wrapper and nested source spans retain exact escaped byte offsets.
) ## Summary - Move whole-Tensor arithmetic and cast dispatch onto `te.Tensor` while scalar TIRx smart constructors decline whole-Tensor operands. - Remove the legacy `tirx.generic` module, TOPI import-time mutation bridge, and obsolete aliases. - Migrate scan and cast callers while preserving identity-gated Thrust sum selection. Whole-Tensor behavior now lives with TE, leaving scalar TIRx construction independent of TOPI initialization.
## Rationale
TIRx variables use inherited `ExprNode::ty` as their single semantic
type. Retaining a primitive handle surrogate erases the distinction
between scalar values, typed pointers, and true opaque pointers, then
forces later passes and code generators to reconstruct information that
the IR already owns.
## Changes
- Remove the duplicate reflected `Var::type_annotation` state and
preserve exact `PrimType` or `PointerType` through construction,
visitors, transforms, specialization, builders, printers, and code
generation.
- Keep scalar-only boundaries explicit through `PrimExpr`, `PrimVar`,
and `PrimType`; pointer-capable values remain general `Expr` or `Var`.
- Keep helper boundaries no broader than their contracts: TE tensor
variable indices use `PrimVar`, while expression deep equality recurses
through general `Expr` only where pointer-bearing `Call` arguments
require it and does not generalize private arithmetic subclasses.
- Keep core statement reflection typed as `Expr`, name general
reinterpret targets as `target_ty`, and preserve exact pointer calls in
the general vectorization path with explicit scalarization behavior.
- Delete `PrimType::Handle()` and `PrimType::IsHandle()`. True opaque
pointers use `PointerType::VoidPointerTy()`; TVMScript renders the
canonical global type as `T.handle`, standalone values as `T.handle()`,
and scoped void pointers with a keyword-only storage scope.
- Make `CodeGenSourceBase::SSAGetID` a single `Type` boundary across
source backends, without a separate primitive-type or runtime-dtype
variant.
- Keep WebGPU semantic argument classification type-aware: storage
buffers are identified from `PointerType`, POD arguments from
`PrimType`, and only the final `FunctionInfo` launch ABI is serialized
to `DLDataType`.
- Preserve exact pointer semantics at runtime boundaries, including
access pointers, packed calls and returns, external calls, storage
rewrites, and target-specific lowering.
## Migration guide
- **Variable types:** In C++, replace `var->type_annotation` with
`var->ty`; in Python, replace `var.type_annotation` with `var.ty`. The
result is the exact `Type`: scalar variables carry `PrimType`, while
pointer variables carry `PointerType`.
- **Scalar boundaries:** Use `PrimVar` and `PrimExpr` for variables and
expressions that are semantically scalar. When starting from a general
view, narrow explicitly with `var.as_or_throw<PrimVar>()` or
`expr.as_or_throw<PrimExpr>()`. Keep pointer-capable fields and call
arguments as `Var` or `Expr`. A default-constructed `PrimVar` is
nullable, so construct local scalar variables explicitly, for example
`PrimVar i("i")`.
- **Opaque pointers:** Replace `PrimType::Handle()` with
`PointerType::VoidPointerTy()`. Replace `IsHandle()` tests with explicit
`PointerType` inspection; use `PointerType(element_type, storage_scope)`
when the pointee type is known instead of erasing it to a runtime handle
dtype.
- **TVMScript handles:** Use `arg: T.handle` for a global void-pointer
annotation and `arg = T.handle()` for a standalone value. Use
`T.handle(storage_scope="shared")` for a scoped void pointer. Typed
pointers use forms such as `T.handle("float32")`, `T.handle("float32",
"global")`, or `T.handle("float32", "shared")`. Legacy
`T.handle("void")` input remains parse-compatible, but the printer
canonicalizes it to `T.handle` (or the keyword-only scoped form).
- The separate `tirx.type_annotation` intrinsic used by access-pointer
APIs is unchanged; this migration removes only the duplicate variable
field.
## Validation
- Complete native C++ test executable: 122/122 passed, including
`IRF.CountVar`.
- Relax binding-rewrite suite: 12/12 passed, including transferred-user
bookkeeping.
- Canonical typed/void/scoped TVMScript handle printer and round-trip
checks: 5/5 passed.
Retire GitHub automation that is disabled, non-functional, unowned, or explicitly no longer wanted, together with its orphaned support code and tests. This removes the optional Dependabot version-update config; automatic team mentions and cc-review requests; last-successful and nightly branch advancement; reviewer pings; the disabled PR-comment bot; and the unused CI-resource upload path. Active CI, release, tvm-bot, and manual Docker-update workflows remain, with narrower permissions and event guards, plus refreshed issue-template and network-resource guidance. The behavior change is intentional: reviewer requests and team routing become manual, and the nightly and last-successful branches are no longer advanced by this repository. Dependabot security updates remain controlled by repository settings.
This PR simplifies Jenkins pytest execution around standard pytest-xdist behavior. - Runs each already-filtered CPU/GPU suite once with `-n auto`; the broad suite keeps load-group scheduling because its order-sensitive cases require it. - Removes external sharding, wrapper/profile code, JUnit XML generation and publication, the skipped-test XML consumer, obsolete suite naming, and orphaned helpers. - Retains one inert `task_clear_pytest.sh` entry point only because PR jobs evaluate their Jenkinsfile from the trusted base branch before checking out the PR; it performs no cleanup or reporting and can be removed after this pipeline lands. - Corrects stale broad-suite paths and explicit target guards, and migrates a scalar stride test to the current `T.handle` pointer semantics while preserving its negative lowering check. - Prevents nested MetaSchedule/XGBoost unit tests from multiplying CPU fanout without serializing the full suite. - Builds only the `tvm_runtime` target for the secondary GPU configuration and removes its unconsumed `gpu2` artifact upload. The result reduces parallelism to one layer managed by pytest-xdist while preserving GPU filtering and native failure visibility.
## Summary - remove redundant parentheses from four VM dtype comparisons - clean up pessimizing moves, missing overrides, unused state, and hidden virtual overloads reported by macOS Clang - preserve `PrimType` across source-codegen type-bearing paths and remove transitional raw dtype adapters - retain raw `DLDataType` only for explicit runtime launch metadata and runtime-helper boundaries - leave the deferred LLVM and CMake compatibility paths unchanged `PrimType::operator==` already compares the represented dtype by code, bits, and lanes, so no backend-specific non-identity equality regression is needed. ## Validation - compiled all original warning-producing translation units with Clang 22 and the relevant warning families promoted to errors - built `tvm_runtime` and `tvm_compiler` with LLVM 15 after the codegen migration - passed 14 focused C-host, bool, OpenCL, Metal, device, and Vulkan codegen tests, with 70 hardware-dependent skips - generated OpenCL C, Metal, WGSL, and CUDA source from the same typed kernel - compiled the changed Hexagon and Vulkan translation units independently - passed pre-commit, Git whitespace/log checks, and a static audit leaving raw codegen `DLDataType` declarations only at the WebGPU runtime ABI boundary
…#19927) This PR updates `IRMutatorWithAnalyzer` and `IRVisitorWithAnalyzer` to add the constraint `loop_extent > 0` while visiting a `ForNode` body. If execution reaches the loop body, the loop must have at least one iteration, so the body context can safely assume the extent is positive. The constraint is scoped only to the loop body, leaving the loop header expressions (`min`, `extent`, `step`) outside of this assumption. While validating this change, it exposed an issue in `IntSetAnalyzer`: one-sided constraints such as `m > 0` could cause existing parametric bounds like `m - 1` to be recursively relaxed to `+inf`. This caused `DomainTouched` to lose finite symbolic bounds. The PR fixes this by preserving existing parametric bounds when recursive interval relaxation would otherwise replace them with infinity.
## Summary - Keep the Python test launcher close to plain `pytest -n auto`, move nightly tests under `tests/nightly/python`, remove obsolete launchers and collection bookkeeping, and partition CPU/GPU jobs with explicit `gpu` marker expressions. - Repair exact-pointer regressions at their owning boundaries: packed raw-string ABI values, CUDA/Metal matrix intrinsic pointers, internal TE extern offsets, MetaSchedule scalar annotations, localized auto-tensorization scope matching, and typed DLTensor fixture fields. - Preserve typed workspace calls in TIR and cast pointer-returning external calls in CodeGenC, covered by a plain-TIRx 1024-byte global workspace that is compiled as C++. - Finish phasing out value-bearing Relax `R.Prim` annotations by requiring an explicit dtype, removing obsolete value-based contracts, and expressing the DISCO rank-dependent slices as explicit scalar `call_tir` inputs. - Gate the distributed callback on the optional DISCO runtime, NCCL, and at least two GPUs so capability-limited jobs skip instead of failing. - Remove the non-demonstrating pointer probe, use direct TVMScript comparison for packed strings, and remove the four designated legacy testing modules. The seven repaired CPU categories cover packed raw strings (7 failures), CUDA/Metal matrix access-pointer types (7), internal TE extern offsets (1), a typed DLTensor fixture (1), MetaSchedule scalar annotations (1), CodeGenC workspace return casts (12), and localized auto-tensorization storage-scope matching (19). ## Validation - Base: `ded6ad8dd212869c881efb5590f8a33fc972728e` - Head: `a7277e86dbcfe0638c8c252d36760859c4ab4297` - All 35 locally available original failing node IDs pass across the focused runs. - The full focused TE, TIR builtin-lowering, and CodeGenC files pass: 61 tests. - The complete touched Relax/TVMScript set plus PlanAndUpdateBufferAllocationLocation passes with 784 passed, 20 skipped, and 1 expected failure. - The DISCO callback collects and skips when its runtime or two-GPU environment is unavailable. - Six direct mapping tests, twelve tensor-core sketches, and the dp4a sketch pass unchanged. - The compiler rebuild, branch-wide pre-commit hooks, and full-range whitespace checks pass. - The 13 broad CBLAS/TFLite nodes remain dependency-gated; their owning TE and generated-C regressions compile. No merge is included in this change.
## Summary Update TVM to tvm-ffi's stable `TVMFFIAny`-backed Optional and Variant layout. - bump `3rdparty/tvm-ffi` from `84ee1a07` to `5411a642` - migrate Optional presence checks from the removed inherited `defined()` API - adapt pointer, cast, JSON, and FFI return sites while preserving missing-value semantics - require `apache-tvm-ffi>=0.1.13` and preserve the source-matched FFI build in the macOS wheel smoke test ## Validation Validated with linked LLVM builds, default and conditional backend compilation, an Emscripten 4.0.23 WebAssembly compile/link, C++ Optional/FFI and TVM suites, focused runtime/IR/Relax Python tests, broad regression coverage, repository format/static checks, production wheel metadata inspection, and an isolated source-FFI wheel install/import smoke test. ## Release compatibility blocker Do not merge or publish this bump until the tvm-ffi 0.1.13 release preserves compatibility for TVM 0.25's by-value JSON `Stringify` consumer, or an equivalent non-matching release/version strategy is in place. The release also needs coordinated handling for the published ORCJIT extension: `apache-tvm-ffi-orcjit==0.1.0` returns `Optional<Function>` by value while the new layout grows from 8 to 16 bytes, which can corrupt return storage when mixed with the new runtime. TVM's wheel now requires `apache-tvm-ffi>=0.1.13`; resolver-driven publish tests remain enabled so production publication fails closed until a compatible release exists.
## Rationale Analyzer constraint scopes continue to provide loop-positive facts to constant-bound and rewrite proofs. During IntSet relaxation, however, a scoped domain constraint is a refinement only for a variable explicitly present in the relaxation map. Applying it to an unmapped variable reinterprets a free parameter as a relaxation domain and can let a loop-local symbol survive recursive interval evaluation. ## Changes - Apply scoped IntSet constraints only to variables already present in the relaxation map; unmapped variables remain free parameters. - Remove the finite-bound restoration fallback and its stronger parametric-bound contract. - Add a direct compact-buffer regression that prevents a loop-local variable from escaping into a function-scope allocation extent.
Remove the Relax-specific Id indirection and use Var/DataflowVar object identity directly. Type-changing rewrites now remap definitions, uses, and binding lookups coherently while preserving reflection, serialization, and the DataflowVar subtype. Existing tests are adjusted for the API change; no new test files or test cases are added. Validation: the compiler and C++ tests build successfully; focused C++ coverage passes 4/4; the affected existing Python matrices pass 549 tests with 2 expected xfails; source censuses, diff checks, and applicable hooks pass.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request adds support for dynamic indices when gathering from a relax.ShapeExpr in the ONNX frontend by materializing the shape as an int64 tensor and performing a dynamic gather. A new test is also added to verify this behavior. Feedback points out a correctness issue in the constant-index fast path where multi-element constant indices are incorrectly handled, and suggests restricting this fast path to constants of size 1.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
The ONNX importer's Gather converter asserted that indices must be a constant whenever the data operand is a ShapeExpr, raising "Only constant indices supported for shape gather." for any runtime-computed index. Detection post-processing graphs such as FasterRCNN feed a dynamic index into a Gather whose data comes from a Shape node, so import failed before compilation could start. Keep the fast path for a single constant index, which resolves one dimension to a PrimValue and preserves shape-specialized handling downstream. Any other index (dynamic, or a constant selecting multiple dimensions) materializes the shape as an int64 tensor via shape_to_tensor and gathers from it at runtime, reusing the existing negative-index normalization. Adds a regression test that gathers a dimension out of a Shape result using a non-constant index, covering positive and negative indices, and checks it against onnxruntime. Fixes part of apache#19965.
hamzaqureshi5
force-pushed
the
fix/onnx-gather-dynamic-shape-index
branch
from
e0aa965 to
44b2dff
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The ONNX importer's Gather converter asserted that indices must be a constant whenever the data operand is a ShapeExpr, raising "Only constant indices supported for shape gather." for any runtime-computed index. Detection post-processing graphs such as FasterRCNN feed a dynamic index into a Gather whose data comes from a Shape node, so import failed before compilation could start.
Keep the constant-index fast path, which resolves a single dimension to a PrimValue and preserves shape-specialized handling downstream. For a dynamic index, materialize the shape as an int64 tensor via shape_to_tensor and gather from it at runtime, mirroring how Reshape already handles ShapeExpr inputs.
Adds a regression test that gathers a dimension out of a Shape result using a non-constant index and checks it against onnxruntime.
Fixes part of #19965.