Replace FHT assembly with AVX intrinsics

AGENTS.md

@@ -194,6 +194,6 @@ unless the C++ contract becomes const-correct.
 
 ### Vendored code
 
-`include/rabitqlib/third/` and `include/rabitqlib/utils/fht_avx.hpp` are imported code. Do not
-reformat, refactor, or include them in project-wide lint fixes unless the task explicitly targets
-the vendor snapshot. Keep attribution and license text intact.
+`include/rabitqlib/third/` contains imported code. Do not reformat, refactor, or include it
+in project-wide lint fixes unless the task explicitly targets the vendor snapshot. Keep
+its attribution and license text intact.

CMakeLists.txt

@@ -126,8 +126,6 @@ set_target_properties(rabitq_core PROPERTIES
 rabitq_enable_warnings(rabitq_core)
 
 if(MSVC)
-    enable_language(ASM_MASM)
-    target_sources(rabitq_core PRIVATE src/simd/fht_avx2_masm.asm)
     set_source_files_properties(
         ${RABITQ_AVX2_SOURCES} ${RABITQ_HNSW_AVX2_SOURCES}
         PROPERTIES COMPILE_OPTIONS "/arch:AVX2"

DEVELOPMENT.md

@@ -28,9 +28,9 @@ Verify formatting without changing files:
 ./scripts/check-format.sh
 ```
 
-The scripts intentionally exclude vendored Eigen code and the imported FFHT
-implementation. To use a nonstandard executable name, set `CLANG_FORMAT`; it
-must still identify itself as clang-format 15.
+The scripts intentionally exclude vendored code in `include/rabitqlib/third/`.
+To use a nonstandard executable name, set `CLANG_FORMAT`; it must still identify
+itself as clang-format 15.
 
 clangd embeds its own formatter, so use clangd 15 in editors such as VS Code
 if format-on-save must exactly match CI. If another clangd version is required,
@@ -77,9 +77,9 @@ Use a fresh build directory when changing compilers. `CMAKE_CXX_COMPILER`
 selects the compiler explicitly; `CXX=c++` on the check command must match it.
 
 The wrapper supplies clang-tidy with that compiler's standard-library include
-paths and reports diagnostics only for first-party files. Vendored Eigen,
-hnswlib, and the imported FFHT implementation are excluded. New checks should
-be added incrementally after their existing first-party findings are fixed.
+paths and reports diagnostics only for first-party files. Vendored Eigen and
+hnswlib are excluded. New checks should be added incrementally after their
+existing first-party findings are fixed.
 
 Focused clang-tidy checks on affected code are sufficient during iteration.
 For first-party C++ changes, run the full check above before merging and report
@@ -275,7 +275,10 @@ must happen in generic code first.
   disabled for distributed builds; release wheels already set it to off.
 - Put calculations and scratch-storage helpers outside the dispatcher. Shared implementation
   headers use internal linkage so independently compiled backends retain their own bodies.
-  Include the FFHT implementation inside the private kernel namespace for the same reason.
+  The AVX FHT intrinsics in `src/simd/fht_kernels.hpp` use the private kernel namespace
+  for the same reason. Rotation uses this implementation on GCC, Clang, and MSVC;
+  no assembler is required. Keep the FFHT attribution and MIT license in the
+  intrinsics header intact.
 - Pass ordinary pointers, sizes, and library-owned query state across ISA boundaries. Never
   pass Eigen matrix/packet objects between backends. The private matrix implementation
   header includes Eigen under a namespace selected by each backend translation unit.
@@ -296,7 +299,7 @@ covers its arithmetic kernels, not every scalar loop in construction and search.
 | Integer scalar quantization, extra-code packing, transpose, sign masks | Existing runtime selection; byte layouts and rounding rules are unchanged |
 | Standard and high-accuracy FastScan | Runtime selection; generic LUT construction stays separate from selection |
 | IVF and float SymphonyQG batch correction | Complete estimator runs in the selected backend; non-float template paths remain generic |
-| FHT/Kac rotation | Complete rotation and scaling run in selected ISA translation units; imported FFHT AVX butterflies are unchanged |
+| FHT/Kac rotation | Complete rotation and scaling run in selected ISA translation units; shared AVX intrinsics preserve the FFHT butterfly order |
 | Float matrix rotation and PiPNN construction | Matrix products, row norms, and lower-triangle pairwise distances use isolated matrix backends |
 | HNSW search and IVF centroid routing | Cached HNSW search selection; centroid routing uses the common raw-distance dispatcher |
 | Quantization orchestration, reconstruction, non-float utilities | Template/control code remains generic; no blanket native tuning or reduction-order rewrite |

docs/docs/rabitq/rotator.md

@@ -5,6 +5,11 @@ RaBitQLib provides two types of random rotation. All implementations sample and
 
 By default, the library uses the `FFHT + Kac’s Walk` method. 
 
+The default rotator accepts input dimensions from 64 through 65,536 inclusive,
+including 16,384. It pads to a multiple of 64 and uses AVX intrinsics
+for power-of-two blocks through `2^16`. Allocate the
+output using `rotator->size()`, which can exceed the input dimension.
+
 The implementation can be found in `rotator.hpp`.
 
 ```css
@@ -49,6 +54,13 @@ rotator -> rotate(x.data(), x_prime.data())
 
 ## FFHT + Kac’s Walk
 ### Description
+Each transform uses the shared AVX intrinsics in `src/simd/fht_kernels.hpp`,
+compiled within the AVX2 or AVX-512 backend selected at runtime. GCC, Clang,
+and MSVC use the same implementation without inline assembly or MASM.
+The FFHT butterfly order, normalization, four sign-flip passes, and saved
+rotation state are preserved. These intrinsics still require x86 AVX;
+support for other CPU architectures requires an additional backend.
+
 This method is a combination of the well-known Fast Johnson-Lindenstrauss Transformation algorithms based on [Fast Hadamard Transform](https://www.cs.princeton.edu/~chazelle/pubs/FJLT-sicomp09.pdf) and ideas in [Kac’s Walk](https://projecteuclid.org/journals/annals-of-applied-probability/volume-27/issue-1/Kacs-walk-on-n-sphere-mixes-in-nlog-n-steps/10.1214/16-AAP1214.full). 
 It first samples 4 sequences of random signs (i.e., Rademacher random variables). Then for each vector, it repeats the following procedures 4 times.
 

include/rabitqlib/index/hnsw/hnsw.hpp

@@ -561,7 +561,8 @@ inline void HierarchicalNSW::load(const char* filename) {
         element_count > loaded.max_elements_ ||
         loaded.num_cluster_ > buffer::kSearchBufferMaxPointCount ||
         (element_count != 0 && loaded.num_cluster_ == 0) || loaded.dim_ < 64 ||
-        loaded.dim_ > 4095 || loaded.padded_dim_ != round_up_to_multiple(loaded.dim_, 64) ||
+        loaded.dim_ > rotator_impl::FhtKacRotator::kMaxDim ||
+        loaded.padded_dim_ != round_up_to_multiple(loaded.dim_, 64) ||
         loaded.ex_bits_ > 8 ||
         (loaded.metric_type_ != METRIC_L2 && loaded.metric_type_ != METRIC_IP) ||
         loaded.M_ == 0 || loaded.M_ > 10000 || loaded.maxM_ != loaded.M_ ||