Skip to content

Simplify name tagger cache and align tests with production flags - #262

Draft
trungminhdo4-glitch wants to merge 1 commit into
opensanctions:mainfrom
trungminhdo4-glitch:fix/issue-241-static-taggers
Draft

Simplify name tagger cache and align tests with production flags#262
trungminhdo4-glitch wants to merge 1 commit into
opensanctions:mainfrom
trungminhdo4-glitch:fix/issue-241-static-taggers

Conversation

@trungminhdo4-glitch

Copy link
Copy Markdown

Problem

The tagger cache in rust/src/names/tagger.rs was an RwLock<HashMap<(TaggerKind, Normalize), Arc<Tagger>>> -- justified by a comment as "same shape as the org_types Replacer cache". The analogy doesn't hold: the Replacer receives arbitrary flags over FFI from Python, while the tagger has no Python surface at all. Its only production caller is analyze_names with the constant TAGGER_FLAGS (CASEFOLD | NAME).

Additionally, the unit tests built automatons with CASEFOLD | SQUASH_SPACES -- a combination pinned to a Python wrapper default that no longer exists. All tagger tests therefore exercised a configuration production never runs.

Changes

  • Remove the flag-keyed cache: The RwLock<HashMap<...>>, TaggerCache type alias, and TAGGER_CACHE static are all removed.
  • Add two LazyLock<Tagger> statics: ORG_TAGGER and PERSON_TAGGER, built with TAGGER_FLAGS (CASEFOLD | NAME).
  • Simplify get_tagger: Now takes only TaggerKind, returns &'static Tagger.
  • Move TAGGER_FLAGS to tagger.rs as pub(crate) const -- it lives where it's actually consumed.
  • Re-pin tests to production flags: Remove the test-local FLAGS constant and use the same statics your production code uses.
  • Update stale comments in reset.py, territories.rs, and person_names.rs.

Why two statics suffice

  • No Python surface for get_tagger
  • The only production call site is apply_tagger with a constant flag combination
  • No other callers request arbitrary Normalize flags
  • The RwLock<HashMap<...>> was guarding against a use-case that never materialised

Test changes

Old test FLAGS constant (CASEFOLD | SQUASH_SPACES) replaced by production statics (CASEFOLD | NAME). Pointer-equality tests renamed from cache_* to reflect static identity. No test expectations required semantic changes -- the test inputs (simple ASCII phrases) normalise identically under both flag combinations.

Validation

cargo fmt --check  # clean
cargo clippy -- -D warnings  # clean
cargo test  # 204 passed, 0 failed

Risk analysis

  • Initialisation order: LazyLock guarantees thread-safe, single-initialization semantics.
  • Deadlocks: No locks remain.
  • Hidden callers: Repository-wide grep confirms no other callers of get_tagger.
  • Public API: No PyO3 surface change.

Out of scope

  • Performance measurement (no benchmarks exist in the repo; static access is categorically faster than RwLock lookups)
  • Further refactoring of the Builder/factory functions

Closes #241

Replace the flag-keyed RwLock<HashMap<(TaggerKind, Normalize), Arc<Tagger>>>
cache with two LazyLock<Tagger> statics (Org, Person) built with the
production-normalisation flags (CASEFOLD | NAME).

The tagger has no Python surface and is only driven by analyze_names
with a constant flag combination, so the per-flag cache was guarding
against a use-case that never materialised. Two statics remove the
lock, map, key hashing, and unnecessary dynamic dispatching.

- Drop the RwLock/HashMap/Arc cache and its read/write lock overhead
- Add ORG_TAGGER and PERSON_TAGGER statics initialised lazily
- Move TAGGER_FLAGS to tagger.rs as pub(crate) const
- Re-pin tests to use the same flags as production
@pudo

pudo commented Jul 27, 2026

Copy link
Copy Markdown
Member

@trungminhdo4-glitch csn you confirm that you've read this PR and are prepared to explain it?

@trungminhdo4-glitch

Copy link
Copy Markdown
Author

Yes, I have read the full PR and I am prepared to explain it.

I also want to be transparent that I used AI assistance to help analyze the issue, inspect the relevant code paths, and develop the proposed solution. I did not intend to present the changes as work I had not reviewed: I read the resulting diff, checked the repository-wide call sites and assumptions, and ran the formatting, Clippy, and Rust test suite myself before opening the PR.

My understanding of the change is:

  • The previous cache supported arbitrary (TaggerKind, Normalize) combinations through an RwLock<HashMap<...>>.
  • get_tagger has no Python or PyO3 surface, and its only production caller uses the fixed TAGGER_FLAGS = CASEFOLD | NAME.
  • Because production only needs the organization and person taggers with that fixed configuration, the dynamic cache can be replaced by two thread-safe LazyLock<Tagger> statics.
  • Arc is no longer required because the taggers live for the process lifetime and can be returned as &'static Tagger.
  • The tests previously built separate taggers with CASEFOLD | SQUASH_SPACES, which did not represent the actual production path. They now exercise the same statics and flags used by analyze_names.
  • The existing test inputs normalize identically under both flag combinations, so no behavioral expectations needed to change.

I ran:

  • cargo fmt --check
  • cargo clippy -- -D warnings
  • cargo test -- 204 passed

If this repository prohibits or requires a specific disclosure for AI-assisted contributions and I missed that rule, I apologize. I am happy to follow your preferred process, revise the PR, provide a more detailed explanation of any part of the change, or close it if that is the appropriate action.

@pudo

pudo commented Jul 27, 2026

Copy link
Copy Markdown
Member

We're not against AI contributions, we just get a ton of fully Claude generated PRs that only exist for contribution farming and that take a ton of time to get merged and make a shit time for our team.

Do you think this one here has a real perf or correctness impact?

@trungminhdo4-glitch

Copy link
Copy Markdown
Author

Thanks for asking directly. I ran a small local A/B benchmark to answer this honestly rather than relying on theory about lock removal.

Setup: rustc 1.93.1, Intel i7-8700 @ 3.20GHz, --release builds with separate target dirs, 15 samples each.

Hot get_tagger access (5M calls/sample):

  • Baseline (RwLock read + HashMap lookup + Arc clone): 37 ns/call
  • PR (static reference, compiler constant-folds after inlining): ~0 ns/call

End-to-end tagging (200K calls/sample, 5 text inputs — "siemens aktiengesellschaft", "acme number one limited", "john smith", "isa bin tarif al bin ali", plus a no-hit case):

  • All baseline/PR ratios are between 0.98 and 1.02, across Org and Person taggers.
  • The isolated get_tagger win does not translate into a measurable E2E difference. The tag() call (100–8000 ns) dominates the 37 ns saved on the accessor.

Cold initialization (15 processes per variant):

  • Org: baseline 63.2 ms / PR 62.0 ms
  • Person: baseline 3.39 s / PR 3.26 s
  • Dominated by AC automaton construction; the lock/map overhead is invisible.

Conclusion: there is a real, measurable microbenchmark difference in the accessor, but it does not produce a meaningful end-to-end performance improvement. The PR also does not fix a reproduced correctness bug; the correctness-adjacent benefit is only that the tests now exercise the same configuration as production.

So I would characterize this as a cleanup, not a material performance or correctness improvement. It was useful for me to investigate, but I understand that this alone may not justify your team's review and maintenance cost. I am fine with closing it if it does not meet your bar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

names/tagger: drop the flag-keyed cache, re-pin tests to production flags

2 participants