[drk-ram]

Summary

Fixes #25.

main already has a safe GGUF header reader (gguf_metadata.py: magic/version/tensor_count/kv_count, general.architecture, general.name, quantization, tokenizer model, HTTP Range fetching of only the header, size/sanity limits) and maps architecture/hyperparameters/quantization into the AIBOM. The remaining gap from the issue was the chat template (# TODO: add chat template field mapping) and the cross-variant consistency signal. This PR adds those.

What is added

• Extraction: tokenizer.chat_template and named templates (tokenizer.chat_template.<name>).
• Integrity signal per file:
  • presence (true / false)
  • hash: sha256:<hex> of the template
  • provenance: gguf:<filename>#tokenizer.chat_template
  • security status: no-indicators-found / indicators-found (+ indicator ids) from a static heuristic scan for dunder/introspection access, URLs, long encoded blobs, hidden/bidi characters and oversized templates. This is a heuristic, not an attestation, and is documented as such; the template text itself is never copied into the AIBOM.
• Consistency across variants: the first shard of up to 8 GGUF files is read (headers only, 4 concurrent range fetches; later shards are skipped). Result consistent / inconsistent / not-comparable, plus examined/total counts, and per-file hashes when inconsistent. A template present in only some variants counts as inconsistent. An unreadable extra variant is skipped and reflected in the counts; an unreadable primary file still yields no GGUF metadata as before.
• AIBOM output: components[0].modelCard.properties entries in owasp:aibom:chatTemplate:* (present, hash, source, securityStatus, securityIndicators, consistency, variantsExamined, variantsTotal, variantHashes). Same custom-namespace approach as owasp:aibom:distribution:*, since the official genai:aibom taxonomy forbids unofficial names.
• Registry-driven scoring: chat_template_hash, chat_template_security_status, chat_template_consistency registered as GGUF-only supplementary fields (counted only when present), so they show in the Field Checklist and do not lower scores for non-GGUF models. The issue's third point (report vs registry-driven scoring) is already handled by the current scoring code.

Checked against real data: the scanner reports no indicators on the chat templates of Qwen2.5, Mistral-7B-v0.3, Phi-3, OpenHermes, Zephyr, Hermes-2-Pro, DeepSeek-R1-Distill and Gemma-2, and a live run against bartowski/Llama-3.2-1B-Instruct-GGUF (18 files) read 8 headers in ~14s and reported consistent.

Behaviour change to note

GGUFFileExtractor now fetches extra variant headers (bounded by MAX_GGUF_VARIANTS_EXAMINED = 8), so GGUF repos take longer to process. The existing test test_extract_metadata_selects_first_gguf_file asserted that only the first file is fetched; it now asserts the first file is the primary source, since the others are intentionally read for the comparison.

Tests

tests/test_gguf_chat_template.py (new, synthetic GGUF bytes from tests/fixtures, no network):

• template/named template extraction, hash equals the existing fixture helper, absent/blank handling
• mapping, template text not leaked, scan indicators (parametrized) and false-positive guards
• consistency matrix (identical, different, missing in one, none, single, empty)
• shard selection, extractor behaviour (tampered variant, unreadable variant, fetch bound, unreadable primary)
• AIBOM properties, registry/Field Checklist behaviour for GGUF vs other formats
• header-only HTTP Range fetch with a stubbed client: first request bytes=0-63, growth requests follow, never reads past the header

Validation

• pytest --no-cov (Python 3.11): 218 passed