Core: Fix hash Variant object keys in ExpressionUtil.sanitize - #17840
Open
waterWang wants to merge 1 commit into
Open
Core: Fix hash Variant object keys in ExpressionUtil.sanitize#17840waterWang wants to merge 1 commit into
waterWang wants to merge 1 commit into
Conversation
Variant object keys are query literals, not schema identifiers, and can hold sensitive values like string literals. sanitizeVariantObject was formatting keys as (hash-<field>) with the original field name in the clear, leaking them into ScanReport filters and operator metrics/logs. Run object keys through sanitizeSimpleString (same as string values) so they are hashed like other predicate constants. Closes apache#17836
anoopj
approved these changes
Aug 27, 2026
anoopj
left a comment
anoopj
left a comment
Member
There was a problem hiding this comment.
Nice catch! Variant object keys are query literals, so leaving them in the clear didn't make sense. The code looks correct to me.
One minor comment; ExpressionUtil is in the API module and the change doesn't touch core/, so the title prefix should be API:[...] rather than Core: [...]
ebyhr
approved these changes
Aug 28, 2026
uros-b
approved these changes
Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ExpressionUtil.sanitizeredacts predicate constants (hashed strings, digit-count for numbers, coarse dates). For Variant objects it was formatting keys as(hash-%s)with the original field name while hashing only the value. Variant object keys are query literals (they can hold the same secrets people put in string literals), and leaked intoSnapshotScanScanReport.filter(REST catalog metrics) and logs.Fix
Run object keys through
sanitizeSimpleString(the same hashing used for string values), and update the unit test expectations to match.api/.../ExpressionUtil.java:sanitizeVariantObjectnow emits(hash-<hash-of-key>): <value>api/.../TestExpressionUtil.java: updatetestSanitizeVariantObject/testSanitizeVariantArrayexpectationsCloses #17836