Skip to content

fix(table): surface a malformed manifest size from DynamoDB - #9023

Open
jackylee-ch wants to merge 1 commit into
lance-format:mainfrom
jackylee-ch:fix/dynamodb-manifest-size-validation
Open

fix(table): surface a malformed manifest size from DynamoDB#9023
jackylee-ch wants to merge 1 commit into
lance-format:mainfrom
jackylee-ch:fix/dynamodb-manifest-size-validation

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

A manifest row's size attribute was read with .ok(), so a value that is not a number silently became None. The sibling version field, parsed from the same item seven lines below, reports the same failure with a descriptive error:

let size = item.get("size").and_then(|attr| match attr {
    AttributeValue::N(size) => size.parse().ok(),   // silently None
    _ => None,
});
...
let version = version.parse().map_err(|e| Error::invalid_input(format!(
    "dynamodb error: could not parse the version number returned {}, error: {}", version, e)))?;

Losing the size is not free either — read_manifest falls back to a HEAD request for the length, so a corrupt row costs an extra round trip on every read and gives no signal why.

Read the attribute in one place: an absent value stays None for rows written before the column existed, and a present one that is not a number is reported.

A manifest row's size attribute was read with .ok(), so a value that is
not a number silently became None. The sibling version field, parsed from
the same item seven lines below, reports the same failure with a
descriptive error. Losing the size is not free either: read_manifest
falls back to a HEAD request for the length, so a corrupt row costs an
extra round trip on every read and gives no signal why.

Read the attribute in one place, keeping an absent value as None for rows
written before the column existed and reporting a present one that is not
a parseable number.
@github-actions github-actions Bot added the bug Something isn't working label Sep 7, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The shared parser preserves the legitimate legacy absence case while consistently rejecting malformed stored metadata in both DynamoDB lookup paths. This enforces the existing (path, size) contract without changing writers or requiring migration.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant