Skip to content

API: Add fallible NestedField builder - #3190

Open
blackmwk wants to merge 1 commit into
apache:mainfrom
blackmwk:ir-3182
Open

blackmwk wants to merge 1 commit into
apache:mainfrom
blackmwk:ir-3182

Conversation

@blackmwk

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

  • Makes NestedField fields private and exposes read-only accessors.
  • Adds a typed NestedField builder whose build method returns Result and validates initial and write defaults against the field type.
  • Routes NestedField constructors, serde conversion, and internal schema rebuilds through the builder.
  • Removes the consuming NestedField with methods and makes NestedField and MapType convenience constructors fallible.
  • Migrates workspace consumers and updates the public API snapshot.

Are these changes tested?

  • Added unit coverage for builder field access and validation of primitive, fixed, struct, list, and map defaults.
  • cargo test -p iceberg --lib: 1,676 passed.
  • cargo test -p iceberg --doc --all-features: 83 passed, 13 ignored.
  • cargo clippy --workspace --all-targets --all-features -- -D warnings: passed.
  • make check-public-api: passed.
  • cargo fmt --all -- --check: passed.

AI Disclosure

This PR was implemented with assistance from OpenAI Codex for code migration, test scaffolding, and verification. The implementation and generated changes were reviewed, and the full core test suite, workspace clippy checks, formatting checks, doctests, and public API checks were run locally. There are no known unresolved uncertainties.

@dannycjones dannycjones left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a quick look, I think this is a much needed change, thanks @blackmwk

Comment on lines +2302 to +2303
.field_type(Type::Primitive(PrimitiveType::Long))
.initial_default(Literal::Primitive(PrimitiveLiteral::Long(114514)))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the type here changed - I think it's probably fine actually

Comment on lines -1848 to +1860
Arc::new(NestedField::new(
1,
"col_float",
Type::Primitive(PrimitiveType::Float),
false,
)),
Arc::new(NestedField::new(
2,
"col_string",
Type::Primitive(PrimitiveType::String),
false,
)),
Arc::new(
NestedField::new(1, "col_float", Type::Primitive(PrimitiveType::Float), false)
.expect("valid nested field"),
),
Arc::new(
NestedField::new(
2,
"col_string",
Type::Primitive(PrimitiveType::String),
false,
)
.expect("valid nested field"),
),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with these, I am wondering if it would be better to maintain these tests by constructing a simple vec, and then mapping with Arc::new and maybe even expect("all fields should be valid").

Comment on lines +723 to +733
pub(crate) fn rebuild(&self, id: i32, field_type: Type) -> Result<Self> {
Self::builder()
.id(id)
.name(self.name.clone())
.required(self.required)
.field_type(field_type)
.doc_opt(self.doc.clone())
.initial_default_opt(self.initial_default.clone())
.write_default_opt(self.write_default.clone())
.build()
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe some Rustdoc to explain in what scenario we may want to rebuild with a new ID or field type.

Comment on lines +1243 to 1247
.write_default(Literal::Primitive(PrimitiveLiteral::UInt128(
Uuid::parse_str("ec5911be-b0a7-458c-8438-c9a3e53cffae")
.unwrap()
.as_u128(),
)))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we can make some ergonomic improvements to some other APIs, by adopting Into<Literal> in write_default setter.

Suggested change
.write_default(Literal::Primitive(PrimitiveLiteral::UInt128(
Uuid::parse_str("ec5911be-b0a7-458c-8438-c9a3e53cffae")
.unwrap()
.as_u128(),
)))
.write_default(PrimitiveLiteral::UInt128(
Uuid::parse_str("ec5911be-b0a7-458c-8438-c9a3e53cffae")
.unwrap()
.as_u128(),
))

This can be a follow-up, and it would be opt-in so new tests and changes could pick it up.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same for impl From<PrimitiveType> for Type

Comment on lines 698 to 706
/// Construct a required field.
pub fn required(id: i32, name: impl ToString, field_type: Type) -> Self {
pub fn required(id: i32, name: impl ToString, field_type: Type) -> Result<Self> {
Self::new(id, name, field_type, true)
}

/// Construct an optional field.
pub fn optional(id: i32, name: impl ToString, field_type: Type) -> Self {
pub fn optional(id: i32, name: impl ToString, field_type: Type) -> Result<Self> {
Self::new(id, name, field_type, false)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be ergonomic here to return a builder, preconfigured as optional or required?

Then call sites are one of these:

let f0 = NestedField::required(0, "f0", PrimitiveType::Int).build().unwrap()
let f1 = NestedField::required(1, "f1", PrimitiveType::Int)
    .write_default(Literal::Primitive(PrimitiveLiteral::Int(5)))
    .build()
    .unwrap()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove pub field accessors in NestedField

2 participants