Skip to content

Support more native datatypes in Walshadow - #92

Merged
harshil-goel merged 2 commits into
mainfrom
harshil/support-data-types
Aug 31, 2026
Merged

Support more native datatypes in Walshadow#92
harshil-goel merged 2 commits into
mainfrom
harshil/support-data-types

Conversation

@harshil-goel

@harshil-goel harshil-goel commented Aug 25, 2026

Copy link
Copy Markdown
Contributor
  • array[]
  • json
  • vector
  • geography
  • map
  • hstore

@harshil-goel
harshil-goel force-pushed the harshil/support-data-types branch from 07694b8 to 62a2051 Compare August 25, 2026 07:32
Comment thread src/catalog/type_bridge.rs Outdated
Comment thread Cargo.toml
Comment thread src/emit/ch_emitter.rs
// Composite columns take a whole PG array/hstore text (oracle-resolved)
// and fan it into nested buffers, independent of the value's own kind.
match buf {
ColumnBuf::Array { .. } => return encode_array(buf, v),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something seems wrong here, that we're taking oracle encoded values & then parsing them

maybe daemon protocol for oracle should be sending CH native blocks instead. extension can use pg-clickhouse-c or clickhouse-c

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harshil-goel
harshil-goel force-pushed the harshil/support-data-types branch from 62a2051 to 11c081d Compare August 25, 2026 08:33
Base automatically changed from harshil/update-configs to main August 25, 2026 08:34
@harshil-goel
harshil-goel force-pushed the harshil/support-data-types branch 3 times, most recently from c685471 to 06df467 Compare August 25, 2026 10:10
// `_<elem>` (PG array convention) → Array of a supported element, else
// the String fallback (geography stays String, rendered WKT at decode).
_ => {
if att.type_name == "hstore" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might need some pg_depend/pg_extension tracking to also check type's extension info

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_ => {
if att.type_name == "hstore" {
"Map(String, Nullable(String))".into()
} else if att.type_name == "vector" || att.type_name == "halfvec" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally we map to QBit, or at least halfvec to Array(BFloat16)

CH & pg disagree here: CH offers BFloat16 which is not 1:1 ieee half float

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#42

@harshil-goel
harshil-goel force-pushed the harshil/support-data-types branch from 06df467 to 5b5bde0 Compare August 26, 2026 12:45
Comment thread src/emit/ch_emitter.rs Outdated
Comment thread src/emit/ch_emitter.rs
@harshil-goel
harshil-goel force-pushed the harshil/support-data-types branch 2 times, most recently from 9205d19 to 3c81793 Compare August 27, 2026 09:59
Comment thread src/xact/xact_buffer.rs
@harshil-goel
harshil-goel force-pushed the harshil/support-data-types branch 2 times, most recently from 2925506 to b93c386 Compare August 31, 2026 11:46
@harshil-goel
harshil-goel force-pushed the harshil/support-data-types branch from b93c386 to f63b3ee Compare August 31, 2026 15:27
@harshil-goel
harshil-goel merged commit 7bd7f96 into main Aug 31, 2026
5 checks passed
@harshil-goel
harshil-goel deleted the harshil/support-data-types branch August 31, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants