Skip to content

A HyperDX row click finds its row, a source may alias its columns, and CSV answers (T-496) - #359

Merged
chasers merged 2 commits into
t-485-macros-per-statementfrom
t-496-with-expression-alias
Sep 20, 2026
Merged

chasers merged 2 commits into
t-485-macros-per-statementfrom
t-496-with-expression-alias

Conversation

@chasers

@chasers chasers commented Sep 20, 2026

Copy link
Copy Markdown
Owner

TL;DR: A HyperDX row click now finds its row, a source may alias its columns (ServiceName as service), and CSV output answers.

Tracker: T-496. Plan PL-66. Stacked on #358.

What changed

HyperDX does Before Now
Sends WITH (expr) AS alias for an aliased select list ❌ parse error ✅ rewritten
Row click: finds the row by the values it was shown ❌ 0 rows ✅ finds it
Asks for CSV (alert sample rows) ❌ unknown format CSV, CSVWithNames, CSVWithNamesAndTypes

Why the row click found nothing

  • The SQL was fine. The edge's answers were the problem.
  • A TIMESTAMP_NS answered 6 digits. Elixir's NaiveDateTime holds 6. HyperDX sent the value back, and it no longer matched the stored 9.
  • A map answered its keys in Elixir's order. DuckDB compares maps entry by entry, in order, as ClickHouse does.

How it works

  • Query reads a nanosecond column as integer nanoseconds. Format writes all 9 digits, in every format.
  • Frame.to_rows/2 takes map_entries: true. The edge answers a map in its stored key order.
  • Rewrite: WITH (expr) AS alias items leave the WITH list. The alias is written as its expression wherever it is read.
  • Rewrite: JSONExtract(json, 'Type') becomes a cast, for scalars, Map(K, V) and Array(T).
  • Rewrite: MD5(x) answers bytes, as ClickHouse's does. HyperDX wraps it in lower(hex(...)) for a long string.
  • Macros: toJSONString, leftUTF8.

Watch out

  • ⚠️ Behavior change: a TIMESTAMP_NS column now answers 9 fractional digits on this edge. It was 6.
  • ⚠️ Behavior change: a map's JSON keys are in stored order. They were sorted by Elixir.
  • ⚠️ An alias is not replaced where a SELECT defines it, after a ., or before a (.
  • ⚠️ WITH expr AS alias without parentheses is still a parse error. HyperDX always writes them.
  • ⚠️ New doc section: on one node with a SQLite catalog, HyperDX's parallel metadata queries hit smolquery's existing database is locked issue. Eight concurrent queries through the plain API reproduce it with no ClickHouse edge running. A Postgres catalog does not have it.

How it was verified

  • ✅ A local node with two rows that differ only in their last 3 nanosecond digits. HyperDX's row-click statement answers the one it was built from.
  • ✅ Through HyperDX's client: the alias histogram, the MD5 predicate, CSV.
  • ✅ Unit tests for each rewrite, the formats, and the round trip end to end.

Checks

  • mix precommit
  • mix ci
  • mix dialyzer

Review fixes

  • ✅ A WITH alias is read in its own scope: not inside a subquery, and not at all when a subquery defines the name for itself.
  • ✅ A later alias may use an earlier one: (a + 1) AS b, (b * 2) AS c.
  • SELECT ServiceName service (no AS) is a definition, not a call.

🤖 Generated with Claude Code

Chase Granberry and others added 2 commits September 20, 2026 02:59
…d CSV answers (T-496)

Four things HyperDX does that the edge did not take, each checked against
HyperDX's own code or the statement it builds:

- WITH (expr) AS alias. HyperDX sends one per alias of a source's select
  list (ServiceName as service), so a filter can say "service" in a
  statement whose SELECT does not define it, as its histogram's does not.
  The engine's WITH takes only subqueries. Rewrite drops those items from
  the list, drops the list when nothing else is in it, and writes the alias
  as its expression wherever it is read: not where a SELECT defines it, not
  after a dot, not before a parenthesis.
- A row click. HyperDX finds the clicked row by the values it was shown:
  ts=parseDateTime64BestEffort('...', 9) AND attrs=JSONExtract('{...}',
  'Map(String, String)') AND ... It found nothing, for two reasons that
  were the edge's answers, not its SQL. A TIMESTAMP_NS answered six digits,
  because a NaiveDateTime holds six; Query now reads such a column as
  integer nanoseconds and Format writes all nine. A map answered its keys
  in Elixir's order, and the engine compares maps entry by entry, as
  ClickHouse does; Frame.to_rows/2 takes map_entries: true and the edge
  answers a map in the order it was stored. JSONExtract(json, 'Type')
  becomes a cast, for scalars, Map(K, V) and Array(T). MD5 answers bytes,
  as ClickHouse's does, since HyperDX wraps it in lower(hex(...)) for a
  long string. toJSONString and leftUTF8 are macros.
- CSV, CSVWithNames and CSVWithNamesAndTypes, which HyperDX asks for alert
  sample rows.

Verified on a local node with two rows differing only in their last three
nanosecond digits: HyperDX's row-click statement answers the one it was
built from. The alias histogram, the MD5 predicate and CSV answer through
HyperDX's client.

docs/clickstack.md gains a section on the SQLite catalog: HyperDX sends its
metadata queries in parallel, and on one node with a SQLite catalog that
meets smolquery's existing "database is locked" issue, which eight
concurrent queries through the plain API reproduce with no ClickHouse edge
running. A Postgres catalog does not have it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… may use an earlier one, and a name defined without AS is not a call

WITH (expr) AS alias was a flat word replace over the statement. Three
statements the review ran showed what that misses:

- A subquery that defines the name for itself. With
  WITH (ServiceName) AS service SELECT service FROM (SELECT ServiceName AS
  service ...) ORDER BY service, the outer "service" is the subquery's
  column, and writing (ServiceName) there named a column that is not in
  scope. An alias a subquery defines with AS is now not substituted
  anywhere, and no name is substituted inside a subquery.
- Chained aliases. (a + 1) AS b, (b * 2) AS c left c as (b * 2), with b
  unresolved. Each expression now has the earlier aliases written into it.
- An alias defined without AS. SELECT ServiceName service became
  ServiceName (ServiceName), a call. A name is read only where a value can
  stand: after a keyword such as SELECT, WHERE, AND or BY, an operator, a
  comma or an opening parenthesis; not after another name, a closing
  parenthesis, a quoted name or a literal.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@chasers
chasers force-pushed the t-496-with-expression-alias branch from 9026a64 to 4129dde Compare September 20, 2026 03:11
@chasers
chasers merged commit 9096f60 into main Sep 20, 2026
12 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant