HyperDX's own code runs against the edge, 17 of 17: parametric aggregates, LIKE's escape, arrays, and the words DuckDB keeps (T-496) - #357
Merged
Conversation
chasers
added this pull request to stack #354
September 20, 2026 01:21
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 37463487 | Triggered | Generic Password | 4eb5520 | scripts/hyperdx-probe/probe.ts | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
…ates, LIKE's escape, arrays, and the words DuckDB keeps (T-496)
Until now the edge was checked against SQL copied from HyperDX's source.
This time HyperDX's code ran: its SQL generator, metadata reader and client
wrapper (hyperdxio/hyperdx @ c42dda8) over the real @clickhouse/client,
against a smolquery node with 300 rows. Search and the histogram passed.
Three things failed and one answered wrongly:
- getAllFields, getMapKeys and getKeyValues: a syntax error on
groupUniqArrayArray(1000)(keys) and groupUniqArray(20)(x).
- A search for user_id found 0 rows of 75. HyperDX writes
LIKE lower('%user\_id%'), and the engine's LIKE has no escape character
unless told, so the backslash matched itself.
- A hand-written FROM default.otel_logs did not parse: default is a word
the engine reserves.
SmolqueryClickHouse.Rewrite now moves a parametric aggregate's parameters
behind its arguments (quantile is quantile_cont, since ClickHouse's
interpolates), adds ESCAPE '\' after a LIKE pattern holding a backslash,
quotes a bare default database, and renames isNull, isNotNull and any,
which are the parser's own words. The macros gain groupUniqArray,
groupUniqArrayArray, groupArray and their -If forms, quantileIf,
getSubcolumn and lowCardinalityKeys.
HyperDX reads keysArr as an array, and a list answered as a String holding
JSON. Format now answers a list as Array(Nullable(T)): a JSON array,
['a','b'] in a tab-separated row, a length and elements in RowBinary.
scripts/hyperdx-probe/probe.ts is the run, with how to repeat it. The three
statements join the fixture, so the integration test keeps them working.
Five chart configurations through HyperDX's chart builder answer too.
Still not run: HyperDX's UI and API server. No container runs on the dev
box, and docs/clickstack.md says so.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…swers as JSON again, not as an Elixir term
- any( was renamed to any_value( wherever it appeared, which is right for
ClickHouse's aggregate and wrong for the quantifier both dialects have.
WHERE x = ANY(SELECT y FROM u) became x = any_value(SELECT ...), a parse
error, and the statement had run before this branch. An ANY( after a
comparison operator is now left alone. "ANY (" with a space never
matched, so the behavior had also depended on whitespace.
- Answering a list as Array(...) caught every {:list, _} dtype, and a map
whose values are not strings crosses the frame as a list of key/value
structs. SELECT MAP {1: 2} answered the text of an Elixir term,
[%{"key" => 1, "value" => 2}], in a tab-separated row, where it had
answered JSON; and inside an array RowBinary wrote it with no Nullable
marker under a type that promised one. A list of structs is a String
holding JSON again, Nullable(String) as an array's element, and every
format writes what its type says.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
chasers
force-pushed
the
t-496-hyperdx-sidebar-syntax
branch
from
September 20, 2026 03:11
970c7ea to
59bb6a9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR: HyperDX's own code now runs against the edge, 17 of 17 steps. This PR fixes the 3 failures and 1 wrong answer that run found.
Tracker: T-496. Plan PL-66. Stacked on #356 (T-500).
How it was found
hyperdxio/hyperdx @ c42dda8), over the real@clickhouse/client.What the run found
groupUniqArray(20)(x),groupUniqArrayArray(1000)(keys)user_idLIKE lower('%user\_id%'): DuckDB'sLIKEhas no default escape characterFROM default.otel_logsdefaultis a reserved word in DuckDBWhat changed
Rewrite: a parametric aggregatef(params)(args)becomesf(args, params).quantileisquantile_cont.Rewrite: aLIKEpattern holding a backslash getsESCAPE '\'.Rewrite: a baredefault.is quoted.isNull,isNotNull,anyare renamed: they are the parser's own words.groupUniqArray,groupUniqArrayArray,groupArray, their-Ifforms,quantileIf,getSubcolumn,lowCardinalityKeys.Format: a list answers asArray(Nullable(T)). It was aStringholding JSON. HyperDX readskeysArras an array.Watch out
Array(...), in every format. Before it was JSON text typedString.quantile_cont(...)), not as ClickHouse does. Onlycount()is renamed.WITH (expr) AS alias,JSONExtract(s, 'Map(...)'),toJSONString,CSVoutput.How to repeat the run
scripts/hyperdx-probe/probe.ts. Its header has the setup: clone HyperDX,npm install, run withtsx. Node only, no container.Tests
quantileby group,avg,max, filteredcountandsum,count(DISTINCT)).Checks
mix precommitmix cimix dialyzerReview fixes
x = ANY(SELECT ...)keeps itsANY. Only ClickHouse's aggregateany(x)is renamed.🤖 Generated with Claude Code