Fix silent error-swallowing in Firebolt scripts; drop bogus t3a.small results#970
Conversation
t3a.small results
rschu1ze
left a comment
There was a problem hiding this comment.
I added updated measurements for firebolt - @thevar1able triggered them in the automation yesterday. These look reasonable, i.e. the updated error handling at least didn't break measurements.
BUT: The the automatic run for firebolt did not run on t3a.small, and it also did not run for firebolt-parquet or on firebolt-parquet-partitioned, so we don't have proof yet that the new error handling really works.
@thevar1able Could you start these extra runs^^ and push results to this PR? Besides that, LGTM.
|
@rschu1ze none of those jobs can run on t3a.small, see PR description. They fail on data ingest. Also the measurements from other tasks are probably not so meaningful, I got them before learning that I need to run the script with branch= var, i.e. they used main branch, without error handling fix. |
Ah, I needed to re-read the PR msg. Also, I found the failing run on
That's still fine. New results will be generated on the next run of the automation. |
Resolves #945
Summary
firebolt/queryandfirebolt/load(and their-parquet/-parquet-partitionedcopies) checked for failure with
jq -e '.errors' ..., butjq -ealso exitsnon-zero on malformed or empty input — which the old code treated identically
to "no .errors key present," i.e. success. A crashed/overloaded engine that
comes back with an empty 200 body or a non-JSON body could silently be
recorded as a fast, error-free query with no timing on stderr, or (for
load)surface only as an opaque
curl: (22) ... error 500with the actual reasondiscarded.
query: now checks HTTP status, non-empty body, and JSON-validityexplicitly before checking
.errors, and validates.statistics.elapsedis actually numeric before declaring success.
load: same idea via arun_sqlhelper — prints the real response bodyon any non-200/invalid-JSON/
.errorsresponse instead of swallowing it.Validated that both cases fail to load dataset on
t3a.small: