Skip to content

feat(model): run long date and timestamp IN lists on SQL Server through STRING_SPLIT - #4329

Merged
bpamiri merged 2 commits into
developfrom
peter/mssql-date-in-list-split
Oct 5, 2026
Merged

bpamiri merged 2 commits into
developfrom
peter/mssql-date-in-list-split

Conversation

@bpamiri

@bpamiri bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Fixes #4318

Summary

On SQL Server, a statement whose IN lists push it past the 2,100-parameter limit binds its largest lists as one STRING_SPLIT parameter each (#4103 / #4321). Date and timestamp lists were left out and still threw Wheels.TooManyParameters. They now split too, matching exactly the rows the same list matches when it fits.

What the driver binds

Before choosing a cast, I measured what each engine's driver sends for a date or time parameter (SQL_VARIANT_PROPERTY of the bound value, SQL Server 2022):

Lucee 6 / 7 Adobe ColdFusion 2023 / 2025 BoxLang
cf_sql_timestamp datetime2(7) datetime2(7) datetime2(7)
cf_sql_date date date date
cf_sql_time datetime time(7), fraction dropped error

What that means for the cast. Because the normal path compares a datetime2(7) value, it misses fractional values on DATETIME columns and anything but whole minutes on SMALLDATETIME columns. Casting split values to the column's own type would match more rows than today. So the split casts to the bound type instead:

  • a timestamp list uses CAST(value AS DATETIME2(7));
  • a date list uses CAST(value AS DATE).

In a probe over every column type, those matched the normal path on 54 of 54 cases on both Lucee 7 and Adobe 2023. The normal-path behaviour itself is filed separately as #4326 and #4327; they will move the split cast together with the normal one.

Changes (MicrosoftSQLServerModel.cfc)

  • Reading the bind types. $stringSplitDateCasts(dataSource) reads the bind types once per datasource and keeps them in application.wheels.sqlServerDateBindTypes, next to the compatibility level, so a reload reads them again. The read itself is in $readDateBindTypes().
  • Mapping them to casts. $dateCastsFor() gives a cast only for the bind types this is proven to mirror: datetime2/7 and date/0. A driver that binds anything else keeps Wheels.TooManyParameters. No engine names are involved.
  • Accepted values. $isoDateTimeValue() accepts yyyy-mm-dd, optionally with HH:nn, :ss and .lll, or CFML's {ts '…'} / {d '…'} form. Each is written as yyyy-mm-ddTHH:nn:ss.lll, which SQL Server reads the same way under every language and DATEFORMAT setting.
  • Refused values. A list with any other value form keeps the error: locale formats, offsets or Z, other fraction lengths, or invalid dates. Those depend on the engine's own date parsing, which the split can't mirror.
  • Times. Time lists aren't split, since engines disagree and two of them error on the normal path.
  • Plumbing. $inListsToSplit, $stringSplitParts and $stringSplitList take an optional dateCasts argument. Without it, dates aren't split, as before.
  • Docs. The Wheels.TooManyParameters advice, the v4-2-0 query-builder guide, and an added changelog fragment.

Tests

New vendor/wheels/tests/specs/model/sqlServerDateStringSplitSpec.cfc (SQL Server only; it calls skip() elsewhere). It uses a table with one column of each type: date, time(7), datetime, smalldatetime, datetime2(7), datetime2(3) and datetimeoffset(7). The stored values include .003, .123 and .999 fractions, 23:59:59.997 and .999, midnight, a NULL row, and :29 / :30 seconds.

  • Differential, every column type. whereIn, whereNotIn and a where string IN return the same ids for a short list (one parameter per value) as for the same list padded past the limit. The padded list can only run if it really splits; otherwise it throws Wheels.TooManyParameters.
  • Date objects. CFML date objects match the same rows, short and padded.
  • Far past the limit. A timestamp list of 2,500 values runs.
  • Errors kept. Time lists and mm/dd/yyyy dates past the limit still throw Wheels.TooManyParameters.
  • Unit specs. They cover the ISO normalisation, the forms refused, the casts per bind type, and the one-read-per-datasource cache.

sqlServerStringSplitSpec's date case now expects an ISO date list past the limit to run, instead of throwing.

RED (these specs on develop's adapter, Lucee 7 + SQL Server): the differential, date-object, far-past-limit and ISO-date specs fail, and the unit specs error because the helpers don't exist yet.

GREEN:

Run Lucee 7 Lucee 6 Adobe 2023 Adobe 2025 BoxLang
model directory on SQL Server (pass / fail / error) 1411 / 0 / 0 1411 / 0 / 0 1408 / 0 / 0 1408 / 0 / 0 1411 / 0 / 0
sqlServerDateStringSplitSpec 9/9 passed 9/9 passed 9/9 passed 9/9 passed 9/9 passed
  • SQLite:
    • full core suite, Lucee 7: 6755 passed;
    • internal specs: 113 passed;
    • RustCFML v0.693.0 model directory: 1375 / 0 / 0. The new spec compiles there and skips.
  • Other checks: the complexity gate passes, and the guides build passes.

🤖 Generated with Claude Code

…gh STRING_SPLIT

A statement whose date or timestamp IN lists push it past SQL Server's
parameter limit threw Wheels.TooManyParameters. On compatibility level
130+, such a list now binds as one STRING_SPLIT parameter, each value cast
to the type the driver binds a cf_sql_timestamp / cf_sql_date as
(datetime2(7) / date), so it matches exactly the rows the same list
matches bound one parameter per value. Casting to the column's own type
would match more rows than that on DATETIME and SMALLDATETIME columns.

The bind types are read once per datasource with SQL_VARIANT_PROPERTY and
kept in application.wheels; a driver that binds other types keeps the
error. Values must be yyyy-mm-dd with an optional HH:nn[:ss[.lll]], or
CFML's {ts}/{d} form; time lists and other forms keep the error.

Fixes #4318

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
@bpamiri bpamiri added this to the v4.2.0 milestone Oct 5, 2026
@bpamiri

bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Review @ aae4cc3: source looks good, no blocking findings. Final verdict after matrix 37278068958 and PR CI

Differential spec (sqlServerDateStringSplitSpec)

  • It covers every comparable column type: d DATE, dt DATETIME, sdt SMALLDATETIME, dt2 DATETIME2(7), dt23 DATETIME2(3) and dto DATETIMEOFFSET(7).
  • The stored values are the awkward ones: .003, .123, .999, 23:59:59.997 / .999, midnight, :29 / :30 (SMALLDATETIME rounding), and a NULL row.
  • For each column, a short whereIn / whereNotIn runs one parameter per value, while the same list padded past 2,150 values can only run if it splits. Otherwise it returns ERROR Wheels.TooManyParameters, which can never equal the short list's ids. The spec asserts inLong == inShort and notInLong == notInShort, plus a where-string IN split by a forced limit (bothWays).
  • That is a real equality proof for IN and NOT IN on every column type, and the NULL row checks NOT IN's NULL semantics.
  • Date objects and a 2,500-value timestamp list are covered.
  • The TIME column is deliberately not in variables.columns. Time lists are covered by the "keeps Wheels.TooManyParameters" case.

The probe fails safe

  • $readDateBindTypes() reads SQL_VARIANT_PROPERTY(CAST(? AS SQL_VARIANT), 'BaseType' / 'Scale') for a bound cf_sql_timestamp and cf_sql_date, and returns {} on any error.
  • $dateCastsFor() grants a cast only for exactly datetime2/7 → DATETIME2(7) and date/0 → DATE. An unknown driver, or a driver configured differently, gets no cast, and a date list keeps Wheels.TooManyParameters. For example, mssql-jdbc's datetimeParameterType=datetime would make the timestamp read datetime/3.
  • No engine names are involved. The result is cached per effective datasource in the application's Wheels settings, so a reload re-reads it.

Strict parsing, no locale dependence

  • $isoDateTimeValue() accepts only yyyy-mm-dd, optionally with HH:nn, :ss and .lll (exactly 3 fraction digits), or the same inside {ts '…'} / {d '…'}.
  • It validates the calendar day ($isCalendarDate, so leap days are handled) and the hour/minute/second ranges, and rewrites each value as yyyy-mm-ddTHH:nn:ss.lll (or yyyy-mm-dd for a date list).
  • SQL Server reads ISO 8601 with T for DATETIME2, and yyyy-mm-dd for DATE, the same way under every LANGUAGE / DATEFORMAT. That is unlike DATETIME's yyyy-mm-dd, which is why the cast target matters.
  • Locale forms, offsets / Z, and other fraction lengths make the whole list fall back to the error.
  • Every value is reformatted from regex groups and bound inside the one nvarchar parameter, so no text from the value reaches the SQL.

Findings

On the matrix I'll read sqlServerDateStringSplitSpec and sqlServerStringSplitSpec on all five SQL Server legs, and the PR CI tally.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Wheels Test Results

221 735 tests   216 876 ✅  1h 27m 14s ⏱️
 17 120 suites    4 859 💤
     32 files          0 ❌

Results for commit ed691cc.

♻️ This comment has been updated with latest results.

@bpamiri

bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Final @ aae4cc3: APPROVE on code and on this matrix, gated on a develop merge and a fresh matrix

Matrix 37278068958 (all 32 legs read)

  • 32/32 legs have 0 fail, 0 error.
  • All five SQL Server legs (lucee6, lucee7, adobe2023, adobe2025, boxlang): sqlServerDateStringSplitSpec passes 9/9 with 0 skipped. That covers:
    • the per-column-type differential (date, datetime, smalldatetime, datetime2(7), datetime2(3), datetimeoffset(7) × whereIn / whereNotIn / where-string IN);
    • date objects;
    • a 2,500-value timestamp list;
    • Wheels.TooManyParameters kept for time lists and non-ISO dates;
    • the ISO normalisation, refused forms and cast-per-bind-type unit cases;
    • the one-probe-per-datasource cache.
  • sqlServerStringSplitSpec passes 12/12 on those legs, including the reworked "runs an ISO date list past the limit".
  • Every other leg: the SQL Server-only cases skip as designed (4/9 and 5/12).

PR CI: 26 pass, 3 skipping, 0 fail, 0 pending.

Gate: the branch isn't up to date with develop.

@bpamiri

bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Final: APPROVE @ ed691cc (develop merged, including #4325; matrix 37281004808, all 32 legs read; PR CI read)

Delta since aae4cc3

Matrix

PR CI: 26 pass, 3 skipping, 0 fail, 0 pending.

The gate from my previous comment is met. No open findings. The INFOs (a failed probe cached until reload; HH:nn vs HH:mm notation) stand, and #4326/#4327 must route the split cast through the shared helper.

@bpamiri
bpamiri merged commit 727c33b into develop Oct 5, 2026
67 checks passed
@bpamiri
bpamiri deleted the peter/mssql-date-in-list-split branch October 5, 2026 08:21
bpamiri added a commit that referenced this pull request Oct 5, 2026
The SQL Server adapter reads two facts once per datasource and keeps them
in application.wheels: the database's compatibility level (STRING_SPLIT,
#4321) and the types the driver binds dates as (#4329). A failed read was
kept too, so one transient error disabled STRING_SPLIT, or date-list
splitting, until the next reload.

A failed read is no longer kept. Its time is recorded, the answer is "not
supported" until $probeRetrySeconds() (60) have passed, and then the probe
runs again; a good read is kept as before. The back-off stops a broken
probe from running before every long statement.

Signed-off-by: Peter Amiri <peter@alurium.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(model): bind long date and time IN lists on SQL Server through STRING_SPLIT

1 participant