Skip to content

fix: honor XSD whitespace facets in numeric and boolean casts - #4355

Open
mvanhorn wants to merge 4 commits into
apache:mainfrom
mvanhorn:fix/4298-jena-xsd-cast-whitespace
Open

mvanhorn wants to merge 4 commits into
apache:mainfrom
mvanhorn:fix/4298-jena-xsd-cast-whitespace

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Oct 26, 2026 •

Copy link
Copy Markdown

Background

It addresses the behavior reported in issue #4298.

Summary of the fix

CastXSD.castByLex used to reject a string with leading or trailing whitespace when casting to a type whose whiteSpace facet is collapse. So xsd:integer(' 1') raised an error, while xsd:date(' 2026-01-01 ') already worked because date and time casts go through XSDFuncOp.dateTimeCast.

XPath and XQuery Functions and Operators 3.1, section 19.2 (casting from xs:string and xs:untypedAtomic), says the input string is first processed according to the whiteSpace facet of the target type. The numeric types, xsd:boolean and the duration types all use collapse, so surrounding whitespace should be stripped before the lexical form is checked.

With this change castByLex strips leading and trailing XML whitespace (#x9, #xA, #xD, #x20) before it validates and builds the term, for the same set of datatypes the old check covered. The result is built from the stripped string, so xsd:integer(' 1 ') gives "1"^^xsd:integer and str() of it is "1". Whitespace inside the value is still an error, for example xsd:integer('1 2').

Behavior change for durations: xsd:duration, xsd:dayTimeDuration and xsd:yearMonthDuration extend XSDAbstractDateTimeType and are cast through castByLex, so they pick up the same rule. xsd:duration(' P1D ') used to be an error and now casts to "P1D"^^xsd:duration. ' P 1D ' still fails.

Date and time casts other than durations are unchanged by this PR.

Verification

Tests for the jena CastXSD change live in jena-arq/src/test/java/org/apache/jena/sparql/expr/TestCastXSD.java, including cast_duration_whitespace for the three duration types. xsd_cast_10 to xsd_cast_13 in TestExpressions2 used to assert that surrounding whitespace is rejected; they now assert that the cast succeeds, and xsd_cast_14 checks that '1 2' still fails. The tests in jena-arq/src/test/java/org/apache/jena/sparql/expr/TestCastXSD.java fail without the fix in the jena CastXSD change (checked by reverting the source changes).

mvn -pl jena-arq -am -Dtest=TestCastXSD,TestExpressions2 test passes, as does a wider run over org.apache.jena.sparql.expr.Test*, org.apache.jena.sparql.function.Test*, Scripts_ARQ and Scripts_SPARQL.

Fixes #4298

By submitting this pull request, I acknowledge that I am making a contribution to the Apache Software Foundation under the terms and conditions of the Contributor's Agreement.

@ThomasThelen

ThomasThelen commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Review

This appears to break existing tests, widens the behaviour change to durations without mentioning anything about it, and leaves the whitespace in the lexical form of the result term. Were the unit tests run before making the PR? I see failures on my end.

[ERROR] Failures:
[ERROR]   TestExpressions2.xsd_cast_10:257 Expected org.apache.jena.sparql.expr.ExprEvalException to be thrown, but nothing was thrown.
[ERROR]   TestExpressions2.xsd_cast_11:258 Expected org.apache.jena.sparql.expr.ExprEvalException to be thrown, but nothing was thrown.
[ERROR]   TestExpressions2.xsd_cast_12:259 Expected org.apache.jena.sparql.expr.ExprEvalException to be thrown, but nothing was thrown.
[ERROR]   TestExpressions2.xsd_cast_13:260 Expected org.apache.jena.sparql.expr.ExprEvalException to be thrown, but nothing was thrown.

The tests sort of imply that rejecting surrounding whitespace was deliberate behaviour rather than an accident, so the PR should state the case for changing it and update or delete those tests, rather than leaving the contradiction for CI to find.

Scope is wider than the title

The removed clause also matched castType instanceof XSDAbstractDateTimeType. Dates and times were never affected, because string-to-temporal casts go through XSDFuncOp.dateTimeCast, which never calls castByLex. That is why the issue saw xsd:date(' 2026-01-01 ') succeed. The three duration types do extend XSDAbstractDateTimeType and do go through castByLex, so xsd:duration(' P1D ') changes from an error to a success with this PR. That is probably the right outcome, but it is untested and not mentioned in the title or description.

The result term keeps the whitespace

castByLex now builds the literal from the untrimmed string, so xsd:integer(' 1 ') returns the term " 1 "^^xsd:integer, and str() of that result gives " 1 " where the spec gives "1". Booleans come out canonical and numerics do not. The new tests are shaped around this:

  • cast_numeric_whitespace compares with sameValueAs plus the datatype URI instead of the exact-term testCast used everywhere else in the file.
  • cast_whitespace_controls asserts ' 2015-10-12 '^^xsd:date as the expected output term, which fixes the odd lexical form in a test.

Collapsing XML whitespace (#x9, #xA, #xD, #x20) before NodeValue.makeNode(lex, castType) would give canonical terms and let the new tests use testCast like the rest of the file.

Minor

  • cast_whitespace_select runs a full query inside an expression-level unit test class, and the isNumeric(...) and + 0 assertions do not exercise the changed code. Query-level tests in this module live in the scripted manifests under jena-arq/testing.
  • Import order: org.apache.jena.query.QueryExecution is listed before org.apache.jena.query.DatasetFactory.
  • The PR description is template text and does not describe the behaviour change. Please add the spec citation and the duration side effect.
  • Commits should be GH-<ticket_number>: Your_commit_message_here. See here for reference

Remove leading and trailing XML whitespace (#x9, #xA, #xD, #x20) before
casting a string to a numeric, boolean or duration datatype, so the
result has the canonical lexical form ("1"^^xsd:integer, not
" 1 "^^xsd:integer).

Update TestExpressions2 xsd_cast_10..13, which asserted the old
rejection, add duration coverage, and drop the query-level test.
@mvanhorn

mvanhorn commented Nov 8, 2026

Copy link
Copy Markdown
Author

Thanks for the careful review, you're right on all of it. I hadn't run the existing TestExpressions2 cases before opening this, and that's on me.

Pushed 6b3cd59:

  • castByLex now strips leading and trailing XML whitespace (#x9, #xA, #xD, #x20) before validating and building the term, for the same set of datatypes the old check covered. So xsd:integer(' 1 ') gives "1"^^xsd:integer and str() of it is "1".
  • The case for the change: XPath F&O 3.1 section 19.2 (casting from xs:string) says the input is first stripped per the target type's whiteSpace facet, and numerics, boolean and the duration types all have collapse. So xsd:integer(' 1') should cast rather than error.
  • xsd_cast_10 to 13 in TestExpressions2 now assert the cast succeeds (13 was a duplicate of 12, so it now covers trailing whitespace on xsd:integer), plus xsd_cast_14 for '1 2', which still fails.
  • Durations: called out in the commit message, with cast_duration_whitespace covering xsd:duration, dayTimeDuration and yearMonthDuration, and ' P 1D ' still failing.
  • The numeric tests use testCast with the exact term. I dropped the xsd:date control that cemented ' 2015-10-12 '^^xsd:date (date casts go through dateTimeCast, so this PR leaves them alone), the query-level test, and the stray imports.
  • The new commit uses the GH-4298: prefix.

Ran mvn -pl jena-arq -am -Dtest=TestCastXSD,TestExpressions2 test and a wider run over org.apache.jena.sparql.expr.Test*, org.apache.jena.sparql.function.Test*, Scripts_ARQ and Scripts_SPARQL, both passing on 6b3cd59.

// Cast by lexical form with checking.
private static NodeValue castByLex(String lex, XSDDatatype castType) {
if ( isWhitespaceCollapsed(castType) )
// These datatypes have the whitespace facet "collapse" so leading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe simply the comment,

// Remove empty white space around values

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b4eda4b, the comment in castByLex now just reads "// Remove empty white space around values". Thanks for the suggestion.

throw exception("Can not cast '"+lex+"' to a "+castType);
}

private static boolean isWhitespaceCollapsed(XSDDatatype castType) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From Andy's comment, it sounds like there's already a method in Jena to perform the white space removal. Is it possible to use that instead of this new method and isXMLWhitespace?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, switched to XML10Char.trim in d3b9b3d and removed trimXMLWhitespace and isXMLWhitespace from CastXSD. TestCastXSD and TestExpressions2 still pass.

Replace the private trimXMLWhitespace and isXMLWhitespace helpers in
CastXSD with the existing XML10Char.trim, which trims the XML 1.0 S
production characters.
@ThomasThelen

Copy link
Copy Markdown
Contributor

The code looks fine to me at this point. Here are some results from comparing URI patterns that are affected by this PR.

In the table below, T is any of integer, decimal, float, double, long, int, short, byte, nonNegativeInteger, positiveInteger, negativeInteger, nonPositiveInteger, unsignedLong, unsignedInt, unsignedShort. D is duration, dayTimeDuration or yearMonthDuration and v is a valid lexical form for the target type.

Pattern main PR
xsd:T(" v ") error Not a valid literal form (has whitespace): ' v ' "v"^^xsd:T
xsd:T("\tv\t") "\tv\t"^^xsd:T "v"^^xsd:T
xsd:T("\nv\n") "\nv\n"^^xsd:T "v"^^xsd:T
xsd:T("\rv\r") "\rv\r"^^xsd:T "v"^^xsd:T
xsd:T(" \t\n\rv \t\n\r") error Not a valid literal form (has whitespace) "v"^^xsd:T
xsd:double(" INF "), xsd:double(" NaN "), xsd:float(" -INF ") error Not a valid literal form (has whitespace) "INF"^^xsd:double, "NaN"^^xsd:double, "-INF"^^xsd:float
xsd:integer(" +1 ") error Not a valid literal form (has whitespace) +1
xsd:decimal(" 1 "^^xsd:integer) error Not a valid literal form (has whitespace): ' 1 ' "1"^^xsd:decimal
xsd:double(" 1 "^^xsd:integer) error Not a valid literal form (has whitespace): ' 1 ' "1"^^xsd:double
xsd:boolean(" true "), also padded with \t, \n, \r or a mix error Invalid cast: " true " -> xsd:boolean true
xsd:boolean(" 1 ") error Invalid cast: " 1 " -> xsd:boolean true
xsd:boolean(" 0 ") error Invalid cast: " 0 " -> xsd:boolean false
xsd:boolean(" true "@en), xsd:boolean(" true "^^<http://example/dt>) error Not a valid literal form (has whitespace): ' true ' true
xsd:boolean("\ttrue\t"@en), also \n, \r, and the same with ^^<http://example/dt> "\ttrue\t"^^xsd:boolean true
xsd:D(" P1D ") error Not a valid literal form (has whitespace): ' P1D ' "P1D"^^xsd:D
xsd:D("\tP1D\t"), also \n, \r "\tP1D\t"^^xsd:D "P1D"^^xsd:D
xsd:D(" \t\n\rP1D \t\n\r") error Not a valid literal form (has whitespace) "P1D"^^xsd:D
xsd:integer(" abc ") error Invalid lexical form: ' abc ' for …#integer error Invalid lexical form: 'abc' for …#integer
xsd:integer(" ") error Invalid lexical form: ' ' for …#integer error Invalid lexical form: '' for …#integer
xsd:integer(" 1.0 "), xsd:integer(" 1e0 "), xsd:integer(" INF "), xsd:decimal(" 1e0 ") error, message quotes the padded string error, message quotes the stripped string
xsd:boolean(" TRUE ") error Invalid cast: " TRUE " -> xsd:boolean error Invalid lexical form: 'TRUE' for …#boolean
xsd:boolean("\ftrue\f"), xsd:boolean("\u00A0true\u00A0") error Invalid cast: "…" -> xsd:boolean error Invalid lexical form: '…' for …#boolean
xsd:date(" 2026-01-01 "), and dateTime, dateTimeStamp, time, gYear, gYearMonth, gMonth, gMonthDay, gDay padded literal returned, e.g. " 2026-01-01 "^^xsd:date same as main
xsd:anyURI(" http://example/ ") " http://example/ "^^xsd:anyURI same as main
xsd:string(" abc "), xsd:string(" 1 "^^xsd:integer) " abc ", " 1 " same as main
xsd:integer(" 1 "^^xsd:integer) " 1 "^^xsd:integer same as main
xsd:D(" P1D "@en), xsd:D(" P1D "^^<http://example/dt>) error Invalid cast same as main
xsd:T("\fv\f"), xsd:T("\u00A0v\u00A0") error Invalid lexical form same as main
xsd:integer("1 2"), xsd:integer("") error Invalid lexical form same as main
unpadded arguments of every type and source same result same as main

@mvanhorn

Copy link
Copy Markdown
Author

Thanks for putting this comparison together, it lines up with what the PR is meant to change. The rows that match main are out of scope on purpose: date and time casts go through dateTimeCast, and \f and U+00A0 still fail because XML whitespace is only #x9, #xA, #xD and #x20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xsd:integer(' 1 ') with leading/trailing whitespace errors, contradicting isNumeric, arithmetic and the datatype's own validator

2 participants