Visitar URL original
GH-4300: Compare SPARQL strings by Unicode code point by shin4141 · Pull Request #4314 · apache/jena · GitHub
Skip to content

GH-4300: Compare SPARQL strings by Unicode code point - #4314

Open
shin4141 wants to merge 2 commits into
apache:mainfrom
shin4141:codex/jena-4300-codepoint-order
Open

shin4141 wants to merge 2 commits into
apache:mainfrom
shin4141:codex/jena-4300-codepoint-order

Conversation

@shin4141

@shin4141 shin4141 commented Oct 8, 2026

Copy link
Copy Markdown

GitHub issue resolved #4300

Pull request Description:

SPARQL uses the Unicode code point collation for string comparisons. XSDFuncOp.compareString used String.compareTo, which compares UTF-16 code units. For example, U+FFFD should sort before U+1D11E, but their UTF-16 order is reversed. This affects string </> comparisons and the value-comparison path used for sorting.

Compare one code point at a time, while retaining the existing behavior for equal strings and prefixes. Add regressions for plain and xsd:string expression comparisons and NodeValueSortKey ordering. The change is confined to three files and adds no dependency or public API.

This follows the SPARQL 1.1 string collation and the Unicode Codepoint Collation.

Validation:

  • On main at d77438478938cc651760f4a61e5d9ee7acff072b, both new regression tests failed. A separate actual-SPARQL probe using U+FFFD, U+FFFE and U+1D11E failed all six checks for <, >, ORDER BY, MIN() and MAX(); the old order was U+1D11E, U+FFFD, U+FFFE.
  • With this change, the two new regression tests and all six actual-SPARQL checks passed. The SPARQL probe is a pre-submit validation artifact outside the three-file PR diff.
  • mvn -B -q -pl jena-arq -am -Dtest=TestGH4300RealSPARQL,TestExpressions2,TestNodeValueSortKey,TestNodeValueOps,TestOrdering,TestSortOrdering,TestComparison,TestStatisticsAggregates,TestNodeValue,TestCustomAggregates -Dsurefire.failIfNoSpecifiedTests=false test passed: 401 tests including the temporary probe, 0 failures/errors/skips, using Java 21 in an isolated Maven container. The prior 198-test subset remains green.
  • git diff --check passed.

The full Jena test suite was not run.


  • Tests are included.
  • Documentation change and updates are provided for the Apache Jena website (not needed for this bug fix).
  • Commits have been squashed to remove intermediate development commit messages (local changes are not committed yet).
  • Key commit messages start with the issue number (to complete before submission).

By submitting this pull request, I acknowledge that I am making a contribution to the Apache Software Foundation under the terms and conditions of the Contributor's Agreement.


See the Apache Jena "Contributing" guide.

@rvesse rvesse left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, may be worth @OyvindLGjesdal sanity checking this PR as he's our resident guru on all things Il8n

@rvesse
rvesse requested a review from OyvindLGjesdal October 8, 2026 09:14
@afs
afs self-requested a review October 8, 2026 13:53
@afs

afs commented Oct 8, 2026

Copy link
Copy Markdown
Member

Looks good.

SPARQL uses the Unicode code point collation for string comparisons.

We have had a lot of discussion in the working groups about this. It is important to get the terminology sorted out, and Unicode has itself changed over time, so for anyone reading this in the future ...

 

RDF Concepts defines "RDF string", which is used for the literal lexical form:

RDF string, or simply string, is used to describe an ordered sequence of zero or more Unicode code points which are Unicode scalar values. Unicode scalar values do not include the surrogate code points.

Surrogates are code points.

"scalar values" - is the concept that excludes the surrogates. Java's "codepoint" usage is really a scalar value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

String </> (and ORDER BY/MIN/MAX) compare by UTF-16 code units instead of Unicode code points

3 participants