Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ versions may change the store format and the API; the release notes say how to u

- A spatial `FILTER` (`geof:sfWithin`, `geof:distance` and the other relations) over a geometry reached by a property path with an alternative, `?`, `*` or `+` (`?x (a|b/c)/geo:asWKT ?p`) gave every candidate geometry for every row, including those outside, repeated (#48). The parser joins such a path to the rest through a blank node, which the spatial rewrite didn't count as a link. Answers now match the evaluator's.
- The same for a GeoSPARQL relation written as a triple (`?g geo:sfWithin ?rg`) on a geometry reached by such a path (`?x (a|b)/geo:hasGeometry ?g`): every geometry for every row (#54).
- GeoSPARQL relation triples (`?g geo:sfWithin ?rg`) inside `FILTER EXISTS`, `FILTER NOT EXISTS` or `BIND(EXISTS …)` were not inferred: `EXISTS` matched only asserted relation triples, so it kept no row and `NOT EXISTS` every row (#56). The spatial rewrite now reaches the patterns inside expressions.

## 0.1.0-rc.1 - 2026-11-17

Expand Down
94 changes: 91 additions & 3 deletions crates/tripletrail-spatial/src/rewrite.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1278,6 +1278,10 @@ struct Rewriter<'a> {
/// How often each variable is written in the query (see [`mentions`]); empty when not
/// known, and then every variable counts as used.
mentions: HashMap<String, usize>,
/// Inside an `EXISTS`: its pattern is evaluated once per outer row with that row's values
/// bound, so relation triples become plain geometry lookups and a test there, without the
/// index (whose candidates would be found once per row, for the whole store).
exists: usize,
}

/// How often each variable name is written in `query`.
Expand Down Expand Up @@ -1345,6 +1349,7 @@ impl<'a> Rewriter<'a> {
estimator: Estimator::new(ctx),
side_rows: HashMap::new(),
mentions: HashMap::new(),
exists: 0,
}
}

Expand Down Expand Up @@ -2229,7 +2234,8 @@ impl<'a> Rewriter<'a> {
&format!("{GEO}hasDefaultGeometry"),
d,
)]);
let any = if matches!(node, TermPattern::Variable(_)) {
// Inside EXISTS, `node` is usually bound by the outer row: a lookup is the cheaper.
let any = if matches!(node, TermPattern::Variable(_)) && self.exists == 0 {
// MINUS rather than FILTER NOT EXISTS: one anti-join instead of a lookup per
// row (100x faster on large joins). They agree because `node` is on both sides.
GraphPattern::Minus {
Expand Down Expand Up @@ -2388,6 +2394,10 @@ impl<'a> Rewriter<'a> {
} else {
inferred
};
// Inside EXISTS, the row's values are bound: the lookups and the test as they are.
if self.exists > 0 {
return body;
}
// The generated FILTER is itself a spatial test; plan it like any other.
let body = self.walk_generated(body);
GraphPattern::Distinct {
Expand Down Expand Up @@ -2482,12 +2492,59 @@ impl<'a> Rewriter<'a> {
}
}

/// `e` with the patterns of its `EXISTS` rewritten as any other: relation triples in
/// `FILTER (NOT) EXISTS { ?g geo:sfWithin ?r }` are inferred too.
fn walk_exists(&mut self, e: Expression) -> Expression {
use Expression as E;
let mut w = |e: Box<E>| Box::new(self.walk_exists(*e));
match e {
E::Exists(p) => {
self.exists += 1;
let p = self.walk(*p);
self.exists -= 1;
E::Exists(Box::new(p))
}
E::Or(a, b) => E::Or(w(a), w(b)),
E::And(a, b) => E::And(w(a), w(b)),
E::Equal(a, b) => E::Equal(w(a), w(b)),
E::SameTerm(a, b) => E::SameTerm(w(a), w(b)),
E::Greater(a, b) => E::Greater(w(a), w(b)),
E::GreaterOrEqual(a, b) => E::GreaterOrEqual(w(a), w(b)),
E::Less(a, b) => E::Less(w(a), w(b)),
E::LessOrEqual(a, b) => E::LessOrEqual(w(a), w(b)),
E::Add(a, b) => E::Add(w(a), w(b)),
E::Subtract(a, b) => E::Subtract(w(a), w(b)),
E::Multiply(a, b) => E::Multiply(w(a), w(b)),
E::Divide(a, b) => E::Divide(w(a), w(b)),
E::UnaryPlus(a) => E::UnaryPlus(w(a)),
E::UnaryMinus(a) => E::UnaryMinus(w(a)),
E::Not(a) => E::Not(w(a)),
E::If(a, b, c) => E::If(w(a), w(b), w(c)),
E::In(a, list) => {
let a = w(a);
E::In(a, list.into_iter().map(|e| self.walk_exists(e)).collect())
}
E::Coalesce(list) => {
E::Coalesce(list.into_iter().map(|e| self.walk_exists(e)).collect())
}
E::FunctionCall(f, args) => {
E::FunctionCall(f, args.into_iter().map(|e| self.walk_exists(e)).collect())
}
other => other,
}
}

fn walk(&mut self, p: GraphPattern) -> GraphPattern {
use GraphPattern as G;
let b = |p: GraphPattern| Box::new(p);
match p {
G::Bgp { patterns } => self.bgp(patterns),
G::Filter { expr, inner } if self.exists > 0 => G::Filter {
expr: self.walk_exists(expr),
inner: b(self.walk(*inner)),
},
G::Filter { expr, inner } => {
let expr = self.walk_exists(expr);
let inner = self.walk(*inner);
// A FILTER applies to its whole group, BINDs included: the BINDs it does not
// read are applied after it, so the tests see the group's join itself (and its
Expand Down Expand Up @@ -2524,6 +2581,22 @@ impl<'a> Rewriter<'a> {
out
}
G::Join { left, right } => match linked_through_paths(G::Join { left, right }) {
// Inside EXISTS, the relation's lookups are evaluated for each row of the rest
// of the group (LATERAL), with its values bound, rather than for every
// geometry of the store and then joined.
G::Join { left, right }
if self.exists > 0 && (has_relation(&left) != has_relation(&right)) =>
{
let (rest, relation) = if has_relation(&right) {
(left, right)
} else {
(right, left)
};
G::Lateral {
left: b(self.walk(*rest)),
right: b(self.walk(*relation)),
}
}
G::Join { left, right } => G::Join {
left: b(self.walk(*left)),
right: b(self.walk(*right)),
Expand All @@ -2537,7 +2610,7 @@ impl<'a> Rewriter<'a> {
} => G::LeftJoin {
left: b(self.walk(*left)),
right: b(self.walk(*right)),
expression,
expression: expression.map(|e| self.walk_exists(e)),
},
G::Lateral { left, right } => G::Lateral {
left: b(self.walk(*left)),
Expand Down Expand Up @@ -2567,7 +2640,7 @@ impl<'a> Rewriter<'a> {
} => G::Extend {
inner: b(self.walk(*inner)),
variable,
expression,
expression: self.walk_exists(expression),
},
G::OrderBy { inner, expression } => G::OrderBy {
inner: b(self.walk(*inner)),
Expand Down Expand Up @@ -2636,6 +2709,21 @@ impl<'a> Rewriter<'a> {
}
}

/// Whether `p`'s join operands hold a GeoSPARQL relation triple.
fn has_relation(p: &GraphPattern) -> bool {
match p {
GraphPattern::Join { left, right } => has_relation(left) || has_relation(right),
GraphPattern::Bgp { patterns } => patterns.iter().any(|t| match &t.predicate {
NamedNodePattern::NamedNode(p) => p
.as_str()
.strip_prefix(GEO)
.is_some_and(|n| RELATIONS.contains(&n) || NOT_INDEXABLE.contains(&n)),
NamedNodePattern::Variable(_) => false,
}),
_ => false,
}
}

/// The join `p` with the blank nodes that link a property path to its siblings written as
/// variables (#48 follow-up). The parser joins `?x (a|b)/geo:hasGeometry ?g` as a path to a
/// blank node and a triple from it; a rewritten BGP projects its variables only, so a blank
Expand Down
126 changes: 126 additions & 0 deletions crates/tripletrail/tests/spatial_paths.rs
Original file line number Diff line number Diff line change
Expand Up @@ -137,3 +137,129 @@ fn relation_triples_keep_paths_joined_through_blank_nodes() {
);
}
}

/// Relation triples inside `FILTER EXISTS`, `FILTER NOT EXISTS` and `BIND(EXISTS ...)` are
/// inferred as outside them: the same features as the join written out.
#[test]
fn relation_triples_inside_exists_are_inferred() {
let e = Engine::new_tripletrail();
e.load(RdfFormat::Turtle, DATA.as_bytes()).unwrap();
let prefixes = "PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX ex: <https://example.org/>";
let all = rows(&e, &format!("{prefixes} SELECT ?x WHERE {{ ?x a ex:P }}"));
for (inner, path) in [
("?x geo:hasGeometry ?g . ?g geo:sfWithin ?rg", "plain"),
(
"?x ex:ref/geo:hasGeometry ?g . ?g geo:sfWithin ?rg",
"sequence",
),
(
"?x (ex:ref|ex:part+)?/geo:hasGeometry ?g . ?g geo:sfWithin ?rg",
"path",
),
("?x geo:sfWithin ex:region", "features"),
] {
let joined = rows(
&e,
&format!(
"{prefixes} SELECT DISTINCT ?x WHERE {{ ex:region geo:hasGeometry ?rg . \
?x a ex:P . {inner} }}"
),
);
assert!(!joined.is_empty(), "{path}");
let exists = rows(
&e,
&format!(
"{prefixes} SELECT ?x WHERE {{ ex:region geo:hasGeometry ?rg . \
?x a ex:P FILTER EXISTS {{ {inner} }} }}"
),
);
assert_eq!(exists, joined, "EXISTS {path}");
let not_exists = rows(
&e,
&format!(
"{prefixes} SELECT ?x WHERE {{ ex:region geo:hasGeometry ?rg . \
?x a ex:P FILTER NOT EXISTS {{ {inner} }} }}"
),
);
let rest: Vec<String> = all
.iter()
.filter(|x| !joined.contains(x))
.cloned()
.collect();
assert_eq!(not_exists, rest, "NOT EXISTS {path}");
let bound = rows(
&e,
&format!(
"{prefixes} SELECT ?x WHERE {{ ex:region geo:hasGeometry ?rg . \
?x a ex:P BIND(EXISTS {{ {inner} }} AS ?in) FILTER(?in) }}"
),
);
assert_eq!(bound, joined, "BIND {path}");
}
}

/// A relation inside EXISTS is looked up for each outer row with its values bound: over a few
/// thousand rows, about as fast as the same test written as a join, not once per row over
/// every geometry of the store.
#[test]
fn relation_triples_inside_exists_use_the_rows_values() {
let mut ttl = String::from(
"@prefix geo: <http://www.opengis.net/ont/geosparql#> .\n@prefix ex: <https://example.org/> .\n",
);
for r in 0..5 {
let x = r * 20;
ttl.push_str(&format!(
"ex:r{r} a ex:R ; geo:hasGeometry ex:rg{r} . ex:rg{r} geo:asWKT \"POLYGON(({x} 0, {} 0, {} 10, {x} 10, {x} 0))\"^^geo:wktLiteral .\n",
x + 10,
x + 10
));
}
for i in 0..2000 {
let (x, y) = ((i * 37) % 100, (i * 11) % 20);
let via = if i % 2 == 0 {
"geo:hasGeometry"
} else {
"ex:ref ex:m{i} . ex:m{i} geo:hasGeometry"
};
let via = via.replace("{i}", &i.to_string());
ttl.push_str(&format!(
"ex:f{i} a ex:P ; {via} ex:g{i} . ex:g{i} geo:asWKT \"POINT({x} {y})\"^^geo:wktLiteral .\n"
));
}
let e = Engine::new_tripletrail();
e.load(RdfFormat::Turtle, ttl.as_bytes()).unwrap();
let prefixes = "PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX ex: <https://example.org/>";
let time = |q: &str| {
let start = std::time::Instant::now();
let n = rows(&e, q);
(start.elapsed(), n)
};
let inner = "?x ex:ref?/geo:hasGeometry ?g . ?g geo:sfWithin ?rg";
let (join, joined) = time(&format!(
"{prefixes} SELECT DISTINCT ?x ?r WHERE {{ ?r a ex:R ; geo:hasGeometry ?rg . ?x a ex:P . {inner} }}"
));
assert!(!joined.is_empty());
let floor = join.max(std::time::Duration::from_millis(100));
for (form, want_joined) in [
("FILTER EXISTS { INNER }", true),
("BIND(EXISTS { INNER } AS ?in) FILTER(?in)", true),
("FILTER NOT EXISTS { INNER }", false),
] {
let q = format!(
"{prefixes} SELECT ?x ?r WHERE {{ ?r a ex:R ; geo:hasGeometry ?rg . ?x a ex:P {} }}",
form.replace("INNER", inner)
);
let (took, n) = time(&q);
if want_joined {
assert_eq!(n, joined, "{form}");
} else {
assert_eq!(n.len(), 5 * 2000 - joined.len(), "{form}");
}
assert!(
took < floor * 20,
"{form}: {took:?} against {join:?} for the join"
);
}
}
5 changes: 4 additions & 1 deletion docs/spatial.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,10 @@ The GeoSPARQL relation properties work as triple patterns too (the Query Rewrite
`?a geo:sfIntersects ?b` holds between features, through their default geometry or any
geometry if they have none, and between geometries. Asserted relation triples count as well.
They work the same in the sub-selects of TripleTrail's own services (`st:`, window, `pgr`,
`path`), which the spatial rewrite and its index reach as they reach the main query (#857).
`path`), which the spatial rewrite and its index reach as they reach the main query (#857),
and inside `FILTER EXISTS`, `FILTER NOT EXISTS`, `BIND(EXISTS { … } AS ?v)` and `HAVING(EXISTS …)` (#56),
where they are looked up for each row with its values bound. An `EXISTS` in an `ORDER BY` or
aggregate expression sees asserted relation triples only.

Geometry properties are virtual too: `geo:hasMetricLength`, `hasLength`, `hasMetricArea`,
`hasArea`, `hasMetricPerimeterLength`, `hasPerimeterLength`, `hasMetricSize`, `hasSize`,
Expand Down
Loading