Spec: pluggable search backends for 5.0 #97

Problem Statement

arocapi can only search through OpenSearch. raspbocrate runs Oni and arocapi on a Raspberry Pi with 4–8 GB of memory, which can't sensibly run OpenSearch, so a repository on small hardware has no usable search. The OpenSearch path also has faults that any consumer can hit:

• Advanced queries go straight to query_string, which reaches every indexed field. A request can search inside Metadata or Content it would be Denied.
• Filter and facet keys can disagree (field vs field.keyword).
• Undeclared filter keys and bad ranges aren't rejected, though the RO-Crate API spec says they MUST be a 400.
• The index name is fixed at entities.
• total stops being exact above 10,000 hits.
• RO-Crate API v0.4.0 gives an implementation no way to say it lacks advanced search, geo or relevance sorting, so a lightweight implementation can't describe itself honestly.

Consumers also have to hand-build an OpenSearch mapping and a DSL visibility query that must agree with fields they index elsewhere, and nothing checks that they do.

Solution

arocapi 5.0 makes search pluggable. A consumer passes a Search backend as a required search option. arocapi ships two reference backends as exported factories, OpenSearchBackend and PostgresBackend, and a consumer may write their own against the exported SearchBackend interface.

The consumer declares their search fields once (a Field declaration per field, with a type and Roles). That single declaration drives the index, /capabilities, request validation, filter and facet keys, advanced-query fields and the typed Search document shape. Search visibility becomes one typed filter language over declared fields that every backend applies, so a Search visibility filter carries over unchanged when switching backends.

arocapi parses advanced queries itself into a tree, so no raw query ever reaches an engine, and it rejects undeclared fields, syntax errors and oversized queries with a 422. It implements RO-Crate API v0.5.0, which publishes searchTypes, geo and sorts in /capabilities.

arocapi 5.0 supports Postgres 18+ only. MySQL is dropped. Every consumer gets the EntitySearch table through the usual prisma/models symlink and prisma migrate. A Pi deployment drops OpenSearch from its Compose file and swaps the factory.

User Stories

Choosing and configuring a backend

1. As a repository operator on a Raspberry Pi, I want to search with Postgres alone, so that I don't have to run OpenSearch on a 4–8 GB device.
2. As a consumer, I want to pass a search backend as a single required search option, so that all search configuration lives in one place.
3. As a consumer, I want OpenSearchBackend({...}) and PostgresBackend({...}) exported from arocapi, so that I can configure a reference backend without writing search code.
4. As a consumer, I want the reference backends customised through options only, so that upgrades within 5.x don't break me through changed internals.
5. As a consumer with unusual needs, I want to write my own backend as an object literal typed SearchBackend, so that I'm not limited to the reference backends.
6. As a consumer, I want to keep owning my OpenSearch client and Prisma client, so that connections, pooling and shutdown stay under my control.
7. As a consumer, I want to set the OpenSearch index name with indexName (default entities), so that several repositories can share a cluster.
8. As a consumer switching from OpenSearch to Postgres, I want to change only the factory call, so that my field declarations, search documents and visibility filter carry over unchanged.
9. As a consumer upgrading from 4.x, I want every breaking change in the release notes to name what was removed and its replacement, so that I can migrate without an upgrade guide.

Declaring fields

10. As a consumer, I want to declare each search field once with a type (text, keyword, date, number, boolean, geo), so that every part of search agrees on it.
11. As a consumer, I want to opt fields into roles (basic, advanced, filter, facet, sort), so that I control exactly what clients can search, filter, facet and sort by.
12. As a consumer, I want every text field in basic search by default, with basic: false to opt out, so that the common case needs no configuration.
13. As a consumer, I want to give text fields a weight of 1–4, so that matches in a title rank above matches in a description.
14. As a consumer, I want advanced, filter, facet and sort to be opt-in, so that no field becomes queryable by accident and leaks Denied Metadata or Content.
15. As a consumer, I want facet to imply filter, so that every facet a client sees can also be filtered on, as the spec requires.
16. As a consumer, I want to set a facet's bucket count with facet: { size } (default 20, any positive size), so that a language facet can list thousands of languages.
17. As a consumer, I want one optional label per field used wherever it's published, so that Oni shows human-friendly names.
18. As a consumer, I want to declare a field with no role, so that I can store a fact used only by my visibility filter without publishing it.
19. As a consumer, I want filter, facet and sort keys always to be the field name, so that I never meet .keyword mismatches.
20. As a consumer, I want a text field to be usable as a filter, facet or sort through a hidden exact form, so that I don't have to declare the same data twice.
21. As a consumer, I want the factory to throw on a bad declaration (a name not matching ^[A-Za-z_][A-Za-z0-9_]*$, id, a relevance sort field, more than one geo field, a bad weight or size), so that configuration errors show up at startup.
22. As a custom backend author, I want capabilitiesFromFields(fields) exported, so that I can derive /capabilities-shaped capabilities from the same declarations.
23. As a custom backend author, I want to write capabilities directly if I don't use field declarations, so that the contract doesn't force the declaration format on me.

Indexing

24. As a consumer, I want to supply one search document per entity, keyed by declared fields and including full text, so that I decide what is searchable and arocapi never guesses from Entity columns or meta.
25. As a consumer, I want index() typed from my field declarations (keyword → string | string[], geo → { lat, lng }, …), so that a wrong key or value type fails to compile.
26. As a consumer, I want index(docs) to upsert a batch idempotently, so that reloading the same data is safe.
27. As a consumer, I want remove(ids) to be idempotent, so that removing something already gone isn't an error.
28. As a consumer, I want index to throw on an undeclared key, a value of the wrong type, an unparseable date, bad coordinates or an array in a sort field, naming the entity and key, so that I find bad data at load time.
29. As a consumer, I want a missing field allowed in a document, so that the entity just doesn't match filters on it and sorts last.
30. As a consumer, I want any non-sort field to accept a scalar or an array, so that multi-valued properties such as languages work.
31. As a consumer, I want dates given as YYYY, YYYY-MM, YYYY-MM-DD or RFC 3339, stored as their first UTC instant (a date-time with no offset is UTC), so that partial dates from RO-Crates index without preprocessing.
32. As a consumer on Postgres, I want to pass a Prisma transaction ({ tx }) to index, so that an entity and its search document commit together.
33. As a consumer on Postgres, I want index to throw an error naming the missing ids when an entity doesn't exist yet, and roll back the batch, so that I see a clear error, not a raw foreign-key violation.
34. As a consumer, I want deleting an entity to remove its Postgres search row by cascade, so that search never returns deleted entities.
35. As a consumer on OpenSearch, I want createIndex() to build the mapping from my field declarations, so that I never hand-write a mapping.
36. As a consumer on OpenSearch, I want createIndex() to error on an existing index whose mapping differs, naming the field, so that a changed declaration doesn't silently mis-index.
37. As a consumer, I want to rebuild an index by looping index over my entities, so that I need no special rebuild API.
38. As a future deposit API, I want index and remove on the contract, so that arocapi can write through any backend.
39. As a read-only custom backend author, I want to throw from index and remove, so that I can still satisfy the contract.

Searching

40. As a client, I want basic search to take a free-text string each backend interprets in its own way, so that typing words just works.
41. As a client, I want an empty query to return every visible entity (filters still apply), for both search types, so that cockatiel and the downloader can browse by filter alone.
42. As a client, I want advanced queries with field: value, AND, OR, brackets, quoted phrases, unary NOT and field groups (field: (a OR b)), so that Oni's advanced search keeps working.
43. As a client, I want precedence NOT > AND > OR, operators uppercase only and whitespace around : ignored, so that queries mean the same thing on every backend.
44. As a client, I want a text field value to match when the field contains all its words, or the exact phrase if quoted, so that advanced matching is predictable.
45. As a client, I want non-text fields in advanced queries to match exactly, so that inLanguage: eng means that language.
46. As a client, I want wildcards, ?, ~, regexes, ranges and boosts treated as ordinary characters, not errors, so that pasted text doesn't fail.
47. As a client, I want a syntax error to be a 422 VALIDATION_ERROR giving the position and what was expected, with details.violations: [{ field: 'query', message }], so that I can show a useful message.
48. As a repository operator, I want an advanced query naming an undeclared field to be a 422 that names it, so that nobody can search inside Denied Metadata or Content.
49. As a repository operator, I want queries capped at 1024 characters, and advanced queries at 10 levels of nesting and 64 leaf clauses (counted after field groups are expanded), so that a hostile query can't exhaust the server.
50. As a client, I want a cap breach to be a 422 VALIDATION_ERROR on query naming the limit, so that I know what to shorten.
51. As a client, I want a backend's own engine limits (OpenSearch too-many-clauses, Postgres stack depth) to come back as a 422, not a 500, so that an unusual query never looks like a server fault.
52. As a client, I want undeclared filter keys, undeclared sorts, undeclared search types, ranges on string or boolean filters and geo parameters on a backend without geo to be a 400 VALIDATION_ERROR, so that mistakes are reported, not ignored.
53. As a client, I want exact-term filters and date and number range filters, with date bounds treated as UTC, so that Oni's year filter 2001-01-01..2001-12-31 matches an entity dated 2001.
54. As a client, I want sort to accept any declared sort or relevance, so that custom sorts such as identifier or originatedOn work.
55. As a client, I want relevance to mean best match first and to stay stable across pages for an empty query, so that paging never repeats or skips results.
56. As a client, I want total to be exact, so that page counts are right above 10,000 hits.
57. As a client, I want every declared facet present in every response (empty when nothing matches), so that my facet UI doesn't flicker.
58. As a client, I want facet buckets named by the value as a string and ordered by count, then name, so that facets are identical across backends.
59. As a client, I want a date facet bucketed by UTC year with names "YYYY", ordered by year, so that I don't have to parse epoch timestamps.
60. As a client, I want bounding-box filtering and a geohash grid when the backend declares a geo field, so that Oni's map works on both backends.
61. As a client, I want searchExtra (score and highlights where the backend provides them) passed through unchanged, so that existing result rendering keeps working.
62. As a client, I want searchTime measured around the backend call, so that it's comparable between backends.
63. As a client, I want each search hit to be a full Record with access attached and the consumer's entity transformers applied, so that search results look exactly like entity listings.

Visibility

64. As a consumer, I want a required top-level searchVisibilityFilter(context) returning, or resolving to, a filter over id and declared fields, so that hidden entities never appear in hits, totals or facets on any backend.
65. As a consumer, I want the filter typed from my backend's field declarations, so that an unknown field or a wrong value type fails to compile.
66. As a consumer, I want equals (or a bare value), not, in and notIn on every type but geo, plus lt, lte, gt and gte on dates and numbers, so that I can express my visibility rules.
67. As a consumer, I want AND, OR and NOT taking arrays of filters, with several keys in one object meaning AND, so that I can combine rules.
68. As a consumer, I want a multi-valued field to match when any value matches, and not/notIn to match when none does (including when the field is missing), so that draft: { not: true } shows entities with no draft flag.
69. As a consumer, I want null to match a missing value and { not: null } a present one, so that I can test for presence.
70. As a consumer of a public repository, I want to opt in explicitly with AllVisibleSearchFilter ({}), so that visibility is never on by accident.
71. As a repository operator, I want an invalid visibility filter (undeclared field, wrong operator for the type, bad value) to fail closed as a 500 that's logged and names the field, so that a misconfiguration never leaks hidden entities.
72. As a repository operator, I want arocapi to re-apply entityVisibilityFilter when loading hits and drop any hit it can't load, so that a mismatch between the two filters gives a short page, not a leak.
73. As a custom backend author, I want request.visibility handed to me already resolved, so that my backend never calls consumer code.

Capabilities

74. As a client, I want /capabilities to report apiVersion 0.5.0, so that I know which spec the server follows.
75. As a client, I want search.searchTypes with basic always present and advanced (listing its queryable fields and labels) only when supported, so that Oni can build or hide its advanced search.
76. As a client, I want search.geo: { supported }, so that Oni can hide its map when geo isn't available.
77. As a client, I want search.sorts listing declared sorts with labels (relevance implied), so that I can offer the right sort options.
78. As a client, I want search.filters and search.facets derived from the declarations, with fields that have no role never published, so that /capabilities matches what the server accepts.

Database and tooling

79. As a consumer, I want arocapi to run on Postgres 18+, with the EntitySearch table in the shared prisma/models, so that setup stays one symlink plus prisma migrate.
80. As an OpenSearch consumer, I want the EntitySearch table to need no extensions and to stay empty, so that the Postgres backend costs me nothing.
81. As a consumer, I want all shipped string columns to be text, so that long ids and names aren't truncated by MySQL-era limits.
82. As a maintainer, I want the test suite to run on Postgres 18 with both reference backends, so that the two backends are held to the same contract.
83. As a maintainer, I want the dev loader script and docs examples moved to createIndex() + index() and Postgres JSON paths, so that they show the supported way.

Documentation

84. As a new consumer, I want Getting started to set up Postgres 18 + OpenSearch, declare fields, call createIndex() and index after adding an entity, so that I have a working search quickly.
85. As a consumer choosing a backend, I want Strategies → Search to compare OpenSearch and Postgres (ranking, analysers, scale vs footprint; identical filters, facets, totals, sorts, grammar and errors) and show how to switch and how to write my own, so that I can decide.
86. As a consumer, I want Configuration → Search to document search, fields and each backend's options, so that I have one full reference.
87. As a consumer, I want Configuration → Visibility to document the search filter language and its semantics, so that I write correct rules.
88. As a client developer, I want Reference → Search queries to document the grammar, caps and 422s, so that I can build queries that work.
89. As a custom backend author, I want Reference → Search backend to document the full contract (capabilities, search(request, context) including visibility, index/remove, entity-must-exist, SearchQueryError, capabilitiesFromFields), so that I can implement it correctly.

Implementation Decisions

The decisions are recorded in full in ADR 0003 (pluggable search backends) and ADR 0001 (entity visibility filters). This section summarises them for implementers and adds how they land in the codebase.

Public API (breaking, 5.0)

• New required option search: SearchBackend. The options opensearch, queryBuilderClass and queryBuilderOptions are removed, along with the OpensearchQueryBuilder export and its QueryBuilderOptions/FilterCapability/FacetCapability/SearchCapability types. There are no REMOVED_OPTIONS runtime migration errors for them.
• searchVisibilityFilter stays a required top-level option but changes type. It now returns a SearchVisibilityWhere in the shared language, typed from the search option's declarations, instead of an OpenSearch QueryContainer. AllVisibleSearchFilter stays and now returns {}. entityVisibilityFilter and AllVisibleEntityFilter are unchanged.
• New exports: the SearchBackend interface and its request/result types, SearchQueryError, OpenSearchBackend, PostgresBackend, capabilitiesFromFields, the field declaration types, SearchDocument<Fields> and SearchVisibilityWhere<Fields>. createIndex() is exposed by OpenSearchBackend and isn't on the contract.
• @opensearch-project/opensearch stays a dependency for OpenSearchBackend, and its types no longer appear in the core options.
• The @prisma/client peer stays ^7 (Prisma 8 can't run arocapi or EntitySearch; revisit for 6.0).

The contract

interface SearchBackend {
  readonly capabilities: BackendSearchCapability; // /capabilities-shaped: filters, facets, sorts, searchTypes, geo
  search(request: SearchRequest, context: RequestContext): Promise<SearchResult>;
  index(docs: SearchDocument[], options?: { tx?: unknown }): Promise<void>;
  remove(ids: string[]): Promise<void>;
}

type SearchResult = {
  total: number; // exact
  hits: { id: string; searchExtra?: SearchExtra }[]; // ordered
  facets?: Record<string, { name: string; count: number }[]>;
  geohashGrid?: Record<string, number>;
};

(The shape comes from the "Search backend contract: functions or class?" decision, amended by the index and visibility decisions.)

• SearchRequest is normalised and valid. It carries the search type, with a raw string for basic or a parsed tree for advanced; filters already split into exact terms and ranges, with date bounds as UTC instants; optional bounding box and geohash precision; limit/offset; sort and order; and visibility.
• capabilities holds only what /capabilities publishes. Weights, facet sizes and text vs keyword stay out of the contract.
• No lifecycle methods. The consumer owns clients.

Who does what

• arocapi (the search route): validates the request against backend.capabilities (400 for undeclared filter keys, sorts and search types, bad ranges, and geo without support); applies defaults and paging; enforces query caps; parses advanced queries; calls searchVisibilityFilter(context) and puts the result on the request; times searchTime around backend.search; maps SearchQueryError to 422 and any other error to 500; loads Records from Prisma under entityVisibilityFilter, dropping hits it can't load (with a warning log); runs access and the transformer pipeline; attaches searchExtra unchanged; makes sure every declared facet key is present.
• The backend: checks request.visibility against its declarations (a plain Error naming the field on any fault, which becomes a 500); applies visibility to hits, total and facets; runs the query; returns ordered ids, exact total, facets and geohash grid.

Advanced query parser (new arocapi module)

• Grammar: field: value (an unquoted value runs to the next operator, ) or end; a: x b: y is a 422), quoted phrases, AND, OR, unary NOT, brackets, field groups rewritten to (field: a OR field: b). Precedence is NOT > AND > OR; operators are uppercase only; whitespace around : is insignificant. Dropped: field: *, bare terms with no field, \ escapes. Wildcards, ?, ~, regexes, ranges and boosts are ordinary characters.
• Fields are checked against capabilities.searchTypes.advanced.fields. An undeclared field is a 422 naming it.
• Caps are fixed constants: 1024 code points for any query; 10 levels of nesting and 64 leaf clauses after the field-group rewrite for advanced. Each breach is a 422 VALIDATION_ERROR naming the limit. The length cap isn't a request-schema bound, so it's never a 400.
• Every query 422 uses code: VALIDATION_ERROR and details.violations: [{ field: 'query', message }], with the position and what was expected for syntax errors.
• Basic queries are never parsed: only the length cap applies.

Field declarations (shared module used by both reference backends)

• Shape: fields: { [name]: { type, label?, weight?, basic?, advanced?, filter?, facet?: true | { size }, sort? } }, inferred as a literal so documents and visibility filters are typed.
• capabilitiesFromFields maps text/keyword → string in /capabilities, includes advanced fields under searchTypes.advanced.fields, sets geo.supported when a geo field exists and omits fields with no role.
• Document validation (shared by both backends' index) covers: undeclared keys, value types, date parsing (partial date → first UTC instant; no offset → UTC), { lat, lng } ranges and arrays in sort fields.
• Facets: value buckets are named by the stringified value and ordered by count desc, then name asc; the default size is 20. A date facet has one bucket per UTC year with matches, named "YYYY" and ordered by year.

Search visibility language (shared module)

• Field conditions: a bare value (= equals), or { equals, not, in, notIn } (every type but geo) and { lt, lte, gt, gte } (date, number). text compares its exact form. Combinators: AND, OR, NOT (arrays); several keys in one object = AND. Usable keys: id and any declared field, including fields with no role.
• Semantics: any value matches; not/notIn match when no value matches, including missing; null = missing, { not: null } = present.
• Each backend translates it: OpenSearch to a bool filter; Postgres to parameterised SQL over EntitySearch. Neither ever takes consumer SQL or DSL.

OpenSearchBackend

• Options: the consumer's OpenSearch client, fields, indexName (default entities).
• createIndex() builds the mapping from declarations (text plus a private exact sub-field when filter/facet/sort is needed; geo_point for geo) and errors, naming the field, on an existing index whose mapping differs.
• Basic: multi_match boosted by weight. Advanced tree: bool / match with operator: and / match_phrase / must_not / match_all. track_total_hits: true. Date facets use a yearly histogram named "YYYY" in UTC. Highlights go into searchExtra.
• An OpenSearch 400 or a too-many-clauses error is translated to SearchQueryError.
• index doesn't check that entities exist.

PostgresBackend and the EntitySearch model

• Options: the consumer's Prisma client and fields.
• EntitySearch ships in the shared prisma/models with @relation to Entity and onDelete: Cascade; Entity gains the back-relation. It holds a weighted tsvector over basic text fields (weight 4→A … 1→D), per-field tsvectors for every declared text field (so each is matchable on its own), declared field values for exact matching, filtering, faceting and sorting, a point behind GiST and a stored full-length geohash. GIN goes through @@index(type: Gin) with no raw ops, so there's no migration drift. No extensions (no PostGIS, no pg_trgm).
• Queries use raw SQL ($queryRaw + Prisma.sql), always parameterised. Basic: websearch_to_tsquery. Advanced: a SQL boolean over per-field matches, phraseto_tsquery for phrases, NOT EXISTS/NOT for negation, no WHERE for empty. Relevance: ts_rank with weights, tie-broken by id so empty-query order is stable.
• index accepts { tx }, upserts in one batch, catches the foreign-key violation, rolls back and throws an error naming the missing entity ids.
• Postgres engine limits (e.g. max_stack_depth) are mapped to SearchQueryError.

Database

• The Prisma datasource becomes postgresql and migrations are regenerated from scratch. All shipped String columns drop their @db.VarChar(n) limits and become text.
• Docs examples and test fixtures use Postgres JSON paths (path: ['draft']).

Capabilities route

• apiVersion becomes 0.5.0. search comes from backend.capabilities and includes the required searchTypes, geo and sorts.
• Spec dependency: RO-Crate API v0.5.0 is proposed in Search capabilities for lightweight implementations (v0.5.0). arocapi implements that proposal.

Release

• Every breaking commit carries a BREAKING CHANGE: footer naming what was removed or changed and its replacement, e.g. opensearch → search: OpenSearchBackend({...}); the searchVisibilityFilter DSL → the shared filter language; MySQL → Postgres 18+. There is no upgrade guide.

Docs

• Getting started: Postgres 18 + OpenSearch Compose, @prisma/adapter-pg, declare fields, createIndex(), add an entity then index it.
• Strategies → Search: backend comparison, the Postgres backend as the lightweight option, a short write-your-own section. All query-builder content goes.
• Configuration → Search: search, fields, each backend's options. Configuration → Visibility: the filter language. Configuration → Data sources: only prisma.
• Reference gains Search queries (grammar, caps, 422s; linked from Errors) and Search backend (the contract; linked from Strategies).

Testing Decisions

Good tests here exercise external behaviour only: HTTP responses and the exported backend methods. They don't assert on generated OpenSearch DSL or SQL, internal module calls or the parser's internals. Relevance and basic-search matching legitimately differ between backends, so shared tests never compare relevance order or which documents fuzzy basic search matches. They hold both backends to the same filters, facets, sorts, exact totals, visibility, advanced-query matching and errors.

Three seams, agreed during planning:

1. The HTTP API with a real backend (main seam). A shared contract suite drives POST /search and GET /capabilities through app.inject on the full plugin. It runs once with Postgres 18 + OpenSearchBackend and once with Postgres 18 + PostgresBackend. Each case sets up its own fields, search documents (via index) and searchVisibilityFilter, then checks:

   • hits, exact totals, facets (presence, naming, ordering, "YYYY" date buckets) and geohash grids
   • declared sorts, and stable order for empty queries across pages
   • exact and range filters with UTC date bounds
   • advanced queries (words, phrases, NOT, field groups, precedence)
   • the visibility language's any/none/null semantics applied to hits, totals and facets
   • 400s, 422s, and the 500 on an invalid visibility filter
   • /capabilities derived from the declarations

   The existing integration tests and their setup are prior art. They move to Postgres, and the whole integration file runs against both backends.

2. The HTTP API with a fake SearchBackend. Route tests register the plugin with an object literal that records each SearchRequest and returns canned results. They cover arocapi's half:

   • validation from capabilities (400s)
   • the advanced parser, by asserting the tree the backend receives, plus syntax 422s with position and expected
   • query caps
   • SearchQueryError → 422, other errors → 500
   • dropping hits Prisma can't load
   • access and transformers on hits, searchExtra passthrough, searchTime
   • filling in missing declared facet keys, and request.visibility carrying the filter's result

   This replaces the mocked OpenSearch client in the current search route tests, which with the capabilities route tests are prior art, and needs no containers.

3. The reference backends' public write surface. Tests call index, remove, createIndex() and the factories' declaration checks directly against the real engines:

   • idempotent upserts and removes
   • each index validation error naming entity and key
   • partial dates stored as their first UTC instant
   • { tx } committing or rolling back with the entity
   • Postgres naming missing entity ids, and cascade delete from Entity
   • createIndex() erroring on a differing mapping
   • factory errors on bad declarations

   Most cases are shared across both backends.

Other test files that mock Prisma stay as they are. Type-level behaviour (typed index documents and typed visibility filters rejecting unknown fields) is covered by the consumer type check and the docs examples type check in pnpm lint. Tooling: postgres:18 replaces mysql:8 in test Compose and CI, and @prisma/adapter-pg replaces @prisma/adapter-mariadb. Everything runs in one job with one lock, generating and migrating once, with a single coverage report kept at 100%.

Out of Scope

• MySQL as a datasource (MySQL users stay on 4.x; requests are wontfix) and SQLite as a datasource or backend.
• A grep or in-memory backend, except possibly as a test fake.
• The deposit API itself (it only motivated putting index/remove on the contract).
• Zero-downtime index rebuilds (e.g. an OpenSearch alias swap); addable after 5.0 without a breaking change.
• Several geo points per entity, and a partial-date type matching the range it covers.
• Consumer-configurable query caps (addable later as an optional setting).
• Letting entityVisibilityFilter reach consumer-owned tables (Visibility: let entityVisibilityFilter reach consumer-owned tables).
• Prisma 8 support.
• Drafting the spec PR on crate-works/ro-crate-api.
• Moving raspbocrate from MySQL to Postgres.
• Follow-ups in other repos: Oni reading searchTypes, geo and sorts from /capabilities, emitting AND NOT, skipping empty advanced lines instead of sending *, trimming its advanced-search help, and reading "YYYY" date-facet names; nabu emitting key_as_string; ldacapi moving to createIndex() + index().

Further Notes

• This spec is the destination of the map Search: pluggable search backends for 5.0. Every decision behind it is linked from that map's Decisions so far. It supersedes the requirements in Search: decide whether to support backends beyond OpenSearch #48 (undeclared filter keys, range filters, configurable index, facet/filter mismatch).
• Measured on a Pi 5 (4 GB, SD card), a full Postgres response (page, exact total, five facets) takes ≤72 ms at 10k entities, ≤280 ms at 50k and about 1 s at 200k.
• Every push to main publishes a beta prerelease, so the breaking changes should land on an integration branch or in an order that keeps main consistent; see RELEASING.md.
• A suggested order of work is: Postgres-only database and tooling → contract + parser + validation with the fake backend → field declarations and visibility language → OpenSearchBackend → PostgresBackend → capabilities v0.5.0 → docs.