feat(search): the server picks the Messages list's order and names its terms

[mbeisser1]

Feature Description

The server now decides the Messages list's order and tells the web app what it decided. With no sort, GET /v1/messages puts the best match first when the query has a positive free-text term, and the newest message first when it has none. Each page also says which order it applied and which terms it ranks by. The web app stops parsing the search itself: web/src/lib/freeTextTerms.ts, its copy of Expr::positive_text_terms, is deleted.

This is option 1 of #1538, decided on 2026-10-05. For the person using the app, nothing changes except where the two parsers disagreed. In those cases the app now shows what the server does, so it can no longer offer Relevance for a search the server would refuse.

User Story

As a person searching my messages, I want the sort menu and the bold matches to follow what the server actually ranks by, so the list never offers an order that fails or hides one that works.

Implementation Details

• Server. search::Filter keeps the positive free-text terms it builds its rank query from (Filter::ranked_terms). list_messages reads sort with no default. Once the query is compiled, it picks relevance when there are terms and -date when there are none (default_message_list_sort). The answer is a ListMessagesResponse: the four page keys plus search: MessageSearch { sort, terms: [FreeTextTerm { text, prefix }] }. sort uses the same spelling as the parameter (message_list_sort_text). A named sort is applied as before, and sort=relevance on a query with no positive term is still validation-failed.

• Field name. I chose search rather than query. query is already a string elsewhere on the wire (an Export Run's query scope), and http-api.md reserves the …Query type suffix for query strings, so ListMessagesQuery would read as the query-string type. search with type MessageSearch follows the naming rule for "a thing named for what it is".

• Document rules. openapi/document_rules.rs has a PAGE_EXCEPTION constant naming GET /v1/messages and ListMessagesResponse. Four checks enforce it:

  • The route must answer that schema.
  • No other route may answer it.
  • The schema must require the four keys and search.
  • Every page, Page_* included, now fails on any property beyond its allowed keys. This check is new.

  I checked that the rule bites by renaming the allowed key in the constant: the test then fails with "has search beside its four keys".

• Web. MessageSearchList sends no sort unless a Date order was picked.

  • The sort menu shows lastPage.search.sort. It offers Relevance only when search.terms is non-empty, and the rows bold search.terms.
  • Before the first page arrives, the menu shows the person's pick, or nothing when the server is picking.
  • useRoutePagedList now also returns lastPage, typed through an optional extra-fields generic on PagedFetchPage. It adds no new cache or hook.
  • effectiveMessageSort and its rankable branch are deleted.

• Picking Relevance clears the pick. Relevance is offered only when the server already ranks by default. Picking it removes sort from the address (pickedSortParam), and sort=relevance in the address reads as no pick. Keeping relevance would carry it to the next search, and the server refuses it for a field-only query. This matches the old behaviour, where effectiveMessageSort dropped a Relevance pick for a query it could not rank.

Key Files Changed

• crates/server/server/src/messages_api.rs: the default order, ListMessagesResponse, MessageSearch, FreeTextTerm
• crates/server/server/src/search/{mod,emit}.rs: Filter::ranked_terms
• crates/server/server/src/db/conversation_messages.rs: default_message_list_sort, message_list_sort_text; DEFAULT_MESSAGE_LIST_SORT removed
• crates/server/server/src/openapi/document_rules.rs: the named page exception, plus the no-extra-keys check on every page
• web/src/screens/MessageSearchList.tsx, web/src/lib/messageSearchSort.ts, web/src/lib/resultsView.ts, web/src/lib/routeQuery.ts, web/src/components/ResultsColumn.tsx
• web/src/lib/freeTextTerms.ts and its test: deleted
• docs/architecture/http-api.md ("Lists": the written exception, with its reason and the rejected alternative), docs/architecture/search.md (the default order rule), CHANGELOG.md

HTTP API Changes

• GET /v1/messages: with no sort, the default order changes from date (oldest first) to relevance when q has a positive free-text term, and -date (newest first) otherwise.
• GET /v1/messages: the answer changes from Page_Message to ListMessagesResponse, which is {items, total, limit, offset, search: {sort, terms: [{text, prefix}]}}. search describes the query, not the rows. It is the one exception to the four-key page, written down in docs/architecture/http-api.md, "Lists".
• docs/src/assets/openapi.json and web/src/lib/serverApi.types.ts are regenerated.

Database Changes

• No schema changes
• schema/sql/*.sql changed (every database rebuilds empty and re-imports; the fingerprint is computed, nothing to bump)

How to Test

1. Start the server and the web app, then log in with Explore Demo Account.
2. Switch the results to Messages and search dinner. The request has no sort. The menu reads Relevance and offers Relevance and Date, and dinner is bold in each row.
3. Pick Date: the address gains sort=-date. Pick Relevance: sort leaves the address and the list is ranked again.
4. Search date:2024 -dinner. The request has no sort. The menu reads Date, Newest first and offers only Date, the first rows are from Dec 31, 2024, and nothing is bold.

I did all four in a real browser (Playwright) against this branch's server on a scratch data directory, port 8090, serving the built web/dist.

New tests, each confirmed to fail without the fix. I ran each against the pre-fix server or web sources (git stash / git checkout origin/main -- on the non-test files):

• messages_api::tests::with_no_sort_the_server_picks_the_order_and_reports_it: fails on the order assertion, since the old default was oldest first.
• messages_api::tests::a_named_sort_is_applied_and_reported
• messages_api::tests::the_page_names_the_terms_the_search_ranks_by: covers -word, not word, a negated group -("…" or …), a field word body:work, date:2024, a phrase and a prefix.
• MessageSearchList.test.tsx: "sends no sort, and shows the order the server applied", "offers only Date when the server returns no terms to rank by", "takes the order and the terms from the server, not from the words typed", and "bolds the terms the server returned". All four failed on the old component.

Updated tests. Two existing server tests assumed four keys or the oldest-first default. The web tests for messageSearchSort, resultsView and useConversationMessages were updated for the new shape and for Relevance not being kept as a pick.

Commands run, all passing:

• ./scripts/check-pr.sh
• cargo test -p message-crate-server: 1293 passed
• cd web && npm test: 2057 passed
• cd web && npm run build
• ./scripts/check-generated-api-types.sh
• cd docs && npm run check && npm run build

Checklist

• Code follows the project's style guidelines
• Self-reviewed the code
• Added unit tests for new functionality
• Added integration tests where applicable
• Existing tests pass locally
• Updated documentation
• ./scripts/check-pr.sh passes and formatter rewrites are committed

Related Issues

Closes #1538

Found in the review of #1526. ADR 0004 (one module parses the search language) and ADR 0002 (one way to fetch data in web/) apply.

Screenshots / Demo

None: nothing changes on screen for a search that both parsers read the same way.

Deployment Notes

The change breaks clients by design, as the no-backwards-compatibility rule allows: a client that relied on the oldest-first default of GET /v1/messages now gets the new default. The only clients in this repository are the web app and the server's own tests, and both are updated. No environment or dependency changes.

🤖 Generated with Claude Code

[mbeisser1]

Review of 3c00441 (the PR merged with main at cb1b2f0).

• Standards: 5 findings, below.
• Spec: 1 finding, below.
• Correctness: no findings. The reported sort and terms always match the applied ORDER BY, search comes back on every page (an empty last page included), lastPage cannot carry a previous query's terms (no placeholderData, and the list remounts per query), and the other callers of GET /v1/messages send an explicit sort.

[mbeisser1]

Re-review of the fix commits (7b345b7..5fb347f), and the review of the CHANGELOG conflict resolution in the first merge of main. Each finding below is answered in its thread.

[mbeisser1]

Review summary

Reviewed and fixed on 3f0539b. CI run 37411846666 passed on that commit.

First review (of 3c00441)

• Standards: 5 findings. 5 fixed (7b345b7, a5e26e3, 108f39d, 7970ba0, 5fb347f), 0 declined, 0 deferred.
• Spec (the decision comment on The browser re-implements which query words the server ranks by #1538): 1 finding. Fixed in a5e26e3 and ccddf87: the page rules now know a page by its keys as well as by its name.
• Correctness: no findings.

Re-review of the fixes

• Standards: 3 findings. 3 fixed (a116ad0, 72d3d8b).
• Correctness: 1 finding. Fixed in ccddf87.

Merges of main

• f1505a0 merged main at cb1b2f0. It resolved a conflict in CHANGELOG.md, keeping both Design entries. The merge review found 1 finding (a stray blank line), fixed in 3c00441.
• 3f0539b merged main at 5bfbdb2 with no conflict.

Other

• No commits were needed for CI failures.
• No issues were deferred.
• No user threads are open.
• The CHANGELOG entry sits under 0.11.0, the version in development. The decision says 0.10.0, which has already been released.