feat(midnight-liquidation): union the market whitelist across sources

[haydenshively]

Why

Prod and staging pointed at different Midnight markets APIs — prod at the in-code default api.morpho.org, staging overriding to api.morpho.dev (which carries extra whitelisted shorter-maturity markets for testing). Staging is supposed to reveal problems that will exist in prod, and an external dependency differing between the two undercuts that.

Measuring the actual difference first narrowed the problem considerably:

	api.morpho.org (prod)	api.morpho.dev (staging)
/v0/midnight/markets?listed=true	6 markets	19 — the same 6 plus 13 test markets maturing daily
/markets/midnight/liquidation-candidates	45 rows, 26 markets	byte-for-byte identical

So the candidates endpoint is chain-wide, not environment-scoped (staging's override of it is a no-op), and the whitelist was the only real difference — additive, on the same chain, with the same cbBTC collateral and USDC loan token in both sets. The gap that actually mattered was that nothing ever exercised api.morpho.org before prod did: a breaking change to the host prod depends on would sail past staging.

What

MARKETS_API_URL now accepts a comma-separated list of endpoints, and the whitelist is the union across them, with the max-age staleness rule applied per source.

• src/discovery/markets.ts — createUnionListedMarketFilter composes per-source filters. current() freezes the fresh-source set for one discovery pass, so a pass is judged against one staleness reading.
• src/config.ts — new urlListEnv (order-preserving, de-duplicated on the parsed URL, fail-loud on any malformed entry or an all-separators value); MarketsConfig.apiUrl → apiUrls.
• src/index.ts — builds one filter per URL and reports the all-expired case per tick.

Post-maturity liquidation-mode choice (faf1e33)

Surfaced by the protocol review and folded in here because the daily-maturity test markets staging now whitelists make it routine: for a matured and unhealthy position the contract opens both liquidation gates, but plan() always picked post-maturity mode, whose LIF ramps WAD → maxLif over an hour. Early in the ramp that repays nearly the full seized value, forfeiting the ~420 bps incentive normal mode pays immediately (~40x haircut at 60s past maturity) — and it made the first sim-passing block the block where profit ≈ 0, inviting sim-to-inclusion reverts.

sizing/plan.ts now builds both candidate plans when both gates are open and keeps the one with the higher expected surplus (seized oracle value minus contract-derived repaid). Ties resolve toward post-maturity: after the ramp it is at least as good (same LIF, no RCF cap), and its gate (now > maturity) cannot close between read and exec, while normal mode's (unhealthy) can if the price recovers. Matured-and-healthy positions are unchanged — post-maturity is their only open gate.

Intended deployment: staging gets the superset, prod stays prod-only

The property that matters is superset, not equality: staging must exercise everything prod touches. Staging reads both endpoints (so it finally exercises api.morpho.org, and its whitelist is a strict superset of prod's); prod keeps the single-endpoint default and does not widen. That fixes the stated problem at zero prod blast radius, and keeps the prod key from spending real gas on markets that exist only for testing.

The default stays the single public endpoint, so this branch is inert for prod — no prod env change is needed at all.

1. Merge → deploy-staging ships this code to staging-bot.
2. Set staging MARKETS_API_URL to https://api.morpho.org/v0/midnight/markets,https://api.morpho.dev/v0/midnight/markets, and delete staging's now-redundant LIQUIDATION_CANDIDATES_API_URL (a no-op override — the payloads are identical). Confirm markets.whitelist { markets: 19, sources: 2, fresh: 2 }.
3. Prod: nothing to do.

⚠️ Set a list value only after the supporting image is live, and clear it back to a single URL before rolling back past this release — older images validate the var as one URL and loadConfig would crash-loop the service. Documented in the README env table.

Safety semantics

Per-source staleness is what makes reading two endpoints safe in both directions: a source that goes down or goes stale drops out of the union (markets.source_expired) rather than either emptying the whitelist and halting all liquidations, or letting a stale set keep a since-delisted market in scope. refresh fans out concurrently and never throws — a per-source failure keeps that source's last-known-good and still lands its healthy peers. The union throws at construction if no source is configured, because an empty union lists nothing while logging nothing.

Log events (all carrying chainId): markets.whitelist (deduplicated combined size — per-source counts overlap and can be neither summed nor maxed into it), markets.listed / markets.refresh_failed / markets.source_expired (all carrying a host/path source label), markets.listed_empty (a source went from some markets to none — authoritative, but a healthy peer would otherwise mask it), markets.whitelist_expired (per tick, all sources stale → fail-closed), markets.refresh_error (at error: the non-throwing contract broke). discover.filtered additionally names the filtered-out market ids (skippedMarkets), so "was market X whitelisted at tick T" is answerable from one log line.

One semantic consequence of union-additivity, now documented in the union JSDoc and the README: an active delisting takes effect only once every configured source has dropped the market — one endpoint delisting it does not remove it while a peer still lists it.

Review

Reviewed by reviewer, protocol-engineer, and product-manager. The scoping decision above came out of that review — the original plan had prod read both endpoints. Fixed in 13f31f0: the per-tick expiry signal that a long MARKETS_REFRESH_MS could outrun, the silent empty-filters case, the unreported empty-response transition, the startup tryCatch that had been dropped, per-pass staleness consistency, the union-size event, and source labels/dedup that collided on same-host endpoints.

Codex follow-ups fixed in ad9fcb5: MARKETS_API_URL now rejects empty entries (a leading/trailing/repeated comma — usually a misinterpolated env var — must fail loud rather than silently narrow the whitelist), and trailing path slashes are canonicalized before de-duplication so two spellings of one endpoint cannot be polled twice and counted as two independent sources.

A second review round (same three agents) confirmed the mode-choice implementation line-by-line against the contract source and found no remaining Critical/Important code issues. Fixed in 1e2f62d: README mode docs brought in line with the both-gates choice (postMaturityMode: false shortly after maturity is by design), delisting-propagation semantics documented, chainId threaded into the union events, skippedMarkets ids on discover.filtered, maxRepaidPreMaturity → maxRepaidNormalMode (normal mode is gated on unhealthiness alone, so it applies after maturity too), and two comment inaccuracies (union source label is host + path; URL dedup collapses only bare-origin trailing slashes and default ports).

Known and deliberately not in scope — pre-existing issue this PR no longer amplifies, now that prod stays narrow:

• The uncounted continue for unplannable positions (runner/tick.ts:140) — prod's silently-dropped set stays at its current size rather than growing. Still worth its own fix.

Verification

• typecheck 0 errors; bun lint 0 warnings; bun format applied; knip clean.
• bun test: 1600 pass / 9 fail — the same 9 failures as the stashed baseline (fork + e2e tests requiring RPC_URL_8453 / FORK_URL), so zero regressions. 14 new tests.
• Vacuity checks: dropped the isFresh guard from the union and confirmed both per-source-expiry tests fail; separately replaced the deduplicated union size with a sum and removed the empty-source guard, and confirmed both of those tests fail; for the mode choice, forced post-maturity unconditionally and confirmed both normal-mode-preference tests fail; for the chainId threading, dropped the field from the markets.whitelist emit and confirmed both of its assertions fail. Reverted after each.
• Mode choice: 4 new plan() tests (normal wins early in the ramp — cap-bound and underwater shapes; post-maturity wins the post-ramp tie; post-maturity wins when the RCF cap binds normal mode). The fork e2e position is matured-and-healthy, so its post-maturity assertion is unaffected.

🤖 Generated with Claude Code

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: faf1e33388

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[linear-code]

BOTS-3