diff --git a/CHANGELOG.md b/CHANGELOG.md index bec90c810..9808f474f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,18 @@ # CHANGELOG +## 2026-10-27 — Preserve refresh outcomes and source-dated browser history (local candidate) + +- Keep the last failed/older refresh outcome in the shared cache so late + optional FX data cannot announce false recovery or replay success to a new + subscriber. Preserve original waits, their timestamp and valid recovery. +- Record browser history at snapshot retrieval time, deduplicate replayed or + older data and reject invalid samples. Keep legacy browser-clock history + unchanged in its old storage key; source-dated history starts separately. +- Add delayed/interrupted/repeated-flow regressions and preserve both history + keys in existing browser-fixture cleanup. The source/data/bundle/security + verification and remaining release hold are in the refresh runbook. +- Local preparation only; no production publication or revenue claim. + ## 2026-10-22 — Verified pedestrian route exceptions (local candidate, issue #59) - Keep the existing Progreso International Bridge (230901) and Santa Teresa diff --git a/docs/REFRESH-OPERATIONS.md b/docs/REFRESH-OPERATIONS.md index c885101ce..477f1c8b7 100644 --- a/docs/REFRESH-OPERATIONS.md +++ b/docs/REFRESH-OPERATIONS.md @@ -1,5 +1,180 @@ # Public data refresh +## Reliability continuation — 2026-10-27 UTC (local, unreleased) + +- Release continuation: the owner explicitly approved the scoped Zod 4 + upgrade and freshness repairs on October 27, conditional on final review + and exact-commit checks. This does not authorize cancelling or approving + the obsolete Pages run. The existing Vercel publisher remains the release + path after a passing guarded merge; do not duplicate-dispatch it. +- Existing PR #150 was fast-forwarded to public-metadata candidate + `cb414c5d63feb5d3fde68a8877bdcff6638bc188`. Initial private-email pushes + were rejected without changing the remote; metadata-only copies using the + verified GitHub noreply identity preserve identical tested file trees and + all original local branches. No email-privacy setting was changed. +- Both exact-head CI jobs passed source tests and security audit, then failed + the unchanged size gate: the same entry compressed to 62.15 KiB on CI's + Node 22 versus 61.96 KiB on the earlier local Node 25. Keep the 62 KiB gate. + The follow-up moves directional history enumeration into the already lazy + Analytics view, reuses safe storage helpers and validates stored history + in one pass without sorting corrupted ordering. Snapshot writes remain + synchronous, and invalid/future/expired/duplicate/older samples still fail + closed. Remove redundant dashboard error wrapping; shared listener and + storage boundaries continue to isolate failures. Official temporary Node + 22/24 runtimes are now used for final size verification. A further independent + check found that an otherwise valid stored sample could retain unrelated + deeply nested fields and make later serialization throw. Keep only its + validated time/wait pair; the new regression and actual cached replay pass. + All 17 focused checks now pass on Node 22; its unchanged entry-size check + reports 61.98/62 KiB. Merge `a18bdd1` also preserves the newer data-only main + `29a4850d891c2908125845bc5272ed9fd3e4e0ee`. Fresh final-head CI and full + customer-path checks remain required before merge. +- Ownership: the canonical checkout and the earlier October 20 worktree have + unrelated changes. This continuation uses the isolated clone + `task-6/borderpulse-candidate`, branch `fix/reliability-candidate-20261027`. + Merge `5abae6ad06c8ed1fd732be2da78dc02a1de9c536` preserves PR #150 head + `1a7433f8c9e1605d20e739d0973f058e064a440f` and collector main + `5f7693a74fb214708a9fd6c7e22c50a12b066d28`. The original branches and + dirty worktrees are unchanged. Source repair is + `07b114e6a1f9950926c92aec2511041af4cd5771`. +- Why this is additional work: PR #150 patches validation dependencies; it + does not repair browser refresh state or browser-history timestamps. A + successful waits request, failed next request and late optional FX response + reproducibly emitted success again for the earlier reading. Cache each + notification's outcome so late enrichment cannot erase failure and a new + subscriber receives the actual last outcome. Valid recovery still works. +- The dashboard also recorded the viewing clock for each successful refresh, + including an old static fallback. Record the supplied snapshot retrieval + time and reject repeated/older, missing, invalid, future or expired samples. + Zero remains a valid wait; missing and closed waits do not become zero. + Version 2 history deliberately starts without reusing version 1's + unverifiable viewing timestamps. The legacy raw value is retained unchanged. + Source retrieval time still does not establish an individual lane report's + date; existing raw CBP report labels and freshness thresholds are unchanged. +- Public evidence at 02:46 UTC: `/api/public/crossings` returned HTTP 200, + 42 crossings and `fetched_at=2026-10-27T02:46:39.503Z`. The publication + manifest still identifies `d214944c11d9b7da0935e4dde2d47b02f2c02e64` and + bundled CBP/history/FX retrievals from October 23 at approximately 17:22 UTC, + about 81.4 hours earlier. Collector success is therefore distinct from + bundled publication freshness. Vercel scheduled run `40346639406` failed + its unchanged audit (one high Zod and two moderate entries); upload was + skipped. At the later 02:59 UTC check, Pages run `40439591945` had been + cancelled without starting a job, successor `40440984634` was pending and + collector `40440913247` had succeeded. Main had advanced to data-only + `6ff010bc172e581a032a6ff5d771bff77888d976`; the local candidate retains + its explicitly tested earlier snapshot. No run was retried or cancelled here. +- A further read-only check found Pages run `36763480106`, dispatched September + 30 at source `9f6197138ffa7b6de4327ff518a12f423726c2f0`, still waiting at + `github-pages` with no steps. Its exact historical workflow and today's + workflow both use concurrency group `pages`. This supports a queue-blocker + diagnosis; GitHub does not expose a definitive pending-reason field here. + Pending-deployment readback reports no reviewer or timer and + `current_user_can_approve=false`; no in-progress run was reported. Do not + approve that obsolete source. Any separately authorized cancellation must + account for queued Pages publication being able to proceed afterward, and + retain environment/access policy and every release gate. +- Clean locked install, normal and Vercel-mode builds, full `npm test` with + 156 source/unit checks and 402 isolated headless browser navigations pass. + The initial addition exceeded the + 62 KiB entry gate by 19 bytes; reusing the existing safe-storage writer and + consolidating cache notification state brought it to 61.96 KiB in the normal + build and 61.93 KiB in Vercel mode without a budget change. Current registry + audit reports zero findings and the + unchanged gate passes; separate synthetic high/unavailable reports both + correctly fail. This later registry result does not lift the dependency + review or publication hold. Audit, bundle and workflow policies are intact. +- Local preview is `http://127.0.0.1:3426/`; shell readback verifies its + San Ysidro canonical, useful visible snapshot and 42-crossing input retrieved + at `2026-10-27T02:37:52.699Z`. It is a localhost artifact, not a hosted + preview or production release. Full route verification used only isolated + headless contexts after parent coordination; EN/ES desktop/mobile covers + retrieval aging, expiry, zero/missing/closed reports, interrupted refresh, + recovery, keyboard menus, Back/Forward, history, no-JavaScript reports and + metadata. Independent targeted browser checks also pass 25 flow groups: + EN/ES at 1440/390px plus denied-storage recovery, actual rendered zero/trend + states, source-dated history, preserved v1 data, repeated/enriched/older + snapshots, failed refresh followed by delayed FX, and keyboard/remount paths. + All 71 live endpoint requests were intercepted and no uncaught page errors + occurred. Eight final screenshots were captured and representative desktop/ + mobile pixels inspected. Synthetic fixtures are local test evidence, not + measured crossing conditions. Native Safari, VoiceOver and physical devices + remain unrun. +- No push, duplicate PR, merge to main, preview/production deployment, + credential, service, account/access setting, audit exception or monitor. + The scoped release decision above supersedes the earlier hold; passing + final exact-head CI remains mandatory before merging or invoking the + existing production publisher. + Fresh public manifest/history/FX readback and two subsequent scheduled runs + must then establish publication recovery. No revenue result is measured. + +## Combined validation security review — 2026-10-24 UTC (unreleased) + +- Isolated integration base main `f5d7a2f1dc4b6a8dbf9fe5da75838f0ec6a2c70a`. + Retains the original PR #149 commit `81e6069b` and Zod candidate + `27b4d1d3` as ancestors; neither source branch is rewritten. The only + integration conflict was adjacent test commands, resolved by keeping both. + Snapshot updates remain from current main. The dated standalone notes below + describe their earlier local passes, not a production receipt. +- Every discovered frontmatter consumer now has CLI regression coverage in + temporary fixtures: the blog index builder, both anomaly templates and the + LLM drafter. Real SDK 0.98.0 runs against an intercepted fetch with a dummy + key and no inherited credentials. No external LLM request is made. + Mocked EN/ES tool responses preserve forced draft/human-review flags; + invalid fields and missing tool responses report failures without a draft. + Template failures preserve field diagnostics and continue to valid entries. + The index still excludes drafts and rejects unreviewed auto analysis. +- The major-version review covers the SDK's declared `^3.25.0 || ^4.0.0` + peer range, actual request/response handling, defaults, URL/string/enum + validation, field diagnostics and generated-frontmatter serialization. + No production schema, generator or client implementation changes are needed. + Package/lock changes remain limited to Zod and the scoped parser backport. +- Clean integrated install/build and full npm test pass: **148 unit tests and + 416 browser navigations**, including EN/ES mobile/desktop regressions. All + 25 JS/CSS assets and normalized metadata for 36 posts match the baseline. + Local entry **61.81/62 KiB** gzip; no budget change. Remote CI still required. +- The upstream parser advisory-range discrepancy remains visible as two + moderate entries. No alert dismissal, accepted-advisory addition, threshold, + account/security setting or credential change. Unbounded collection parsing + is unchanged by the Zod fix; mocks do not validate real provider behavior. + Review and exact integrated CI are required before any later release. + This continuation permits a draft review PR, not merge or deployment. + +## Zod validation candidate — 2026-10-24 UTC (local, unreleased) + +- Separate from PR #149; base main `3e803b6ef6e13f2e4ec213c1233821859f3d0cbe`. + Replace only Zod **3.25.76 → 4.6.7** and its declared range. Official npm + lists no stable 3.x release beyond 3.25.76. The + [reviewed advisory](https://github.com/advisories/GHSA-mrhx-c7xc-m3w5) + lists 4.6.6 as patched; the [maintainer fix](https://github.com/colinhacks/zod/pull/6652) + rejects oversized bounded collections before parsing their elements. + [4.6.7](https://github.com/colinhacks/zod/releases/tag/v4.6.7) is the next + published patch. This is an investigated major change, not a same-major fix. +- Inspected application use is `src/lib/blog-schema.js`, consumed by the blog + index builder and both draft generators, including an LLM tool response. + No browser/API validation consumer was found. The installed Anthropic SDK + 0.98.0 declares peer support for `^3.25.0 || ^4.0.0`; it resolves the same + new Zod version without changing the SDK or any other locked package. + The SDK module import was checked without making an API call or using a key. +- Reviewed the [migration guide](https://zod.dev/v4/changelog), especially + defaults and error formatting. All 36 checked-in posts retain identical + normalized metadata. Defaults, explicit draft values, EN/ES, invalid-field + diagnostics and the existing six-tag boundary have focused regression tests. + On 10,000 invalid tags, the old version returned 10,001 issues; the candidate + returns only the size issue. The controlled case does not exhaust memory. +- Clean baseline/candidate installs and builds pass. All 25 generated JS/CSS + assets are byte-identical; no visitor-payload increase or budget change. + Candidate audit removes the Zod finding, reports zero high/critical and + zero omit-dev entries, and passes the unchanged audit gate. The two existing + Typography/parser moderate entries remain; PR #149 is still separate. +- Full local npm test passes: **138 unit tests and 416 browser navigations**, + including existing EN/ES mobile/desktop fixtures. Entry JS **61.81/62 KiB** + gzip and CSS **19.09/20 KiB**; budgets and audit policy are unchanged. +- Unbounded collections are unchanged by the upstream fix. This review does + not establish zero resource-exhaustion risk or complete application security. + Before publication, require review of this major dependency change and fresh + final integration/CI. No push, merge, deployment, alert dismissal, exception, + credential, account/security-setting or paid-service change in this pass. + ## Walk-or-drive inventory candidate — 2026-10-22 UTC (issue #59, unreleased) - Review base: `a10706b5eb5ae48468e32c5fc21c657d63bd8153`, rebased cleanly @@ -336,6 +511,41 @@ and this performance follow-up. Version 6.1.5 is the CSS parser, not Router. An inverse-patch check against main passed with a temporary index; no rollback was executed. Avoid rolling the whole site back to an older data bundle. +## Local Typography parser backport candidate — 2026-10-23 UTC + +This local candidate scopes an npm override to `@tailwindcss/typography` only: +its parser changes from 6.0.10 to **6.1.5**. Typography stays at 0.5.19; no other +package or dependency edge changes. npm hoists the replacement from the nested +Typography directory to `node_modules/postcss-selector-parser`. This is a local +review candidate, not a production publication receipt. + +The [maintainer advisory](https://github.com/postcss/postcss-selector-parser/security/advisories/GHSA-rj75-hqrm-r3gf) +and [6.1.5 release](https://github.com/postcss/postcss-selector-parser/releases/tag/6.1.5) +identify the security backport for the 6.x line. The separate +[reviewed advisory range](https://github.com/advisories/GHSA-rj75-hqrm-r3gf) +still says `<7.1.6`, so npm may keep reporting the two moderate package entries. +Keep those findings visible: no advisory exception, audit threshold change, +forced downgrade, or unrelated 7.x migration is part of this candidate. + +The inspected consumer is the CSS build plugin in `src/index.css`, parsing +repository sources. No browser/API import or visitor-controlled selector +parsing endpoint was found. That limited review does not establish zero risk +for future consumers or other dependencies. + +`npm run test:css-parser` resolves the parser through Typography itself, checks +the patched version and pseudo-element transformations, and parses a flat +400KB class/id selector in a heap-limited worker with a five-second deadline. +The old parser fails that bounded control; the backport completes it. The test +stops only its own worker if parsing regresses, without leaving a blocked test +process. It is also part of the full `npm test` gate. + +For review or a future parent upgrade, perform a clean `npm ci`, build and full +test suite; compare generated CSS against the same source/data input before +the override. Recheck the actual Typography resolution and advisory metadata. +Remove this override only when the parent itself supplies a patched compatible +parser, then repeat the output and route checks. Preserve the existing JS/CSS +budgets and the dated historical review below. + ## Bounded dependency review — 2026-10-22 UTC Official npm audit reports **eight moderate package entries**, zero high or diff --git a/package-lock.json b/package-lock.json index 16c9fc526..0b2407560 100644 --- a/package-lock.json +++ b/package-lock.json @@ -59,7 +59,7 @@ "tailwind-merge": "^3.6.0", "tailwindcss-animate": "^1.0.7", "vaul": "^1.1.2", - "zod": "^3.24.2" + "zod": "^4.6.7" }, "devDependencies": { "@anthropic-ai/sdk": "^0.98.0", @@ -3400,18 +3400,6 @@ "tailwindcss": ">=3.0.0 || insiders || >=4.0.0-alpha.20 || >=4.0.0-beta.1" } }, - "node_modules/@tailwindcss/typography/node_modules/postcss-selector-parser": { - "version": "6.0.10", - "dev": true, - "license": "MIT", - "dependencies": { - "cssesc": "^3.0.0", - "util-deprecate": "^1.0.2" - }, - "engines": { - "node": ">=4" - } - }, "node_modules/@types/babel__core": { "version": "7.20.5", "dev": true, @@ -5860,6 +5848,20 @@ "node": "^10 || ^12 || >=14" } }, + "node_modules/postcss-selector-parser": { + "version": "6.1.5", + "resolved": "https://registry.npmjs.org/postcss-selector-parser/-/postcss-selector-parser-6.1.5.tgz", + "integrity": "sha512-TEHFzb5KRDKaOgJ4TYE+0jyJtFbjLvkdVOu0fZ2b6MTWzoqZd4Le+61aEppWkvTlYePgOX4ATL1wsAIPMLTw6Q==", + "dev": true, + "license": "MIT", + "dependencies": { + "cssesc": "^3.0.0", + "util-deprecate": "^1.0.2" + }, + "engines": { + "node": ">=4" + } + }, "node_modules/prop-types": { "version": "15.8.1", "license": "MIT", @@ -6933,7 +6935,9 @@ "license": "ISC" }, "node_modules/zod": { - "version": "3.25.76", + "version": "4.6.7", + "resolved": "https://registry.npmjs.org/zod/-/zod-4.6.7.tgz", + "integrity": "sha512-g8W2D2kU6a8untc053fX1uVBKzM7yqAxW/3zwk83+6RyF2+SKOuV+c0wBDhUwzV1PaGapmZH+rmcUzPf+3CGwg==", "license": "MIT", "funding": { "url": "https://github.com/sponsors/colinhacks" diff --git a/package.json b/package.json index 77d53dabf..db2c2ce4c 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "type": "module", "scripts": { "dev": "vite", - "test": "npm run test:ads && npm run test:writer && npm run test:refresh && npm run test:publish && npm run test:hours && npm run test:trust && npm run test:mdx && npm run test:metadata && npm run test:audit && npm run test:bundle && npm run test:routes", + "test": "npm run test:ads && npm run test:writer && npm run test:refresh && npm run test:publish && npm run test:hours && npm run test:trust && npm run test:mdx && npm run test:css-parser && npm run test:metadata && npm run test:audit && npm run test:bundle && npm run test:routes", "test:metadata": "node --test scripts/test-faq-scope.mjs scripts/test-hreflang.mjs scripts/test-crossing-snapshot.mjs", "test:ads": "node --test scripts/test-ad-slot-diagnostics.mjs && node scripts/check-google-ads.mjs", "test:publish": "node --test scripts/test-publish-data.mjs scripts/test-public-crossings.mjs", @@ -35,8 +35,9 @@ "news:classify": "node scripts/build-curation-queue.mjs", "test:bundle": "node scripts/check-bundle-size.mjs", "test:audit": "node scripts/check-audit.mjs", - "test:trust": "node --test scripts/test-trust-state.mjs scripts/test-local-comparison.mjs scripts/test-browser-state.mjs scripts/test-historical-pattern.mjs scripts/test-router-navigation.mjs", - "test:mdx": "node --test scripts/test-mdx-frontmatter.mjs", + "test:trust": "node --test scripts/test-trust-state.mjs scripts/test-local-comparison.mjs scripts/test-browser-state.mjs scripts/test-historical-pattern.mjs scripts/test-router-navigation.mjs scripts/test-wait-history.mjs", + "test:mdx": "node --test scripts/test-mdx-frontmatter.mjs scripts/test-blog-schema.mjs scripts/test-blog-generation.mjs", + "test:css-parser": "node --test scripts/test-typography-parser.mjs", "test:writer": "node --test scripts/test-generated-writer.mjs", "test:refresh": "node --test scripts/test-data-refresh.mjs" }, @@ -92,7 +93,7 @@ "tailwind-merge": "^3.6.0", "tailwindcss-animate": "^1.0.7", "vaul": "^1.1.2", - "zod": "^3.24.2" + "zod": "^4.6.7" }, "devDependencies": { "@anthropic-ai/sdk": "^0.98.0", @@ -113,5 +114,10 @@ "tailwindcss": "^4.3.3", "typescript": "^5.8.2", "vite": "^6.4.4" + }, + "overrides": { + "@tailwindcss/typography": { + "postcss-selector-parser": "6.1.5" + } } } diff --git a/scripts/check-document-language.mjs b/scripts/check-document-language.mjs index 94b24d330..a6fa55ecb 100644 --- a/scripts/check-document-language.mjs +++ b/scripts/check-document-language.mjs @@ -4,7 +4,7 @@ import fs from 'node:fs'; const crossingPath = '/crossing/san-ysidro/'; const enArticle = '/blog/san-ysidro-vs-otay-mesa/'; const esArticle = '/blog/san-ysidro-vs-otay-cual-es-mas-rapida/'; -const keys = ['borderPulse_language', 'borderPulse_region', 'borderPulse_theme', 'borderPulse_geoConsent', 'borderPulse_waitHistory_v1']; +const keys = ['borderPulse_language', 'borderPulse_region', 'borderPulse_theme', 'borderPulse_geoConsent', 'borderPulse_waitHistory_v1', 'borderPulse_waitHistory_v2']; // Run only against a local preview. Every data response is synthetic; the // caller blocks external providers and service workers before opening a page. diff --git a/scripts/check-report-times.mjs b/scripts/check-report-times.mjs index b38c2677a..d0e969bdc 100644 --- a/scripts/check-report-times.mjs +++ b/scripts/check-report-times.mjs @@ -7,7 +7,7 @@ const FEED_PATHS = ['/data/crossings.json', '/api/public/crossings']; const FIXTURE_HEADER = 'x-borderpulse-report-time-fixture'; const STORAGE_KEYS = [ 'borderPulse_language', 'borderPulse_region', 'borderPulse_geoConsent', - 'borderPulse_waitHistory_v1', 'borderPulse_theme', + 'borderPulse_waitHistory_v1', 'borderPulse_waitHistory_v2', 'borderPulse_theme', ]; const LANE_LABELS = { passenger_standard: { en: 'Standard', es: 'Estándar' }, diff --git a/scripts/test-blog-generation.mjs b/scripts/test-blog-generation.mjs new file mode 100644 index 000000000..314f7cb25 --- /dev/null +++ b/scripts/test-blog-generation.mjs @@ -0,0 +1,188 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { spawnSync } from 'node:child_process'; +import { fileURLToPath } from 'node:url'; +import test from 'node:test'; +import matter from 'gray-matter'; + +const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +const valid = { + title: 'A test crossing guide', + description: 'Check current official crossing reports before traveling. This fixture explains the steps and sources to verify.', + slug: 'test-crossing-guide', date: '2026-10-24', author: 'sebastian', + pillar: 'data-analysis', lang: 'en', +}; +const anomaly = { + type: 'lowest_today', port_name: 'Test Port', port_slug: 'test-port', + port_number: '9901', state: 'Test', finding: 'Controlled fixture', + evidence: { day_label: 'Monday', best_hour: 7, best_median: 10, + overall_median: 30, samples: 12, peak_hour: 18, ratio: 2, peak_median: 60 }, +}; +const mdx = fm => '---\n' + Object.entries(fm).map(([key, value]) => + key + ': ' + JSON.stringify(value)).join('\n') + '\n---\n\nFixture body.\n'; + +function fixture(t) { + const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'blog-generation-')); + t.after(() => fs.rmSync(directory, { recursive: true, force: true })); + const put = (file, value) => { + const full = path.join(directory, file); + fs.mkdirSync(path.dirname(full), { recursive: true }); + fs.writeFileSync(full, value); + }; + put('package.json', '{"type":"module"}'); + for (const file of [ + 'src/lib/blog-schema.js', 'src/lib/slugs.js', 'scripts/build-blog-index.mjs', + 'scripts/draft-from-anomaly.mjs', 'scripts/draft-from-anomaly-llm.mjs', + 'scripts/lib/anthropic-client.mjs', 'scripts/lib/prompts/anomaly-draft.md', + ]) put(file, fs.readFileSync(path.join(root, file))); + fs.symlinkSync(path.join(root, 'node_modules'), path.join(directory, 'node_modules'), 'dir'); + fs.mkdirSync(path.join(directory, 'src/content/blog'), { recursive: true }); + put('public/data/crossings.json', '{"crossings":[]}'); + put('drafts/blog/auto-a/queue.json', JSON.stringify({ anomalies: [anomaly] })); + // Run the real SDK/client while intercepting every fetch before it can use + // the network. Child processes receive a dummy key and no inherited secrets. + put('mock-fetch.mjs', [ + "import fs from 'node:fs';", + "import assert from 'node:assert/strict';", + 'globalThis.fetch = async (url, options) => {', + " assert.equal(String(url), 'https://api.anthropic.com/v1/messages');", + " assert.equal(options.method, 'POST');", + ' const request = JSON.parse(options.body);', + " assert.equal(request.tool_choice.name, 'write_post_draft');", + ' assert.equal(request.max_tokens, 3000);', + " fs.appendFileSync('mock-calls.txt', 'intercepted\\n');", + " const content = JSON.parse(fs.readFileSync('mock-content.json', 'utf8'));", + ' return new Response(JSON.stringify({', + " id: 'msg_fixture', type: 'message', role: 'assistant', model: 'fixture',", + " content, stop_reason: 'tool_use', stop_sequence: null,", + ' usage: { input_tokens: 1, output_tokens: 1 },', + " }), { status: 200, headers: { 'content-type': 'application/json' } });", + '};', + ].join('\n')); + const run = (script, { content } = {}) => { + const args = []; + const env = { PATH: process.env.PATH, TZ: 'UTC' }; + if (content) { + put('mock-content.json', JSON.stringify(content)); + args.push('--import', path.join(directory, 'mock-fetch.mjs')); + env.ANTHROPIC_API_KEY = 'test-only-no-network'; + } + const result = spawnSync(process.execPath, [...args, path.join(directory, script)], { + cwd: directory, env, encoding: 'utf8', timeout: 20_000, + }); + assert.ifError(result.error); + return result; + }; + const drafts = () => fs.readdirSync(path.join(directory, 'drafts/blog/auto-a')) + .filter(file => file.endsWith('.mdx')).map(file => matter(fs.readFileSync( + path.join(directory, 'drafts/blog/auto-a', file), 'utf8'))); + return { directory, put, run, drafts }; +} +const toolResponse = frontmatter => [{ + type: 'tool_use', id: 'tool_fixture', name: 'write_post_draft', + input: { frontmatter, body: 'Mocked draft body.' }, +}]; + +test('template generation writes validated drafts for both existing anomaly templates', t => { + const f = fixture(t); + f.put('drafts/blog/auto-a/queue.json', JSON.stringify({ + anomalies: [anomaly, { ...anomaly, type: 'sunday_spike' }], + })); + const result = f.run('scripts/draft-from-anomaly.mjs'); + assert.equal(result.status, 0, result.stderr); + assert.match(result.stdout, /2 drafts written, 0 skipped/); + assert.equal(f.drafts().length, 2); + for (const draft of f.drafts()) { + assert.equal(draft.data.draft, true); + assert.equal(draft.data.humanEdited, false); + assert.equal(draft.data.source, 'auto-a'); + } +}); + +test('template generation reports invalid metadata and continues to a valid anomaly', t => { + const f = fixture(t); + f.put('drafts/blog/auto-a/queue.json', JSON.stringify({ + anomalies: [{ ...anomaly, port_name: 'x'.repeat(100) }, anomaly], + })); + const result = f.run('scripts/draft-from-anomaly.mjs'); + assert.equal(result.status, 0); + assert.match(result.stderr, /frontmatter validation failed/); + assert.match(result.stderr, /title/); + assert.match(result.stdout, /1 drafts written, 1 skipped/); + assert.equal(f.drafts().length, 1); +}); + +test('real SDK with mocked EN/ES tool responses retains the draft and human-review flags', t => { + for (const lang of ['en', 'es']) { + const f = fixture(t); + const result = f.run('scripts/draft-from-anomaly-llm.mjs', { + content: toolResponse({ ...valid, lang, draft: false, humanEdited: true }), + }); + assert.equal(result.status, 0, result.stderr); + assert.match(result.stdout, /1 drafts written, 0 failed/); + assert.equal(fs.readFileSync(path.join(f.directory, 'mock-calls.txt'), 'utf8'), 'intercepted\n'); + assert.equal(f.drafts().length, 1); + assert.deepEqual({ ...f.drafts()[0].data }, { + ...valid, lang, draft: true, humanEdited: false, source: 'auto-a', + }); + assert.equal(fs.readdirSync(path.join(f.directory, 'src/content/blog')).length, 0); + } +}); + +test('mocked LLM invalid fields retain flattened diagnostics and write no draft', t => { + const f = fixture(t); + const result = f.run('scripts/draft-from-anomaly-llm.mjs', { + content: toolResponse({ ...valid, lang: 'fr', tags: Array(7).fill('tag'), + officialSources: [{ label: 'CBP', url: 'invalid-url' }] }), + }); + assert.equal(result.status, 0); + assert.match(result.stdout, /0 drafts written, 1 failed/); + for (const field of ['lang', 'tags', 'officialSources']) assert.match(result.stderr, new RegExp(field)); + assert.equal(f.drafts().length, 0); +}); + +test('a mocked response without the draft tool is reported without creating a file', t => { + const f = fixture(t); + const result = f.run('scripts/draft-from-anomaly-llm.mjs', { + content: [{ type: 'text', text: 'No tool in this fixture.' }], + }); + assert.equal(result.status, 0); + assert.match(result.stdout, /0 drafts written, 1 failed/); + assert.match(result.stderr, /Model did not call write_post_draft tool/); + assert.equal(f.drafts().length, 0); +}); + +test('blog index retains draft exclusion and requires human review before publishing auto analysis', t => { + const f = fixture(t); + const frontmatter = { ...valid, draft: true, source: 'auto-a', humanEdited: false }; + const file = 'src/content/blog/test-crossing-guide.mdx'; + f.put(file, mdx(frontmatter)); + assert.equal(f.run('scripts/build-blog-index.mjs').status, 0); + const index = path.join(f.directory, 'public/data/blog/index.json'); + assert.deepEqual(JSON.parse(fs.readFileSync(index)).posts, []); + f.put(file, mdx({ ...frontmatter, draft: false })); + const rejected = f.run('scripts/build-blog-index.mjs'); + assert.equal(rejected.status, 1); + assert.match(rejected.stderr, /requires humanEdited: true/); + assert.deepEqual(JSON.parse(fs.readFileSync(index)).posts, []); + f.put(file, mdx({ ...frontmatter, draft: false, humanEdited: true })); + assert.equal(f.run('scripts/build-blog-index.mjs').status, 0); + const posts = JSON.parse(fs.readFileSync(index)).posts; + assert.equal(posts.length, 1); + assert.equal(posts[0].slug, valid.slug); + assert.deepEqual(posts[0].tags, []); +}); + +test('blog index rejects malformed source metadata with the existing field diagnostics', t => { + const f = fixture(t); + f.put('src/content/blog/test-crossing-guide.mdx', mdx({ ...valid, lang: 'fr', + officialSources: [{ label: 'CBP', url: 'invalid-url' }] })); + const result = f.run('scripts/build-blog-index.mjs'); + assert.equal(result.status, 1); + assert.match(result.stderr, /frontmatter errors/); + assert.match(result.stderr, /lang/); + assert.match(result.stderr, /officialSources/); + assert.equal(fs.existsSync(path.join(f.directory, 'public/data/blog/index.json')), false); +}); diff --git a/scripts/test-blog-schema.mjs b/scripts/test-blog-schema.mjs new file mode 100644 index 000000000..8839acacd --- /dev/null +++ b/scripts/test-blog-schema.mjs @@ -0,0 +1,72 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { frontmatterSchema } from '../src/lib/blog-schema.js'; + +const valid = { + title: 'A border crossing guide', + description: 'Check current official crossing reports before traveling. This guide explains the steps and sources to verify.', + slug: 'a-border-crossing-guide', + date: '2026-10-24', + author: 'sebastian', + pillar: 'crossing-guides', + lang: 'en', +}; + +test('EN/ES frontmatter keeps defaults and strips unknown fields', () => { + for (const lang of ['en', 'es']) { + assert.deepEqual(frontmatterSchema.parse({ ...valid, lang, extra: 'ignored' }), { + ...valid, + lang, + tags: [], + relatedCrossings: [], + officialSources: [], + draft: false, + source: 'human', + humanEdited: true, + }); + } +}); + +test('draft metadata retains explicit values and official sources', () => { + const input = { + ...valid, + lang: 'es', + tags: ['planning'], + relatedCrossings: ['san-ysidro'], + officialSources: [{ label: 'CBP', url: 'https://www.cbp.gov/' }], + draft: true, + source: 'auto-a', + humanEdited: false, + }; + assert.deepEqual(frontmatterSchema.parse(input), input); +}); + +test('invalid frontmatter still supplies field diagnostics used by build and drafters', () => { + const result = frontmatterSchema.safeParse({ + ...valid, + title: '', + description: 'short', + slug: 'Invalid Slug', + lang: 'fr', + officialSources: [{ label: 'CBP', url: 'invalid-url' }], + }); + assert.equal(result.success, false); + const fields = result.error.flatten().fieldErrors; + assert.deepEqual(Object.keys(fields).sort(), ['description', 'lang', 'officialSources', 'slug', 'title']); + assert.ok(Object.values(fields).every(messages => messages.length > 0)); +}); + +test('the existing six-tag limit accepts its boundary and rejects the next item', () => { + assert.equal(frontmatterSchema.safeParse({ ...valid, tags: Array(6).fill('planning') }).success, true); + assert.equal(frontmatterSchema.safeParse({ ...valid, tags: Array(7).fill('planning') }).success, false); +}); + +test('oversized invalid tags report the size bound without accumulating element errors', () => { + // Exercise the advisory on the real bounded schema, with a small controlled input. + // This verifies bounded diagnostics rather than timing or exhausting the process. + const result = frontmatterSchema.safeParse({ ...valid, tags: Array(10_000).fill(0) }); + assert.equal(result.success, false); + assert.equal(result.error.issues.length, 1); + assert.equal(result.error.issues[0].code, 'too_big'); + assert.deepEqual(result.error.issues[0].path, ['tags']); +}); diff --git a/scripts/test-data-refresh.mjs b/scripts/test-data-refresh.mjs index f26cab095..6ee2aba5a 100644 --- a/scripts/test-data-refresh.mjs +++ b/scripts/test-data-refresh.mjs @@ -133,6 +133,55 @@ test('an initial failure never invents a fresh timestamp', async () => { assert.deepEqual(data.crossings, []); }); +for (const interruptedBy of ['failed refresh', 'older fallback']) { + test(`delayed optional data cannot erase a ${interruptedBy}, including subscriber replay`, async t => { + t.mock.timers.enable({ apis: ['setInterval'] }); + const service = new DataService(); + let finishFx; + const fx = new Promise(resolve => { finishFx = resolve; }); + const originalTime = '2026-09-28T01:00:00Z'; + const doc = fetched_at => ({ fetched_at, crossings: [{ port_number: '250401', current_wait_time: 0 }] }); + service.fetchCrossingsDoc = async () => ({ doc: doc(originalTime), fromFallback: false }); + service.fetchJson = async path => path === '/data/exchange-rate.json' ? fx : null; + const received = []; + service.addListener(data => received.push(data)); + await service.getBorderData(); + + service.fetchCrossingsDoc = interruptedBy === 'failed refresh' + ? async () => { throw new Error('interrupted waits refresh'); } + : async () => ({ doc: doc('2026-09-28T00:00:00Z'), fromFallback: true }); + const retained = await service.getBorderData(); + assert.equal(retained.success, false); + assert.equal(retained.fromFallback, true); + assert.equal(retained.timestamp, originalTime); + assert.equal(retained.crossings[0].current_wait_time, 0); + assert.equal(service.cache, retained); + + finishFx({ rate: 18, fetched_at: originalTime }); + await new Promise(resolve => setImmediate(resolve)); + assert.equal(received.length, 2, 'the old enrichment must not announce false recovery'); + assert.equal(service.cache, retained); + + const replayed = []; + const stop = subscribeToBorderData(data => replayed.push(data), { + service, page: new EventTarget(), connection: new EventTarget(), + }); + assert.equal(replayed[0], retained, 'a new view replays the latest failure state'); + await service.inFlight; + stop(); + assert.equal(replayed.at(-1).success, false); + + const recoveredTime = '2026-09-28T01:15:00Z'; + service.fetchCrossingsDoc = async () => ({ doc: doc(recoveredTime), fromFallback: false }); + const recovered = await service.getBorderData(); + assert.equal(recovered.success, true); + assert.equal(recovered.fromFallback, false); + assert.equal(recovered.timestamp, recoveredTime); + assert.equal(service.cache.timestamp, recoveredTime); + assert.equal(service.cache.success, true); + }); +} + test('live failure retains a newer browser reading, then accepts live recovery', async t => { const service = new DataService({ liveApi: true }); let liveTime = '2026-09-29T01:00:00Z'; diff --git a/scripts/test-typography-parser.mjs b/scripts/test-typography-parser.mjs new file mode 100644 index 000000000..e6133f059 --- /dev/null +++ b/scripts/test-typography-parser.mjs @@ -0,0 +1,70 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createRequire } from 'node:module'; +import { Worker } from 'node:worker_threads'; + +// Resolve the parser used by the CSS plugin, including a nested dependency. +const require = createRequire(import.meta.url); +const typographyRequire = createRequire(require.resolve('@tailwindcss/typography/package.json')); +const parserPath = typographyRequire.resolve('postcss-selector-parser'); +const version = typographyRequire('postcss-selector-parser/package.json').version; +const utils = typographyRequire('./src/utils.js'); + +test('Typography resolves a version patched for GHSA-rj75-hqrm-r3gf', () => { + assert.match(version, /^\d+\.\d+\.\d+$/); + const [major, minor, patch] = version.split('.').map(Number); + const patched = (major === 6 && (minor > 1 || (minor === 1 && patch >= 5))) + || major > 7 || (major === 7 && (minor > 1 || (minor === 1 && patch >= 6))); + assert.ok(patched, 'Typography actually resolves vulnerable parser ' + version); +}); + +test('Typography preserves pseudo-element and selector-list transformations', () => { + const cases = [ + ['.alpha::before', ['::before', '.alpha']], + ['.alpha::before, .beta::before', ['::before', '.alpha, .beta']], + ['.alpha:hover', [null, '.alpha:hover']], + ['.alpha:not(.beta)::after', ['::after', '.alpha:not(.beta)']], + ['[data-state="open"] > a::before', ['::before', '[data-state="open"] > a']], + ['section > p::before, section > a::after', [null, 'section > p::before, section > a::after']], + ]; + for (const [selector, expected] of cases) { + assert.deepEqual(utils.commonTrailingPseudos(selector), expected); + } +}); + +test('flat class/id selectors parse within a bounded worker deadline', { timeout: 8_000 }, async t => { + // The advisory uses a flat 400KB selector: nesting guards cannot protect it. + // Bound both time and heap; a regression stops only this test's own worker. + const source = [ + "const { parentPort, workerData } = require('node:worker_threads');", + "const { performance } = require('node:perf_hooks');", + 'const parser = require(workerData.parserPath);', + "const selector = '.a#b'.repeat(100_000);", + 'const started = performance.now();', + 'const ast = parser().astSync(selector);', + 'parentPort.postMessage({ nodes: ast.first.nodes.length,', + ' roundTrip: ast.toString() === selector, elapsedMs: performance.now() - started });', + ].join('\n'); + const worker = new Worker(source, { + eval: true, + workerData: { parserPath }, + resourceLimits: { maxOldGenerationSizeMb: 256, maxYoungGenerationSizeMb: 32 }, + }); + let deadline; + try { + const result = await new Promise((resolve, reject) => { + deadline = setTimeout(() => reject(new Error('Flat selector parsing exceeded the 5-second safety deadline')), 5_000); + worker.once('message', resolve); + worker.once('error', reject); + worker.once('exit', code => { + if (code !== 0) reject(new Error('Parser worker exited with code ' + code)); + }); + }); + assert.equal(result.nodes, 200_000); + assert.equal(result.roundTrip, true); + t.diagnostic('Parser ' + version + ': 200000 selector nodes in ' + Math.round(result.elapsedMs) + 'ms'); + } finally { + clearTimeout(deadline); + await worker.terminate(); + } +}); diff --git a/scripts/test-wait-history.mjs b/scripts/test-wait-history.mjs new file mode 100644 index 000000000..4fb018e30 --- /dev/null +++ b/scripts/test-wait-history.mjs @@ -0,0 +1,136 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + recordSnapshot, getHistory, getHistoryForDirection, getPreviousWait, +} from '../src/components/utils/waitTimeHistory.js'; +import { getAllHistory } from '../src/components/utils/historyAnalytics.js'; + +const KEY = 'borderPulse_waitHistory_v2'; +const LEGACY_KEY = 'borderPulse_waitHistory_v1'; +const NOW = Date.parse('2026-10-27T03:00:00Z'); +const minute = 60_000; +const day = 24 * 60 * minute; +const iso = time => new Date(time).toISOString(); +const crossing = wait => ({ port_number: '250401', current_wait_time: wait }); + +function fixture(t, values = {}) { + const previous = Object.getOwnPropertyDescriptor(globalThis, 'localStorage'); + const stored = new Map(Object.entries(values)); + const storage = { + getItem: key => stored.get(key) ?? null, + setItem: (key, value) => stored.set(key, value), + }; + Object.defineProperty(globalThis, 'localStorage', { configurable: true, writable: true, value: storage }); + t.after(() => { + if (previous) Object.defineProperty(globalThis, 'localStorage', previous); + else delete globalThis.localStorage; + }); + t.mock.method(Date, 'now', () => NOW); + return { stored, storage }; +} + +test('history records source retrievals once across refresh, enrichment, fallback and recovery', t => { + fixture(t); + const sourceTime = NOW - 2 * day; + recordSnapshot([crossing(0)], 'northbound', iso(sourceTime)); + assert.deepEqual(getHistory('250401'), [{ t: sourceTime, wait: 0 }], + 'an old published reading stays dated at its actual retrieval'); + t.mock.method(Date, 'now', () => NOW + 5 * minute); + recordSnapshot([crossing(0)], 'northbound', iso(sourceTime)); + recordSnapshot([crossing(99)], 'northbound', iso(sourceTime)); + recordSnapshot([crossing(60)], 'northbound', iso(sourceTime - minute)); + assert.equal(getHistory('250401').length, 1, 'replayed and older snapshots create no observations'); + recordSnapshot([crossing(20)], 'northbound', iso(NOW)); + recordSnapshot([crossing(20)], 'northbound', iso(NOW + minute)); + assert.deepEqual(getHistory('250401'), [ + { t: sourceTime, wait: 0 }, { t: NOW, wait: 20 }, { t: NOW + minute, wait: 20 }, + ], 'a later retrieval is a real sample even if the wait is unchanged'); + assert.equal(getPreviousWait('250401'), 20); +}); + +test('missing, malformed, future and expired source times never become current history', t => { + const { stored } = fixture(t); + for (const sourceTime of [undefined, null, '', 'invalid', NOW, new Date(NOW), iso(NOW + 1), iso(NOW - 7 * day - 1)]) { + recordSnapshot([crossing(20)], 'northbound', sourceTime); + } + assert.deepEqual(getAllHistory(), {}); + assert.equal(stored.has(KEY), false); + recordSnapshot([crossing(20)], 'northbound', iso(NOW - minute)); + assert.deepEqual(getHistory('250401'), [{ t: NOW - minute, wait: 20 }]); +}); + +test('unverifiable legacy browser-clock history is preserved without being reused', t => { + const legacy = JSON.stringify({ '250401': [{ t: NOW - minute, wait: 99 }] }); + const { stored } = fixture(t, { [LEGACY_KEY]: legacy }); + assert.deepEqual(getAllHistory(), {}); + assert.equal(getPreviousWait('250401'), null); + recordSnapshot([crossing(0)], 'northbound', iso(NOW - minute)); + recordSnapshot([crossing(20)], 'northbound', iso(NOW)); + assert.equal(getPreviousWait('250401'), 0); + assert.equal(stored.get(LEGACY_KEY), legacy, 'retain the old raw value unchanged'); + assert.equal(JSON.parse(stored.get(KEY))['250401'][0].wait, 0); +}); + +test('malformed stored data cannot break history consumers or introduce invalid samples', t => { + const { stored } = fixture(t); + for (const value of ['invalid', 'null', '42', '[]', '"history"', '{"250401":{}}']) { + stored.set(KEY, value); + assert.deepEqual(getAllHistory(), {}); + assert.deepEqual(getHistory('250401'), []); + assert.equal(getPreviousWait('250401'), null); + } + stored.set(KEY, JSON.stringify({ '250401': [ + null, { t: NOW + minute, wait: 99 }, { t: NOW - 8 * day, wait: 99 }, + { t: NOW - 7 * day - 0.5, wait: 99 }, + { t: NOW, wait: -1 }, { t: 'invalid', wait: 99 }, { t: NOW, wait: '99' }, + { t: NOW - 2 * minute, wait: 0 }, { t: NOW, wait: 20 }, + { t: NOW - minute, wait: 99 }, { t: NOW, wait: 99 }, + ] })); + assert.deepEqual(getHistory('250401'), [{ t: NOW - 2 * minute, wait: 0 }, { t: NOW, wait: 20 }], + 'out-of-order stored entries are discarded rather than reordered into source history'); + recordSnapshot([crossing(30)], 'northbound', iso(NOW)); + assert.equal(getHistory('250401').length, 2, 'a duplicate does not overwrite its recorded source'); +}); + +test('stored samples retain only source time and wait before later writes', t => { + const { stored } = fixture(t); + const nested = '{"extra":'.repeat(10_000) + '0' + '}'.repeat(10_000); + stored.set(KEY, `{"250401":[{"t":${NOW - minute},"wait":0,"extra":${nested}}]}`); + assert.deepEqual(getHistory('250401'), [{ t: NOW - minute, wait: 0 }]); + assert.doesNotThrow(() => recordSnapshot([crossing(20)], 'northbound', iso(NOW)), + 'unrelated nested storage fields cannot escape the cached subscriber replay'); + assert.deepEqual(JSON.parse(stored.get(KEY)), { + '250401': [{ t: NOW - minute, wait: 0 }, { t: NOW, wait: 20 }], + }); +}); + +test('directional samples remain separate and unavailable or closed waits are not zero', t => { + fixture(t); + const row = { ...crossing(10), southbound_wait_time: 30 }; + recordSnapshot([row], 'northbound', iso(NOW)); + recordSnapshot([row], 'southbound', iso(NOW)); + assert.deepEqual(getAllHistory('northbound'), { '250401': [{ t: NOW, wait: 10 }] }); + assert.deepEqual(getHistoryForDirection('250401', 'southbound'), [{ t: NOW, wait: 30 }]); + for (const wait of [null, undefined, NaN, Infinity, -1]) { + recordSnapshot([{ port_number: '250601', current_wait_time: wait }], 'northbound', iso(NOW)); + } + recordSnapshot([{ port_number: '250601', current_wait_time: 0, port_status: 'Closed' }], 'northbound', iso(NOW)); + assert.deepEqual(getHistory('250601'), []); + t.mock.method(Date, 'now', () => NOW + 7 * day + 1); + assert.deepEqual(getAllHistory(), {}, 'expired readings do not persist in analytics'); +}); + +test('storage denial and quota failures leave history usable and later writes can recover', t => { + const { stored, storage } = fixture(t); + t.mock.method(console, 'warn', () => {}); + Object.defineProperty(globalThis, 'localStorage', { configurable: true, get() { throw new Error('blocked'); } }); + assert.doesNotThrow(() => recordSnapshot([crossing(20)], 'northbound', iso(NOW))); + assert.deepEqual(getAllHistory(), {}); + Object.defineProperty(globalThis, 'localStorage', { configurable: true, writable: true, value: storage }); + storage.setItem = () => { throw new Error('quota'); }; + assert.doesNotThrow(() => recordSnapshot([crossing(20)], 'northbound', iso(NOW))); + assert.deepEqual(getHistory('250401'), []); + storage.setItem = (key, value) => stored.set(key, value); + recordSnapshot([crossing(20)], 'northbound', iso(NOW)); + assert.deepEqual(getHistory('250401'), [{ t: NOW, wait: 20 }]); +}); diff --git a/src/components/dashboard/AnalyticsView.jsx b/src/components/dashboard/AnalyticsView.jsx index 6ad6de2ee..ea76b6c79 100644 --- a/src/components/dashboard/AnalyticsView.jsx +++ b/src/components/dashboard/AnalyticsView.jsx @@ -12,7 +12,7 @@ import { XAxis, YAxis, } from 'recharts'; -import { getAllHistory } from '@/components/utils/waitTimeHistory'; +import { getAllHistory } from '@/components/utils/historyAnalytics'; import { getStorageKey } from '@/components/utils/crossingDirection'; // Auto-derived volume proxy: sum of CBP-published lanes_open across all lane diff --git a/src/components/utils/dataService.js b/src/components/utils/dataService.js index 293283eb5..4168011b1 100644 --- a/src/components/utils/dataService.js +++ b/src/components/utils/dataService.js @@ -63,6 +63,9 @@ export class DataService { addListener(fn) { this.listeners.add(fn); } removeListener(fn) { this.listeners.delete(fn); } notify(payload) { + // Replay the latest outcome to new subscribers. Replacing the cached + // object also invalidates older requests' pending optional enrichment. + this.cache = payload; for (const fn of this.listeners) { try { fn(payload); } catch (e) { console.warn('listener failed', e); } } @@ -82,7 +85,7 @@ export class DataService { if (!res.ok) throw new Error(`HTTP ${res.status} for ${LIVE_DATA_PATH}`); return { doc: validateSnapshot(await res.json()), fromFallback: false }; } catch (error) { - console.warn('[dataService] live CBP endpoint failed; using published snapshot', error); + console.warn('[dataService] CBP fallback', error); return { doc: validateSnapshot(await this.fetchJson(DATA_PATH)), fromFallback: true }; } } @@ -123,7 +126,6 @@ export class DataService { southbound_note: sbDoc?.note || null, fromFallback, }; - this.cache = payload; this.notify(payload); // Current waits do not depend on the optional feeds. Enrich the same // reading later, without overwriting a newer refresh or its timestamp. @@ -138,9 +140,8 @@ export class DataService { southbound_timestamp: sb?.fetched_at || null, southbound_note: sb?.note || null, }; - this.cache = enriched; this.notify(enriched); - }).catch(error => console.warn('[dataService] optional feed failed', error)); + }).catch(error => console.warn('[dataService] optional feed', error)); } return payload; } catch (err) { diff --git a/src/components/utils/historyAnalytics.js b/src/components/utils/historyAnalytics.js new file mode 100644 index 000000000..81543294c --- /dev/null +++ b/src/components/utils/historyAnalytics.js @@ -0,0 +1,9 @@ +import { readAllHistory } from './waitTimeHistory.js'; + +// Only the deferred Analytics view needs to enumerate directional history. +export function getAllHistory(direction = null) { + const all = readAllHistory(); + if (!direction) return all; + return Object.fromEntries(Object.entries(all) + .filter(([key]) => key.endsWith(':sb') === (direction === 'southbound'))); +} diff --git a/src/components/utils/waitTimeHistory.js b/src/components/utils/waitTimeHistory.js index f610cf6e2..8e76e9366 100644 --- a/src/components/utils/waitTimeHistory.js +++ b/src/components/utils/waitTimeHistory.js @@ -1,65 +1,82 @@ /** * waitTimeHistory - localStorage-backed per-crossing history. - * Keeps last 7 days of { t, wait } samples per port_number. + * Keeps last 7 days of { t, wait } samples per port_number, where t is the + * snapshot retrieval time, not the time the browser displayed the reading. * Used for sparklines, trend arrows, and the Analytics view. */ -import { getStorageKey, getWaitMinutes } from '@/components/utils/crossingDirection'; +import { getStorageKey, getWaitMinutes } from './crossingDirection.js'; +import { readStorage, writeStorage } from '../../lib/browserStorage.js'; -const STORAGE_KEY = 'borderPulse_waitHistory_v1'; -const MAX_DAYS = 7; -const MAX_SAMPLES_PER_CROSSING = 7 * 24 * 4; // 15-min grain × 7d +// v1 stamped the browser clock and cannot establish source retrieval times. +// Leave that raw value intact, but do not reuse it as verified history. +const STORAGE_KEY = 'borderPulse_waitHistory_v2'; +const RETENTION_MS = 7 * 24 * 60 * 60 * 1000; +const MAX_SAMPLES_PER_CROSSING = 7 * 24 * 4; -function readAll() { +export function readAllHistory() { try { - const raw = localStorage.getItem(STORAGE_KEY); - return raw ? JSON.parse(raw) : {}; + const parsed = JSON.parse(readStorage(STORAGE_KEY, '{}')); + if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return {}; + const now = Date.now(); + const cutoff = now - RETENTION_MS; + const entries = []; + for (const [key, value] of Object.entries(parsed)) { + if (!Array.isArray(value)) continue; + const series = []; + let previous = cutoff - 1; + for (const sample of value) { + // v2 writes increasing retrieval times. Discard corrupted ordering; + // sorting it would manufacture a sequence the recorder did not save. + if (Number.isInteger(sample?.t) && sample.t > previous && sample.t <= now + && Number.isFinite(sample.wait) && sample.wait >= 0) { + series.push({ t: sample.t, wait: sample.wait }); + previous = sample.t; + } + } + if (series.length) entries.push([key, series.slice(-MAX_SAMPLES_PER_CROSSING)]); + } + return Object.fromEntries(entries); } catch { return {}; } } -function writeAll(obj) { - try { - localStorage.setItem(STORAGE_KEY, JSON.stringify(obj)); - } catch (e) { - console.warn('[waitTimeHistory] write failed', e); - } -} - -export function recordSnapshot(crossings, direction = 'northbound') { +export function recordSnapshot(crossings, direction = 'northbound', fetchedAt = null) { if (!Array.isArray(crossings) || crossings.length === 0) return; const now = Date.now(); - const cutoff = now - MAX_DAYS * 24 * 60 * 60 * 1000; - const all = readAll(); + const cutoff = now - RETENTION_MS; + const sourceTime = typeof fetchedAt === 'string' ? Date.parse(fetchedAt) : NaN; + if (!(sourceTime >= cutoff && sourceTime <= now)) return; + const all = readAllHistory(); for (const c of crossings) { const id = getStorageKey(c.port_number || c.id, direction); if (!id) continue; const wait = getWaitMinutes(c, direction); - if (wait == null) continue; + if (!Number.isFinite(wait) || wait < 0) continue; const series = all[id] || []; const last = series[series.length - 1]; - // Dedupe: skip if identical to last entry written in the last minute - if (last && now - last.t < 60 * 1000 && last.wait === wait) continue; + // Reopening, refreshes and optional-feed enrichment can replay one source + // snapshot. Neither those repeats nor an older fallback is a new sample. + if (last && sourceTime <= last.t) continue; - series.push({ t: now, wait }); - const trimmed = series.filter((s) => s.t >= cutoff).slice(-MAX_SAMPLES_PER_CROSSING); - all[id] = trimmed; + series.push({ t: sourceTime, wait }); + all[id] = series.slice(-MAX_SAMPLES_PER_CROSSING); } - writeAll(all); + writeStorage(STORAGE_KEY, JSON.stringify(all)); } export function getHistory(crossingId) { if (!crossingId) return []; - const all = readAll(); + const all = readAllHistory(); return all[crossingId] || []; } export function getHistoryForDirection(crossingId, direction = 'northbound') { const key = getStorageKey(crossingId, direction); if (!key) return []; - const all = readAll(); + const all = readAllHistory(); return all[key] || []; } @@ -69,21 +86,6 @@ export function getPreviousWait(crossingId, direction = 'northbound') { return h[h.length - 2].wait; } -export function getAllHistory(direction = null) { - const all = readAll(); - if (!direction) return all; - - const filtered = {}; - const suffix = direction === 'southbound' ? ':sb' : null; - - for (const [key, value] of Object.entries(all)) { - if (suffix && key.endsWith(suffix)) filtered[key] = value; - if (!suffix && !key.endsWith(':sb')) filtered[key] = value; - } - - return filtered; -} - export function clearHistory() { try { localStorage.removeItem(STORAGE_KEY); } catch {} } diff --git a/src/pages/Dashboard.jsx b/src/pages/Dashboard.jsx index 9863d5b8e..6785ca4ea 100644 --- a/src/pages/Dashboard.jsx +++ b/src/pages/Dashboard.jsx @@ -156,11 +156,7 @@ export default function Dashboard() { fetchedAt: data.timestamp, }); if (data.success) { - try { - recordSnapshot(data.crossings || [], 'northbound'); - } catch (e) { - console.warn('[dashboard] snapshot error', e); - } + recordSnapshot(data.crossings, 'northbound', data.timestamp); } }; return subscribeToBorderData(applyData);