Skip to content

Make every chart say where its figures came from - #840

Closed
guidovizoso wants to merge 7 commits into
mainfrom
guido/chart-sources
Closed

guidovizoso wants to merge 7 commits into
mainfrom
guido/chart-sources

Conversation

@guidovizoso

@guidovizoso guidovizoso commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

What this changes

A chart is the most confident-looking answer a Bot can give, and a figure it recalled from training looks exactly like one it read this morning. Now every chart says which it is.

  • A sources line under every chart. Bar, pie, line, area and progress charts take three new fields: basis (read, given or own-knowledge), sources (title and optional address) and asOf. The card draws them as one line under the plot, for example "Sources: World Bank World Development Indicators API — GDP (current US$), 2025 · As of 2025". Only http(s) addresses become links.
  • Unchecked figures say so. With basis: own-knowledge the line starts, in amber, "From the Bot's own knowledge. Not checked against a source."
  • The model has to choose. basis is required in the schema the model is given, so it cannot draw a chart without saying where the numbers came from. The renderer treats all three as optional, so charts already in a conversation draw as before, with no line.
  • Bots look figures up first. PROVENANCE_GUIDANCE gains a paragraph: before charting figures that change over time (prices, statistics, rankings, results), look them up with a search tool if you hold one and list what you read as the sources; otherwise say so in basis.
  • Odd arguments don't break the card. The arguments are drawn as the model wrote them, so a source without a title, a repeated source, a non-string date or a non-web address is skipped or shown as text rather than throwing. A repeated source is named once.
  • Headless turns get the same schema. server/src/components/headless.ts mirrors the app's chart schema (a routine's chart is drawn by the app later), and headless-components.test.ts keeps the two in step.

Where it runs

  • New state that outlives a request? None. The fields travel in the chart's tool call arguments, which are already stored in the conversation like any other tool call.
  • What happens on the second replica? Nothing different. The schema and the prompt are the same on every process, and the line is drawn in the browser from the stored arguments.
  • Anything serialised? No.
  • Anything fanned out to a browser? No new fan-out. A chart reaches other viewers the way it already did, as part of the conversation.
  • New listener, port, or schedule? No.

Boundary and audit

  • Every acting call still goes through the gateway. Charts are drawn, not acted with; no gateway path changes.
  • New refusals and new failures each write a row. There are none: a chart with malformed provenance is drawn without the bad parts, not refused.
  • Nothing new is trusted from the client that the server can resolve itself. The new fields come from the model and are shown, not acted on. Worth saying plainly: the server does not check that a chart marked read matches the tool calls in the turn. The line is the Bot's own account, which is why own-knowledge is required to be said out loud and drawn as unchecked.

Changelog

  • "Every chart says where its figures came from" under Unreleased.

How it was checked

  • New tests: app/tests/chart-sources.test.tsx (sources named and linked for a chart drawn from what the Bot read, the unchecked note for its own knowledge and kept apart from the sources, wrongly shaped sources skipped, links only for web addresses, no line on a chart from before, and basis required of the model). server/tests/headless-components.test.ts and its integration test cover the mirrored schema.
  • In a running app on a combined branch of this and five sibling PRs (local stack, 2026-10-26): asked for a chart of G7 GDP. The Bot searched first and the card rendered "Sources: World Bank World Development Indicators API — GDP (current US$), 2025 · As of 2025".
  • The full server suite on the combined branch: 5,821 pass. The only failures were the assertion-shape guard, since fixed here (the allow-list follows the moved headless-components lines), and one Mastra test that needs a package not installed locally. The app suite: 1,675 pass.

Slack and Teams

A chart drawn natively in Slack or Teams has no caption, so the reply now carries the same line after each chart: "Sources: World Bank · As of 2025", or the own-knowledge warning. Covered in server/tests/delivery-routes.test.ts.

A chart now carries a basis (read, given, own knowledge), the sources it was
read from and an as-of date, and draws them under the plot. Figures the Bot
recalled without looking up are marked as unchecked. The model has to give a
basis; older conversations without one still render, with no footer.
Separate the unchecked note from the sources list, name a repeated source once,
and skip sources or dates that are not the shape the schema asks for instead of
breaking the card.
@guidovizoso

Copy link
Copy Markdown
Collaborator Author

Landed in #847.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant