Skip to content

feat(docs): inject SEO metadata into published guide bundles - #33

Open
simonkrol wants to merge 1 commit into
mainfrom
seo-investigation
Open

simonkrol wants to merge 1 commit into
mainfrom
seo-investigation

Conversation

@simonkrol

@simonkrol simonkrol commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

SEO / discoverability:

  • Added tools/build-user-guide/inject-seo-metadata.py — patches the outer bundler-shell <head> of docs/index.html and docs/site/user-guide.html (the bytes a crawler sees before any JS runs) with a real <title>, <meta name="description">, Open Graph tags, a Twitter Card type, and a canonical link, all sourced from the page's own real content. Replaces the generic "This page requires JavaScript to display" <noscript> fallback with the real <h1> and intro paragraphs. Idempotent (marker-based) and never touches the inner __bundler/template JSON span.
  • tools/build-user-guide/build-user-guide.sh: wires the injector into the build pipeline (default build mode, after check-bundle-js.py passes; and --record mode, before the manifest is pinned), so both published bundles always carry the injected metadata and the manifest hash reflects it.
  • docs/index.html, docs/site/user-guide.html: regenerated through the pipeline (not hand-edited). Only the outer shell changed — the inner page content is byte-identical to main (verified via SHA-256 of the extracted __bundler/template span).
  • docs/site/.user-guide.manifest: re-pinned to the new output hash so build-user-guide.sh --check / make guide-check see no drift from this change.
  • Added docs/social-preview.png (1200×1200) — a static PNG render of the logo mark already used on the bundler's own loading screen (#__bundler_thumbnail's inline SVG, identical in both bundles). Used as og:image, so link unfurls (Slack, LinkedIn, Facebook, and Twitter/X via its OG fallback) show a real image instead of none.
  • Added docs/robots.txt — standard allow-all robots file pointing at the sitemap.
  • Added docs/sitemap.xml — lists the two genuinely distinct published URLs (https://aws-solutions.github.io/konductor/ and https://aws-solutions.github.io/konductor/site/user-guide.html). Deliberately excludes internal hash-fragment routes (e.g. #/tasks), which aren't separately crawlable.

Social card tags, specifically: twitter:card is set to summary_large_image (the layout only — it carries no image of its own). No twitter:title, twitter:description, or twitter:image are set; Twitter/X falls back to og:title/og:description/og:image when those are absent, so duplicating the same values under both prefixes would have been redundant.

Known, deliberate deviation from "single bundled page, no network calls": that requirement (in tools/build-user-guide/BUILD_USER_GUIDE_PROMPT.md) is about the guide opening via file:// with no build step and no network fetch (besides the already-accepted Google Fonts link). og:image is a web convention that only works by pointing at a separately-fetchable HTTP(S) URL — it cannot be satisfied by an embedded data URI in practice (most unfurlers, including Twitter's, expect a real URL, not a data: URI). docs/social-preview.png is therefore a second static file living alongside the bundle, not embedded in it. This does not affect the file://-offline use case — nothing in the rendered page loads this image; it's metadata read only by external link-unfurling crawlers after the page is shared, never fetched by a browser opening the HTML file itself. It does mean the guide is no longer strictly a single file. Flagging this explicitly as an intentional, scoped exception rather than an oversight.

Deliberately excluded from this PR:

Fixes # (no linked issue)

Type of Change

Build tooling (tools/build-user-guide/) and generated GitHub Pages output (docs/).

Testing

  • Ran make guide-check before and after this change. One failure persists in both cases — check [10], `config` is listed as withdrawn from the guide, but cli/README.md no longer declares it — confirmed via git stash to be identical on clean main, i.e. pre-existing and unrelated to this change. No new failures introduced.
  • Verified the inner __bundler/template span of both regenerated bundles is byte-identical (SHA-256) to the same span on main — only the outer shell changed.
  • Verified docs/social-preview.png is a valid PNG (1200×1200, RGBA) and is staged/committed, not left untracked, so the referenced URL will resolve once published.
  • Confirmed idempotency: running inject-seo-metadata.py a second time against already-patched files produces byte-identical output.
  • New or changed code files carry the required SPDX header; formats without comment syntax (e.g. JSON, PNG) are exempt.

Not applicable here: npm test (no JS/TS source changed), cd cli && make test (no cli/ Rust/Python source changed — this touches tools/build-user-guide/, a separate script, not the cli/ crate), agent/skill benchmark harness (no agent or skill behavior changed).

Checklist

  • My code follows the style guidelines of this project (see AGENTS.md's Code Style & Conventions)
  • I have performed a self-review of my own changes
  • I have commented my code where necessary
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings — build is clean
  • I have added or updated tests that prove my change works (see Testing above for the exact commands)

Notes for Reviewers

  • Known pre-existing failure, not caused by this PR: make guide-check check [10] fails on main as well as on this branch (confirmed via git stash): `config` is listed as withdrawn from the guide, but cli/README.md no longer declares it — drop it from WITHDRAWN_COMMANDS. Out of scope here; flagging so it isn't mistaken for a regression from this change.
  • The injector sources all copy (title, description, OG tags, noscript fallback) from the page's own real <h1> and paragraph text — nothing is invented. social-preview.png is likewise a faithful render of an asset already present in the bundle (the loading-screen SVG), not a new design.
  • See the "Known, deliberate deviation" note above re: og:image and the single-file-bundle requirement — happy to discuss if a different tradeoff is preferred (e.g. dropping the image tags entirely to stay strictly single-file).
  • docs/index.html and docs/site/user-guide.html are generated artifacts; please don't review them line-by-line — review inject-seo-metadata.py and the build-user-guide.sh wiring, and treat the two HTML diffs as their output.

Example Linkedin Post:
image

Tested Pages build: https://simonkrol.github.io/konductor/

Patch real <title>, <meta name="description">, OG tags, Twitter Card
tags, and a canonical link into the outer shell of docs/index.html and
docs/site/user-guide.html — the bytes a crawler sees before any JS
runs. The bundler leaves that shell with an empty <title>Bundled
Page</title> and no description, so search results and link previews
had nothing to show.

tools/build-user-guide/inject-seo-metadata.py sources every value from
the page's own real content (title/h1/paragraphs), never fabricated
copy, and is idempotent via a marker comment so re-running it replaces
rather than duplicates. It also replaces the old generic "This page
requires JavaScript to display" <noscript> fallback with the real h1
and intro paragraphs. It never touches the inner __bundler/template
span, so the page itself is unchanged — verified byte-identical to
the inner content on main.

build-user-guide.sh now runs the injector after check-bundle-js.py in
the default build path, and again before the manifest is pinned in
--record mode, so both bundles always carry the metadata and
.user-guide.manifest's hash reflects it.

Add docs/robots.txt and docs/sitemap.xml, an allow-all robots file
pointing at a sitemap listing the two actually-crawlable published
URLs (the root page and the user guide) — not the internal
hash-fragment routes, which aren't separately crawlable.
@simonkrol
simonkrol marked this pull request as ready for review October 2, 2026 18:32

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant