Skip to content

fix(serializer): stop escaping Obsidian syntax and plain prose on save - #51

Merged
evangress merged 2 commits into
mainfrom
fix/obsidian-syntax-escaping
Nov 10, 2026
Merged

evangress merged 2 commits into
mainfrom
fix/obsidian-syntax-escaping

Conversation

@evangress

Copy link
Copy Markdown
Collaborator

Problem

Saving an edited note backslash-escaped Obsidian syntax. Obsidian renders the result as literal text, so every link, embedded image and callout in that note broke:

Written Saved by Toril (before)
[[Pneumothorax]] \[\[Pneumothorax]]
![[scan.png]] !\[\[scan.png]]
> [!tip] Pearl > \[!tip] Pearl

The same defensive escaping also rewrote ordinary prose: figure [1], ~3 cm, 5 * 3, snake_case. Notes that were only opened were never affected. This is a §3.2 round-trip bug, the same class as the front-matter corruption fixed in v1.0.0.

Fix

src/editor/escapes.ts, called only from docToMarkdown in serializer.ts, removes an escape only when re-parsing proves the document unchanged. It never re-implements CommonMark/GFM escaping rules, so it cannot get them wrong. It works in two passes:

  1. Per block: bisection over candidate escapes, memoized by block text, so between keystrokes only the edited block re-parses.
  2. Whole document: one parse as the proof. If it fails (cross-block meaning, such as a reference definition), a bounded whole-document search runs instead.

Escapes that are actually needed stay: task markers, heading/list/quote markers, raw HTML.

A note damaged by an earlier save is repaired by the next edit and save. This is pinned as a fixture, because the changelog tells users it works.

Gates

  • tests/escapes.test.ts:
    • The search: joint verification, bisection, the whole-document fallback, the parse budget, and a throwing parser counting as inequality.
    • A 200-case randomized property over the real parser: the saved file loads exactly as the fully escaped file would, and it is stable wherever the old output was.
  • tests/roundtrip.test.ts:
    • New obsidianPreserved class (22 constructs) and escapesKept class (6 constructs).
    • The prose cases move to preserved.
    • normalized entries updated: <autolink> → bare URL, superfluous escapes dropped, table padding beside [[x\|y]], and the repair fixture.
  • Teeth: forcing the verifier to accept every removal fails 8 tests.
  • pnpm typecheck passes and pnpm test passes (632 tests). No Rust changes.

Cost

About one extra full parse per serialization. In jsdom: ~12 ms at 800 words, ~47 ms at 3k, ~180 ms at 12k. It is paid on the debounced change listener. The worst case (one needed escape among many) went from ~2.2 s to ~0.2 s at 12k words once verification moved to per-block. A follow-up is noted in ROADMAP H0: take the whole-document proof off the dirty check if very long notes feel slow.

Found while gating, not fixed here

Both reproduce on main and are recorded in CLAUDE.md §0 and ROADMAP H0:

  • A bare URL ending in \ gains a backslash on every save.
  • remark moves a space out of a strikethrough boundary.

Also in this PR (docs)

  • ROADMAP:
    • A 2026-11-09 status update and recommended order.
    • Track H (hardening): repair tool, save compare-and-swap, image insertion, Obsidian attachment settings, rendering, trash/purge.
    • Track M (medical & radiology students): templates, case log, image viewing and annotation, privacy kit, self-quiz, Anki export, review queue, and more.
    • §9, value areas Toril does not offer today.
  • CLAUDE.md: gate list and §0 notes.
  • CHANGELOG: an Unreleased entry.
  • README: removed a stale "front matter not built" line.

Not yet verified

The fix has not been checked in the running app against a real vault. That is in progress.

🤖 Generated with Claude Code

evangress and others added 2 commits November 9, 2026 21:43
remark-stringify escapes every `[`, `*`, `_` and `~` defensively. For an
Obsidian vault that is destructive: saving an edited note rewrote
`[[Note]]` as `\[\[Note]]`, `![[img.png]]` as `!\[\[img.png]]` and
`> [!tip]` as `> \[!tip]`, which Obsidian renders as literal text. It also
rewrote ordinary prose (`figure [1]`, `~3 cm`, `snake_case`).

src/editor/escapes.ts removes each escape only when re-parsing proves the
document unchanged: per-block bisection (memoized), then one whole-document
parse as the proof, with a bounded whole-document fallback. It never
re-implements CommonMark's escaping rules.

Gates: tests/escapes.test.ts (search, fallback, parse budget, 200-case
property over the real parser) and new obsidianPreserved / escapesKept
classes plus a repair fixture in roundtrip.test.ts.

Also adds Track H (hardening), Track M (medical & radiology students) and
§9 (unplaced value areas) to ROADMAP.md, and fixes a stale README line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cape fix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@evangress

Copy link
Copy Markdown
Collaborator Author

Real-vault check (headless)

I ran every note in a 179-note personal Obsidian vault through the same split → round-trip → rejoin that main.ts performs, on a copy. Nothing was written to the vault.

main this PR
Wikilinks / embeds / callout markers surviving a save 296 of 352 352 of 352
Notes whose links break on save 23 0
Notes changed by a save 156 152
Notes that load differently or are unstable — 0

Across the whole vault, the only differences between main's output and this PR's are 1,175 removed backslashes and 81 removed <…> autolink pairs. Line counts are identical in every file.

Pre-existing issues found by the same run are recorded in ROADMAP H0b and are not fixed here:

  • A note containing ®, © or ™ fails to load in jsdom. It comes from the emoji plugin; on-device confirmation is pending.
  • An empty bullet - is saved as - <br /> (56 notes).
  • Trailing whitespace is stripped, tab-indented nested lists become two spaces, and emphasis is split around inline code.

Interactive verification in the dev app is next.

@evangress
evangress merged commit 9ccb4f8 into main Nov 10, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant