Skip to content

feat(iwork): elementary support for apple iwork, starting with .pages - #732

Merged
andiwand merged 15 commits into
mainfrom
feat/iwork-pages
Aug 24, 2026
Merged

feat(iwork): elementary support for apple iwork, starting with .pages#732
andiwand merged 15 commits into
mainfrom
feat/iwork-pages

Conversation

@andiwand

@andiwand andiwand commented Aug 23, 2026

Copy link
Copy Markdown
Member

🤖 Generated with Claude Code

Stages 1 and 2 of src/odr/internal/iwork/PLAN.md: the container comes apart, and a .pages opens as a text document and renders its body instead of coming back as the zip it is made of.

What lands

  • iwork_snappy — Apple's own .iwa framing (0x00 plus a little-endian 24-bit compressed length, repeated to EOF) plus Snappy block decompression. Stock Snappy stream framing is not present, so a conan dependency would be the wrong shape as well as an extra one in the wasm/android/apple builds.
  • iwork_protobuf — the wire format only. Apple has never published the .proto schemas, so a code generator would have nothing to generate; archives are read against hand-written accessors by field number.
  • iwork_archive — an .iwa is an object graph, not a tree. The component list comes from Index/Metadata.iwa (file names carry identifier suffixes, so globbing for them does not work), and objects resolve by identifier across components.
  • Three FileTypes — iwork_pages, iwork_numbers, iwork_keynote, with rows in file_type_table.cpp and the three bindings that mirror the enum by ordinal. Only .pages detects and opens; the other two are classification-only, because which app wrote a package is read off its root archive type and neither has a fixture to pin that against.
  • Pages body text — paragraphs from the storage's paragraph run table rather than from splitting on \n, U+2028 as a line break, and the run tables' UTF-16 indices translated against the UTF-8 text.

What does not

Styles, page geometry, drawables, images, tables, Numbers, Keynote, encryption — stages 3 to 7 of the plan. src/odr/internal/iwork/AGENTS.md records what the two landed stages decided and why; PLAN.md marks them and the three places plan and code disagreed.

Testing

The container readers are covered by inline byte strings — hand-built Snappy blocks, varint edge cases, an unknown archive type that must be skipped rather than thrown on, truncated framing. The two committed .pages fixtures cover detection, the empty-body regression, the body text, and cross-component object resolution.

Full suite green; the only new reference output is output/pages/, pinned in test/data.cmake.

andiwand and others added 15 commits August 23, 2026 17:53
@andiwand @claude
feat(iwork): unpack the snappy framing of an apple .iwa
6fbdb0e 
The framing an iWork package uses is Apple's own — a four-byte header per
block, `0x00` and a little-endian 24-bit compressed length — so stock Snappy
stream decoding does not apply and only the block decoder does. That is a
varint length plus literal and copy tags, which is less code than a dependency
would be and keeps it out of the wasm, android and apple builds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GADNSpk1CY88GMpqkafN6z
@andiwand @claude
feat(iwork): read the protobuf wire format
afa2079 
An iWork archive is protobuf, but Apple has never published the `.proto`
schemas, so there is nothing for a code generator to generate and linking conan
`protobuf` would drag it into every downstream build to replace this. Only the
wire format is needed: varints, the three fixed and length-delimited forms, and
unknown fields carried along rather than dropped. A group means the parse went
wrong, so it throws.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GADNSpk1CY88GMpqkafN6z
@andiwand @claude
feat(iwork): index the objects an iwork package holds
e42219f 
An `.iwa` is not a tree but a sequence of archived objects, each a
`TSP.ArchiveInfo` naming an identifier and the type of the messages that
follow. Objects reference each other by identifier across components, so the
package reads its component list from `Index/Metadata.iwa` first — the file
names carry identifier suffixes often enough that globbing for them finds
nothing — and decompresses a component when something in it is asked for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GADNSpk1CY88GMpqkafN6z
@andiwand @claude
feat(iwork): name the three apple iwork file types
e571f7f 
`.pages`, `.numbers` and `.key` are zips, so today they are reported as `[zip]`
and open as an archive rather than a document. Naming them gives a caller the
extensions and MIME types to route one and hand a file picker, which is what
has to be decided before the file is held.

Classification only for now: which app wrote a package is read off its root
archive, and only `.pages` has a fixture to pin that against, so the rows
declare no capabilities and nothing detects or decodes one yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GADNSpk1CY88GMpqkafN6z
@andiwand @claude
feat(pages): open a .pages document and read its body text
1ee90b6 
A `.pages` package now opens as a text document rather than as the zip it is
made of, and its body comes out as paragraphs. Which app wrote the package is
read off the type of the root archive in `Index/Document.iwa` — the extension
is not consulted, since a caller may not have one.

Paragraph boundaries come from the storage's paragraph run table rather than
from splitting the text on `\n`, and `U+2028` inside a paragraph becomes a line
break. The anchor a drawable leaves in the text is dropped: styles, page
geometry, drawables, images and tables are all still to come, so this is the
text and nothing else.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GADNSpk1CY88GMpqkafN6z
@andiwand @claude
docs(iwork): record what the first two stages decided
32284fd 
`PLAN.md` was written before any of it existed; this is the module's `AGENTS.md`
alongside it — why a fixture is the citation here rather than a spec section,
why snappy and protobuf are in-tree, the `Message` lifetime the whole engine
rests on, and where the run tables sit. `PLAN.md` marks the two landed stages
and the three places the plan and the code disagreed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GADNSpk1CY88GMpqkafN6z
@andiwand @claude
test(data): pin the reference output the two pages fixtures render
98c418d 
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GADNSpk1CY88GMpqkafN6z
@andiwand
Merge branch 'main' of github.com:opendocument-app/OpenDocument.core …
3661b29 
…into feat/iwork-pages

# Conflicts:
#	CHANGELOG.md
#	test/CMakeLists.txt
@andiwand @claude
docs(readme): name pages, numbers and key in the format lists
ddcf8e8 
The branch decodes `.pages` and classifies `.numbers` and `.key`, but README
still listed `pages` as unsupported and named neither of the other two — the
one place a caller looks to decide which MIME types to advertise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfnhZeFgZh84WMmeKK5cp2
@andiwand @claude
refactor(iwork): drop what nothing reads and reuse what exists
4d1a27d 
`ComponentInfo::identifier` was filled from every component info and never
read: `component` finds by name, `load_` keys on the locator, and `object`
goes through the object map. It goes back in when something reads it.

`without_anchors` was `util::string::replace_all` written out, and
`paragraph_starts` said "the body starts at its first character" twice — once
as an early return for a missing table, once as the normalisation that already
covers it.

The three iWork alias arrays had landed between `table`'s doc comment and
`table`, so the comment read as if it described them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfnhZeFgZh84WMmeKK5cp2
@andiwand @claude
fix(iwork): bound a snappy block by what it can hold, not what it claims
ceb791f 
`reserve` was sized from the block header's 32-bit uncompressed length before
a single tag was read, so a zip carrying `Index/Document.iwa` with the ten
bytes `00 06 00 00 | FF FF FF FF 0F | 00` reserved ~4 GiB and only then threw.
Every zip a caller opens reaches this, decoding no iWork content at all.

The reservation is now capped at what the compressed bytes could expand to,
and a tag is checked against what the block has left to give before anything
is appended, so nothing is written or allocated against a length the file
merely claims.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfnhZeFgZh84WMmeKK5cp2
@andiwand @claude
test(iwork): build packages inline instead of only through fixtures
7f3fc67 
Both `.pages` fixtures are ASCII with no line break, so the UTF-16 index
translation, its three throws and the `U+2028` branch were reachable but never
run — `unit += length == 4 ? 2 : 1` could be dropped with the suite green.
Nothing reached `Package` or `read_iwa`'s error paths either, and nothing
pinned the `.numbers`/`.key` fallback both `PLAN.md` and `CHANGELOG.md`
promise.

`iwork_test_util.hpp` assembles the layers a package is made of — protobuf
fields, `TSP.ArchiveInfo`, a literal-only Snappy block, the component list —
so a storage of any shape is stated inline over a `VirtualFilesystem`. It is a
test-only assembler and must never grow into a writer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfnhZeFgZh84WMmeKK5cp2
@andiwand @claude
refactor(iwork): read bytes and utf-16 indices through the utils
f4dad47 
`iwork_snappy.cpp` and `iwork_protobuf.cpp` each carried the same
variable-width little-endian accumulation loop; `util::byte::from_little_endian`
now takes a size for a field narrower than the type it is read into.

`iwork_parser.cpp` decoded UTF-8 by hand to count UTF-16 code units, which
utfcpp — already a dependency behind `util::string` — does properly.
`util::string::utf16_offsets` is that walk, with its own tests.

Also cuts the doc comments in `iwork_archive.hpp` and the ones the last three
commits added back to a line or two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfnhZeFgZh84WMmeKK5cp2
@andiwand @claude
style(iwork): define members in their namespace, not by qualified name
79587bb 
`iwork::Component::locator`, `iwork::IworkFile::document`, `iwork::Message::field`
and the rest are now defined inside `namespace odr::internal::iwork` like every
sibling engine; only the free functions keep the qualified form the root
`AGENTS.md` asks for. That drops the `iwork::` prefix off the types they use,
and off the test helpers via a namespace alias.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfnhZeFgZh84WMmeKK5cp2
@andiwand @claude
refactor(util): take the utf-16 indices as a span
787e311 
`utf16_offsets` only reads them, so an array or a subrange is as good as the
vector the one caller happens to hold.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CfnhZeFgZh84WMmeKK5cp2
@andiwand
andiwand merged commit 0fa644c into main Aug 24, 2026
33 of 35 checks passed
@andiwand
andiwand deleted the feat/iwork-pages branch August 24, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@andiwand