Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: CI

# Editable install against a freshly built cdylib at the pinned engine rev,
# then the full corpus-parity suite. Written, not yet run (no GitHub repo yet).
# Editable install against the Ruby tier cdylib at FONTISAN_REV,
# then the full corpus-parity suite.

on:
push:
Expand All @@ -21,24 +21,24 @@ jobs:
python: ['3.9', '3.12']
steps:
- uses: actions/checkout@v4
- name: Read the engine pin
- name: Read the Ruby source pin
id: pin
shell: bash
run: echo "rev=$(cat ENGINE_REV)" >> "$GITHUB_OUTPUT"
run: echo "rev=$(cat FONTISAN_REV)" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@v4
with:
repository: fontist/fontisan-rs
repository: fontist/fontisan
ref: ${{ steps.pin.outputs.rev }}
path: fontisan-rs
path: fontisan-source
- uses: dtolnay/rust-toolchain@stable
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Build the C-ABI cdylib
run: cargo build -p fontisan-ffi --release
working-directory: fontisan-rs
- name: Stage the cdylib and stamp the engine rev
run: python3 scripts/stage_cdylib.py fontisan-rs/target/release/libfontisan${{ matrix.os == 'macos-15' && '.dylib' || '.so' }}
run: cargo build --release --manifest-path ext/fontisan_cdylib/Cargo.toml
working-directory: fontisan-source
- name: Stage and verify the cdylib
run: python3 scripts/stage_cdylib.py --fontisan-source fontisan-source fontisan-source/ext/fontisan_cdylib/target/release/libfontisan_cdylib${{ matrix.os == 'macos-15' && '.dylib' || '.so' }}
- name: Fetch the shared corpus
run: python3 scripts/corpus_fetch.py
- name: Install
Expand Down
38 changes: 24 additions & 14 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ on:
- 'pyproject.toml'
- 'src/**'
- 'ENGINE_REV'
- 'FONTISAN_REV'
- 'CORPUS_REV'
- 'scripts/**'
- '.github/workflows/wheels.yml'
Expand All @@ -34,38 +35,47 @@ jobs:
fail-fast: false
matrix:
include:
- { name: linux-glibc-x86_64, os: ubuntu-22.04, lib: libfontisan.so, platform_tag: manylinux_2_35_x86_64 }
- { name: linux-musl-x86_64, os: ubuntu-22.04, lib: libfontisan.so, platform_tag: musllinux_1_2_x86_64, container: 'alpine:3.20' }
- { name: macos-arm64, os: macos-15, lib: libfontisan.dylib, platform_tag: macosx_11_0_arm64 }
- { name: macos-x64, os: macos-15-intel, lib: libfontisan.dylib, platform_tag: macosx_11_0_x86_64 }
- { name: windows-x64, os: windows-2022, lib: fontisan.dll, platform_tag: win_amd64 }
- { name: linux-glibc-x86_64, os: ubuntu-22.04, lib: libfontisan_cdylib.so, platform_tag: manylinux_2_35_x86_64 }
- { name: linux-musl-x86_64, os: ubuntu-22.04, lib: libfontisan_cdylib.so, platform_tag: musllinux_1_2_x86_64, container: 'alpine:3.20' }
- { name: macos-arm64, os: macos-15, lib: libfontisan_cdylib.dylib, platform_tag: macosx_11_0_arm64 }
- { name: macos-x64, os: macos-15-intel, lib: libfontisan_cdylib.dylib, platform_tag: macosx_11_0_x86_64 }
- { name: windows-x64, os: windows-2022, lib: fontisan_cdylib.dll, platform_tag: win_amd64 }
container: ${{ matrix.container }}
env:
# Rust's musl target defaults to static CRT linkage, incompatible with cdylib.
RUSTFLAGS: ${{ matrix.container && '-C target-feature=-crt-static' || '' }}
steps:
- name: Alpine prerequisites (musl)
if: matrix.container
run: apk add --no-cache bash git curl build-base python3 py3-pip py3-virtualenv rust cargo nodejs
run: apk add --no-cache bash git curl build-base python3 py3-pip py3-virtualenv nodejs
- name: Install stable Rust for musl
if: matrix.container
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs -o /tmp/rustup-init.sh
sh /tmp/rustup-init.sh -y --profile minimal --default-toolchain stable
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- uses: actions/checkout@v4
- name: Read the engine pin
- name: Read the Ruby source pin
id: pin
shell: bash
run: echo "rev=$(cat ENGINE_REV)" >> "$GITHUB_OUTPUT"
run: echo "rev=$(cat FONTISAN_REV)" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@v4
with:
repository: fontist/fontisan-rs
repository: fontist/fontisan
ref: ${{ steps.pin.outputs.rev }}
path: fontisan-rs
path: fontisan-source
- uses: dtolnay/rust-toolchain@stable
if: ${{ !matrix.container }}
- uses: actions/setup-python@v5
if: ${{ !matrix.container }}
with:
python-version: '3.12'
- name: Build the cdylib
run: cargo build -p fontisan-ffi --release
working-directory: fontisan-rs
- name: Stage the vendored engine (stamps the engine rev)
run: cargo build --release --manifest-path ext/fontisan_cdylib/Cargo.toml
working-directory: fontisan-source
- name: Stage and verify the vendored engine
shell: bash
run: python3 scripts/stage_cdylib.py "fontisan-rs/target/release/${{ matrix.lib }}"
run: python3 scripts/stage_cdylib.py --fontisan-source fontisan-source "fontisan-source/ext/fontisan_cdylib/target/release/${{ matrix.lib }}"
- name: Build and retag the wheel
shell: bash
run: |
Expand Down
2 changes: 1 addition & 1 deletion ENGINE_REV
Original file line number Diff line number Diff line change
@@ -1 +1 @@
614dca354822aa5a20df50bbfb15e5e2f64f5c8a
f45057a12e15ba9f2c02526cd8ca0603c80edaf9
1 change: 1 addition & 0 deletions FONTISAN_REV
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2f4966d791354432d3780600188d026bfb61e864
52 changes: 39 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

The Python face of the [fontisan](https://github.com/fontist/fontisan) font
engine (fontist/fontisan#166). A setuptools package that rides the existing
fontisan C-ABI cdylib (`fontisan-rs`, the same one the Ruby `:ffi` tier uses)
through `ctypes`, shipped as per-platform package data
(`fontisan/lib/libfontisan.{so,dylib}` / `fontisan.dll`). No second engine, no
new FFI surface. Modeled on `parsanol/parsanol-python`.
Ruby `:ffi` tier’s C-ABI cdylib (`fontist/fontisan`
`ext/fontisan_cdylib`) through `ctypes`, shipped as per-platform package data
(`fontisan/lib/libfontisan_cdylib.{so,dylib}` / `fontisan_cdylib.dll`).
Modeled on `parsanol/parsanol-python`.

> **Provisional.** The package name `fontisan-python` and the Python floor
> (`requires-python >= 3.9`) are placeholders. The PyPI name (`fontisan` vs
Expand Down Expand Up @@ -38,21 +38,45 @@ with `status: "unsupported"`. Errors carry `kind` (`ErrorKind`, from

## Engine pin

`ENGINE_REV` holds the fontisan-rs commit the cdylib is built from. The pinned
engine exports no rev symbol, so `scripts/stage_cdylib.py` stamps
`fontisan/lib/ENGINE_REV` next to the vendored library; the loader refuses a
cdylib whose stamp (or `fontisan_cdylib_engine_rev`, where a newer engine
exports it) differs from the pin, or whose ABI version is not 1.
`FONTISAN_REV` pins the `fontist/fontisan` source commit containing
`ext/fontisan_cdylib`. Its `Cargo.toml` pins the `fontist/fontisan-rs` engine
dependency. `ENGINE_REV` and `src/fontisan/_engine.py` are generated from that
dependency using `scripts/stage_cdylib.py --update-engine-rev`; do not edit
them by hand.

Staging requires a checkout at `FONTISAN_REV` with its committed Cargo.toml,
checks that both generated engine pins match the dependency, and loads the
built binary to check its `fontisan_cdylib_engine_rev` export before copying
it. The runtime loader also requires that export to match `ENGINE_REV` and
requires ABI version 1. There is no revision stamp file.

## Develop

Building requires Rust and Cargo. The binding tests also require `rustc` on
`PATH` to compile small cdylib fixtures for revision-validation checks.

```sh
cargo build -p fontisan-ffi --release # in a fontisan-rs checkout at ENGINE_REV
python scripts/stage_cdylib.py <fontisan-rs>/target/release/libfontisan.dylib
git clone https://github.com/fontist/fontisan /tmp/fontisan-source
git -C /tmp/fontisan-source checkout "$(cat FONTISAN_REV)"
cargo build --release --manifest-path /tmp/fontisan-source/ext/fontisan_cdylib/Cargo.toml
# macOS; use libfontisan_cdylib.so on Linux or fontisan_cdylib.dll on Windows
python scripts/stage_cdylib.py --fontisan-source /tmp/fontisan-source \
/tmp/fontisan-source/ext/fontisan_cdylib/target/release/libfontisan_cdylib.dylib
python scripts/corpus_fetch.py # shared fontisan corpus -> .corpus/fontisan
pip install -e . pytest && pytest -q -s
```

To change the source pin, update `FONTISAN_REV`, check out that commit, then
regenerate the engine pins before building and staging:

```sh
python scripts/stage_cdylib.py --fontisan-source /tmp/fontisan-source --update-engine-rev
```

The generator and staging script support Python 3.9 without a TOML package.
They expect the pinned crate’s inline Git dependency declaration and fail if
upstream changes that layout.

`FONTISAN_CDYLIB` points at a cdylib elsewhere; `FONTISAN_CORPUS_DIR` at an
existing corpus root (the directory that contains `corpus/`).

Expand All @@ -70,5 +94,7 @@ not vendored; only the ci tier is used.
`.github/workflows/wheels.yml` builds one wheel per platform (linux glibc and
musl x86_64, macOS arm64 and x64, windows x64), retags each with its real
platform tag, and smokes the installed wheel with the parity suite.
`ci.yml` runs the suite against a freshly built cdylib. Neither has been run
yet; only the local macOS arm64 build is verified.
`ci.yml` runs the suite against a freshly built cdylib. Both workflows check
out `fontist/fontisan` at `FONTISAN_REV`, build `ext/fontisan_cdylib`, and
verify the binary and Cargo dependency pin during staging. The wheels
workflow also runs when `FONTISAN_REV` changes.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Repository = "https://github.com/fontist/fontisan-python"
where = ["src"]

[tool.setuptools.package-data]
fontisan = ["lib/*.so", "lib/*.dylib", "lib/*.dll", "lib/ENGINE_REV"]
fontisan = ["lib/libfontisan_cdylib.so", "lib/libfontisan_cdylib.dylib", "lib/fontisan_cdylib.dll"]

[tool.pytest.ini_options]
testpaths = ["tests"]
20 changes: 17 additions & 3 deletions scripts/corpus_fetch.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,21 @@
import json
import os
import shutil
import stat
import subprocess
import sys
import urllib.request
from pathlib import Path


def _remove_readonly(function, path, exc_info):
"""Windows cannot unlink Git's read-only pack files until made writable."""
if not isinstance(exc_info[1], PermissionError):
raise exc_info[1]
os.chmod(path, os.stat(path).st_mode | stat.S_IWRITE)
function(path)


root = Path(__file__).resolve().parents[1]
dest = Path(sys.argv[1]) if len(sys.argv) > 1 else root / ".corpus" / "fontisan"
rev = (root / "CORPUS_REV").read_text().strip()
Expand All @@ -31,10 +41,14 @@
shutil.copytree(Path(src) / "corpus", corpus)
else:
tmp = dest / "_clone"
subprocess.check_call(["git", "clone", "--filter=blob:none", "--no-checkout", "https://github.com/fontist/fontisan", str(tmp)])
subprocess.check_call(["git", "-C", str(tmp), "checkout", rev, "--", "corpus"])
# Fetch all objects at the pin before checkout; no lazy blob hydration.
tmp.mkdir()
subprocess.check_call(["git", "-C", str(tmp), "init"])
subprocess.check_call(["git", "-C", str(tmp), "remote", "add", "origin", "https://github.com/fontist/fontisan"])
subprocess.check_call(["git", "-C", str(tmp), "fetch", "--depth=1", "origin", rev])
subprocess.check_call(["git", "-C", str(tmp), "checkout", "FETCH_HEAD", "--", "corpus"])
shutil.move(str(tmp / "corpus"), str(corpus))
shutil.rmtree(tmp)
shutil.rmtree(tmp, onerror=_remove_readonly)

manifest = json.loads((corpus / "manifest.json").read_text())
n = 0
Expand Down
128 changes: 109 additions & 19 deletions scripts/stage_cdylib.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,114 @@
#!/usr/bin/env python3
"""Vendor a built fontisan cdylib into the package and stamp the engine rev.

usage: stage_cdylib.py <path-to-cdylib> [<engine-rev>]

The rev defaults to the repo-root ENGINE_REV pin; the stamp (lib/ENGINE_REV)
is what the loader checks, because the pinned engine exports no rev symbol.
"""
"""Derive the engine pin from the pinned Ruby crate and verify a built cdylib."""
import argparse
import ast
import ctypes
import re
import shutil
import subprocess
import sys
from pathlib import Path

root = Path(__file__).resolve().parents[1]
src = Path(sys.argv[1])
rev = sys.argv[2] if len(sys.argv) > 2 else (root / "ENGINE_REV").read_text().strip()
if rev != (root / "ENGINE_REV").read_text().strip():
sys.exit(f"refusing to stage rev {rev}: repo pins {(root / 'ENGINE_REV').read_text().strip()}")
dest = root / "src" / "fontisan" / "lib"
dest.mkdir(parents=True, exist_ok=True)
for old in dest.iterdir():
old.unlink()
shutil.copy2(src, dest / src.name)
(dest / "ENGINE_REV").write_text(rev + "\n")
print(f"staged {src.name} @ {rev} -> {dest}")
ROOT = Path(__file__).resolve().parents[1]
LIBRARIES = ("libfontisan_cdylib.so", "libfontisan_cdylib.dylib", "fontisan_cdylib.dll")
CARGO_PATH = "ext/fontisan_cdylib/Cargo.toml"


def source_engine_rev(source):
"""Read the inline dependency used by the pinned standalone Rust crate.

Keep this deliberately narrow so Python 3.9 needs no TOML dependency. If
upstream changes the dependency layout, fail instead of guessing a pin.
"""
source = source.resolve()
expected = (ROOT / "FONTISAN_REV").read_text().strip()
actual = subprocess.check_output(
["git", "-C", str(source), "rev-parse", "HEAD"], text=True
).strip()
if actual != expected:
raise ValueError(f"source checkout is {actual}; FONTISAN_REV pins {expected}")
manifest = (source / CARGO_PATH).read_text()
committed = subprocess.check_output(
["git", "-C", str(source), "show", f"HEAD:{CARGO_PATH}"], text=True
)
if manifest != committed:
raise ValueError("source Cargo.toml differs from the pinned commit")
section = re.search(r"(?ms)^\[dependencies\]\s*\n(.*?)(?=^\[|\Z)", manifest)
dependency = re.search(r"(?ms)^fontisan\s*=\s*\{([^}]+)\}", section[1]) if section else None
if dependency is None:
raise ValueError("expected an inline fontisan dependency in Cargo.toml")
git = re.search(r'\bgit\s*=\s*"([^"]+)"', dependency[1])
rev = re.search(r'\brev\s*=\s*"([0-9a-f]{40})"', dependency[1])
if git is None or git[1] != "https://github.com/fontist/fontisan-rs" or rev is None:
raise ValueError("expected fontist/fontisan-rs with an exact 40-character rev")
return rev[1]


def check_engine_pin(rev):
pinned = (ROOT / "ENGINE_REV").read_text().strip()
module = ast.parse((ROOT / "src/fontisan/_engine.py").read_text())
generated = [
ast.literal_eval(node.value)
for node in module.body
if isinstance(node, ast.Assign)
and any(isinstance(target, ast.Name) and target.id == "ENGINE_REV" for target in node.targets)
]
if pinned != rev or generated != [rev]:
raise ValueError("engine pins differ from Cargo.toml; run with --update-engine-rev")


def update_engine_pin(rev):
(ROOT / "ENGINE_REV").write_text(rev + "\n")
(ROOT / "src/fontisan/_engine.py").write_text(
'\"\"\"Generated from the pinned fontisan cdylib Cargo.toml.\n\n'
'Regenerate with scripts/stage_cdylib.py --fontisan-source <checkout> --update-engine-rev.\n\"\"\"\n\n'
f'ENGINE_REV = "{rev}"\n'
)


def stage(binary, rev):
binary = binary.resolve()
if binary.name not in LIBRARIES:
raise ValueError(f"unexpected library name: {binary.name}")
library = ctypes.CDLL(str(binary))
exported_rev = library.fontisan_cdylib_engine_rev
exported_rev.argtypes = []
exported_rev.restype = ctypes.c_char_p
actual = exported_rev()
if actual != rev.encode("ascii"):
raise ValueError(f"binary exports engine rev {actual!r}; Cargo.toml pins {rev}")
dest = ROOT / "src/fontisan/lib"
dest.mkdir(parents=True, exist_ok=True)
# Validate everything before replacing any previously staged library.
target = dest / binary.name
if binary != target.resolve():
shutil.copy2(binary, target)
for name in (*LIBRARIES, "libfontisan.so", "libfontisan.dylib", "fontisan.dll", "ENGINE_REV"):
old = dest / name
if old != target:
old.unlink(missing_ok=True)
print(f"staged {binary.name} @ {rev} -> {dest}")


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("binary", nargs="?", type=Path, help="built cdylib to verify and stage")
parser.add_argument("--fontisan-source", required=True, type=Path, help="checkout at FONTISAN_REV")
parser.add_argument("--update-engine-rev", action="store_true", help="regenerate ENGINE_REV and _engine.py from Cargo.toml")
args = parser.parse_args()
if args.binary is None and not args.update_engine_rev:
parser.error("binary is required unless --update-engine-rev is used")
try:
rev = source_engine_rev(args.fontisan_source)
if args.update_engine_rev:
update_engine_pin(rev)
print(f"generated engine pins @ {rev}")
check_engine_pin(rev)
if args.binary is not None:
stage(args.binary, rev)
except (OSError, ValueError, AttributeError, subprocess.CalledProcessError) as exc:
sys.exit(f"refusing to stage: {exc}")


if __name__ == "__main__":
main()
8 changes: 5 additions & 3 deletions src/fontisan/_engine.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
"""The fontisan-rs rev this package is built against (kept in step with the
repo-root ENGINE_REV file by tests/test_engine_rev.py)."""
"""Generated from the pinned fontisan cdylib Cargo.toml.

ENGINE_REV = "614dca354822aa5a20df50bbfb15e5e2f64f5c8a"
Regenerate with scripts/stage_cdylib.py --fontisan-source <checkout> --update-engine-rev.
"""

ENGINE_REV = "f45057a12e15ba9f2c02526cd8ca0603c80edaf9"
Loading
Loading