Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .agents/skills/crash-triage/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@ description: >

There are now two evidence paths. When crash-report consent and a compiled endpoint are both
present, the patched Sentry Native ARM32 handler wakes the shipped `sentry-crash` process. That
process reads the stopped target, suspends the other Linux LWPs with `ptrace`, captures their
registers, then walks the crashing and captured threads' frame chains before resuming them. It
records up to 128 frames for the crashing thread plus up to 32 for each captured non-crashing
thread,
registers, modules, Linux/webOS versions and build ids. Its transport is disabled; the next
process reads the stopped target, walks the crashing thread's frame chain from a copy of its
stack, and unwinds every other Linux LWP remotely through libunwind's ptrace accessors (DWARF,
with function names), attaching and detaching per thread. It records up to 128 frames for the
crashing thread plus up to 32 for each other thread, registers, modules, Linux/webOS versions and
build ids. Its transport is disabled; the next
healthy PlxNative launch sanitises the envelope and sends it from the ordinary telemetry spool.
That Sentry event **is a
backtrace** when it contains multiple frames.
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ WERROR ?= -Werror
# The out-of-process crash walker follows ARM's APCS frame chain. Keeping r11 frame pointers in
# every C frame is therefore part of the crash-reporting ABI, not debug-only codegen; without it a
# valid envelope contains only the faulting PC. Rust's matching flag is in RUST_ENV/config.toml.
CFLAGS = --sysroot=$(SYSROOT) -O2 -fno-omit-frame-pointer -Wall -Wextra $(WERROR) -Iinclude -Isrc -Ivendor/nanosvg -D_GNU_SOURCE
CFLAGS = --sysroot=$(SYSROOT) -O2 -fno-omit-frame-pointer -funwind-tables -Wall -Wextra $(WERROR) -Iinclude -Isrc -Ivendor/nanosvg -D_GNU_SOURCE
# DEBUG=1 keeps DWARF in the binary so a crash PC symbolizes to file:line instead of just
# a function name (tools/crash-report.sh / the crash-triage skill). Same codegen, bigger
# binary — deploy it only while chasing a crash.
Expand Down
8 changes: 6 additions & 2 deletions ci/build-sentry-native.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,17 @@
set -euo pipefail

ROOT=$(cd "$(dirname "$0")/.." && pwd)
VERSION=0.13.9
VERSION=0.16.5
ARCHIVE="$ROOT/vendor/sentry-native-$VERSION.tar.gz"
SOURCE="$ROOT/vendor/sentry-native-src"
BUILD="$ROOT/vendor/sentry-native-build"
PREFIX="$ROOT/vendor/sentry-native-prefix"
PATCH="$ROOT/vendor/sentry-native/webos-arm32.patch"
# Two hunks of that patch are upstream PRs and drop out of it once they land in a release we pin:
# pointer-width stack reads (getsentry/sentry-native#2052) and the ARM32 registers + both
# frame-record shapes (#2053). The rest is webOS-only and stays.
URL="https://github.com/getsentry/sentry-native/archive/refs/tags/$VERSION.tar.gz"
SHA256=d43a41197ffaa218ceaef8cfcc7ecf584ca1a2c5bda2426b7ab4032875c67167
SHA256=8d3f63f092ab24ab7f5d30cd8f0e80dc78670a3b3be3f1237948667907cdc3a4

WEBOS_SDK=${WEBOS_SDK:-"$HOME/webos-ndk/arm-webos-linux-gnueabi_sdk-buildroot"}
CC="$WEBOS_SDK/bin/arm-webos-linux-gnueabi-gcc"
Expand Down Expand Up @@ -68,6 +71,7 @@ patch -d "$SOURCE" -p1 < "$PATCH"
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY \
-DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_FLAGS="-funwind-tables -fno-omit-frame-pointer" \
-DSENTRY_BACKEND=native \
-DSENTRY_TRANSPORT=none \
-DSENTRY_BUILD_SHARED_LIBS=OFF \
Expand Down
27 changes: 17 additions & 10 deletions docs/agent-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -670,16 +670,23 @@ which the linking section explains is load-bearing rather than tidy.
the suspend/reload pairing if you touch playback or routing.
- **Crash forensics has two layers.** With error-report consent and a compiled Sentry endpoint,
Sentry Native's patched ARM32 backend replaces the signal disposition and wakes the shipped
`sentry-crash` daemon. The dying process stays stopped while the daemon copies its `ucontext`,
enumerates the other Linux LWPs, `PTRACE_ATTACH`es them, snapshots PC/SP/FP with
`PTRACE_GETREGS`, and keeps them stopped while it walks every APCS frame chain. That lifetime is
the Linux counterpart of KSCrash's suspend → context → unwind → resume sequence; enumerating
`/proc/<pid>/task` alone only produces names and zeroed contexts. The crashed thread keeps up to
128 frames and each other thread up to 32 to reduce pressure on the 256 KiB durable-record
ceiling; the importer still rejects an oversized envelope rather than claiming a bound for an
arbitrary 256-LWP process. The JSON therefore carries ARM registers and real multi-frame stacks
for all successfully captured threads, plus modules and both Linux-kernel and webOS firmware
context.
`sentry-crash` daemon. The dying process stays stopped while the daemon copies its `ucontext`
and walks the crashed thread's APCS frame chain out of a copy of its stack; for every other
Linux LWP in `/proc/<pid>/task` it `PTRACE_ATTACH`es, unwinds **remotely through libunwind's
ptrace accessors** (DWARF, with `function` names from the ELF symbol tables), and detaches — that
is upstream sentry-native's own machinery since 0.16 (#1747), and it replaced a hand-written
suspend/`PTRACE_GETREGS`/frame-walk block this repo carried against 0.13.9 until 2026-09-02. The
crashed thread keeps up to 128 frames and each other thread up to 32 to reduce pressure on the
256 KiB durable-record ceiling; the importer still rejects an oversized envelope rather than
claiming a bound for an arbitrary 256-LWP process. The JSON therefore carries ARM registers and
real multi-frame stacks for all successfully captured threads, plus modules and both Linux-kernel
and webOS firmware context. The pin is **0.16.5** (`ci/build-sentry-native.sh`), and the patch
beside it (`vendor/sentry-native/webos-arm32.patch`) is down to what upstream does not do: a
`process_vm_readv` wrapper for glibc 2.12, ARM32 registers in the event, a frame-pointer walk that
reads BOTH ARM32 frame records — GCC leaves `fp` on the LR slot (`[fp-4]`/`[fp]`), rustc/LLVM on
the saved-fp slot (`[fp]`/`[fp+4]`), and one process here holds both — pointer-width stack reads, the 32-frame cap for non-crashed threads,
the 30 s handler budget, and two webOS-only escapes in the signal handler (no in-process libunwind,
no SDK hooks — both reproduced a recursive SIGSEGV through `getenv`).
The SDK has **no HTTP transport and writes no minidump**: it launches the
same `plxnative` binary in spool-only mode, which moves the bounded envelope into the install's
runtime root. A healthy launch rejects user/request scope, strips path prefixes and queues the
Expand Down
4 changes: 2 additions & 2 deletions rust-modules/src/telemetry/native.rs
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,7 @@ pub(crate) fn preview_event() -> Vec<u8> {
"release": concat!("plxnative@", env!("PLX_VERSION")),
"environment": super::sender::ENVIRONMENT,
"dist": "<ELF build id>",
"sdk": {"name": "plxnative", "version": "0.13.9"},
"sdk": {"name": "plxnative", "version": "0.16.5"},
"contexts": {
"os": {"type": "os", "name": "Linux", "version": "<kernel release>",
"build": "<kernel build suffix>", "kernel_version": "<kernel release>"},
Expand Down Expand Up @@ -922,7 +922,7 @@ mod tests {
"request": {"url": "must-not-pass"},
"extra": {"future_sdk_field": "must-not-pass"},
"arbitrary": "must-not-pass",
"sdk": {"name": "plxnative", "version": "0.13.9", "future": "must-not-pass"},
"sdk": {"name": "plxnative", "version": "0.16.5", "future": "must-not-pass"},
"contexts": {
"os": {"name": "Linux", "future": "must-not-pass"},
"webos": {"type": "webos", "name": "webOS TV", "release": "4.10.2",
Expand Down
Loading
Loading