diff --git a/develop-docs/README.md b/develop-docs/README.md new file mode 100644 index 0000000000..946a62999b --- /dev/null +++ b/develop-docs/README.md @@ -0,0 +1,104 @@ +# Develop Docs + +This folder holds internal developer documentation for the Sentry Java/Android SDK: +architecture notes, feature deep-dives, design decisions, and cross-module concepts +that don't belong in the public [Sentry docs](https://docs.sentry.io) or in inline +code comments. + +If you are documenting **how** or **why** something works for the people who maintain +this SDK, it goes here. If you are documenting **how to use** the SDK for end users, +it belongs in the public docs instead. + +## Rules + +These rules keep the docs consistent, easy to navigate, and easy to grep. + +### Directory structure + +- Use a **flat directory structure**. All documents live directly in `develop-docs/`. + Do not create nested category folders. +- Group and namespace documents with **filename prefixes** instead of folders. Common + prefixes: + - `general-` — cross-cutting topics (e.g. `general-development.md`, `general-architecture.md`) + - `feature-` — a specific SDK feature (e.g. `feature-errors.md`, `feature-profiling.md`) + - `integration-` — a specific integration or module (e.g. `integration-opentelemetry.md`, `integration-spring.md`) + - `platform-` — platform-specific concerns (e.g. `platform-android.md`, `platform-jvm.md`) + - `process-` — team processes and workflows (e.g. `process-release.md`) + + Add a new prefix only when an existing one clearly does not fit, and keep the list + above up to date. + +### File naming + +- File names are **lowercase**. +- Use **dashes** (`-`) as separators, never underscores or spaces. For example, use + `feature-profiling.md`, not `feature_profiling.md` or `Feature Profiling.md`. +- Use the `.md` extension for all documents. +- Choose short, descriptive names. The prefix conveys the category, so the rest of the + name only needs to describe the topic (`feature-session-replay.md`, not + `feature-session-replay-how-it-works.md`). + +### Images and other assets + +- When a document embeds images (or other binary assets), store them in a **folder with + the same base name** as the document. For example, `feature-profiling.md` keeps its + assets in `feature-profiling/`: + + ```text + develop-docs/ + feature-profiling.md + feature-profiling/ + pipeline.png + overview.svg + ``` + +- Reference assets with **relative paths**: `![Profiling pipeline](feature-profiling/pipeline.png)`. +- Asset file names follow the same rules as documents: lowercase, dashes, descriptive. +- Prefer **vector formats** (SVG) for diagrams and screenshots where practical +- Prefer **Mermaid** over a static image whenever a diagram can be expressed as one + (see below) — it lives in the document, is versioned as text, and is easy to update. + +### Writing style + +- Write in the **present tense** and the **active voice**. Describe how the system + behaves now ("The transport retries failed envelopes"), not how it will or did behave. +- Keep one **top-level `# ` heading** per document (the title), and nest sections with + `##`, `###`, etc. Do not skip heading levels. +- Keep documents focused on a **single topic**. Split large topics into multiple + prefixed documents and link between them rather than growing one giant file. +- Use fenced **code blocks with a language identifier** (```kotlin `, + ` ```bash `) so syntax highlighting works. +- Prefer Kotlin snippets over Java. +- When referencing code, link to the file with a **relative path** (e.g. + `../sentry/src/main/java/io/sentry/Sentry.java`) rather than pasting large excerpts + that fall out of date. +- Avoid pinning content to a specific SDK version or date unless it is genuinely + version-specific; keep docs evergreen. +- Cross-link related documents with relative links (e.g. `[profiling](feature-profiling.md)`). + +### Diagrams with Mermaid + +- Prefer [Mermaid](https://mermaid.js.org/) for diagrams. It renders directly on GitHub + and lives in the document as text, so it versions and reviews like code. +- Embed a Mermaid diagram in a fenced block tagged `mermaid`: + + ````markdown + ```mermaid + flowchart LR + Event[SentryEvent] --> Processor[EventProcessors] + Processor --> Transport + Transport --> Sentry[(Sentry)] + ``` + ```` + +- For complex diagrams, include a link to the [Mermaid Live Editor](https://mermaid.live/) + so reviewers can iterate quickly. +- Fall back to a static image (stored per the asset rules above) only when a diagram + cannot reasonably be expressed in Mermaid. + +## Adding a new document + +1. Pick the right prefix (or introduce a new one and document it above). +2. Create `develop-docs/-.md` with a single `# ` title. +3. If it embeds assets, create the matching `develop-docs/-/` folder. +4. Link to it from related documents so it is discoverable. diff --git a/develop-docs/feature-perfetto-profiling.md b/develop-docs/feature-perfetto-profiling.md new file mode 100644 index 0000000000..01d756b7e4 --- /dev/null +++ b/develop-docs/feature-perfetto-profiling.md @@ -0,0 +1,223 @@ +# Perfetto profiling on Android + +This document describes how continuous profiling works on Android when the SDK +captures traces through the OS-level [`android.os.ProfilingManager`](https://developer.android.com/reference/android/os/ProfilingManager) +API (available on API 35+), and how a captured **profile chunk** flows all the way +from the device to a downloadable profile in Sentry. + +## What Perfetto is + +[Perfetto](https://perfetto.dev/) is Android's system-wide tracing stack. Its +[stack-sampling profiler](https://perfetto.dev/docs/data-sources/native-heap-profiler) +records periodic samples of native and Java call stacks and writes them to a binary +`.pftrace` trace file (a serialized [Perfetto protobuf](https://perfetto.dev/docs/reference/trace-packet-proto)). +Starting with Android 15, apps can request such traces at +runtime via `ProfilingManager` without root or `adb`, which is what makes on-device +continuous profiling possible. + +Useful Perfetto references: + +- Perfetto docs: https://perfetto.dev/docs/ +- Trace format (`TracePacket` proto): https://perfetto.dev/docs/reference/trace-packet-proto +- Perfetto UI (to open a downloaded `.pftrace`): https://ui.perfetto.dev/ + +## Pipeline overview + +A profile chunk is captured on the device, embedded into a Sentry envelope, expanded and +routed by Relay, and finally symbolicated and stored by the monolith so it can be served +as a flamegraph and downloaded as a raw Perfetto trace. + +```mermaid +flowchart TD + subgraph device["Android device — sentry-java"] + PM[android.os.ProfilingManager] + PP[PerfettoProfiler] + PCP[PerfettoContinuousProfiler] + PC[ProfileChunk] + ENV["Envelope item
[JSON metadata][raw .pftrace]
header: meta_length"] + PM --> PP --> PCP --> PC --> ENV + end + + subgraph relay["Relay (processing mode)"] + SPLIT[Split payload at meta_length] + CONV[Convert Perfetto → Sample v2] + OS1[Upload raw .pftrace to object store] + KAFKA[["Kafka topic: profiles
ProfileChunkKafkaMessage
(Sample v2 + attachment stored_id)"]] + SPLIT --> CONV --> KAFKA + SPLIT --> OS1 + end + + subgraph monolith["Monolith — getsentry/sentry"] + TASK[process_profile_task] + SYM[Symbolicate / deobfuscate] + VR[vroomrs: parse + normalize] + OS2[(Object store)] + SNUBA[(Snuba: function metrics)] + DB[(ProfileChunkAttachment row)] + TASK --> SYM --> VR + VR --> OS2 + VR --> SNUBA + TASK --> DB + end + + ENV -->|envelope| relay + KAFKA --> TASK + OS1 -.stored_id.-> DB + VROOM[getsentry/vroom
serve + merge flamegraphs] + OS2 --> VROOM + SNUBA --> VROOM +``` + +## SDK (getsentry/sentry-java) + +On API 35+, [`AndroidOptionsInitializer`](../sentry-android-core/src/main/java/io/sentry/android/core/AndroidOptionsInitializer.java) +wires up `PerfettoContinuousProfiler` automatically. On older devices the SDK falls back +to the legacy `Debug`-based [`AndroidContinuousProfiler`](../sentry-android-core/src/main/java/io/sentry/android/core/AndroidContinuousProfiler.java), +gated by the `enableLegacyProfiling` option (manifest key +`io.sentry.profiling.enable-legacy-profiling`, defaults to `true`). Only **continuous +profiling** is supported on the Perfetto path — transaction-based and app-start profiling +are not. + +### Capturing chunks + +Continuous profiling emits a stream of independent [`ProfileChunk`](../sentry/src/main/java/io/sentry/ProfileChunk.java)s +rather than one profile per transaction. `PerfettoContinuousProfiler` drives a chained +loop: each chunk runs for `MAX_CHUNK_DURATION_MILLIS` (60s) via `PerfettoProfiler`, which +calls `ProfilingManager.requestProfiling(PROFILING_TYPE_STACK_SAMPLING, …)` at +`PROFILING_FREQUENCY_HZ` (101 Hz). When a chunk's trace file is ready, a new chunk starts, +so profiling runs continuously. + +A chunk keeps a stable `profilerId` across the session and a per-chunk `chunkId`. When the +OS produces the trace file, the profiler builds a `ProfileChunk` tagged with the Perfetto +content type: + +```kotlin +ProfileChunk.Builder(profilerId, chunkId, measurements, traceFile, timestamp, ProfileChunk.PLATFORM_ANDROID) + .setContentType(ProfileChunk.CONTENT_TYPE_PERFETTO) // "application/x-perfetto-trace" + .build() +``` + +The chunk is captured via `scopes.captureProfileChunk(...)` and sent as its own envelope +with item type [`SentryItemType.ProfileChunk`](../sentry/src/main/java/io/sentry/SentryItemType.java) +(wire name `profile_chunk`). + +### Envelope format and the `meta_length` header + +A legacy chunk base64-encodes its trace into the `ProfileChunk` JSON. A Perfetto chunk is +much larger, so [`SentryClient`](../sentry/src/main/java/io/sentry/SentryClient.java) instead +routes it through the new `SentryEnvelopeItem.fromPerfettoProfileChunk(...)` factory, which +avoids base64 by sending the raw binary alongside the JSON. + +The trick is a single envelope **item** whose payload concatenates the JSON metadata and +the raw `.pftrace` bytes with **no delimiter**: + +```text +[ProfileChunk JSON bytes][raw .pftrace binary bytes] +``` + +A new `meta_length` property on the [envelope item header](../sentry/src/main/java/io/sentry/SentryEnvelopeItemHeader.java) +tells the server where the JSON ends and the binary begins. The standard envelope item +structure (header line + newline + payload) is unchanged; `meta_length` simply subdivides +the payload: + +```text +{"type":"profile_chunk","content_type":"application/x-perfetto-trace","filename":"…","length":,"meta_length":} + +``` + +- `length` — total payload size (JSON + binary), as for any envelope item. +- `meta_length` — byte length of the JSON prefix. It is only known after the payload is + serialized, so the header computes it lazily (via a `Callable`) and omits the + field entirely for non-Perfetto items, keeping the change backward compatible. + +## Relay (getsentry/relay) + +Relay processing is gated behind the `organizations:continuous-profiling-perfetto` feature +flag; without it the chunk is dropped. In processing mode Relay: + +1. **Splits** the compound item payload at `meta_length` into `(metadata JSON, raw profile)` + and reads `content_type: "perfetto"` from the metadata. +2. **Converts** the binary Perfetto trace into the existing **Sample v2** profile JSON + format (`relay_profiling::expand_perfetto(...)`, backed by a checked-in subset of the + Perfetto protobuf definitions). +3. **Uploads** the raw `.pftrace` blob to object store (usecase `profiles`, keyed per + org/project, with an attachment-retention TTL). +4. **Produces** a `ProfileChunkKafkaMessage` to the `profiles` Kafka topic. The message + carries the expanded Sample v2 JSON as `payload` plus an `attachments` array, where each + attachment records: + - `name` (e.g. `profile.perfetto`), + - `content_type` (e.g. `application/x-perfetto-trace`), + - `stored_id` — the object store key of the uploaded raw blob. + +```json +{ + "organization_id": 1, + "project_id": 42, + "received": 1720000000, + "retention_days": 30, + "payload": "", + "attachments": [ + { + "name": "profile.perfetto", + "content_type": "application/x-perfetto-trace", + "stored_id": "" + } + ] +} +``` + +The monolith later uses `stored_id` to fetch the raw trace back. + +## Monolith (getsentry/sentry) + +A profiling task consumes the `profiles` topic. `process_profile_task` (in +`src/sentry/profiles/task.py`): + +1. Unpacks the message and parses the Sample v2 `payload`. +2. Runs **deobfuscation** (Android only for now). +3. Calls `vroomrs` to parse and normalize the chunk + (`vroomrs.profile_chunk_from_json_str(...)`), compresses it, and saves it to the profile + **object store** bucket. +4. Extracts function metrics and emits them to **Snuba** (for querying and flamegraph + metadata). +5. Persists a lightweight **`ProfileChunkAttachment`** DB row for each attachment on the + message — storing `project_id`, `profiler_id`, `chunk_id`, `name`, `content_type`, and + the `stored_id` object store key — so the raw Perfetto trace can be downloaded by ID + rather than exposing the `stored_id` directly. + +`getsentry/vroom` is an extra service on top, which reads the stored chunk and Snuba-indexed metadata to serve and merge multiple profile chunks into a single flamegraph. The API endpoint is served by the monolith, but it just passes the request through to vroom. + +### Perfetto format dispatch (vroom / vroomrs) + +Older Android SDKs emit the legacy Android trace format tagged as a "faulty" `version=2`, +and the pipeline historically keyed off the platform rather than the version. To +distinguish legacy from Sample v2 chunks, `ProfileChunk` carries a dedicated `version` +field, and both `vroom` and `vroomrs` now dispatch on it instead of the platform: + +- Version `""` or `2.android-trace` → legacy Android trace format. +- Any other version → Sample v2. + +## Downloading a Perfetto profile + +The monolith exposes two feature-gated endpoints (both require the +`organizations:continuous-profiling-perfetto` flag): + +- **List attachments** — `GET /organizations/{org}/profiling/chunk-attachments/` + (`sentry-api-0-organization-profiling-chunk-attachments`). Requires a `project` and + `profiler_id`; resolves the visible `chunk_id`s (same logic as the flamegraph) and returns + the matching `ProfileChunkAttachment` metadata. +- **Download** — `GET /projects/{org}/{project}/profiling/chunks/{profiler_id}/{chunk_id}/attachments/{attachment_id}/?download` + (`sentry-api-0-project-profiling-chunk-attachment`). The `?download` param is required; it + streams the raw blob back from object store via the stored `stored_id`. Access requires + the org's configured attachments role, analogous to generic event attachments. + +In the flamegraph UI, a toolbar button (added for continuous profiles when the flag is on +and at least one attachment exists) lists and downloads these traces. + +## References + +- SDK: [sentry-java#5251](https://github.com/getsentry/sentry-java/pull/5251) — Android `ProfilingManager` (Perfetto) support +- Relay: [#5659](https://github.com/getsentry/relay/pull/5659), [#5932](https://github.com/getsentry/relay/pull/5932), [#6099](https://github.com/getsentry/relay/pull/6099), [#6102](https://github.com/getsentry/relay/pull/6102) — Perfetto parsing, pipeline, and object-store routing +- vroom: [#672](https://github.com/getsentry/vroom/pull/672) — version dispatch for Android trace profiles +- vroomrs: [#93](https://github.com/getsentry/vroomrs/pull/93) — accept Android profiles in Sample v2 format +- Monolith: [sentry#118029](https://github.com/getsentry/sentry/pull/118029) (chunk attachments + endpoints), [sentry#118071](https://github.com/getsentry/sentry/pull/118071) (flamegraph download button)