docs: Add develop-docs with Perfetto profiling documentation

develop-docs/README.md

@@ -0,0 +1,104 @@
+# Develop Docs
+
+This folder holds internal developer documentation for the Sentry Java/Android SDK:
+architecture notes, feature deep-dives, design decisions, and cross-module concepts
+that don't belong in the public [Sentry docs](https://docs.sentry.io) or in inline
+code comments.
+
+If you are documenting **how** or **why** something works for the people who maintain
+this SDK, it goes here. If you are documenting **how to use** the SDK for end users,
+it belongs in the public docs instead.
+
+## Rules
+
+These rules keep the docs consistent, easy to navigate, and easy to grep.
+
+### Directory structure
+
+- Use a **flat directory structure**. All documents live directly in `develop-docs/`.
+  Do not create nested category folders.
+- Group and namespace documents with **filename prefixes** instead of folders. Common
+  prefixes:
+  - `general-` — cross-cutting topics (e.g. `general-development.md`, `general-architecture.md`)
+  - `feature-` — a specific SDK feature (e.g. `feature-errors.md`, `feature-profiling.md`)
+  - `integration-` — a specific integration or module (e.g. `integration-opentelemetry.md`, `integration-spring.md`)
+  - `platform-` — platform-specific concerns (e.g. `platform-android.md`, `platform-jvm.md`)
+  - `process-` — team processes and workflows (e.g. `process-release.md`)
+
+  Add a new prefix only when an existing one clearly does not fit, and keep the list
+  above up to date.
+
+### File naming
+
+- File names are **lowercase**.
+- Use **dashes** (`-`) as separators, never underscores or spaces. For example, use
+  `feature-profiling.md`, not `feature_profiling.md` or `Feature Profiling.md`.
+- Use the `.md` extension for all documents.
+- Choose short, descriptive names. The prefix conveys the category, so the rest of the
+  name only needs to describe the topic (`feature-session-replay.md`, not
+  `feature-session-replay-how-it-works.md`).
+
+### Images and other assets
+
+- When a document embeds images (or other binary assets), store them in a **folder with
+  the same base name** as the document. For example, `feature-profiling.md` keeps its
+  assets in `feature-profiling/`:
+
+  ```text
+  develop-docs/
+    feature-profiling.md
+    feature-profiling/
+      pipeline.png
+      overview.svg
+  ```
+
+- Reference assets with **relative paths**: `![Profiling pipeline](feature-profiling/pipeline.png)`.
+- Asset file names follow the same rules as documents: lowercase, dashes, descriptive.
+- Prefer **vector formats** (SVG) for diagrams and screenshots where practical
+- Prefer **Mermaid** over a static image whenever a diagram can be expressed as one
+  (see below) — it lives in the document, is versioned as text, and is easy to update.
+
+### Writing style
+
+- Write in the **present tense** and the **active voice**. Describe how the system
+  behaves now ("The transport retries failed envelopes"), not how it will or did behave.
+- Keep one **top-level `# ` heading** per document (the title), and nest sections with
+  `##`, `###`, etc. Do not skip heading levels.
+- Keep documents focused on a **single topic**. Split large topics into multiple
+  prefixed documents and link between them rather than growing one giant file.
+- Use fenced **code blocks with a language identifier** (```kotlin `,
+  ` ```bash `) so syntax highlighting works.
+- Prefer Kotlin snippets over Java.
+- When referencing code, link to the file with a **relative path** (e.g.
+  `../sentry/src/main/java/io/sentry/Sentry.java`) rather than pasting large excerpts
+  that fall out of date.
+- Avoid pinning content to a specific SDK version or date unless it is genuinely
+  version-specific; keep docs evergreen.
+- Cross-link related documents with relative links (e.g. `[profiling](feature-profiling.md)`).
+
+### Diagrams with Mermaid
+
+- Prefer [Mermaid](https://mermaid.js.org/) for diagrams. It renders directly on GitHub
+  and lives in the document as text, so it versions and reviews like code.
+- Embed a Mermaid diagram in a fenced block tagged `mermaid`:
+
+  ````markdown
+  ```mermaid
+  flowchart LR
+    Event[SentryEvent] --> Processor[EventProcessors]
+    Processor --> Transport
+    Transport --> Sentry[(Sentry)]
+  ```
+  ````
+
+- For complex diagrams, include a link to the [Mermaid Live Editor](https://mermaid.live/)
+  so reviewers can iterate quickly.
+- Fall back to a static image (stored per the asset rules above) only when a diagram
+  cannot reasonably be expressed in Mermaid.
+
+## Adding a new document
+
+1. Pick the right prefix (or introduce a new one and document it above).
+2. Create `develop-docs/<prefix>-<topic>.md` with a single `# ` title.
+3. If it embeds assets, create the matching `develop-docs/<prefix>-<topic>/` folder.
+4. Link to it from related documents so it is discoverable.

develop-docs/feature-perfetto-profiling.md

@@ -0,0 +1,223 @@
+# Perfetto profiling on Android
+
+This document describes how continuous profiling works on Android when the SDK
+captures traces through the OS-level [`android.os.ProfilingManager`](https://developer.android.com/reference/android/os/ProfilingManager)
+API (available on API 35+), and how a captured **profile chunk** flows all the way
+from the device to a downloadable profile in Sentry.
+
+## What Perfetto is
+
+[Perfetto](https://perfetto.dev/) is Android's system-wide tracing stack. Its
+[stack-sampling profiler](https://perfetto.dev/docs/data-sources/native-heap-profiler)
+records periodic samples of native and Java call stacks and writes them to a binary
+`.pftrace` trace file (a serialized [Perfetto protobuf](https://perfetto.dev/docs/reference/trace-packet-proto)).
+Starting with Android 15, apps can request such traces at
+runtime via `ProfilingManager` without root or `adb`, which is what makes on-device
+continuous profiling possible.
+
+Useful Perfetto references:
+
+- Perfetto docs: https://perfetto.dev/docs/
+- Trace format (`TracePacket` proto): https://perfetto.dev/docs/reference/trace-packet-proto
+- Perfetto UI (to open a downloaded `.pftrace`): https://ui.perfetto.dev/
+
+## Pipeline overview
+
+A profile chunk is captured on the device, embedded into a Sentry envelope, expanded and
+routed by Relay, and finally symbolicated and stored by the monolith so it can be served
+as a flamegraph and downloaded as a raw Perfetto trace.
+
+```mermaid
+flowchart TD
+    subgraph device["Android device — sentry-java"]
+        PM[android.os.ProfilingManager]
+        PP[PerfettoProfiler]
+        PCP[PerfettoContinuousProfiler]
+        PC[ProfileChunk]
+        ENV["Envelope item<br/>[JSON metadata][raw .pftrace]<br/>header: meta_length"]
+        PM --> PP --> PCP --> PC --> ENV
+    end
+
+    subgraph relay["Relay (processing mode)"]
+        SPLIT[Split payload at meta_length]
+        CONV[Convert Perfetto → Sample v2]
+        OS1[Upload raw .pftrace to object store]
+        KAFKA[["Kafka topic: profiles<br/>ProfileChunkKafkaMessage<br/>(Sample v2 + attachment stored_id)"]]
+        SPLIT --> CONV --> KAFKA
+        SPLIT --> OS1
+    end
+
+    subgraph monolith["Monolith — getsentry/sentry"]
+        TASK[process_profile_task]
+        SYM[Symbolicate / deobfuscate]
+        VR[vroomrs: parse + normalize]
+        OS2[(Object store)]
+        SNUBA[(Snuba: function metrics)]
+        DB[(ProfileChunkAttachment row)]
+        TASK --> SYM --> VR
+        VR --> OS2
+        VR --> SNUBA
+        TASK --> DB
+    end
+
+    ENV -->|envelope| relay
+    KAFKA --> TASK
+    OS1 -.stored_id.-> DB
+    VROOM[getsentry/vroom<br/>serve + merge flamegraphs]
+    OS2 --> VROOM
+    SNUBA --> VROOM
+```
+
+## SDK (getsentry/sentry-java)
+
+On API 35+, [`AndroidOptionsInitializer`](../sentry-android-core/src/main/java/io/sentry/android/core/AndroidOptionsInitializer.java)
+wires up `PerfettoContinuousProfiler` automatically. On older devices the SDK falls back
+to the legacy `Debug`-based [`AndroidContinuousProfiler`](../sentry-android-core/src/main/java/io/sentry/android/core/AndroidContinuousProfiler.java),
+gated by the `enableLegacyProfiling` option (manifest key
+`io.sentry.profiling.enable-legacy-profiling`, defaults to `true`). Only **continuous
+profiling** is supported on the Perfetto path — transaction-based and app-start profiling
+are not.
+
+### Capturing chunks
+
+Continuous profiling emits a stream of independent [`ProfileChunk`](../sentry/src/main/java/io/sentry/ProfileChunk.java)s
+rather than one profile per transaction. `PerfettoContinuousProfiler` drives a chained
+loop: each chunk runs for `MAX_CHUNK_DURATION_MILLIS` (60s) via `PerfettoProfiler`, which
+calls `ProfilingManager.requestProfiling(PROFILING_TYPE_STACK_SAMPLING, …)` at
+`PROFILING_FREQUENCY_HZ` (101 Hz). When a chunk's trace file is ready, a new chunk starts,
+so profiling runs continuously.
+
+A chunk keeps a stable `profilerId` across the session and a per-chunk `chunkId`. When the
+OS produces the trace file, the profiler builds a `ProfileChunk` tagged with the Perfetto
+content type:
+
+```kotlin
+ProfileChunk.Builder(profilerId, chunkId, measurements, traceFile, timestamp, ProfileChunk.PLATFORM_ANDROID)
+    .setContentType(ProfileChunk.CONTENT_TYPE_PERFETTO) // "application/x-perfetto-trace"
+    .build()
+```
+
+The chunk is captured via `scopes.captureProfileChunk(...)` and sent as its own envelope
+with item type [`SentryItemType.ProfileChunk`](../sentry/src/main/java/io/sentry/SentryItemType.java)
+(wire name `profile_chunk`).
+
+### Envelope format and the `meta_length` header
+
+A legacy chunk base64-encodes its trace into the `ProfileChunk` JSON. A Perfetto chunk is
+much larger, so [`SentryClient`](../sentry/src/main/java/io/sentry/SentryClient.java) instead
+routes it through the new `SentryEnvelopeItem.fromPerfettoProfileChunk(...)` factory, which
+avoids base64 by sending the raw binary alongside the JSON.
+
+The trick is a single envelope **item** whose payload concatenates the JSON metadata and
+the raw `.pftrace` bytes with **no delimiter**:
+
+```text
+[ProfileChunk JSON bytes][raw .pftrace binary bytes]
+```
+
+A new `meta_length` property on the [envelope item header](../sentry/src/main/java/io/sentry/SentryEnvelopeItemHeader.java)
+tells the server where the JSON ends and the binary begins. The standard envelope item
+structure (header line + newline + payload) is unchanged; `meta_length` simply subdivides
+the payload:
+
+```text
+{"type":"profile_chunk","content_type":"application/x-perfetto-trace","filename":"…","length":<total>,"meta_length":<json bytes>}
+<ProfileChunk JSON><raw perfetto binary>
+```
+
+- `length` — total payload size (JSON + binary), as for any envelope item.
+- `meta_length` — byte length of the JSON prefix. It is only known after the payload is
+  serialized, so the header computes it lazily (via a `Callable<Integer>`) and omits the
+  field entirely for non-Perfetto items, keeping the change backward compatible.
+
+## Relay (getsentry/relay)
+
+Relay processing is gated behind the `organizations:continuous-profiling-perfetto` feature
+flag; without it the chunk is dropped. In processing mode Relay:
+
+1. **Splits** the compound item payload at `meta_length` into `(metadata JSON, raw profile)`
+   and reads `content_type: "perfetto"` from the metadata.
+2. **Converts** the binary Perfetto trace into the existing **Sample v2** profile JSON
+   format (`relay_profiling::expand_perfetto(...)`, backed by a checked-in subset of the
+   Perfetto protobuf definitions).
+3. **Uploads** the raw `.pftrace` blob to object store (usecase `profiles`, keyed per
+   org/project, with an attachment-retention TTL).
+4. **Produces** a `ProfileChunkKafkaMessage` to the `profiles` Kafka topic. The message
+   carries the expanded Sample v2 JSON as `payload` plus an `attachments` array, where each
+   attachment records:
+   - `name` (e.g. `profile.perfetto`),
+   - `content_type` (e.g. `application/x-perfetto-trace`),
+   - `stored_id` — the object store key of the uploaded raw blob.
+
+```json
+{
+  "organization_id": 1,
+  "project_id": 42,
+  "received": 1720000000,
+  "retention_days": 30,
+  "payload": "<expanded Sample v2 profile JSON>",
+  "attachments": [
+    {
+      "name": "profile.perfetto",
+      "content_type": "application/x-perfetto-trace",
+      "stored_id": "<object store key of the raw .pftrace blob>"
+    }
+  ]
+}
+```
+
+The monolith later uses `stored_id` to fetch the raw trace back.
+
+## Monolith (getsentry/sentry)
+
+A profiling task consumes the `profiles` topic. `process_profile_task` (in
+`src/sentry/profiles/task.py`):
+
+1. Unpacks the message and parses the Sample v2 `payload`.
+2. Runs **deobfuscation** (Android only for now).
+3. Calls `vroomrs` to parse and normalize the chunk
+   (`vroomrs.profile_chunk_from_json_str(...)`), compresses it, and saves it to the profile
+   **object store** bucket.
+4. Extracts function metrics and emits them to **Snuba** (for querying and flamegraph
+   metadata).
+5. Persists a lightweight **`ProfileChunkAttachment`** DB row for each attachment on the
+   message — storing `project_id`, `profiler_id`, `chunk_id`, `name`, `content_type`, and
+   the `stored_id` object store key — so the raw Perfetto trace can be downloaded by ID
+   rather than exposing the `stored_id` directly.
+
+`getsentry/vroom` is an extra service on top, which reads the stored chunk and Snuba-indexed metadata to serve and merge multiple profile chunks into a single flamegraph. The API endpoint is served by the monolith, but it just passes the request through to vroom.
+
+### Perfetto format dispatch (vroom / vroomrs)
+
+Older Android SDKs emit the legacy Android trace format tagged as a "faulty" `version=2`,
+and the pipeline historically keyed off the platform rather than the version. To
+distinguish legacy from Sample v2 chunks, `ProfileChunk` carries a dedicated `version`
+field, and both `vroom` and `vroomrs` now dispatch on it instead of the platform:
+
+- Version `""` or `2.android-trace` → legacy Android trace format.
+- Any other version → Sample v2.
+
+## Downloading a Perfetto profile
+
+The monolith exposes two feature-gated endpoints (both require the
+`organizations:continuous-profiling-perfetto` flag):
+
+- **List attachments** — `GET /organizations/{org}/profiling/chunk-attachments/`
+  (`sentry-api-0-organization-profiling-chunk-attachments`). Requires a `project` and
+  `profiler_id`; resolves the visible `chunk_id`s (same logic as the flamegraph) and returns
+  the matching `ProfileChunkAttachment` metadata.
+- **Download** — `GET /projects/{org}/{project}/profiling/chunks/{profiler_id}/{chunk_id}/attachments/{attachment_id}/?download`
+  (`sentry-api-0-project-profiling-chunk-attachment`). The `?download` param is required; it
+  streams the raw blob back from object store via the stored `stored_id`. Access requires
+  the org's configured attachments role, analogous to generic event attachments.
+
+In the flamegraph UI, a toolbar button (added for continuous profiles when the flag is on
+and at least one attachment exists) lists and downloads these traces.
+
+## References
+
+- SDK: [sentry-java#5251](https://github.com/getsentry/sentry-java/pull/5251) — Android `ProfilingManager` (Perfetto) support
+- Relay: [#5659](https://github.com/getsentry/relay/pull/5659), [#5932](https://github.com/getsentry/relay/pull/5932), [#6099](https://github.com/getsentry/relay/pull/6099), [#6102](https://github.com/getsentry/relay/pull/6102) — Perfetto parsing, pipeline, and object-store routing
+- vroom: [#672](https://github.com/getsentry/vroom/pull/672) — version dispatch for Android trace profiles
+- vroomrs: [#93](https://github.com/getsentry/vroomrs/pull/93) — accept Android profiles in Sample v2 format
+- Monolith: [sentry#118029](https://github.com/getsentry/sentry/pull/118029) (chunk attachments + endpoints), [sentry#118071](https://github.com/getsentry/sentry/pull/118071) (flamegraph download button)