Add DICOM data loader tutorial

README.md

@@ -30,6 +30,8 @@ Below is a list of all tutorials available in this repository:
 - 📊 [Visualize](https://app.foxglove.dev/~/view?ds=foxglove-sample-stream&ds.recordingId=vqKKQcot421Kwg84&ds.overrideLayoutId=b7513959-1d46-4a89-bc24-1584d9677ca1&ds.start=2023-09-01T13:19:45.047438263Z&ds.end=2023-09-01T13:20:15.047438263Z)
 
 ## Foxglove SDK
+### [Play DICOM CT series in Foxglove](foxglove_sdk/dicom_data_loader/README.md)
+- 📝 A Rust data loader that opens uncompressed DICOM files and plays slices or breathing phases on the timeline.
 ### [Using Foxglove to Visualize Ethernet/IP data](foxglove_sdk/ethernet_ip_integration/README.md)
 - 📝 Using Foxglove data, it's easier than ever to stream time series data. In this project, we show you how.
 - 🔗 [Related Blog Post](https://foxglove.dev/blog/use-foxglove-sdk-for-real-time-industrial-plc-data-visualization-and-playback)

foxglove_sdk/dicom_data_loader/.gitignore

@@ -0,0 +1,9 @@
+dist
+node_modules
+rust/Cargo.lock
+rust/target
+data/*
+!data/README.md
+*.foxe
+scripts/.smoke
+.venv

foxglove_sdk/dicom_data_loader/.prettierrc.yaml

@@ -0,0 +1,5 @@
+arrowParens: always
+printWidth: 100
+trailingComma: "all"
+tabWidth: 2
+semi: true

foxglove_sdk/dicom_data_loader/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 0.1.0
+
+- Initial DICOM data loader: sweep through a CT series, or play a 4D breathing study, on the Foxglove timeline.

foxglove_sdk/dicom_data_loader/README.md

@@ -0,0 +1,87 @@
+---
+title: "Play DICOM CT series in Foxglove"
+short_description: "A Rust data loader that opens uncompressed DICOM files and plays slices or breathing phases on the timeline."
+---
+
+# Play DICOM CT series in Foxglove
+
+Foxglove does not open DICOM. This tutorial is a data loader extension, written in Rust and compiled to WebAssembly, that does. Drag a folder of `.dcm` files into Foxglove and the timeline becomes the navigation control: a sweep through axial slices, or a short animation of a patient breathing. No MCAP conversion and no custom panel.
+
+The loader publishes lung-window images, a 3D point cloud of bone and lung, a plot of lung area or volume, and a metadata message.
+
+## Prerequisites
+
+- Rust stable, with the `wasm32-unknown-unknown` target (`rustup target add wasm32-unknown-unknown`)
+- Node.js 18 or newer
+- [uv](https://docs.astral.sh/uv/getting-started/installation/), only for downloading the sample series
+
+## Download sample data
+
+Both series are public, CC BY 3.0, and hosted by the NCI Imaging Data Commons. The script uses the `idc-index` package. Nothing is committed to the repository.
+
+From this directory:
+
+```bash
+uv run --directory scripts download_sample_data.py --dataset chest      # one 133-slice chest CT, ~70 MB
+uv run --directory scripts download_sample_data.py --dataset breathing  # ten phases, ~750 MB
+uv run --directory scripts download_sample_data.py --dataset all
+```
+
+Files are written under `data/chest` and `data/breathing`. The breathing study is about 750 MB (ten phases, 142 slices each). From `rust/`, `cargo test --release` reads those two folders and skips a dataset when its folder is missing. `DICOM_DATA_DIR` overrides the parent directory.
+
+## Build and install
+
+From this directory:
+
+```bash
+npm install
+npm run package
+```
+
+`npm run package` builds the release WebAssembly module and writes `foxglove.foxglove-dicom-data-loader-0.1.0.foxe`. In Foxglove, open Settings → Extensions and install that file. For the desktop app you can instead run `npm run local-install`.
+
+## Open the data
+
+Open one dataset at a time. Select or drag every `.dcm` file from `data/chest` or from `data/breathing` together. The loader only sees the files you hand it, so a partial selection is a partial volume.
+
+Import `foxglove_layouts/dicom_layout.json` for a layout with axial, coronal, and sagittal images, a 3D view, the stats plot, and the metadata panel. The coronal and sagittal panels subscribe to `/dicom/coronal/annotations` and `/dicom/sagittal/annotations`. Those topics exist in sweep mode and draw a line at the current axial slice. 4D mode does not publish them.
+
+Press play.
+
+- **Chest CT** is sweep mode. Each frame is the next axial slice, inferior to superior, at 100 ms per slice (about 13 seconds). Coronal and sagittal views are the centre multi-planar reconstruction, published once. The line on those views tracks the slice. The 3D panel shows bone, lung, and the current slice as a point cloud.
+- **4D lung** is breathing mode, using 4D-Lung patient `100_HM10395`, study S300 (142 slices per phase, so the diaphragm stays in view). The loader finds the ten series that share a frame of reference and the same geometry, ordered by the phase percentage in `SeriesDescription` (`Gated, 40.0%A`). Each frame is one phase at 400 ms, and the cycle repeats three times so playback looks continuous. The centre coronal image and the lung cloud update every phase. Bone, the bounding box, and the metadata message stay at the start of the timeline; Foxglove keeps them visible by backfill when you seek.
+
+## How it works
+
+`rust/src/lib.rs` exports a [`DataLoader`](https://docs.rs/foxglove_data_loader). Foxglove calls `initialize` with the paths you opened. The loader first reads each file only up to the pixel-data tag, groups the headers by `SeriesInstanceUID`, and decides sweep or 4D mode. It then re-opens one series at a time, encodes that phase, and drops the HU volume before the next series. The result is a timeline of encoded messages. Repeated breathing cycles share the same bytes.
+
+Slices in a series are ordered along the slice normal (`ImagePositionPatient` dotted with the cross product of the image orientation). The first frame is the inferior end.
+
+Mode detection:
+
+- **4D** when at least two series share `FrameOfReferenceUID` and have the same rows, columns, and slice count. Phases are ordered by a percentage parsed from the series description, such as `Gated, 40.0%A`, and otherwise by series number.
+- **Sweep** otherwise, using the series with the most slices. Extra series produce a warning.
+
+`StudyDate` and `StudyTime` become the timeline origin, interpreted as UTC. If they do not parse, the origin is zero. `create_iter` walks that timeline from the start time, and `get_backfill` returns the latest message at or before the seek time on each requested channel so panels are not empty when playback starts in the middle.
+
+Images are windowed with a CT lung window (level -600 HU, width 1500) and encoded as `mono8`. Coronal and sagittal views resample along the stack so the spacing matches the in-plane pixels, with superior at the top. Positions are DICOM LPS millimetres converted to metres in a `patient` frame whose origin is the centre of the volume, so the anatomy sits at the origin of Foxglove's z-up view. Every 4D phase uses the first phase's origin. Recentering each phase would hide the breathing motion.
+
+The lung mask is a demo heuristic. A flood fill from the image border marks air connected to the outside of the body (voxels below -400 HU). Voxels that are not outside air and sit between -1000 and -400 HU count as lung. Body is everything that is not outside air. Bone is HU of 250 or higher. Point clouds step 4 pixels in-plane and, for a whole volume, every second slice, which keeps each cloud under 300,000 points for these series.
+
+## Optional: smoke test
+
+`scripts/wasm_smoke.mjs` plays `data/chest` and `data/breathing` through the release module. Build it from `rust/` with `cargo build --release --target wasm32-unknown-unknown`. You also need [`wasm-tools`](https://github.com/bytecodealliance/wasm-tools) and [`@bytecodealliance/jco`](https://www.npmjs.com/package/@bytecodealliance/jco). `DICOM_DATA_DIR` overrides the data parent, `WASM_TOOLS` and `JCO` select the tools, and `PREVIEW_DIR` (optional) is where grayscale PNGs are written.
+
+```bash
+cargo build --release --target wasm32-unknown-unknown --manifest-path rust/Cargo.toml
+node scripts/wasm_smoke.mjs
+```
+
+## Limitations
+
+Only uncompressed little-endian transfer syntaxes are read (Implicit VR and Explicit VR). JPEG and other encapsulated pixel data are skipped, with a warning. Multi-frame files are skipped. The lung and body masks are not a segmentation. The loader does not publish patient name, ID, or other identifying fields.
+
+## Data
+
+- Chest CT: LIDC-IDRI, series `1.3.6.1.4.1.14519.5.2.1.6279.6001.179049373636438705059720603192`. Armato SG 3rd, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA, Kazerooni EA, MacMahon H, Van Beek EJR, Yankelevitz D, et al.: The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans. Medical Physics, 38: 915--931, 2011. [CC BY 3.0](https://creativecommons.org/licenses/by/3.0/). Data from [The Cancer Imaging Archive](https://www.cancerimagingarchive.net/).
+- Breathing CT: 4D-Lung, patient `100_HM10395`, study S300 (ten gated series, 142 slices of 512×512, 0.9766 mm pixels, 3 mm slices, Implicit VR Little Endian). Hugo GD, Weiss E, Sleeman WC, Balik S, Keall PJ, Lu J, Williamson JF. (2016). Data from 4D Lung Imaging of NSCLC Patients. The Cancer Imaging Archive. [CC BY 3.0](https://creativecommons.org/licenses/by/3.0/).

foxglove_sdk/dicom_data_loader/config.ts

@@ -0,0 +1,9 @@
+module.exports = {
+  webpack: (config) => {
+    config.module.rules.push({
+      test: /\.wasm$/i,
+      type: "asset/inline",
+    });
+    return config;
+  },
+};

foxglove_sdk/dicom_data_loader/data/README.md

@@ -0,0 +1,3 @@
+Sample CT series downloaded by `scripts/download_sample_data.py` land in this directory.
+
+`data/chest` is one LIDC-IDRI axial series. `data/breathing` is ten 4D-Lung phases from study S300 (about 750 MB). The files are not part of the repository.

foxglove_sdk/dicom_data_loader/eslint.config.js

@@ -0,0 +1,23 @@
+// @ts-check
+
+const foxglove = require("@foxglove/eslint-plugin");
+const globals = require("globals");
+const tseslint = require("typescript-eslint");
+
+module.exports = tseslint.config({
+  files: ["src/**/*.ts", "src/**/*.tsx"],
+  extends: [foxglove.configs.base, foxglove.configs.react, foxglove.configs.typescript],
+  languageOptions: {
+    globals: {
+      ...globals.es2020,
+      ...globals.browser,
+    },
+    parserOptions: {
+      project: "tsconfig.json",
+      tsconfigRootDir: __dirname,
+    },
+  },
+  rules: {
+    "react-hooks/exhaustive-deps": "error",
+  },
+});

foxglove_sdk/dicom_data_loader/foxglove_layouts/dicom_layout.json

@@ -0,0 +1,151 @@
+{
+  "configById": {
+    "Image!axial": {
+      "foxglovePanelTitle": "Axial",
+      "imageMode": {
+        "imageTopic": "/dicom/axial",
+        "minValue": 0,
+        "maxValue": 255
+      },
+      "followMode": "follow-none",
+      "synchronize": false
+    },
+    "Image!coronal": {
+      "foxglovePanelTitle": "Coronal",
+      "imageMode": {
+        "imageTopic": "/dicom/coronal",
+        "minValue": 0,
+        "maxValue": 255,
+        "annotations": {
+          "/dicom/coronal/annotations": {
+            "visible": true
+          }
+        }
+      },
+      "followMode": "follow-none",
+      "synchronize": false
+    },
+    "Image!sagittal": {
+      "foxglovePanelTitle": "Sagittal",
+      "imageMode": {
+        "imageTopic": "/dicom/sagittal",
+        "minValue": 0,
+        "maxValue": 255,
+        "annotations": {
+          "/dicom/sagittal/annotations": {
+            "visible": true
+          }
+        }
+      },
+      "followMode": "follow-none",
+      "synchronize": false
+    },
+    "3D!volume": {
+      "foxglovePanelTitle": "Volume",
+      "followTf": "patient",
+      "followMode": "follow-pose",
+      "fixedFrame": "world",
+      "scene": {
+        "backgroundColor": "#111111"
+      },
+      "cameraState": {
+        "perspective": true,
+        "distance": 1.6,
+        "phi": 60,
+        "thetaOffset": 35,
+        "fovy": 50,
+        "near": 0.01,
+        "far": 20,
+        "target": [0, 0, 0],
+        "targetOffset": [0, 0, 0]
+      },
+      "topics": {
+        "/dicom/bone": {
+          "visible": true,
+          "colorMode": "rgba-fields",
+          "pointSize": 2
+        },
+        "/dicom/lungs": {
+          "visible": true,
+          "colorMode": "rgba-fields",
+          "pointSize": 2
+        },
+        "/dicom/slice": {
+          "visible": true,
+          "colorMode": "rgba-fields",
+          "pointSize": 2
+        },
+        "/dicom/scene": {
+          "visible": true
+        }
+      }
+    },
+    "Plot!stats": {
+      "foxglovePanelTitle": "Stats",
+      "paths": [
+        {
+          "value": "/dicom/stats.mean_hu",
+          "enabled": true,
+          "timestampMethod": "receiveTime",
+          "color": "#f7df71"
+        },
+        {
+          "value": "/dicom/stats.lung_area_cm2",
+          "enabled": true,
+          "timestampMethod": "receiveTime",
+          "color": "#4e98e2"
+        },
+        {
+          "value": "/dicom/stats.lung_volume_ml",
+          "enabled": true,
+          "timestampMethod": "receiveTime",
+          "color": "#5cd6a9"
+        }
+      ],
+      "showXAxisLabels": true,
+      "showYAxisLabels": true,
+      "showLegend": true,
+      "legendDisplay": "floating",
+      "showPlotValuesInLegend": false,
+      "isSynced": true,
+      "xAxisVal": "timestamp",
+      "sidebarDimension": 240
+    },
+    "RawMessages!meta": {
+      "foxglovePanelTitle": "Metadata",
+      "topicPath": "/dicom/metadata",
+      "diffEnabled": false,
+      "showFullMessageForDiff": false
+    }
+  },
+  "globalVariables": {},
+  "userNodes": {},
+  "playbackConfig": {
+    "speed": 1
+  },
+  "layout": {
+    "first": {
+      "first": "Image!axial",
+      "second": {
+        "first": "Image!coronal",
+        "second": "Image!sagittal",
+        "direction": "row"
+      },
+      "direction": "column",
+      "splitPercentage": 55
+    },
+    "second": {
+      "first": "3D!volume",
+      "second": {
+        "first": "Plot!stats",
+        "second": "RawMessages!meta",
+        "direction": "column",
+        "splitPercentage": 62
+      },
+      "direction": "column",
+      "splitPercentage": 68
+    },
+    "direction": "row",
+    "splitPercentage": 48
+  }
+}