diff --git a/README.md b/README.md index 76fdc5cc..43ee673f 100644 --- a/README.md +++ b/README.md @@ -9,15 +9,16 @@ This repository contains reference drivers and configurations for Intel MIPI CSI
  • Supported Sensors
  • Supported Ubuntu and Kernel Version
  • Directory Structure
    • -
  • Getting Started Guide
    • -
  • Software Dependencies
    • -
  • - Setup Procedure +
  • Setup Procedure
  • Contributing
    • @@ -27,94 +28,208 @@ This repository contains reference drivers and configurations for Intel MIPI CSI + ## Supported Sensors -| Sensor | Sensor Type | Vendor | IPU Version | -|-----------------|-------------|-----------------|----------------------------------| -| AR0233+GW5300 | GMSL | Sensing | IPU6EPMTL, IPU75XA | -| AR0234 | GMSL | D3 Embedded | IPU6EPMTL, IPU75XA, IPU8 | -| AR0234 | MIPI CSI-2 | D3 Embedded | IPU6EPMTL, IPU75XA, IPU8 | -| AR0820+GW5300 | GMSL | Sensing | IPU6EPMTL, IPU75XA | -| AR0830+AP1302 | MIPI CSI-2 | Leopard Imaging | IPU6EPMTL, IPU75XA | -| ISX031 | GMSL | D3 Embedded | IPU6EP, IPU6EPMTL, IPU75XA, IPU8 | -| ISX031 | GMSL | Leopard Imaging | IPU6EP, IPU6EPMTL, IPU75XA, IPU8 | -| ISX031 | GMSL | Sensing | IPU6EP, IPU6EPMTL, IPU75XA | -| ISX031 | MIPI CSI-2 | D3 Embedded | IPU6EP, IPU6EPMTL, IPU75XA, IPU8 | -| ISX031 | MIPI CSI-2 | Sensing | IPU6EP, IPU6EPMTL, IPU75XA | -| IMX415 | MIPI CSI-2 | Leopard Imaging | IPU6EPMTL | -| IMX586 | MIPI CSI-2 | Leopard Imaging | IPU6EPMTL | -| OV13B10 | MIPI CSI-2 | Leopard Imaging | IPU8 | +| GMSL Sensor | User Guide | Vendor | IPU6EP | IPU6EPMTL | IPU75XA | IPU8 | +|---------------------------------------------|---------------------------------------------|-----------------|:------:|:---------:|:-------:|:----:| +| [AR0233+GW5300](doc/ar0233/kernelspace.md) | [User Guide](doc/ar0233/userspace-gmsl.md) | Sensing |❌ |✅|✅|❌| +| [AR0234](doc/ar0234/kernelspace.md) | [User Guide](doc/ar0234/userspace-gmsl.md) | D3 Embedded |✅*|✅|✅|✅| +| [AR0820+GW5300](doc/ar0820/kernelspace.md) | [User Guide](doc/ar0820/userspace-gmsl.md) | Sensing |❌ |✅|✅|❌| +| D457 | [User Guide](doc/d4xx/userspace-gmsl.md) | RealSense |❌ |✅|✅|✅| +| [ISX031](doc/isx031/kernelspace.md) | [User Guide](doc/isx031/userspace-gmsl.md) | D3 Embedded |✅*|✅|✅|✅| +| [ISX031](doc/isx031/kernelspace.md) | [User Guide](doc/isx031/userspace-gmsl.md) | Leopard Imaging |✅*|✅|✅|✅| +| [ISX031](doc/isx031/kernelspace.md) | [User Guide](doc/isx031/userspace-gmsl.md) | Sensing |✅*|✅|✅|✅| + + +| MIPI Sensor | User Guide | Vendor | IPU6EP | IPU6EPMTL | IPU75XA | IPU8 | +|---------------------------------------------|---------------------------------------------|-----------------|:------:|:---------:|:-------:|:----:| +| [AR0234](doc/ar0234/kernelspace.md) | [User Guide](doc/ar0234/userspace-mipi.md) | D3 Embedded |✅*|✅ |✅ |✅| +| [AR0830+AP1302](doc/ar0830/kernelspace.md) | [User Guide](doc/ar0830/userspace-mipi.md) | Leopard Imaging |❌ |✅*|✅*|❌| +| IMX415 | [User Guide](doc/imx415/userspace-mipi.md) | Leopard Imaging |❌ |✅*|❌ |❌| +| [IMX586](doc/imx586/kernelspace.md) | [User Guide](doc/imx586/userspace-mipi.md) | Leopard Imaging |❌ |✅*|❌ |❌| +| [ISX031](doc/isx031/kernelspace.md) | [User Guide](doc/isx031/userspace-mipi.md) | D3 Embedded |✅*|✅ |✅ |✅| +| [ISX031](doc/isx031/kernelspace.md) | [User Guide](doc/isx031/userspace-mipi.md) | Sensing |✅*|✅*|✅*|❌| +| OV13B10 | [User Guide](doc/ov13b10/userspace-mipi.md) | Leopard Imaging |❌ |❌ |✅ |✅| > **Note:** \ -IPU6EP represents ADL, TWL, ASL and RPL platforms; \ +> Items marked with ✅ are enabled by BIOS and ASL method. \ +> Items marked with ✅* are enabled by BIOS method ONLY.\ +> Items marked with ❌ are not enabled by BIOS or ASL method. + + +> **Note:** \ +IPU6EP represents TWL platforms; \ IPU6EPMTL represents MTL and ARL platforms; \ IPU75XA represents PTL platforms; \ IPU8 represents NVL platforms. +--- ## Supported Ubuntu and Kernel Version -| IPU Version | Ubuntu Version | Kernel Version | -|--------------------|-----------------|-----------------| -| IPU6EP / IPU6EPMTL | 24.04.4 | 6.12 Intel BKC | -| | 24.04.4 | 6.17 Canonical | -| | 26.04 | 7.0 Canonical | -| IPU75XA | 24.04.4 | 6.17 Intel BKC | -| | 24.04.4 | 6.17 Canonical | -| | 24.04.4 | 6.18 Intel BKC | -| | 26.04 | 7.0 Canonical | -| IPU8 | 24.04.4 | 6.18 Intel BKC | -| | 24.04.4 | 7.0 IOT Next | +| IPU Version | Ubuntu Version | Intel Kernel Overlay | Canonical Kernel | BIOS support | ASL support | +|:-----------:|-----------------|:--------------------:|:----------------:|:------------:|:-----------:| +| IPU6EP | 24.04.4 | 6.12 | |✅|❌| +| | 24.04.4 | 6.18 (BKC) | 6.17 |✅|❌| +| | 26.04 | 6.18 | 7.0 |✅|❌| +| IPU6EPMTL | 24.04.4 | 6.12 | |✅|❌| +| | 24.04.4 | 6.18 (BKC) | 6.17 |✅|✅*| +| | 26.04 | 6.18 | 7.0 |✅|✅*| +| IPU75XA | 24.04.4 | 6.17 | |✅|❌| +| | 24.04.4 | 6.18 (BKC) | 6.17 |✅|✅*| +| | 26.04 | 6.18 | 7.0 |✅|✅*| +| IPU8 | 24.04.4 | 7.0 | |✅|✅| +| | 26.04 | 7.0 | |✅|✅| +> **Note:** \ +> ✅* indicates that ASL support is **NOT AVAILABLE** for 6.18 (BKC) **by default**. Please rebuild the 6.18 Kernel Overlay with ASL support enabled. For more details, please refer to [doc/acpi/kernelspace.md](doc/acpi/kernelspace.md).\ +> To use Intel BKC, please refer [here](#intel-bkc-using-getting-started-guide-gsg) for more details. + +--- ## Directory Structure -| Directory | Description | -|-----------|-------------| -| [drivers/](drivers) | Host Linux kernel drivers for supported sensors | -| [config/](config) | Host middleware configuration files | -| [doc/](doc) | Documentation guide for kernelspace and userspace configuration | -| [include/](include) | Header files for driver compilation | +| Directory | Description | +|---------------------|-------------| | [acpi/](acpi) | Host ASL source files for different configurations | +| [config/](config) | Host middleware configuration files | +| [drivers/](drivers) | Host Linux kernel drivers for supported sensors | +| [doc/](doc) | Host Documentation guide for kernelspace and userspace configuration | +| [include/](include) | Host Header files for driver compilation | +| [script/](script) | Host Utility scripts | + +--- +## Setup Procedure + +1. Find your desired target platform and desired Ubuntu/Kernel from the table [above](#supported-ubuntu-and-kernel-version). + +2. Setup your target platform. Follow section [below](#software-setup---ubuntu-and-kernel). + > To use **Intel BKC**, Follow section [below](#intel-bkc-using-getting-started-guide-gsg). + +3. DKMS build your kernel drivers. Follow section [below](#software-setup---kernel-driver-dkms-build). + +4. Install required software dependencies on your target platform. Follow section [below](#software-setup---userspace). + +5. Setup your Hardware. Follow section [below](#hardware-setup). + +6. Configure ACPI to match with Hardware setup. -## Getting Started Guide + > For **BIOS Configuration** method, go to [BIOS Configuration](#acpi-setup---bios-configuration). -1. Download `Getting Started Guide` (table below), and setup according to your target `Platform`. -2. Inside the guide, follow all instructions under section `Getting Started with Ubuntu with Kernel Overlay`. -3. Under section `Auto Script Installation`, download and use `Ubuntu Kernel Overlay Auto Installar Script` from platform-respective Software Packages. + > For **ACPI ASL Configuration** method, go to [ACPI ASL Configuration](#acpi-setup---asl-configuration). -> **Note:** All collaterals belows can be downloaded in [rdc.intel.com](https://www.intel.com/content/www/us/en/resources-documentation/developer.html) with proper granted access. +7. Boot and verify the setup using `media-ctl`. Go to [Setup Verification](#setup-verification). + +8. Enjoy your camera stream! Go to [Stream Verification](#stream-verification). + +--- +
    +Expand to show detailed setup steps + +--- +### Software Setup - Ubuntu and Kernel + +#### Intel BKC using Getting Started Guide (GSG) +
    + Show details + +> **Note:** GSG requires granted access to Intel® RDC portal. Please contact your Intel representative for access. + +> **Skip this section** if you do not have access. Use default Canonical Kernel version that comes with the Ubuntu Image instead. + +1. Download **Getting Started Guide** from table below and setup according to your target **Platform**. +2. Under section **Getting Started with Ubuntu with Kernel Overlay**, follow **ALL** instructions to avoid missing dependencies. +3. Under section **Auto Script Installation**, download `Ubuntu Kernel Overlay Auto Installer Script` from platform-respective Software Packages. +4. Run installer script to install Intel Kernel Overlay and Intel BKC on your target platform. + +> **Reminder:** All collaterals below can be downloaded in [rdc.intel.com](https://www.intel.com/content/www/us/en/resources-documentation/developer.html) with proper granted access. | Platform | Getting Started Guide | Software Package | -|---|---|---| +|:---:|:---:|:---:| | ARL | [828853](https://www.intel.com/content/www/us/en/secure/content-details/828853/ubuntu-with-kernel-overlay-on-intel-core-ultra-200u-and-200h-series-processors-code-named-arrow-lake-u-h-for-edge-platforms-get-started-guide.html?DocID=828853) | [831484](https://www.intel.com/content/www/us/en/secure/design/confidential/software-kits/kit-details.html?kitId=831484) | | MTL | [779460](https://www.intel.com/content/www/us/en/secure/content-details/779460/ubuntu-with-kernel-overlay-on-intel-core-mobile-processors-code-named-meteor-lake-u-h-for-edge-platforms-get-started-guide.html?DocID=779460) | [790840](https://www.intel.com/content/www/us/en/secure/content-details/790840/meteor-lake-ps-ubuntu-with-kernel-overlay-software-packages.html?DocID=790840) | | TWL | [793827](https://www.intel.com/content/www/us/en/secure/content-details/793827/ubuntu-with-kernel-overlay-intel-atom-x7000re-x7000c-x7000fe-processor-series-intel-processor-n150-n250-intel-core-3-processor-n355-for-edge-applications-get-started-guide-amston-lake-mr5-amston-lake-fusa-pv-twin-lake-mr2.html?DocID=793827) | [803960](https://www.intel.com/content/www/us/en/secure/design/confidential/software-kits/kit-details.html?kitId=803960) | -| PTL | [858119](https://edc.intel.com/content/www/us/en/secure/design/confidential/products-and-solutions/processors-and-chipsets/panther-lake-h/with-linux-os-get-started-guide-for-edge-compute-applications/) | [871556](https://www.intel.com/content/www/us/en/secure/design/confidential/software-kits/kit-details.html?kitId=860689) | +| PTL | [858119](https://edc.intel.com/content/www/us/en/secure/design/confidential/products-and-solutions/processors-and-chipsets/panther-lake-h/with-linux-os-get-started-guide-for-edge-compute-applications/) | [860689](https://www.intel.com/content/www/us/en/secure/design/confidential/software-kits/kit-details.html?kitId=860689) | Reference: [Intel® IPU6 Enabling Partners Technical Collaterals Advisory](https://www.intel.com/content/www/us/en/secure/content-details/817101/intel-ipu6-enabling-partners-technical-collaterals-advisory.html?DocID=817101) -## Software Dependencies +

    (Back to Setup Procedure)

    -Install these software dependencies in your target system: +
    + +#### Canonical Kernel + +Canonical Kernel version comes with the Ubuntu Image by default. Make sure the kernel version is supported [here](#supported-ubuntu-and-kernel-version). + +

    (Back to Setup Procedure)

    + +--- +### Software Setup - Userspace + +Build and install these software dependencies in your target system: + +#### Utility Tools + +
    + Show details -- ipu-camera-bins (E.g. [ipu6-camera-bins](https://github.com/intel/ipu6-camera-bins/tree/iotg_ipu6) / [ipu7-camera-bins](https://github.com/intel/ipu7-camera-bins)) -- ipu-camera-hal (E.g. [ipu6-camera-hal](https://github.com/intel/ipu6-camera-hal/tree/iotg_ipu6) / [ipu7-camera-hal](https://github.com/intel/ipu7-camera-hal)) +> v4l-utils with version >= 1.30 is **mandatory** for GMSL setup. \ +> acpica with version >= 20260408 is **mandatory** for ACPI ASL setup. + +| Tool | Repository | Tag | Steps to setup | +|------|------------|---------| -----| +| v4l-utils | [v4l-utils](https://github.com/gjasny/v4l-utils) | stable-1.30 | Build from repo | +| acpica | [acpica](https://github.com/open-acpica/acpica.git) | 20260408 | Refer [here](doc/acpi/kernelspace.md#compile-and-load) | + +
    + +#### Intel IPU Camera Software Stack + +
    +Show details + +- ipu-camera-bins (E.g. [IPU6EP, IPU6EPMTL](https://github.com/intel/ipu6-camera-bins/tree/iotg_ipu6) / [IPU75XA, IPU8](https://github.com/intel/ipu7-camera-bins)) +- ipu-camera-hal (E.g. [IPU6EP, IPU6EPMTL](https://github.com/intel/ipu6-camera-hal/tree/iotg_ipu6) / [IPU75XA, IPU8](https://github.com/intel/ipu7-camera-hal)) - [icamerasrc](https://github.com/intel/icamerasrc/tree/icamerasrc_slim_api) -| IPU Version | ipu-camera-bins | ipu-camera-hal | icamerasrc | -|-------------|------------------------------------------|------------------------------------------|------------------------------------------| -| IPU6 | d9421fef539f24fc80c27002d5da753e193b0670 | f93eec544a5234bf0b610b3f76d64c8fa711c364 | 867c5b6ab7925c9b69b8374873a832266d97d7e5 | -| IPU7 | cead7320d84ee9ade4f60d74e935b16b5a760945 | ef30767553685b83034e42325992a2442c5fcb2c | 867c5b6ab7925c9b69b8374873a832266d97d7e5 | -| IPU8 | adf55525ab9d370828723b1ff8bee76ed7a492e8 | fd1efb6f2ef25fd24775a303cc71af6139d797eb | 867c5b6ab7925c9b69b8374873a832266d97d7e5 | +| IPU Version | ipu-camera-bins | ipu-camera-hal | icamerasrc | +|-------------------|------------------------------------------|------------------------------------------|------------------------------------------| +| IPU6EP, IPU6EPMTL | d9421fef539f24fc80c27002d5da753e193b0670 | 5aa9a3bd3d5582667915d93b0128a02953adbbbc | 7517af78f49a18dde6de86042055aa14ebe6d184 | +| IPU75XA | cead7320d84ee9ade4f60d74e935b16b5a760945 | ef30767553685b83034e42325992a2442c5fcb2c | 7517af78f49a18dde6de86042055aa14ebe6d184 | +| IPU8 | adf55525ab9d370828723b1ff8bee76ed7a492e8 | fd1efb6f2ef25fd24775a303cc71af6139d797eb | 7517af78f49a18dde6de86042055aa14ebe6d184 | -## Setup Procedure +
    + +#### Media Driver + +> **Note:** \ +> Build from source only if you are using **Canonical Kernel with PTL**. \ +> Skip this step if you are using **Intel BKC**. + +> **RECOMMENDED** if you want to have **DMABuf** support. + +
    +Show List + +| Component | Repository | Tag | Build Step | +|--------------|------------|---------|------------| +| libva | [libva](https://github.com/intel/libva) | 2.23.0 | Follow build steps in repo | +| gmmlib | [gmmlib](https://github.com/intel/gmmlib) | intel-gmmlib-22.10.0 | Follow build steps in repo | +| media-driver | [media-driver](https://github.com/intel/media-driver) | intel-media-26.1.5 | Follow build steps in repo | + +
    +

    (back to Setup Procedure)

    -### Kernel Driver DKMS Build +--- +### Software Setup - Kernel Driver DKMS Build -Initialize and update current repository recursively to ensure all dependencies are correctly fetched. +**Initialize** and update current repository recursively to ensure all dependencies are correctly fetched. git checkout main git submodule update --init --recursive -> **Note:** Workaround required for IPU7/IPU8 RAW sensor due to tuning dependency. +
    + Workaround for IPU7/IPU8 AR0234 RAW sensor + +> **Note:** Workaround required for IPU7/IPU8 **RAW sensor** due to tuning dependency. By default, `ipu7-drivers` submodule tracks the tip of its `main` branch. For IPU75XA, @@ -129,7 +244,9 @@ For IPU8, git checkout bed9b71d5be56d0e24c771c7a6ec83fd0a9db433 cd .. -Build and install modules using DKMS +
    + +**Build and install** modules using DKMS sudo dkms remove ipu-camera-sensor/0.1 sudo rm -rf /usr/src/ipu-camera-sensor-0.1/ @@ -138,23 +255,58 @@ Build and install modules using DKMS sudo dkms build -m ipu-camera-sensor -v 0.1 sudo dkms install -m ipu-camera-sensor -v 0.1 --force -### BIOS Configuration +

    (back to Setup Procedure)

    + +--- +### Hardware Setup + +1. Reference Hardware setup can be found in [doc/{sensor}](doc/) and respective `userspace.md` files. + - Hardware Connection for [Common GMSL setup](doc/isx031/userspace-gmsl.md#hardware-connection) + +

    (back to Setup Procedure)

    + +--- +### ACPI Setup - BIOS Configuration 1. Power cycle target system with sensors connected. -2. Import sensor profile into BIOS, under section `BIOS Configuration Table` in recommended userspace.md. - - E.g. ISX031 GMSL using [userspace-gmsl.md](doc/isx031/userspace-gmsl.md) - - E.g. AR0234 MIPI using [userspace-mipi.md](doc/ar0234/userspace-mipi.md) +2. Import sensor profile into BIOS, under section `BIOS Configuration Table` in respective userspace.md. + - E.g. ISX031 GMSL using [userspace-gmsl.md](doc/isx031/userspace-gmsl.md#acpi-setup---bios-configuration) + - E.g. AR0234 MIPI using [userspace-mipi.md](doc/ar0234/userspace-mipi.md#bios-configuration-table) + +

    (back to Setup Procedure)

    -### Camera Configuration File Setup +--- +### ACPI Setup - ASL Configuration -Setup camera configuration file under section `Camera Configuration File Setup` in recommended userspace.md. +ASL Configuration is added since release/26Q2.1. +1. Turn off all Camera related BIOS setting. +2. Create own ASL source file using reference ASL source file in `../../acpi/{ipu}`. Follow section [here](doc/acpi/kernelspace.md#reference-asl-source-files). +3. Compile and load ASL source into GRUB. Follow section [here](doc/acpi/kernelspace.md#compile-and-load) + +

    (back to Setup Procedure)

    + +--- ### Setup Verification -Verify sensor setup using `media-ctl`. +> **Note:** Minimum media-ctl version required = ([1.30](https://github.com/gjasny/v4l-utils/tree/stable-1.30)) + +1. Verify sensor setup using `media-ctl -p`. + +

    (back to Setup Procedure)

    -> **Note:** Minimum version required = ([1.30](https://github.com/gjasny/v4l-utils/tree/stable-1.30)) +--- +### Stream Verification + +1. Reference stream verification for different sensor setup can be found in [doc/{sensor}](doc/) and respective userspace.md files. + - E.g. ISX031 GMSL using [userspace-gmsl.md](doc/isx031/userspace-gmsl.md#stream-verification) + - E.g. AR0234 MIPI using [userspace-mipi.md](doc/ar0234/userspace-mipi.md#sample-userspace-command) + +

    (back to Setup Procedure)

    + +
    +--- ## Contributing Please read [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on contributing to this project. diff --git a/doc/acpi/kernelspace.md b/doc/acpi/kernelspace.md index 241f0efd..8b722a44 100644 --- a/doc/acpi/kernelspace.md +++ b/doc/acpi/kernelspace.md @@ -1,113 +1,296 @@ -## User Guide for ACPI SSDT ASL Compilation and Loading for Imaging Sensors - ## Description -This document contains information of imaging specific ACPI SSDT ASL sources compilation and loading. It also contains information of how to construct media-ctl pipeline for imaging sensors described in ACPI SSDT, and how to verify the streaming of sensors. +This document contains information of imaging specific ACPI SSDT ASL sources compilation and loading. -## Create ACPI ASL Source Files +## Table of Contents -ASL source file needs to be created based on current hardware setup using reference ASL source file in ../../acpi/{ipu}. -Below sections describe the common ASL source files for different use cases and IPU generations, and the required defines in ASL source file for deserializer, serializer and camera sensor. + +
    + Table of Contents +
      +
    1. ACPI Configuration for Camera Imaging
      2. +
    2. Legacy BIOS Method
      3. +
    3. SSDT ASL Method
      4. +
    4. Advantage of SSDT ASL Method
      5. +
    5. Requirement to use SSDT ASL method
      6. +
    6. What are ACPI ASL Source Files?
      7. +
    7. Reference ASL Source Files
      8. +
    8. ASL Source Files for Different Use Cases
      9. +
    9. Compile and Load ACPI ASL Source Files
      10. +
    +
    + -### Common ASL to be shared across different use cases and IPU generations -These ASL source files should be able to be reused for different use cases. +## ACPI Configuration for Camera Imaging -> **Note:** If there is a need to be modify common ASL, make sure the changes are generic enough to be reused for different use cases. +The ACPI namespace is a hierarchical representation of platform devices and resources exposed by the BIOS firmware, enabling the operating system to discover, configure, and manage hardware. -> **Note:** These ASL source files are **NOT** meant to be compiled into AML file directly. They should be included by use case specific ASL source files. +Both [Legacy BIOS method](#legacy-bios-method) and [SSDT ASL method](#ssdt-asl-method) create ACPI devices for camera imaging. There can be conflicts if both methods are used simultaneously, so it is recommended to use only one method at a time. -| ASL source file | Description | Usage | -| --- | --- | --- | -| _des_common_max96724.asl | Common ASL for MAX96724 deserializer | Included by caller ASL source file for each MAX96724 deserializer | -| _des_ch_common_isx031.asl | Common ASL for single Deserializer Channel / Link with ISX031 2D camera sensor | Included by caller ASL source file for each link with ISX031 sensor | -| _des_ch_common_d457.asl | Common ASL for single Deserializer Channel / Link with D457 3D camera sensor | Included by caller ASL source file for each link with D457 sensor | -| _ser_common_max9295.asl | Common ASL for MAX9295 serializer | Included by _des_ch_common_*.asl | -| _cam_common_isx031.asl | Common ASL for ISX031 2D camera sensor | Included by _des_ch_common_isx031.asl | -| _cam_common_d457.asl | Common ASL for D457 3D camera sensor | Included by _des_ch_common_d457.asl | +### Legacy BIOS method + +Using Legacy BIOS method, the camera imaging configuration is done through BIOS settings. BIOS has pre-defined fields and methods to store the configuration done on the BIOS menu page, which primarily caters to MIPI Direct camera setups. + +For the MIPI Direct case, the `ipu-bridge` driver will read the BIOS settings and create an FWnode for each camera ACPI device. The sensor is then probed through the ACPI framework and registered into the V4L2 subdev framework. + +For the GMSL case, the `ipu-acpi` driver will read the BIOS settings and create a static I2C platform data structure. The `ipu-driver` will then use the I2C platform data to probe `max9x` GMSL drivers and then sensor drivers through the I2C framework. The SerDes and sensors are then registered into the V4L2 subdev framework. + +Limitations: +This method primarily caters to MIPI Direct camera setups ONLY +To properly describe GMSL setup, existing fields are not sufficient and some fields need to be repurposed. +The mechanism for creating the I2C platform data structure is fixed in `ipu-acpi`, it can **ONLY** populate multiple **SAME sensors** on a single deserializer. + +### SSDT ASL method + +Using SSDT ASL method, the camera imaging configuration is done through ACPI ASL source files. The ASL source files describe the hardware configuration of the system. They are compiled into SSDT binary, packaged into an early initramfs image, and loaded by the kernel during boot as an ACPI table **override**. + +ASL files already have macros that describe the FWnode and I2C information for each camera device. The kernel `drivers/acpi/mipi-disco-img.c` will handle the FWnode creation. Sensor drivers and/or GMSL drivers will be probed through the ACPI framework and registered into the V4L2 subdev framework. + +

    IMPORTANT: DISABLE all camera related BIOS settings when using SSDT ASL method

    + +### Advantage of SSDT ASL method + +- Offers flexibility: ASL source files are configurable according to the current hardware setup. +- Advanced configuration: Supports mix-and-match configurations of different GMSL sensors on a single deserializer. +- Modularity: Users do not need to recompile the `ipu-acpi` driver to add new sensors. Removes the dependency on BIOS. + +### Requirement to use SSDT ASL method +- To compile ASL source files, you need to install `acpica` tools version 20260408 or later. Refer to [Compile and Load](#compile-and-load) section for details. +- GMSL setup enabled through this method requires **CONFIG_I2C_ATR** turned on in base kernel. If you are using Intel BKC, you might want to rebuild the kernel overlay with this config enabled. + +## What are ACPI ASL Source Files? + +ASL (ACPI Source Language) source files are used to describe the hardware configuration of the system to the operating system. In the context of imaging, these ASL source files define the configuration of camera sensors, (deserializers, serializers for GMSL setups), and their connections to the system's MIPI CSI ports. + +The ASL source files are compiled into AML (ACPI Machine Language) binary tables, which are then loaded by the operating system during boot. These tables provide the necessary information for the OS to correctly initialize and manage the imaging hardware. + +

    IMPORTANT: ASL source file must match or be subset of actual hardware setup. +Any mismatch between the ASL source files and the actual hardware can lead to failures in device registration and streaming.

    -### ASL source file for different use cases +The ASL source files are specific to the hardware configuration of the system. The common description of the hardware includes: +- PHY type +- MIPI Port +- Lane configuration +- I2C information (using I2cSerialBusV2) +- GPIO resource allocation (using GpioIo) +- FWnode linkage (using CSI2Bus) -The reference ASL source files are located in ../../acpi/{ipu}. Make sure below ASL source files is at least **subset** of your current hardware setup. -If you connect 4x 2D sensor with max96724, the ASL source file can have 1/2/3/4 channels. -If you only connect 1x 2D sensor with max96724, the ASL source file can only have 1 channel. +For GMSL setups, the description includes the number of sensors, and extended I2C information and FWnode between deserializers and each serializer-sensor module. -> **WARNING**: If ASL specified more than actual Hardware connection, or there is probe failure in any one of the links, the whole v4l2 subdev registration will fail, and subsequent streaming will not be able to work. +## Reference ASL Source Files -> There are ../../acpi/ipu6 , ../../acpi/ipu7 , ../../acpi/ipu8 folders. The files are mostly similar across different IPU generations, but with minor difference such as CPHY/DPHY for deserializer, or IPU MIPI port. Make sure to pick the right reference ASL source file based on your IPU generation. +The ASL source files in [acpi/](../../acpi/) are provided as a starting point for creating your own ASL source files based on your specific hardware setup. -| Use case | Reference ASL source file | +

    You should use these reference files to understand the structure and required defines, and then create your own ASL source files that accurately reflect your hardware configuration.

    + +
    + Hierarchy + +The hierarchy of the ASL Source Files for MIPI setup looks like this: + + Parent MIPI.asl (e.g. mipi_isx031.asl) + ├── _ipu.asl + └── _mipi_cam_common_*.asl + +The hierarchy of the ASL Source Files for GMSL setup looks like this: + + Parent GMSL.asl (e.g. max96724_*.asl) + ├── _ipu.asl + └── _des_common_*.asl + ├── _des_ch_common_*.asl + │ ├── _ser_common_*.asl + │ └── _cam_common_*.asl + └── _des_ch_common_*.asl + ├── _ser_common_*.asl + └── _cam_common_*.asl + +
    + +
    + Common ASL + +These Common ASL source files should be shared and reused across different use cases and IPU generations. They are **NOT** meant to be compiled into AML file directly and are included by parent ASL source files. They contain common definitions and methods that can be used by multiple ASL source files, reducing redundancy and improving maintainability. + +> **Note:** If there is a need to modify common ASL, make sure the changes are generic to be reused for different use cases. + +Future enhancements will further reduce redundancy and improve maintainability. + +| Common ASL | Description | | --- | --- | -| 1x 2D sensor | max96724_li_isx031.asl, max96724_sensing_isx031.asl | -| Nx 2D sensor | max96724_d3_isx031.asl, max96724_dphy_d3_isx031.asl, max9296_d3_isx031.asl | -| Nx 3D sensor | max96724_rs_d457.asl, max9296_rs_d457.asl | -| Nx 2D sensors and 3D sensor | max96724_mixed.asl, max9296_mixed.asl | +| _cam_common_ar0234.asl | Common ASL for AR0234 2D camera sensor | +| _cam_common_d457.asl | Common ASL for D457 3D GMSL camera sensor | +| _cam_common_isx031.asl | Common ASL for ISX031 2D GMSL camera sensor | +| _des_ch_common_ar0234.asl | Common ASL for single GMSL Link / Channel with AR0234 2D GMSL camera sensor | +| _des_ch_common_isx031.asl | Common ASL for single GMSL Link / Channel with ISX031 2D GMSL camera sensor | +| _des_ch_common_d457.asl | Common ASL for single GMSL Link / Channel with D457 3D camera sensor | +| _des_common_max96724.asl | Common ASL for MAX96724 deserializer | +| _des_common_max9296.asl | Common ASL for MAX9296 deserializer | +| _ipu.asl | Common ASL for IPU | +| _mipi_cam_common_isx031.asl | Common ASL for ISX031 2D MIPI camera sensor | +| _ser_common_max9295.asl | Common ASL for MAX9295 serializer | + +
    + +
    + ASL defines -### Deserializer Specific Defines in ASL +This section is purely informative to describe the ASL defines used in the reference ASL source files. The defines are expected to be defined in the caller ASL source file, and will be used by the common ASL source files. The defines are grouped into 4 categories: Deserializer specific, Channel specific, Serializer specific, and Sensor specific. -Deserializer specific defines are used to describe the Connection to Board and SOC. These defines are expected to be defined in the caller ASL source file for each deserializer, and will be used by the common ASL source files. Below table describes the required defines for deserializer, and the correlation to legacy BIOS setting if applicable. +
    +Deserializer Specific Defines + +### Deserializer Specific Defines + +Deserializer-specific defines are used to describe the connection to the board and SOC. These defines are expected to be defined in the caller ASL source file for each deserializer and will be used by the common ASL source files. The table below describes the required defines for a deserializer and the correlation to the legacy BIOS setting, if applicable. | Deserializer Define | Description | Value | Correlate to legacy BIOS setting | | --- | --- | --- | --- | -| DES_PHY_TYPE | PHY connection to Board | 0 for CPHY, 1 for DPHY | PhyConfiguration | -| DES_I2C_ADDR | I2C address of the deserializer | 0x0027 for MAX96724, 0x0048 for MAX9296 | I2C Device 0 | -| DES_LANES | Number of lanes used by the deserializer | 2 or 4 depending on use case | Ppr Value | -| DES_INTERNAL_PHY | Internal PHY + 4 | 4/5/6/7 for PHY0/1/2/3 | Rotation (Rotation/90 + 4) | -| DES_TO_MIPI_PORT | Connected to MIPI port of SOC | 0/1/2/3/4/5 based on SOC and Hardware design | MIPI Port | -| DES_I2C_BUS | I2C bus number for deserializer | "\\_SB.PC00.I2Cx" | I2C Channel | -| DES_PATH | ACPI Path for Deserializer | "\\_SB.PC00.DESx" | - | -| DES_REF | ACPI Reference for Deserializer | \_SB.PC00.DESx | - | +| DES_PHY_TYPE | PHY connection to Board | 0 for CPHY, 1 for DPHY | PhyConfiguration | +| DES_I2C_ADDR | I2C address of the deserializer | 0x0027 for MAX96724, 0x0048 for MAX9296 | I2C Device 0 | +| DES_LANES | Number of lanes used by the deserializer | 2 or 4 depending on use case | Ppr Value | +| DES_INTERNAL_PHY | Internal PHY + 4 | 4/5/6/7 for PHY0/1/2/3 | Rotation (Rotation/90 + 4) | +| DES_TO_MIPI_PORT | Connected to MIPI port of SOC | 0/1/2/3/4/5 based on SOC and Hardware design | MIPI Port | +| DES_I2C_BUS | I2C bus number for deserializer | "\\_SB.PC00.I2Cx" | I2C Channel | +| DES_PATH | ACPI Path for Deserializer | "\\_SB.PC00.DESx" | - | +| DES_REF | ACPI Reference for Deserializer | \_SB.PC00.DESx | - | | DES_PIPE_STR_AUTOSELECT | (Optional) Setting for MAX96724 to configure pipe, useful for 3D camera | 0 to disable fixed pipe, 1 to enable fixed pipe (default) | - | -### Channel Specific Defines in ASL +
    + +
    +Channel Specific Defines + +### Channel Specific Defines -Channel specific defines are common for all links, with index increment. For example, if there are 4 deserializer links, the ASL source file should have 4 sets of channel specific defines with link number from 0 to 3. These defines are expected to be defined in the caller ASL source file for each link, and will be used by the common ASL source files. Below table describes the required defines for each channel. +Channel-specific defines are common for all links, with an incrementing index. For example, if there are four deserializer links, the ASL source file should have four sets of channel-specific defines with link numbers from 0 to 3. These defines are expected to be defined in the caller ASL source file for each link and will be used by the common ASL source files. The table below describes the required defines for each channel. | Channel Define | Description | Value | | --- | --- | --- | -| DESCH_LINK_NUM | Link number for current channel | 0/1/2/3 | -| DESCH_CH | Channel Device for current link | CH00/CH01/CH02/CH03 | -| DESCH_SER | Serializer Device for current link | SER0/SER1/SER2/SER3 | -| DESCH_CAM | Camera Device for current link | CAM0/CAM1/CAM2/CAM3 | -| DESCH_CH_PATH | ACPI Path for Channel Device | "\\_SB.PC00.DESx.CH0x" | -| DESCH_SER_PATH | ACPI Path for Serializer Device | "\\_SB.PC00.DESx.SER0/1/2/3" | -| DESCH_SER_REF | ACPI Reference for Serializer Device | \_SB.PC00.DESx.SER0/1/2/3 | +| DESCH_LINK_NUM | Link number for current channel | 0/1/2/3 | +| DESCH_CH | Channel Device for current link | CH00/CH01/CH02/CH03 | +| DESCH_SER | Serializer Device for current link | SER0/SER1/SER2/SER3 | +| DESCH_CAM | Camera Device for current link | CAM0/CAM1/CAM2/CAM3 | +| DESCH_CH_PATH | ACPI Path for Channel Device | "\\_SB.PC00.DESx.CH0x" | +| DESCH_SER_PATH | ACPI Path for Serializer Device | "\\_SB.PC00.DESx.SER0/1/2/3" | +| DESCH_SER_REF | ACPI Reference for Serializer Device | \_SB.PC00.DESx.SER0/1/2/3 | | DESCH_SER_GPIOREF | ACPI Reference for Serializer GPIO | ^^SER0/1/2/3 | -| CAM_ALIAS | Sensor Alias address | 0x54/0x55/0x56/0x57 depends on hardware design. | +| CAM_ALIAS | Sensor Alias address | 0x54/0x55/0x56/0x57 depends on hardware design. | -### Serializer Specific Defines in ASL +
    -Serializer Specific Defines in ASL is used to specify the configuration for serializer. These defines are expected to be defined in the channel specific ASL source file for each link with serializer, and will be used by the common serializer ASL source file. Below table describes the required defines for serializer, and the correlation to legacy setup if applicable. +
    +Serializer Specific Defines + +### Serializer Specific Defines + +Serializer-specific defines in ASL are used to specify the serializer configuration. These defines are expected to be defined in the channel-specific ASL source file for each link with a serializer and will be used by the common serializer ASL source file. The table below describes the required defines for a serializer and the correlation to the legacy setup, if applicable. | Serializer Define | Description | Value | Correlate to legacy setup | | --- | --- | --- | --- | -| DESCH_SER_I2C | I2C address of the serializer | 0x40 or 0x62 based on MAX9295 design | ser_physical_addr in ipu-acpi.c | +| DESCH_SER_I2C | I2C address of the serializer | 0x40 or 0x62 based on MAX9295 design | ser_physical_addr in ipu-acpi.c | | DESCH_SER_EXTRA_GPIO_PIN | (Optional) Extra MFP pin to be configured in Serializer | 1-11 for for mfp1-11 in max9295 | ser_gpio.chip_hwnum in ipu-acpi.c | -| DESCH_SER_X_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe X | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe X | - | -| DESCH_SER_Y_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe Y | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe Y | - | -| DESCH_SER_Z_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe Z | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe Z | - | -| DESCH_SER_U_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe U | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe U | - | +| DESCH_SER_X_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe X | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe X | - | +| DESCH_SER_Y_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe Y | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe Y | - | +| DESCH_SER_Z_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe Z | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe Z | - | +| DESCH_SER_U_VC | (Optional) Setting for max9295 serializer to configure VC filter for Pipe U | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe U | - | + +
    -### Sensor Specific Defines in ASL +
    +Sensor Specific Defines + +### Sensor Specific Defines | Sensor Define | Description | Value | Correlate to legacy setup | | --- | --- | --- | --- | -| CAM_LANES | Number of lanes used by the camera sensor | 2 or 4 depending on use case | LaneUsed in BIOS | +| CAM_LANES | Number of lanes used by the camera sensor | 2 or 4 depending on use case | LaneUsed in BIOS | | EXTERNAL_FRAME_SYNC | (Optional) Use of FSIN GPIO. Requires an external pulse supplied to deserializer MFP pin | 1 if needed (DESCH_SER_EXTRA_GPIO_PIN and DES_FSIN_GPIO_PIN should also be defined in this case), 0 (default) | - | -#### Sample values for different models of sensors +#### Sample values for different sensor models | Sensor Model | DESCH_SER_I2C | DESCH_SER_EXTRA_GPIO_PIN | EXTERNAL_FRAME_SYNC | CAM_LANES | DESCH_SER_X/Y/Z/U_VC | | --- | --- | --- | --- | --- | --- | | D3 ISX031 | 0x40 | - | - | 4 | - | | LI ISX031 | 0x62 | - | - | 4 | - | | Sensing ISX031| 0x40 | 7 | 1 | 4 | - | -| RS D457 | 0x40 | - | - | 2 | Package () { 0/1/2/3 } for VC0/1/2/3 on Pipe X/Y/Z/U | +| RS D457 | 0x40 | - | - | 2 | Package () { 0 } for VC0 on Pipe X
    Package () { 1 } for VC1 on Pipe Y
    Package () { 2 } for VC2 on Pipe Z
    Package () { 3 } for VC3 on Pipe U | + +
    + +
    + +## ASL Source Files for Different Use Cases + +The reference ASL source files are located in [../../acpi/](../../acpi/) and are grouped according to the platform. The files only have minor differences such as PHY type and MIPI port. + +Make sure the below ASL source files are at least a **subset** of your current hardware setup. + +> Example 1: If you connected sensors on all 4 links with max96724, the ASL source file can have any 1/2/3/4 channels. + +> Example 2: If you only connected on 1 link with max96724, the ASL source file can have only 1 channel, and must be that specific channel. + +> **WARNING**: \ +> If ASL specified more than actual Hardware connection, or there is probe failure in any one of the links, the whole v4l2 subdev registration will fail, and subsequent streaming will not be able to work. + +
    + IPU6EPMTL + +- [ISX031 MIPI YUV](../../acpi/ipu6/mipi_isx031.asl) +- [AR0234 GMSL RAW on MAX9296](../../acpi/ipu6/max9296_d3_ar0234.asl) +- [ISX031 GMSL YUV on MAX9296](../../acpi/ipu6/max9296_d3_isx031.asl) +- [ISX031 GMSL YUV on MAX96724 DPHY](../../acpi/ipu6/max96724_dphy_d3_isx031.asl) +- [D457 GMSL 3D on MAX9296](../../acpi/ipu6/max9296_rs_d457.asl) +- [LI ISX031 GMSL on MAX9296](../../acpi/ipu6/max9296_li_isx031.asl) +- [Sensing ISX031 GMSL on MAX9296](../../acpi/ipu6/max9296_sensing_isx031.asl) +- [2D+2D, 2D+3D GMSL mix-and-match on MAX9296](../../acpi/ipu6/max9296_mixed.asl) + +

    (Go to Compile and Load)

    + +
    + +
    + IPU75XA +- [ISX031 MIPI YUV](../../acpi/ipu7/mipi_isx031.asl) +- [AR0234 GMSL RAW on MAX96724](../../acpi/ipu7/max96724_d3_ar0234.asl) +- [ISX031 GMSL YUV on MAX96724](../../acpi/ipu7/max96724_d3_isx031.asl) +- [ISX031 GMSL YUV on MAX96724 DPHY](../../acpi/ipu7/max96724_dphy_d3_isx031.asl) +- [D457 GMSL 3D on MAX96724](../../acpi/ipu7/max96724_rs_d457.asl) +- [LI ISX031 GMSL on MAX96724](../../acpi/ipu7/max96724_li_isx031.asl) +- [Sensing ISX031 GMSL on MAX96724](../../acpi/ipu7/max96724_sensing_isx031.asl) +- [2D+2D, 2D+3D GMSL mix-and-match on MAX96724](../../acpi/ipu7/max96724_mixed.asl) +- [FrameSync on MAX96724](../../acpi/ipu7/max96724_sensing_isx031_fs.asl) + +

    (Go to Compile and Load)

    + +
    + +
    + IPU8 + +- [ISX031 MIPI YUV](../../acpi/ipu8/mipi_isx031.asl) +- [AR0234 GMSL RAW on MAX96724](../../acpi/ipu8/max96724_d3_ar0234.asl) +- [ISX031 GMSL YUV on MAX96724](../../acpi/ipu8/max96724_d3_isx031.asl) +- [ISX031 GMSL YUV on MAX96724 DPHY](../../acpi/ipu8/max96724_dphy_d3_isx031.asl) +- [D457 GMSL 3D on MAX96724](../../acpi/ipu8/max96724_rs_d457.asl) +- [LI ISX031 GMSL on MAX96724](../../acpi/ipu8/max96724_li_isx031.asl) +- [Sensing ISX031 GMSL on MAX96724](../../acpi/ipu8/max96724_sensing_isx031.asl) +- [2D+2D, 2D+3D GMSL mix-and-match on MAX96724](../../acpi/ipu8/max96724_mixed.asl) + +

    (Go to Compile and Load)

    + +
    + +

    (Go to Compile and Load)

    + +## Compile and Load + +This section contains steps to compile your ASL source files into AML binary tables, and load them into the kernel during boot. The steps are as follows: + +### Prerequisite on Canonical Ubuntu 24.04 or 26.04: -### Compile and Load ACPI ASL source on Canonical Ubuntu 24.04 or 26.04 -Pre-requisite: sudo apt-get install flex bison -Install acpica tools version 20260408 https://github.com/acpica/acpica/releases/tag/20260408 +Install acpica tools version [20260408](https://github.com/acpica/acpica/releases/tag/20260408) wget https://github.com/acpica/acpica/releases/download/20260408/acpica-unix-20260408.tar.gz tar zxf ./acpica-unix-20260408.tar.gz @@ -115,22 +298,30 @@ Install acpica tools version 20260408 https://github.com/acpica/acpica/releases/ make sudo make install +### Compile ASL source file + Run helper script to generate initramfs image from ASL source file and copy to /boot - ../../script/gen_ssdt.sh ../../acpi/{ipu}/sensor.asl + ../../script/gen_ssdt.sh ../../acpi/{create-your-own.asl} -Add following line to /etc/default/grub for GRUB to load SSDT initramfs. Update and reboot. Make sure all camera related BIOS setting are disabled. +### Load SSDT initramfs + +Add the following line to /etc/default/grub for GRUB to load the SSDT initramfs. Update and reboot. Make sure all camera-related BIOS settings are disabled. echo 'GRUB_EARLY_INITRD_LINUX_CUSTOM="img_ssdt.img"' | sudo tee -a /etc/default/grub sudo update-grub sudo reboot ->To revert back to Legacy setup using BIOS and ipu-acpi, remove below line from in /etc/default/grub, update-grub and reboot. +#### Unload SSDT initramfs + +To revert to the legacy setup using BIOS and ipu-acpi, remove the line below from /etc/default/grub, run update-grub, and reboot. sudo sed -i '/GRUB_EARLY_INITRD_LINUX_CUSTOM/d' /etc/default/grub sudo update-grub sudo reboot +### SSDT Verification + Inspect loading of SSDT in kernel dmesg dmesg | grep SSDT @@ -141,3 +332,8 @@ You should see log similar to below: ... [ 0.009609] ACPI: Table Upgrade: install [SSDT- - IMG_IPU] [ 0.009611] ACPI: SSDT 0x00000000678E6000 00143D (v02 IMG_IPU 20260513 INTL 20250404) + + +[Proceed to userspace-gmsl.md](userspace-gmsl.md) for media-ctl pipeline construction and sensor streaming verification. + +

    (back to top)

    diff --git a/doc/acpi/userspace-gmsl.md b/doc/acpi/userspace-gmsl.md index d62c34fe..e5144e61 100644 --- a/doc/acpi/userspace-gmsl.md +++ b/doc/acpi/userspace-gmsl.md @@ -4,88 +4,685 @@ This document details the steps to validate ACPI-enumerated GMSL sensors, provid For Kernel space ASL source file creation and compilation, please refer to [kernelspace.md](./kernelspace.md). -## Construct pipeline using mc-setup.sh +## Table of Contents -Instead of using XML or JSON file to describe the media-ctl pipeline, and let libcamhal to perform the configuration, mc-setup script is provided to directly construct media-ctl pipeline based on the current ACPI enumeration based on ASL source file. This way, libcamhal config files (xml or json) can be generic without the naming and topology dependency, and can be reused for different hardware setup as long as mc-setup is maintained properly and extended when there is new hardware or configuration need. +- [Introduction to ASL configured system](#introduction-to-asl-configured-system) + - [new mc-setup script](#new-mc-setup-script) + - [new libcamhal configuration files](#new-libcamhal-configuration-files) +- [How to use mc-setup.sh](#how-to-use-mc-setupsh) + - [Construct pipeline](#construct-pipeline) + - [Sample Commands](#sample-commands) + - [Advanced - Per stream Configuration](#advanced---per-stream-configuration) +- [Stream verification](#stream-verification) + - [Sanity Streaming Test using v4l2-ctl](#sanity-streaming-test-using-v4l2-ctl) + - [GStreamer streaming using v4l2src](#gstreamer-streaming-using-v4l2src) + - [GStreamer streaming using icamerasrc](#gstreamer-streaming-using-icamerasrc) -Currently mc-setup only supports d4xx and isx031 GMSL sensors, max9295a serializer, max96724 and max9296 deserializer. Feel free to extend mc-setup.sh for other sensors or different use case based on current implementation. The script is located in ../../script/acpi/mc-setup.sh. Please refer to the script for more details of the implementation and usage. Below sections describe how to construct media-ctl pipeline for 2D YUV sensors and 3D sensors, and how to verify the streaming of sensors. +## Introduction to ASL configured system -Provided that ACPI enumeration is successful, just run below command to construct media-ctl pipeline for all available links on all available deserializers. +### new mc-setup script +When using an ASL-configured system, the I2C bus enumerated by the GMSL drivers is dynamic and can differ on every boot. If the legacy method of creating a static media-ctl pipeline in the libcamhal configuration is used, the pipeline will not be valid on every boot and sensor streaming will fail. To address this issue, a new script called **mc-setup.sh** is introduced to dynamically construct the media-ctl pipeline based on the current ACPI enumeration. mc-setup.sh uses the ACPI path of each hardware component (deserializer, serializer, sensor) to construct the media-ctl pipeline. + +> 2D sensors refer to sensors that only have single stream, such as ISX031 (YUV), AR0234 (RAW Bayer). + +> 3D sensors refer to sensors that have multiple streams, such as D457 (Depth, RGB, IR, IMU). + +
    + Default Stream for 2D and 3D sensors + +| Sensor | Default Stream | +| --- | --- | +| AR0234 | Single | +| D457 | Depth+RGB | +| ISX031 | Single | + +The script can be used for MIPI and GMSL sensors. There are default streams for each sensor type. For 2D sensors, the default stream is the only stream available; for 3D sensors, the default stream is Depth+RGB. + +
    + +
    + Supported Sensors + +GMSL Deserializers: MAX9296A, MAX96724 +GMSL Serializers: MAX9295A +GMSL 2D RAW sensors: AR0234 +GMSL 2D YUV sensors: ISX031 +GMSL 3D YUV sensors: D457 + +MIPI 2D YUV sensors: ISX031 +MIPI 2D RAW sensors: OV13B10 + +
    + +

    ( How to use mc-setup.sh)

    + +### new libcamhal configuration files + +> 2D YUV sensors (e.g. ISX031 GMSL) : acpi.xml or acpi.json + +A new libcamhal configuration file called **acpi.xml or acpi.json** is created. It is designed to be sensor agnostic so it can be reused for different 2D YUV sensors, provided that mc-setup is maintained properly and extended when new hardware or configuration needs arise. + +> 2D RAW sensors (e.g. AR0234 GMSL) : [ar0234.xml](../../config/ar0234/ipu6epmtl/sensors/ar0234.xml) + +For 2D RAW sensor that requires 3A and PSYS support, it will require dedicated Graph Setting and Tuning Files, hence a dedicated libcamhal config file is needed instead of sharing acpi.xml or acpi.json with YUV sensors. + +> 3D YUV sensors (e.g. D457 GMSL) : d4xx.xml or d4xx.json + +A new libcamhal configuration file called **d4xx.xml or d4xx.json** is created. It is **NOT** sensor agnostic as 3D sensors might have different subdev and streams supported. + +## How to use mc-setup.sh + +Prerequisites: + +1. ACPI ASL created correctly. +2. ACPI ASL loaded correctly. +3. Kernel driver probed successfully. + +### Construct pipeline + +Before elaborating, here are some definitions of 2D and 3D sensors and the kinds of configurations that mc-setup.sh can support. + +2D sensors are sensors that only have single stream, such as ISX031 (YUV), AR0234 (RAW Bayer). \ +3D sensors are sensors that have multiple streams, such as D457 (Depth, RGB, IR, IMU). \ +Default streams for 3D sensors are **Depth+RGB** unless specified. + +There are **limitations** of only having 4 internal pipes for deserializer, for both MAX9296 and MAX96724. Hence, **total streams cannot exceed 4 streams per deserializer**. This limitation will be observed when there are 3D sensors involved in one of the links. + +Examples below will use MAX96724 as example. + +**2D setup** + +- DES{N} + - Link 0 - 2D sensor (model A) + - Link 1 - 2D sensor (model A) + - Link 2 - 2D sensor (model A) + - Link 3 - 2D sensor (model A) + +**3D setup** + +- DES{N} + - Link 0 - 3D sensor (model A) + - Link 1 - 3D sensor (model A) + +**2D+2D setup (mix-and-match)** + +- DES{N} + - Link 0 - 2D sensor (model A) + - Link 1 - 2D sensor (model B) + - Link 2 - 2D sensor (model C) + - Link 3 - 2D sensor (model D) + +**2D+3D setup (mix-and-match)** + +- DES{N} + - Link 0 - 2D sensor (model A) + - Link 1 - 2D sensor (model B) + - Link 2 - 3D sensor (model C) + +#### Sample Commands + +
    + All default streams + +### Command ../../script/acpi/mc-setup.sh -### Construct pipeline for 2D YUV sensors +### Output + +>2D setup + + Configuration summary: + DES0 LINK0 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video0 + DES0 LINK1 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video1 + DES0 LINK2 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video2 + DES0 LINK3 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video3 + +>3D setup + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + DES0 LINK1 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video1 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video5 + DES1 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video16 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video20 + DES1 LINK1 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video17 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video21 + +>2D+2D setup + + Configuration summary: + DES0 LINK0 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video0 + DES0 LINK1 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video1 + DES0 LINK2 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video2 + DES0 LINK3 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video3 + +>2D+3D setup + + Configuration summary: + DES1 LINK0 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video16 + DES1 LINK1 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video17 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + +
    + +
    + Selected Deserializer + +### Command + +> command 1: \ +DES0, all links, default streams + + ../../script/acpi/mc-setup.sh des=0 + +> command 2: \ +DES1, all links, default streams + + ../../script/acpi/mc-setup.sh des=1 + +### Output + +> DES0 2D+2D, DES1 2D+3D setup + + # command 1 + Configuration summary: + DES0 LINK0 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video0 + DES0 LINK1 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video1 + DES0 LINK2 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video2 + DES0 LINK3 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video3 + + # command 2 + Configuration summary: + DES1 LINK0 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video16 + DES1 LINK1 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video17 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + + +
    + +
    + Selected links (default stream) + +### Command + +> command 1: \ +DES0, \ +link 0 + + ../../script/acpi/mc-setup.sh des=0,link=0 + +> command 2: \ +DES0, \ +link 1 \ +link 2 + + ../../script/acpi/mc-setup.sh des=0,link=1 des=0,link=2 + +> command 3: \ +DES0, \ +link 2 \ +DES1, \ +link 2 + + ../../script/acpi/mc-setup.sh des=0,link=2 des=1,link=2 + +### Output + +> DES0 2D+2D, \ +DES1 2D+3D setup + + # command 1: + Configuration Summary: + DES0 LINK0 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video0 + + # command 2 + Configuration Summary: + DES0 LINK1 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video1 + DES0 LINK2 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video2 + + # command 3 + Configuration Summary: + DES0 LINK2 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video2 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + +
    + +
    + Selected Stream from Selected Link - 3D ONLY + +### Command + +> command 1: \ +DES1, \ +link 2 (3D ONLY), \ +\- Depth + + ../../script/acpi/mc-setup.sh des=1,link=2,stream=depth + +> command 2: \ +DES1, \ +link 2 (3D ONLY), \ +\- Depth \ +\- RGB + + ../../script/acpi/mc-setup.sh des=1,link=2,stream=depth,rgb + +> command 3: \ +DES1, \ +link 2 (3D ONLY), \ +\- Depth \ +\- RGB \ +\- IR \ +\- IMU + + ../../script/acpi/mc-setup.sh des=1,link=2,stream=depth,rgb,ir,imu + +> command 4: \ +DES1, \ +link 0 (2D), \ +\- YUV \ +link 2 (3D ONLY), \ +\- Depth \ +\- RGB \ +\- IR + + ../../script/acpi/mc-setup.sh des=1,link=0 des=1,link=2,stream=depth,rgb,ir + +### Output +> DES1 2D+3D setup + + # command 1: + Configuration summary: + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + + # command 2: + Configuration summary: + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + + # command 3: + Configuration summary: + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video20 + Stream [imu] 38x1 Y8_1X8 50.000 FPS --> /dev/video16 + + # command 4: + Configuration summary: + DES1 LINK0 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video16 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video20 + +
    + +### Advanced - Per stream Configuration + +>**IMPORTANT**: \ +Make sure you are familiar with how to select different links and/or streams from the previous section. This section only focus on how to configure the streams for selected link(s) and/or stream(s). + +
    + Change Resolution + +>IMPORTANT: For supported resolution and format, please refer to ../{sensor}/userspace-{interface}.md for each sensor if applicable. + +Commands in this section only change the resolution of the stream, if applicable. + +### Command + +> command 1: \ +DES0, \ +link 0 (2D) -> 1280x720 + + ../../script/acpi/mc-setup.sh des=0,link=0,res=1280x720 + +> command 2: \ +DES0, \ +link 0 (2D) -> 1280x720 \ +link 1 (2D) -> reuse current + + ../../script/acpi/mc-setup.sh des=0,link=0,res=1280x720 des=0,link=1 + +> command 3: \ +DES1, \ +link 2 (3D ONLY), \ +\- Depth stream -> 1280x720 \ +\- RGB stream -> reuse current \ +\- IR stream -> 1280x720. + + ../../script/acpi/mc-setup.sh des=1,link=2,stream=[depth,res=1280x720],[rgb],[ir,res=1280x720] + +> command 4: \ +DES1, \ +link 1 (2D) -> 1920x1080 \ +link 2 (3D ONLY), \ +\- Depth stream -> 1280x720 \ +\- RGB stream -> default \ +\- IR stream -> 1280x720 + + ../../script/acpi/mc-setup.sh \ + des=1,link=1,res=1920x1080 \ + des=1,link=2,stream=[depth,res=1280x720],[rgb],[ir,res=1280x720] + +### Output + +> DES0 2D+2D, \ +DES1 2D+3D setup + + # command 1: + Configuration summary: + DES0 LINK0 Sensor model isx031 + Stream [single] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + + # command 2: + Configuration summary: + DES0 LINK0 Sensor model isx031 + Stream [single] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + DES0 LINK1 Sensor model isx031 + Stream [single] 1920x1536 UYVY8_1X16 30.000 FPS --> /dev/video1 + + # command 3: + Configuration summary: + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video20 + + # command 4: + Configuration summary: + DES1 LINK1 Sensor model isx031 + Stream [single] 1920x1080 UYVY8_1X16 60.000 FPS --> /dev/video17 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video20 + +
    + +
    + Change Format -| Use Case | Sample command | -| --- | --- | -| all streams on all available links on all available DES | ../../script/acpi/mc-setup.sh | -| all streams on all available links on particular DES | DES_BUSADDR=2-0027 ../../script/acpi/mc-setup.sh | -| single stream on single link on DES0 | ../../script/acpi/mc-setup.sh des=0,link=0,stream=yuv | +Commands in this section only change the format of the stream, if applicable. -### Construct pipeline for 3D sensors +### Command -| Use Case | Sample command | -|--- | --- | -| Depth+RGB on all available links on all available DES | ../../script/acpi/mc-setup.sh | -| Depth+RGB on all available links on particular DES | DES_BUSADDR=2-0027 ../../script/acpi/mc-setup.sh | -| Depth+RGB on single link on DES0 | ../../script/acpi/mc-setup.sh des=0,link=0,stream=depth,rgb | -| all streams on single link on DES0 | ../../script/acpi/mc-setup.sh des=0,link=0,stream=depth,rgb,ir,imu | -| 2x RGB streams on link0,link1 on DES0 | ../../script/acpi/mc-setup.sh des=0,link=0,stream=rgb des=0,link=1,stream=rgb | -| 4x RGB stream on link0,link1 on DES0,DES1 | ../../script/acpi/mc-setup.sh des=0,link=0,stream=rgb des=0,link=1,stream=rgb des=1,link=0,stream=rgb des=1,link=1,stream=rgb | +> command 1: \ +DES0, \ +link 0 -> UYVY8_1X16 ->**Note:** MAX96724 only supports 4 active routing (matching with 4 internal pipes) at the same time, so maximum only 4 streams can be enabled on each DES, and by default it is 2 streams (Depth+RGB) from each D457 (assuming 2x D457 connected on each DES). + ../../script/acpi/mc-setup.sh des=0,link=0,format=UYVY8_1X16 -Below is the sample output of mc-setup.sh for 2x D457 sensors on DES0, 2x D457 sensors on DES1. -From there it shows that which stream from the sensor can be captured from which /dev/videoX node, which user can get the frames using v4l2-ctl, v4l2src, icamerasrc or any other v4l2 compatible application. +> command 2: \ +DES1, \ +link 1 (2D), -> reuse current \ +link 2 (3D ONLY), \ +\- Depth -> FIXED -![mc-setup.sh output](../d4xx/mc-setup-output.png) + ../../script/acpi/mc-setup.sh des=1,link=1 \ + des=1,link=2,stream=depth,format=FIXED -## Sensor stream verification +> command 3: \ +DES1, \ +link 2 (3D ONLY), \ +\- Depth stream -> UYVY8_1X16 \ +\- RGB stream -> YUYV8_1X16 \ +\- IR stream -> Y8_1X8 \ +\- IMU stream -> Y8_1X8 + + ../../script/acpi/mc-setup.sh des=1,link=2,stream=[depth,format=UYVY8_1X16],[rgb,format=YUYV8_1X16],[ir,format=Y8_1X8],[imu,format=Y8_1X8] + +### Output + +> DES1 2D+3D setup + + # command 1: + Configuration summary: + DES0 LINK0 Sensor model isx031 + Stream [single] 1280x720 UYVY8_1X16 n/a FPS --> /dev/video0 + + # command 2: + Configuration summary: + DES1 LINK1 Sensor model isx031 + Stream [single] 1920x1080 UYVY8_1X16 60.000 FPS --> /dev/video17 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 FIXED 30.000 FPS --> /dev/video18 + + # command 3: + Configuration summary: + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video20 + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video16 + +
    + +
    + Change FPS + +Commands in this section only change the FPS of the stream, if applicable. + +### Command + +> command 1: \ +DES1, \ +link 0 -> 60fps + + ../../script/acpi/mc-setup.sh des=1,link=0,fps=60 + +> command 2: \ +DES1, \ +link 1 (2D), -> reuse current \ +link 2 (3D ONLY), \ +\- Depth -> 5fps + + ../../script/acpi/mc-setup.sh des=1,link=1 \ + des=1,link=2,stream=depth,fps=5 + +> command 3: \ +DES1, \ +link 2 (3D ONLY), \ +\- Depth stream -> 30fps \ +\- RGB stream -> 5fps \ +\- IR stream -> 30fps \ +\- IMU stream -> 400fps + + ../../script/acpi/mc-setup.sh des=1,link=2,stream=[depth,fps=30],[rgb,fps=5],[ir,fps=30],[imu,fps=400] + +### Output + +> 2D+3D setup + + # command 1: + Configuration summary: + DES1 LINK0 Sensor model isx031 + Stream [single] 1920x1080 UYVY8_1X16 60.000 FPS --> /dev/video16 + + # command 2: + Configuration summary: + DES1 LINK1 Sensor model isx031 + Stream [single] 1920x1080 UYVY8_1X16 60.000 FPS --> /dev/video17 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 5.000 FPS --> /dev/video18 + + # command 3: + Configuration summary: + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 5.000 FPS --> /dev/video22 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video20 + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video16 + +
    + +
    + Mixed Combination to configure Resolution, Format and FPS + +Commands in this section show mixed combinations for configuring resolution, format, and FPS. +The commands **combine the tokens** from the previous sections into the same command. + +### Command + +> command 1: \ +DES1, \ +link 0 -> 1920x1080 @ 60fps + + ../../script/acpi/mc-setup.sh des=1,link=0,res=1920x1080,fps=60 + +> command 2: \ +DES1, \ +link 1 (2D), -> UYVY8_1X16, 1920x1080 @ 60fps \ +link 2 (3D ONLY), \ +\- Depth -> FIXED, 1280x720 @ 5fps + + ../../script/acpi/mc-setup.sh des=1,link=1,format=UYVY8_1X16,res=1920x1080,fps=60 \ + des=1,link=2,stream=depth,format=FIXED,res=1280x720,fps=5 + +> command 3: \ +DES1, \ +link 2 (3D ONLY), \ +\- Depth stream -> UYVY8_1X16, 1280x720 @ 30fps \ +\- RGB stream -> YUYV8_1X16, 1280x800 @ 30fps \ +\- IR stream -> Y8_1X8, 1280x720 @ 30fps \ +\- IMU stream -> Y8_1X8, 38x1 @ 400fps + + ../../script/acpi/mc-setup.sh des=1,link=2,stream=[depth,format=UYVY8_1X16,res=1280x720,fps=30],[rgb,format=YUYV8_1X16,res=1280x800,fps=30],[ir,format=Y8_1X8,res=1280x720,fps=30],[imu,format=Y8_1X8,res=38x1,fps=400] + +### Output + +> DES1 2D+3D setup + + # command 1: + Configuration summary: + DES1 LINK0 Sensor model isx031 + Stream [single] 1920x1080 UYVY8_1X16 60.000 FPS --> /dev/video16 + + # command 2: + Configuration summary: + DES1 LINK1 Sensor model isx031 + Stream [single] 1920x1080 UYVY8_1X16 60.000 FPS --> /dev/video17 + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 FIXED 5.000 FPS --> /dev/video18 + + # command 3: + Configuration summary: + DES1 LINK2 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video18 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video22 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video20 + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video16 + +
    + + + +--- +## Stream verification ### Sanity Streaming Test using v4l2-ctl ->Pro: v4l2-ctl can be used directly once mc-setup is completed; it is also more lightweight than v4l2src since it does not require GStreamer. +>**Pro:** \ +v4l2-ctl can be used directly once mc-setup is completed; it is also more lightweight than v4l2src since it does not require GStreamer. + +>**Con:** \ +no preview window; it only shows streaming status in terminal. + +#### Sample Command ->Con: no preview window; it only shows streaming status in terminal. + v4l2-ctl -d /dev/video{X} --stream-mmap --stream-count=150 -#### Sample Command for v4l2-ctl +#### Sample output -v4l2-ctl -d /dev/video{X} --stream-mmap + v4l2-ctl -d /dev/video5 --stream-mmap --stream-count=150 + <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 29.99 fps + <<<<<<<<<<<<<<<<<<<<<<<<<<<< 29.99 fps + <<<<<<<<<<<<<<<<<<<<<<<<<<<< 29.99 fps + <<<<<<<<<<<<<<<<<<<<<<<<<<<< 29.99 fps + <<<<<<<<<<<<<<<<<<<<<<<<<<<< 29.99 fps + <<<<<<<<<<<<<<<<<<<<<<<<<<<< -You should see something like ![v4l2-ctl-stream-output](./v4l2-ctl-stream-output.png) in terminal, which means the stream is working properly. +--- +### GStreamer streaming using v4l2src -### Gstreamer streaming using v4l2src +>**Pro:**\ +v4l2src can directly be used once mc-setup is completed without any libcamhal configuration files setup. ->Pro: v4l2src can directly be used once mc-setup is completed without any libcamhal configuration files setup. +>**Con:**\ +v4l2src does not support DMABuf which might hit some performance issue. ->Con: v4l2src does not support DMABuf which might hit some performance issue. +#### Sample Command -#### Sample Command for v4l2src + gst-launch-1.0 v4l2src device=/dev/video{X} ! 'video/x-raw,format={FORMAT},width={WIDTH},height={HEIGHT},framerate={FPS}/1,pixel-aspect-ratio=1/1' ! glimagesink -| Sensor | Link Number | Command Pipeline | -| --- | --- | --- | -| isx031 | des0 link 0 | gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=UYVY,width=1920,height=1536,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| isx031 | des0 link 1 | gst-launch-1.0 v4l2src device=/dev/video1 ! 'video/x-raw,format=UYVY,width=1920,height=1536,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| d457 depth | des0 link 0 | gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=UYVY,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| d457 depth | des0 link 1 | gst-launch-1.0 v4l2src device=/dev/video1 ! 'video/x-raw,format=UYVY,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| d457 rgb | des0 link 0 | gst-launch-1.0 v4l2src device=/dev/video4 ! 'video/x-raw,format=YUY2,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| d457 rgb | des0 link 1 | gst-launch-1.0 v4l2src device=/dev/video5 ! 'video/x-raw,format=YUY2,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | +#### Sample Output -### Gstreamer streaming using icamerasrc + Setting pipeline to PAUSED ... + Pipeline is live and does not need PREROLL ... + Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0"; + Pipeline is PREROLLED ... + Setting pipeline to PLAYING ... + New clock: GstSystemClock + Redistribute latency... + Got EOS from element "pipeline0". + Execution ended after 0:00:05.426540632 + Setting pipeline to NULL ... + Freeing pipeline ... ->Pro: icamerasrc support DMABuf which can have better performance. +--- +### GStreamer streaming using icamerasrc ->Con: Have dependency on [ipu7-camera-hal PR44](https://github.com/intel/ipu7-camera-hal/pull/44) +>**Pro:** \ +icamerasrc supports DMABuf which offers better performance. -Pre-requisite: +>**Con:** \ +Have dependency on [ipu7-camera-hal PR64](https://github.com/intel/ipu7-camera-hal/pull/64) ->**ipu7-camera-hal already cloned and compiled previously** +Prerequisites: + +**ipu7-camera-hal already cloned and compiled previously** cd ipu7-camera-hal - git fetch origin pull/44/head:pr-44 - git checkout pr-44 + git fetch origin pull/64/head:pr-64 + git checkout pr-64 -**Rebuild and Reinstall** ipu7-camera-hal + # Rebuild and Reinstall ipu7-camera-hal Export environment variables below @@ -101,25 +698,96 @@ Export environment variables below export logSink=terminal rm -rf ~/.cache/gstreamer-1.0 -#### Sample Command for icamerasrc +#### Sample Command for 2D GMSL YUV sensor + +Replace the device-name, format, width and height with the correct values for the sensor. + +DMA Command + + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-{X} printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format={FORMAT},width={WIDTH},height={HEIGHT}' ! glimagesink sync=false + +MMAP Command + + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-{X} printfps=true io-mode=mmap ! 'video/x-raw,format={FORMAT},width={WIDTH},height={HEIGHT}' ! glimagesink sync=false + + + +
    + device-name + +|Link| device-name | +|:---:|:---: | +|0/A| acpi-1 | +|1/B| acpi-2 | +|2/C| acpi-3 | +|3/D| acpi-4 | +|4/E| acpi-5 | +|5/F| acpi-6 | +|6/G| acpi-7 | +|7/H| acpi-8 | + +
    + +
    + io-mode + +|use case| io-mode | caps | +|:---:|:---: |---| +|DMA| dma_mode | 'video/x-raw(memory:DMABuf),drm-format={},width={},height={}' | +|MMAP| mmap | 'video/x-raw,format={},width={},height={}' | + +
    + +
    + num-vc + +num-vc property is used to start the stream concurrently. +For multi-streaming case, it is possible to set num-vc=1 and start the stream one by one. + +|use case| num-vc | +|:---:|:---: | +|1x stream | 1 | +|2x stream | 2 | +|3x stream | 3 | +|4x stream | 4 | +|5x stream | 5 | +|6x stream | 6 | +|7x stream | 7 | +|8x stream | 8 | + +
    + +
    + configuration + +| Format | Resolution | +| :---: |:---: | +| UYVY8_1X16 | 1280x720, 1920x1080, 1920x1536 | + +
    + + + +#### Sample Command for 2D GMSL RAW sensor + +- (e.g.) AR0234 GMSL refer to [../ar0234/userspace-gmsl.md](../ar0234/userspace-gmsl.md#sensor-verification). + + +#### Sample Command for 3D GMSL YUV sensor + +- (e.g.) D457 GMSL refer to [../d4xx/userspace-gmsl.md](../d4xx/userspace-gmsl.md#sensor-verification). + -The list below keeps track of the validated sensor and their respective configuration for icamerasrc. -3D sensors require separate config file, and can be found in their respective directories, i.e. d4xx sensor config file is located in ../../config/d4xx/ . +#### Sample Command for Mix-and-Match 2D+3D GMSL YUV sensor -| Sensor | drm-format | width | height | -| --- | --- | --- | --- | -| isx031 | UYVY | 1920 | 1536 | +
    + link 0 (2D) + link 2 (3D, Depth+RGB streams) -Replace the drm-format, width, height in the command pipeline below based on the supported sensor configuration above. + gst-launch-1.0 \ + icamerasrc num-buffers=-1 num-vc=3 scene-mode=normal device-name=acpi-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format={FORMAT},width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=3 scene-mode=normal device-name=d4xx-3-depth printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format={FORMAT},width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=3 scene-mode=normal device-name=d4xx-3-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format={FORMAT},width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -| Link Number | Sample Command Pipeline | -| --- | --- | -| link 0 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -| link 1 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -| link 2 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-3 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -| link 3 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-4 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -| link 4 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-5 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -| link 5 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-6 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -| link 6 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-7 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -| link 7 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=acpi-8 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +
    +

    (back to top)

    diff --git a/doc/d4xx/img-entity-d4xx-gmsl.png b/doc/d4xx/image/img-entity-d4xx-gmsl.png similarity index 100% rename from doc/d4xx/img-entity-d4xx-gmsl.png rename to doc/d4xx/image/img-entity-d4xx-gmsl.png diff --git a/doc/d4xx/realsense-viewer-output.png b/doc/d4xx/image/realsense-viewer-output.png similarity index 100% rename from doc/d4xx/realsense-viewer-output.png rename to doc/d4xx/image/realsense-viewer-output.png diff --git a/doc/d4xx/rs-color-output.png b/doc/d4xx/image/rs-color-output.png similarity index 100% rename from doc/d4xx/rs-color-output.png rename to doc/d4xx/image/rs-color-output.png diff --git a/doc/d4xx/rs-depth-output.png b/doc/d4xx/image/rs-depth-output.png similarity index 100% rename from doc/d4xx/rs-depth-output.png rename to doc/d4xx/image/rs-depth-output.png diff --git a/doc/d4xx/rs-multicam-output.png b/doc/d4xx/image/rs-multicam-output.png similarity index 100% rename from doc/d4xx/rs-multicam-output.png rename to doc/d4xx/image/rs-multicam-output.png diff --git a/doc/d4xx/mc-setup-media-ctl-output.png b/doc/d4xx/mc-setup-media-ctl-output.png deleted file mode 100644 index 8495061c..00000000 Binary files a/doc/d4xx/mc-setup-media-ctl-output.png and /dev/null differ diff --git a/doc/d4xx/mc-setup-output.png b/doc/d4xx/mc-setup-output.png deleted file mode 100644 index 4fde60a3..00000000 Binary files a/doc/d4xx/mc-setup-output.png and /dev/null differ diff --git a/doc/d4xx/userspace-gmsl.md b/doc/d4xx/userspace-gmsl.md index 1068ed45..669a3f8e 100644 --- a/doc/d4xx/userspace-gmsl.md +++ b/doc/d4xx/userspace-gmsl.md @@ -1,79 +1,458 @@ ## Description -This document provides details of the configuration settings for the D457 GMSL sensor using maxim-serdes driver. +This document describes the configuration settings for the D457 GMSL sensor using the maxim-serdes driver. Before going through this document, make sure you have completed the prerequisites, fully understand the ASL configurations, and know how to use mc-setup.sh. ->**Note:** Before going through this document, please make sure you have gone through [../acpi/kernelspace.md](../acpi/kernelspace.md) and [../acpi/userspace-gmsl.md](../acpi/userspace-gmsl.md) to understand the ACPI enumeration and userspace configuration for GMSL sensors. +> Prerequisites:\ +> [acpi/kernelspace.md](../acpi/kernelspace.md) \ +> [acpi/userspace-gmsl.md](../acpi/userspace-gmsl.md) -## Compile ACPI ASL file based on use case +--- +## ACPI Setup - ASL Configuration -### SSDT for 2x + 2x D457 on MAX96724 for IPU75XA +

    IMPORTANT: BIOS setting needs to be turned OFF if using ASL method

    -Currently we only provide [SSDT](../../acpi/ipu7/max96724_rs_d457.asl) for 2x + 2x D457 use case on MAX96724 for IPU75XA. If you want to have different connection, please modify the SSDT before compiling. +ASL source files provided are for **reference only**. ->**Note:** MAX96724 only has 4 pipes, and for now only legacy mode is supported, which means only 4 streams can be streamed at the same time per deserializer. This is why we only provide SSDT for 2x on each DES use case. +> **NOTE:** \ +> Always make sure ASL configuration matches actual hardware connection. -> Follow [This section in ../acpi/kernelspace.md](../acpi/kernelspace.md#compile-and-load-acpi-asl-source-on-canonical-ubuntu-2404-or-2604) to compile and load the ASL source file into kernel. +--- +### ASL Configuration for IPU6EPMTL -## Configure pipeline using mc-setup.sh +
    + MAX9296 DPHY + 4x D457 GMSL sensor use case -Run `../../script/acpi/mc-setup.sh` or refer to [Configure pipeline for 3D sensors](../acpi/userspace-gmsl.md#construct-pipeline-for-3d-sensors) for configuring the streams separately. +- DES0 (MAX9296) + - Link 0: D457 + - Link 1: D457 +- DES1 (MAX9296) + - Link 0: D457 + - Link 1: D457 -After configuring the pipeline, it is recommended to run [sanity streaming test using v4l2-ctl](./userspace-gmsl.md#sanity-streaming-test-using-v4l2-ctl) on each node based on the output of mc-setup.sh. +>**ASL:** [max9296_rs_d457.asl](../../acpi/ipu6/max9296_rs_d457.asl)\ +For compilation and loading, please refer to [kernelspace.md](../acpi/kernelspace.md#compile-and-load). ->**Note:** MAX96724 only supports 4 active routing (matching with 4 internal pipes) at the same time, so maximum only 4 streams can be enabled on each DES, and by default it is 2 streams (Depth+RGB) from each D457 (assuming 2x D457 connected on each DES). +
    +

    (Go to Advanced Pipeline Configuration)

    + +--- +### ASL Configuration for IPU75XA + +
    + MAX96724 CPHY + 4x D457 GMSL sensor use case + +- DES0 (MAX96724) + - Link 0: D457 + - Link 1: D457 +- DES1 (MAX96724) + - Link 0: D457 + - Link 1: D457 + +>**ASL:** [max96724_rs_d457.asl](../../acpi/ipu7/max96724_rs_d457.asl)\ +For compilation and loading, please refer to [kernelspace.md](../acpi/kernelspace.md#compile-and-load). + +
    + +

    (Go to Advanced Pipeline Configuration)

    + +--- +### ASL Configuration for IPU8 + +
    + MAX96724 CPHY + 4x D457 GMSL sensor use case + +- DES0 (MAX96724) + - Link 0: D457 + - Link 1: D457 +- DES1 (MAX96724) + - Link 0: D457 + - Link 1: D457 + +>**ASL:** [max96724_rs_d457.asl](../../acpi/ipu8/max96724_rs_d457.asl)\ +For compilation and loading, please refer to [kernelspace.md](../acpi/kernelspace.md#compile-and-load). + +
    + +

    (Go to Advanced Pipeline Configuration)

    + +--- ## Sensor Verification Each of the entity should have their own subdev node. If there is mismatch in the ASL and actual hardware connection (hardware is less, or probe failed), all of the sensor subdev node will not be created and mc-setup script will fail to execute. -Upon Bootup, run `media-ctl -p` command should show +After boot, running `media-ctl -p` should show: -![media-ctl output](img-entity-d4xx-gmsl.png) +![media-ctl output](image/img-entity-d4xx-gmsl.png) -Upon running mc-setup script, `media-ctl -p` command should show +--- +## Supported Configuration for Each Stream -![media-ctl output after mc-setup](mc-setup-media-ctl-output.png) +| Stream | Format | Resolution | FPS | +| --- | --- | --- | --- | +| Depth | FIXED* , UYVY8_1X16 | 1280x720 | 30, 15, 5 | +| | | 848x480 | 90, 60, 30, 15, 5 | +| | | 848x100 | 100 | +| | | 640x480* | 90, 60, 30*, 15, 5 | +| | | 848x360 | 90, 60, 30, 15, 5 | +| | | 480x270 | 90, 60, 30, 15, 5 | +| | | 424x240 | 90, 60, 30, 15, 5 | +| | | 256x144 | 90 | +| RGB | YUYV8_1X16* | 1280x800 | 30, 15, 10, 5 | +| | | 1280x720 | 30, 15, 10, 5 | +| | | 848x480 | 60, 30, 15, 5 | +| | | 640x480* | 60, 30*, 15, 5 | +| | | 640x360 | 90, 60, 30, 15, 5 | +| | | 480x270 | 90, 60, 30, 15, 5 | +| | | 424x240 | 90, 60, 30, 15, 5 | +| IR | Y8_1X8* , VYUY8_1X16 | 1280x720 | 30, 15, 5 | +| | | 848x480 | 90, 60, 30, 15, 5 | +| | | 640x480* | 90, 60, 30*, 15, 5 | +| | | 640x360 | 90, 60, 30, 15, 5 | +| | | 480x270 | 90, 60, 30, 15, 5 | +| | | 424x240 | 90, 60, 30, 15, 5 | +| | RGB888_1X24 | 1280x800 | 30, 15 | +| IMU | Y8_1X8* | 38x1* | 400, 200, 100, 50* | -## Sensor stream verification +>**Note:** The format, resolution, and FPS marked with * are the default configuration for each stream after a power cycle. Reboot keeps the configuration from the previous session. -### Sanity Streaming Test using v4l2-ctl +## Advanced Pipeline Configuration - Per-Stream Configuration + +>Prerequisites: \ +> Go through [how to use mc-setup.sh](../acpi/userspace-gmsl.md#how-to-use-mc-setupsh) \ +> Go through [Advanced Pipeline Configuration](../acpi/userspace-gmsl.md#advanced---per-stream-configuration) + +This section will provide more stream combinations other than default streams. +The combinations are based on [IPU7 ASL](#asl-configuration-for-ipu75xa). + +

    Please modify the commands accordingly to fit your use case.

    + +
    + 1x Depth Stream + +#### Command for GStreamer streaming + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[depth,format=UYVY8_1X16,res=1280x720,fps=30] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + +

    (Go to Sensor Stream Verification)

    + +#### Command for RealSense SDK use + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[depth,format=FIXED,res=1280x720,fps=30] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + +> FIXED format is mapped as UYVY8_1X16 in the pipeline, but Video Node is configured as 'Z16 '. + +

    (Go to Sensor Stream Verification)

    + +
    + +
    + 1x RGB Stream + +#### Command for GStreamer streaming and RealSense SDK use + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[rgb,format=YUYV8_1X16,res=1280x800,fps=30] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + +

    (Go to Sensor Stream Verification)

    + +
    + +
    + 1x IR Stream + +#### Command 1 for GStreamer streaming + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[ir,format=Y8_1X8,res=1280x720,fps=30] + +#### Command 2 for GStreamer streaming + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[ir,format=RGB888_1X24,res=1280x800,fps=30] + +#### Output + + # Output for Command 1 + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video6 + + # Output for Command 2 + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [ir] 1280x800 RGB888_1X24 30.000 FPS --> /dev/video6 + +

    (Go to Sensor Stream Verification)

    + + +#### Command for RealSense SDK streaming + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[ir,format=VYUY8_1X16,res=1280x720,fps=30] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [ir] 1280x720 VYUY8_1X16 30.000 FPS --> /dev/video6 + +

    (Go to Sensor Stream Verification)

    + +
    + +
    + 1x IMU Stream + +#### Command for GStreamer and RealSense SDK streaming + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[imu,format=Y8_1X8,res=38x1,fps=400] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video2 + +

    (Go to Sensor Stream Verification)

    + +
    + +
    + 1 D457 with all 4 streams (4 current configuration) + +#### Command for GStreamer + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[depth],[rgb],[ir],[imu] + + # or + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=depth,rgb,ir,imu + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + Stream [ir] 1280x720 VYUY8_1X16 30.000 FPS --> /dev/video6 + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video2 + +#### Command for RealSense SDK streaming + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[depth,format=FIXED],[rgb],[ir],[imu] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + Stream [ir] 1280x720 VYUY8_1X16 30.000 FPS --> /dev/video6 + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video2 + +> FIXED format is mapped as UYVY8_1X16 in the pipeline, but Video Node is configured as 'Z16 '. + +

    (Go to Sensor Stream Verification)

    + +
    -Follow [Sanity Streaming Test using v4l2-ctl](../acpi/userspace-gmsl.md#sanity-streaming-test-using-v4l2-ctl) according to mc-setup.sh output. +
    + 1 D457 with all 4 streams (2 current configuration, 2 update configuration) -#### Sample Command for v4l2-ctl +#### Command for GStreamer ->Note: Please use the respective video node that is shown in the output of mc-setup.sh. + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[depth,format=UYVY8_1X16,res=1280x720,fps=30],\ + [rgb],\ + [ir,format=Y8_1X8,res=1280x720,fps=30],\ + [imu] -| Stream | Link Number | Command Pipeline | -| --- | --- | --- | -| depth | des0 link 0 | v4l2-ctl -d /dev/video0 --stream-mmap | -| depth | des0 link 1 | v4l2-ctl -d /dev/video1 --stream-mmap | -| rgb | des0 link 0 | v4l2-ctl -d /dev/video4 --stream-mmap | -| rgb | des0 link 1 | v4l2-ctl -d /dev/video5 --stream-mmap | +#### Output -### Gstreamer streaming using v4l2src + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video6 + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video2 -Follow [Gstreamer streaming using v4l2src](../acpi/userspace-gmsl.md#gstreamer-streaming-using-v4l2src) according to mc-setup.sh output. +#### Command for RealSense SDK streaming -#### Sample Command for v4l2src + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[depth,format=FIXED,res=1280x720,fps=30],\ + [rgb],\ + [ir,format=Y8_1X8,res=1280x720,fps=30],\ + [imu] ->Note: Please use the respective video node that is shown in the output of mc-setup.sh. +#### Output -| Stream | Link Number | Command Pipeline | + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video6 + Stream [imu] 38x1 Y8_1X8 400.000 FPS --> /dev/video2 + +> FIXED format is mapped as UYVY8_1X16 in the pipeline, but Video Node is configured as 'Z16 '. + +

    (Go to Sensor Stream Verification)

    + +
    + +
    + 1 D457 with 3 streams (2 current configuration, 1 update configuration) + +#### Command + + ../../script/acpi/mc-setup.sh des=0,\ + link=0,stream=[depth],\ + [rgb,format=YUYV8_1X16,res=1280x800,fps=30],\ + [ir] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + Stream [ir] 1280x720 Y8_1X8 30.000 FPS --> /dev/video6 + +

    (Go to Sensor Stream Verification)

    + +
    + +
    + 2 D457 with 2 streams (4 update configuration) + +#### Command + + ../../script/acpi/mc-setup.sh \ + des=0,link=0,stream=[depth,format=UYVY8_1X16,res=1280x720,fps=30],\ + [rgb,format=YUYV8_1X16,res=1280x800,fps=30] \ + des=0,link=1,stream=[depth,format=UYVY8_1X16,res=640x480,fps=90],\ + [rgb,format=YUYV8_1X16,res=424x240,fps=5] + +#### Output + + Configuration summary: + DES0 LINK0 Sensor model d4xx + Stream [depth] 1280x720 UYVY8_1X16 30.000 FPS --> /dev/video0 + Stream [rgb] 1280x800 YUYV8_1X16 30.000 FPS --> /dev/video4 + DES0 LINK1 Sensor model d4xx + Stream [depth] 640x480 UYVY8_1X16 90.000 FPS --> /dev/video1 + Stream [rgb] 424x240 YUYV8_1X16 5.000 FPS --> /dev/video5 + +

    (Go to Sensor Stream Verification)

    + +
    + +## Stream Verification + +Below are the sample Device Node mapping for each stream for 4 D457 on a single Deserializer. + +| Link Number | Stream | device | | --- | --- | --- | -| depth | des0 link 0 | gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=UYVY,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| depth | des0 link 1 | gst-launch-1.0 v4l2src device=/dev/video1 ! 'video/x-raw,format=UYVY,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| rgb | des0 link 0 | gst-launch-1.0 v4l2src device=/dev/video4 ! 'video/x-raw,format=YUY2,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | -| rgb | des0 link 1 | gst-launch-1.0 v4l2src device=/dev/video5 ! 'video/x-raw,format=YUY2,width=640,height=480,framerate=30/1,pixel-aspect-ratio=1/1' ! glimagesink | +| DES0 Link 0 | depth | video0 | +| DES0 Link 1 | depth | video1 | +| DES0 Link 2 | depth | video2 | +| DES0 Link 3 | depth | video3 | +| DES0 Link 0 | rgb | video4 | +| DES0 Link 1 | rgb | video5 | +| DES0 Link 2 | rgb | video6 | +| DES0 Link 3 | rgb | video7 | +| DES0 Link 0 | ir | video6 | +| DES0 Link 1 | ir | video7 | +| DES0 Link 2 | ir | video4 | +| DES0 Link 3 | ir | video5 | +| DES0 Link 0 | imu | video2 | +| DES0 Link 1 | imu | video3 | +| DES0 Link 2 | imu | video0 | +| DES0 Link 3 | imu | video1 | + +Note that there are conflicting video nodes for IR and RGB streams, and IMU and Depth streams. +The limiting factors that result in this design are: +1. V4L2_FRAME_DESC_ENTRY_MAX is FIXED to 8, so ACTIVE routing is limited to 8 streams. +2. MAX96724 legacy mode only supports MAXIMUM 4 pipes. + +>FUTURE TODO: +>1. Add support for dynamic routing to include active and disabled routes. +>2. Increase Intel IPU Video Node per MIPI port to 16. +>3. Add Intel IPU Extended Virtual Channel support. +>4. Add MAX96724 Extended Virtual Channel support. + +--- +### Sanity Streaming Test using v4l2-ctl + +Follow [Sanity Streaming Test using v4l2-ctl](#sanity-streaming-test-using-v4l2-ctl) according to the mc-setup.sh output. + +Sample Command + + v4l2-ctl -d /dev/video{X} --stream-mmap --stream-count=150 + +--- +### GStreamer streaming using v4l2src + +>**IMPORTANT**: The video node varies depending on the hardware board design and the ASL configuration. + +Please refer to the output of mc-setup.sh for the correct video node to use. + +#### Sample Command -### Gstreamer streaming using icamerasrc + # DEPTH (UYVY8_1X16) + gst-launch-1.0 v4l2src device=/dev/video{X} ! 'video/x-raw,format=UYVY,width={WIDTH},height={HEIGHT},framerate={FPS}/1,pixel-aspect-ratio=1/1' ! glimagesink ->Pro: icamerasrc support DMABuf which can have better performance. + # RGB (YUYV8_1X16) + gst-launch-1.0 v4l2src device=/dev/video{X} ! 'video/x-raw,format=YUY2,width={WIDTH},height={HEIGHT},framerate={FPS}/1,pixel-aspect-ratio=1/1' ! glimagesink ->Con: Have dependency on [ipu7-camera-hal PR](https://github.com/intel/ipu7-camera-hal/pull/44) + # IR (Y8_1X8) + gst-launch-1.0 v4l2src device=/dev/video{X} ! 'video/x-raw,format=GRAY8,width={WIDTH},height={HEIGHT},framerate={FPS}/1,pixel-aspect-ratio=1/1' ! glimagesink -Follow section [Camera Configuration File Setup for IPU75XA](./userspace-gmsl.md#camera-configuration-file-setup-for-ipu75xa) to setup config file for icamerasrc. + # IR (RGB888_1X24) + gst-launch-1.0 v4l2src device=/dev/video{X} ! 'video/x-raw,format=BGR,width={WIDTH},height={HEIGHT},framerate={FPS}/1,pixel-aspect-ratio=1/1' ! glimagesink + # IMU (Y8_1X8) + NOT SUPPORTED BY V4L2SRC. + +--- +### GStreamer streaming using icamerasrc + +>Pro: icamerasrc supports DMABuf, which can have better performance. + +>Con: +>1. IPU75XA and IPU8 depend on [ipu7-camera-hal pull request #64](https://github.com/intel/ipu7-camera-hal/pull/64) +>2. IPU6EPMTL depends on [ipu6-camera-hal pull request #175](https://github.com/intel/ipu6-camera-hal/pull/175) + +> Note: icamerasrc now enables all Depth, RGB, IR, and IMU streams.\ +> The Depth stream still requires additional conversion to produce meaningful data. Please refer to the [RealSense SDK section](#verify-stream-using-realsense-sdk) for more details. + +Follow the [Libcamhal Configuration File Setup](#libcamhal-configuration-file-setup) section to set up the config file for icamerasrc. + +--- #### Environment Setup Export environment variables below @@ -90,90 +469,201 @@ Export environment variables below export logSink=terminal rm -rf ~/.cache/gstreamer-1.0 +--- +## Libcamhal Configuration File Setup + +#### Libcamhal Config for IPU6EPMTL + +Rebuild [ipu6-camera-hal](https://github.com/intel/ipu6-camera-hal) with pull request [#175](https://github.com/intel/ipu6-camera-hal/pull/175) + +--- +#### Libcamhal Config for IPU75XA + +Rebuild [ipu7-camera-hal](https://github.com/intel/ipu7-camera-hal) with pull request [#64](https://github.com/intel/ipu7-camera-hal/pull/64) + +--- +#### Libcamhal Config for IPU8 + +Rebuild [ipu7-camera-hal](https://github.com/intel/ipu7-camera-hal) with pull request [#64](https://github.com/intel/ipu7-camera-hal/pull/64) + +--- #### Sample Command for icamerasrc -> Note: Icamerasrc and libcamhal config only enabled RGB stream from a D457 sensor, since only RGB stream is human-viewable frame. The Depth stream requires additional conversion to be meaningful data. Please refer to [Realsense SDK section](./userspace-gmsl.md#verify-stream-using-realsense-sdk) for more details. +Make sure to run mc-setup.sh to configure the pipeline before running the command below. Refer to [Pipeline Configuration](#advanced-pipeline-configuration---per-stream-configuration) for more details. -##### Sensor Device Selection +DMA Command -| Link Number | Command Pipeline | -|---|---| -| des0 link 0 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| des0 link 1 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| des0 link 2 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-3 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| des0 link 3 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-4 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| des1 link 0 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-5 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| des1 link 1 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-6 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| des1 link 2 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-7 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| des1 link 3 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-8 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | + # DEPTH (UYVY8_1X16) + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-{X}-depth printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -> **Note**: Refer to icamerasrc device-name property for more sensor details. + # RGB (YUYV8_1X16) + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-{X}-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -###### How to relate Sensor Number with AIC Link Port +MMAP Command -| AIC Link Port | Sensor Number | -|--- |--- | -| A | 1 | -| B | 2 | -| C | 3 | -| D | 4 | + # DEPTH (UYVY8_1X16) + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-{X}-depth printfps=true io-mode=mmap ! 'video/x-raw,format=UYVY,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -For AIC MAX96724 + # RGB (YUYV8_1X16) + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-{X}-rgb printfps=true io-mode=mmap ! 'video/x-raw,format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -![link-port](../images/max96724-faba-cphy.png) + # IR (Y8_1X8) + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-{X}-ir printfps=true io-mode=mmap ! 'video/x-raw,format=GRAY8,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -##### Frame Buffer Memory Type (IO Mode) Selection + # IR (RGB888_1X24) + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-{X}-ir printfps=true io-mode=mmap ! 'video/x-raw,format=BGR,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -| Stream | IO Mode | Command Pipeline | -|---|---|---| -| RGB | MMAP | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-1 printfps=true io-mode=mmap ! 'video/x-raw,format=YUY2,width=640,height=480' ! glimagesink sync=false | -| RGB | DMABUF | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | + # IMU (Y8_1X8) + gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-{X}-imu printfps=true io-mode=mmap ! 'video/x-raw,format=GRAY8,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false -> **Note**: Refer to icamerasrc io-mode property for more sensor details. +>NOTE: Y8_1X8 (GRAY8), RGB888_1X24 (BGR3) are not supported in DMA mode. -##### Sensor Resolution Selection +
    +device-name -| Stream| Resolution | Command Pipeline | -| --- | --- | --- | -| RGB | 640x480 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | +| AIC Link | Stream | device-name | +| A or DES0 Link 0 | Depth | d4xx-1-depth | +| A or DES0 Link 0 | RGB | d4xx-1-rgb | +| A or DES0 Link 0 | IR | d4xx-1-ir | +| A or DES0 Link 0 | IMU | d4xx-1-imu | +| B or DES0 Link 1 | Depth | d4xx-2-depth | +| B or DES0 Link 1 | RGB | d4xx-2-rgb | +| B or DES0 Link 1 | IR | d4xx-2-ir | +| B or DES0 Link 1 | IMU | d4xx-2-imu | +| N | Depth | d4xx-N-depth | +| N | RGB | d4xx-N-rgb | +| N | IR | d4xx-N-ir | +| N | IMU | d4xx-N-imu | -##### Sensor Format Selection +For more details, please refer to icamerasrc device-name property for more details. -| Stream | Format | Command Pipeline | +
    + +
    + io-mode + +| use case | io-mode | caps | | --- | --- | --- | -| RGB | YUYV | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | +| DMA | dma_mode | 'video/x-raw(memory:DMABuf),drm-format={},width={},height={}' | +| MMAP | mmap | 'video/x-raw,format={},width={},height={}' | + +
    -##### Number of Stream (Single Stream / Multi Stream) Selection +
    + num-vc -| Stream | Number of Stream | Command Pipeline | +| use case | num-vc | | --- | --- | --- | -| RGB | x1 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 device-name=d4xx-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| RGB | x2 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=2 device-name=d4xx-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=2 device-name=d4xx-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | -| RGB | x4 (x2 + x2) | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-5 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-6 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width=640,height=480' ! glimagesink sync=false | +| 1x stream | 1 | +| 2x stream | 2 | +| 3x stream | 3 | +| 4x stream | 4 | +| 5x stream | 5 | +| 6x stream | 6 | +| 7x stream | 7 | +| 8x stream | 8 | + +>Note: this num-vc is total number across 2 MIPI ports. Single MAX96724 currently only supports 4 pipes, so the maximum num-vc is 4 per MIPI port. + +
    + +--- +## Sample Use cases + +
    + DEPTH + RGB + IR from 1 D457 + +>**IMPORTANT**: Depth and IR must be configured to the same resolution, otherwise streaming will fail. The FPS will be the same and will take the lower configured value. + +> MAX9296 does not support more than 2 streams per link yet. This will be implemented in the future. + +#### Command + + gst-launch-1.0 \ + icamerasrc num-buffers=-1 num-vc=3 device-name=d4xx-{X}-depth printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=3 device-name=d4xx-{X}-ir printfps=true io-mode=mmap ! 'video/x-raw,format=GRAY8,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=3 device-name=d4xx-{X}-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false + +
    + +
    + DEPTH + RGB + IMU from 1 D457 + +> MAX9296 does not support more than 2 streams per link yet. This will be implemented in the future. -### Verify stream using RealSense SDK +#### Command -Pre-requisite: + gst-launch-1.0 \ + icamerasrc num-buffers=-1 num-vc=3 device-name=d4xx-{X}-depth printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=3 device-name=d4xx-{X}-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=3 device-name=d4xx-{X}-imu printfps=true io-mode=mmap ! 'video/x-raw,format=GRAY8,width=38,height=1' ! glimagesink sync=false --Completed [Pipeline Configuration](./userspace-gmsl.md#configure-pipeline-using-mc-setupsh) +
    --Completed [Symlinks Creation](./userspace-gmsl.md#create-symlinks-using-upstream-rs-enumsh) +
    + DEPTH + RGB + IR + IMU from 1 D457 --Completed [librealsense SDK compilation](./userspace-gmsl.md#compile-librealsense-sdk-from-source) +>**IMPORTANT**: Depth and IR must be configured to the same resolution, otherwise streaming will fail. The FPS will be the same and will take the lower configured value. -#### Create symlinks using upstream-rs-enum.sh +> MAX9296 does not support more than 3 streams per link yet. This will be implemented in the future. ->**Note:** This step is only necessary to stream with RealSense SDK. If you are using v4l2src or v4l2-ctl, you can skip this step and use the video device directly. +#### Command -By running the script below, it creates symlinks for video devices that will be used for streaming with RealSense SDK. Symlink that is created will need to be used with librealsense PR [#15007](https://github.com/IntelRealSense/librealsense/pull/15007). + gst-launch-1.0 \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-{X}-depth printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-{X}-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-{X}-ir printfps=true io-mode=mmap ! 'video/x-raw,format=GRAY8,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-{X}-imu printfps=true io-mode=mmap ! 'video/x-raw,format=GRAY8,width=38,height=1' ! glimagesink sync=false + +
    + +
    + 4x RGB from 4 D457 + +#### Command + + gst-launch-1.0 \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-1-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-2-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-3-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false \ + icamerasrc num-buffers=-1 num-vc=4 device-name=d4xx-4-rgb printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=YUYV,width={WIDTH},height={HEIGHT}' ! glimagesink sync=false + +
    + +--- +### Verify Stream Using RealSense SDK + +>Prerequisites:\ +>1. Completed [Pipeline Configuration](./userspace-gmsl.md#configure-pipeline-using-mc-setupsh)\ +>2. Completed [Symlinks Creation](./userspace-gmsl.md#create-symlinks-using-upstream-rs-enumsh)\ +>3. Completed [librealsense SDK compilation](./userspace-gmsl.md#compile-librealsense-sdk-from-source) + +>IMPORTANT: RealSense SDK requires streams to be configured to the expected format to be recognized. \ + +|Stream | Format | +| --- | --- | +| Depth | Z16 | +| RGB | YUYV8_1X16 | +| IR | VYUY8_1X16 | +| IMU | Y8_1X8 | + +> Z16 is MEDIA_BUS_FMT_FIXED, but IPU only recognize Z16 as UYVY8_1X16. \ +Hence, pipeline will be configured as UYVY8_1X16 for Depth stream, but Video Node is configured as Z16 for RealSense SDK to recognize the Depth stream. + +--- +#### Create Symlinks Using upstream-rs-enum.sh + +>**Note:** This step is only necessary when streaming with the RealSense SDK. If you are using v4l2src or v4l2-ctl, you can skip it and use the video device directly. + +Running the script below creates symlinks for the video devices used by the RealSense SDK. The symlink must be used with librealsense pull request [#15007](https://github.com/IntelRealSense/librealsense/pull/15007). sudo ../../script/d4xx/upstream-rs-enum.sh ##### Sample Video Node Symlink -The symlink for capture node should be the same as the output of mc-setup.sh command. The syntax looks like video-rs-{stream-type}-{index}, and the stream type can be depth, color, ir or imu. The index starts from 0 for link 0 on DES0, then incremented by 1 for link 1, link 2 and link 3. +The capture-node symlink should match the output of the mc-setup.sh command. The syntax is video-rs-{stream-type}-{index}, and the stream type can be depth, color, ir, or imu. The index starts at 0 for link 0 on DES0 and increments by 1 for link 1, link 2, and link 3. -For example, if Depth and RGB stream from Link 0 on DES0 are enabled, the symlink for Depth stream will be video-rs-depth-0 and the symlink for RGB stream will be video-rs-color-0, and both of them will point to the corresponding video node allocated by kernel. +For example, if the Depth and RGB streams from link 0 on DES0 are enabled, the Depth symlink will be video-rs-depth-0 and the RGB symlink will be video-rs-color-0. Both point to the corresponding video node allocated by the kernel. | Sample Capture Node | Sample symlink | | --- | --- | @@ -182,7 +672,7 @@ For example, if Depth and RGB stream from Link 0 on DES0 are enabled, the symlin ##### Sample Subdev Symlink -The subdev symlink is created with syntax video-rs-{stream-type}-sd-{index}, and the stream type and index follow the same rule as capture node symlink, except that it is pointing to the subdev node instead of video capture node. The subdev node is used for configuration of the sensor, and it is required to be used with librealsense PR [#15007](https://github.com/IntelRealSense/librealsense/pull/15007). +The subdev symlink uses the syntax video-rs-{stream-type}-sd-{index}. The stream type and index follow the same rule as the capture-node symlink, except that the target is the subdev node instead of the video capture node. The subdev node is used to configure the sensor, and it is required for librealsense pull request [#15007](https://github.com/IntelRealSense/librealsense/pull/15007). | Sample Entity | Sample symlink | | --- | --- | @@ -191,9 +681,10 @@ The subdev symlink is created with syntax video-rs-{stream-type}-sd-{index}, and | D4XX rgb 19-0010 | /dev/video-rs-color-sd-0 -> /dev/v4l-subdev12 | | D4XX imu 19-0010 | /dev/video-rs-imu-sd-0 -> /dev/v4l-subdev13 | -#### Compile librealsense SDK from source +--- +#### Compile librealsense SDK From Source -There are changes in SDK to support Intel IPU that is currently in review. +There are SDK changes to support Intel IPU that are currently under review. git clone https://github.com/realsenseai/librealsense.git cd librealsense @@ -204,44 +695,59 @@ There are changes in SDK to support Intel IPU that is currently in review. make -j2 cd Release -#### Sample Tools from Realsense SDK +--- +#### Sample Tools From RealSense SDK + +

    DISCLAIMER: Current driver might not be able to support D457 control update through RealSense Viewer. Only stream viewing is tested.

    -Verify stream using realsense-viewer (output in graphical interface). +> TODO: Work In Progress to add support. + +Verify a stream using realsense-viewer (output appears in the graphical interface). ./realsense-viewer +Make sure the resolution and format match the configuration in mc-setup.sh. +A mismatch will cause the stream to fail to start. + +Changing the resolution or format can only be done in mc-setup.sh for now. +Resolution and format selection in RealSense Viewer will not be reflected in the pipeline configuration. + Sample Output as shown below -![realsense-viewer output](realsense-viewer-output.png) +![realsense-viewer output](image/realsense-viewer-output.png) -Verify multiple streams using rs-multicam (output in graphical interface) +Verify multiple streams using rs-multicam (output appears in the graphical interface). ./rs-multicam Sample Output as shown below -![rs-multicam output](rs-multicam-output.png) +![rs-multicam output](image/rs-multicam-output.png) -Verify single Depth Stream using rs-depth (only Terminal output) +Verify a single Depth stream using rs-depth (terminal output only). ./rs-depth Sample Output as shown below -![rs-depth output](rs-depth-output.png) +![rs-depth output](image/rs-depth-output.png) -Verify single Color Stream using rs-color (only Terminal output) +Verify a single Color stream using rs-color (terminal output only). ./rs-color Sample Output as shown below -![rs-color output](rs-color-output.png) +![rs-color output](image/rs-color-output.png) +--- ## Known Issue -1. RGB stream can only be streamed one time if no Depth stream is configured and streamed. - Workaround: To start RGB stream repetitively, you need to reconfigure pipeline using ../../script/acpi/mc-setup.sh with Depth stream enabled, then you need to start & stop Depth stream first before any stream can work properly. +1. The RGB stream can only be started once if no Depth stream is configured and running. \ + **Workaround**: To start the RGB stream repeatedly, reconfigure the pipeline with ../../script/acpi/mc-setup.sh and enable the Depth stream first. Then start and stop the Depth stream before starting any other stream. + +2. D457 may hit an unrecoverable I2C error (-121). Rebooting will not resolve the issue. \ + **Workaround**: Specify I2C_SPEED as 100000 in ASL. Power Cycle the sensor to recover from the error. -2. D457 might hit unrecoverable I2C error over long period of read/write. Reboot will not resolve the issue. - Workaround: Power Cycle the sensor. +3. There are conflicting video nodes for IR and RGB streams, and for IMU and Depth streams. \ + **Workaround**: Do not run streams with conflicting video nodes at the same time. For example, the Link 0 RGB stream and the Link 2 IR stream cannot run at the same time because they share the same video node. Please refer to [Video Node Reference](#stream-verification) for more details. diff --git a/doc/isx031/userspace-gmsl.md b/doc/isx031/userspace-gmsl.md index a6ad7432..837b24d2 100644 --- a/doc/isx031/userspace-gmsl.md +++ b/doc/isx031/userspace-gmsl.md @@ -6,36 +6,39 @@ This document details the configuration settings for the ISX031 GMSL sensor, pro - [Hardware Connection](#hardware-connection) - [MAX9296 (REV B) Connection](#max9296-rev-b-connection) - - [MAX96724 AIC (REV A) Connection](#max96724-aic-rev-a-connection) - - [MAX96724 AIC (D-PHY) (REV B) Connection](#max96724-aic-d-phy-rev-b-connection) + - [MAX96724 AIC (C-PHY) (REV A) Connection](#max96724-aic-c-phy-rev-a-connection) - [MAX96724 AIC (C-PHY) (REV B) Connection](#max96724-aic-c-phy-rev-b-connection) + - [MAX96724 AIC (D-PHY) (REV B) Connection](#max96724-aic-d-phy-rev-b-connection) - [MAX96724 AIC (C-PHY to D-PHY Adapter) (REV B) Connection](#max96724-aic-c-phy-to-d-phy-adapter-rev-b-connection) -- [BIOS Configuration Table](#bios-configuration-table) +- [ACPI Setup - BIOS Configuration](#acpi-setup---bios-configuration) - [Disable C States](#disable-c-states) - [Sensor ACPI HID](#sensor-acpi-hid) - - [MIPI Camera Configuration for IPU6EP](#mipi-camera-configuration-for-ipu6ep) - - [MIPI Camera Configuration for IPU6EPMTL](#mipi-camera-configuration-for-ipu6epmtl) - - [Connected to MAX9296 AIC](#connected-to-max9296-aic) - - [Connected to D-PHY of MAX96724 AIC (REV B)](#connected-to-d-phy-of-max96724-aic-rev-b) - - [MIPI Camera Configuration for IPU75XA](#mipi-camera-configuration-for-ipu75xa) - - [Connected to C-PHY of MAX96724 AIC (REV B)](#connected-to-c-phy-of-max96724-aic-rev-b) - - [Connected to D-PHY of MAX96724 AIC (REV B) (via C-to-D-PHY adapter)](#connected-to-d-phy-of-max96724-aic-rev-b-via-c-to-d-phy-adapter) - - [MIPI Camera Configuration for IPU8](#mipi-camera-configuration-for-ipu8) - - [Connected to C-PHY of MAX96724 AIC (REV A)](#connected-to-c-phy-of-max96724-aic-rev-a) -- [Camera Configuration File Setup](#camera-configuration-file-setup) - - [Setup for IPU6EP](#setup-for-ipu6ep) - - [Setup for IPU6EPMTL](#setup-for-ipu6epmtl) - - [Setup for IPU75XA](#setup-for-ipu75xa) -- [Environment Setup](#environment-setup) + - [BIOS Settings for IPU6EP](#bios-settings-for-ipu6ep) + - [BIOS Settings for IPU6EPMTL](#bios-settings-for-ipu6epmtl) + - [BIOS Settings for IPU75XA](#bios-settings-for-ipu75xa) + - [BIOS Settings for IPU8](#bios-settings-for-ipu8) +- [ACPI Setup - ASL Configuration](#acpi-setup---asl-configuration) + - [ASL Configuration for IPU6EPMTL](#asl-configuration-for-ipu6epmtl) + - [ASL Configuration for IPU75XA](#asl-configuration-for-ipu75xa) + - [ASL Configuration for IPU8](#asl-configuration-for-ipu8) +- [Libcamhal Configuration File Setup (BIOS configured systems)](#libcamhal-configuration-file-setup-bios-configured-systems) + - [Libcamhal Config for IPU6EP](#libcamhal-config-for-ipu6ep) + - [Libcamhal Config for IPU6EPMTL](#libcamhal-config-for-ipu6epmtl) + - [Libcamhal Config for IPU75XA](#libcamhal-config-for-ipu75xa) + - [Libcamhal Config for IPU8](#libcamhal-config-for-ipu8) - [Sensor Verification](#sensor-verification) -- [Sample Userspace Command](#sample-userspace-command) - - [Sensor Device Selection](#sensor-device-selection) - - [How to relate Sensor Number with AIC Link Port](#how-to-relate-sensor-number-with-aic-link-port) - - [Frame Buffer Memory Type (IO Mode) Selection](#frame-buffer-memory-type-io-mode-selection) - - [Sensor Resolution Selection](#sensor-resolution-selection) - - [Sensor Format Selection](#sensor-format-selection) - - [Number of Stream (Single Stream / Multi Stream) Selection](#number-of-stream-single-stream--multi-stream-selection) +- [Supported Configurations](#supported-configurations) +- [Stream Verification](#stream-verification) + - [Environment Setup](#environment-setup) + - [Stream with GStreamer icamerasrc](#stream-with-gstreamer-icamerasrc) + - [device-name Selection](#device-name-selection) + - [io-mode Selection](#io-mode-selection) + - [Sensor Resolution Selection](#sensor-resolution-selection) + - [Sensor Format Selection](#sensor-format-selection) + - [Number of Streams (Single-Stream / Multi-Stream) Selection](#number-of-streams-single-stream--multi-stream-selection) - [Streaming Result](#streaming-result) + - [Highest Bandwidth Configuration](#highest-bandwidth-configuration) + ## Hardware Connection @@ -57,7 +60,7 @@ This section describes the physical AIC (Add-In Card) hardware setup, including > **Note:** The MAX96724 AIC (REV B) supports both C-PHY and D-PHY connections, selectable via the J14 jumper. -Image below shows the C-PHY setup. +The image below shows the C-PHY setup. ![link-port](../images/max96724-fabb-cphy.png) @@ -65,47 +68,58 @@ Image below shows the C-PHY setup. > **Note:** Ensure the J14 jumper pins are oriented toward the D-PHY connector, as shown in the image below. -Image below shows the D-PHY setup. +The image below shows the D-PHY setup. ![link-port](../images/max96724-fabb-dphy.png) ### MAX96724 AIC (C-PHY to D-PHY Adapter) (REV B) Connection -Image below shows the C-PHY to D-PHY adapter setup. +The image below shows the C-PHY to D-PHY adapter setup. ![link-port](../images/max96724-fabb-cphy-dphy.png) -## BIOS Configuration Table +The following sections show the BIOS settings for each platform. Configure the BIOS for the platform in use. -> **Note:** No External Clock required. +>**Note:** If you want to configure ACPI using ASL, please refer to the [ASL configuration](#acpi-setup---asl-configuration) section below. + +--- +## ACPI Setup - BIOS Configuration + +Using the BIOS configuration exercises the [ipu-acpi](../../drivers/media/platform/intel/) and [max9x](../../drivers/media/i2c/max9x/) drivers. ### Disable C States -Config path: `Intel Advanced Menu`->`Power & Performance`->`CPU - Power Management Control` +Disabling C-states prevents the system from entering low-power states, helping maintain consistent performance and reduce latency during camera operation. -| | Options | -|--- |--- | -| C states | Disabled | +> **Note:** This option applies only to IPU6EP platforms. -> **Note:** : This option is only applicable for IPU6EP platforms (ADL, TWL, ASL and RPL). +>**BIOS path**: `Intel Advanced Menu`->`Power & Performance`->`CPU - Power Management Control` -> C states -> **Disabled** +--- ### Sensor ACPI HID -| Vendor | Sensor ACPI HID | -|--- |--- | -| D3 Embedded | INTC031M | -| Leopard Imaging | INTC031L | -| Otobrite | INTC031O | -| Sensing | INTC031S | - -> **Note:** Sensor ACPI HID value will be used for `MIPI Camera Configuration`. +Use the sensor ACPI HID in the **Custom HID** field. -### MIPI Camera Configuration for IPU6EP +| Vendor | Sensor ACPI HID | +|-----------------|:---------------:| +| D3 Embedded | INTC031M | +| Leopard Imaging | INTC031L | +| Otobrite | INTC031O | +| Sensing | INTC031S | -Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` +--- +### BIOS Settings for IPU6EP
    -Click to expand BIOS camera link options + MAX9296 DPHY + 4x ISX031 +

    (Back to Hardware Setup)

    +> **Note:** No control logic or external clock is required. + +>**BIOS Camera Option 1 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 1` -> **Enabled** + +>**BIOS Camera Option 2 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 2` -> **Enabled** | | Camera1 Link options | Camera2 Link Options | |--- |--- | --- | @@ -142,15 +156,25 @@ Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Cam | Flash Driver Selection | Disabled | Disabled |
    +

    (Go to Libcamhal Config)

    -### MIPI Camera Configuration for IPU6EPMTL +--- +### BIOS Settings for IPU6EPMTL -Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` +
    +MAX9296 DPHY + 4x ISX031 +

    (Back to Hardware Setup)

    -#### Connected to MAX9296 AIC +>**Connection:**\ + Refer to [MAX9296 (REV B) Connection](#max9296-rev-b-connection) for hardware connection and jumper setup. -
    -Click to expand BIOS camera link options +> **Note:** No control logic or external clock is required. + +>**BIOS Camera Option 1 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 1` -> **Enabled** + +>**BIOS Camera Option 2 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 2` -> **Enabled** | | Camera1 Link options | Camera2 Link Options | |--- |--- | --- | @@ -190,12 +214,15 @@ Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Cam
    -#### Connected to D-PHY of MAX96724 AIC (REV B) +
    +MAX96724 DPHY + 8x ISX031 +

    (Back to Hardware Setup)

    -> **Note:** Refer to [MAX96724 AIC (D-PHY) (REV B) Connection](#max96724-aic-d-phy-rev-b-connection) for hardware connection and jumper setup. +>**Connection:**\ + Refer to [MAX96724 AIC (D-PHY) (REV B) Connection](#max96724-aic-d-phy-rev-b-connection) for hardware connection and jumper setup. -
    -Click to expand BIOS camera link options +>**BIOS path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` | | Camera1 Link options | Camera2 Link Options | |--- |--- | --- | @@ -235,16 +262,25 @@ Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Cam
    -### MIPI Camera Configuration for IPU75XA +

    (Go to Libcamhal Config)

    -Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` +--- +### BIOS Settings for IPU75XA -#### Connected to C-PHY of MAX96724 AIC (REV B) +
    +MAX96724 CPHY + 8x ISX031 +

    (Back to Hardware Setup)

    -> **Note:** Refer to [MAX96724 AIC (C-PHY) (REV B) Connection](#max96724-aic-c-phy-rev-b-connection) for hardware connection and jumper setup. +>**Connection:**\ + Refer to [MAX96724 AIC (C-PHY) (REV B) Connection](#max96724-aic-c-phy-rev-b-connection) for hardware connection and jumper setup. -
    -Click to expand BIOS camera link options +> **Note:** No control logic or external clock is required. + +>**BIOS Camera Option 1 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 1` -> **Enabled** + +>**BIOS Camera Option 2 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 2` -> **Enabled** | | Camera1 Link options | Camera2 Link Options | |--- |--- | --- | @@ -286,9 +322,15 @@ Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Cam
    -#### Connected to D-PHY of MAX96724 AIC (REV B) (via C-to-D-PHY adapter) +
    +MAX96724 DPHY (via C-to-D-PHY adapter) + 6x ISX031 +

    (Back to Hardware Setup)

    + +>**Connection:**\ + Refer to [MAX96724 AIC DPHY (via C-PHY to D-PHY Adapter) (REV B) Connection](#max96724-aic-c-phy-to-d-phy-adapter-rev-b-connection) for hardware connection and jumper setup. -> **Note:** Refer to [MAX96724 AIC (C-PHY to D-PHY Adapter) (REV B) Connection](#max96724-aic-c-phy-to-d-phy-adapter-rev-b-connection) for hardware connection and jumper setup. +>**BIOS path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration`
    Click to expand BIOS camera link options @@ -331,19 +373,28 @@ Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Cam | Customize Device ID Number | 19 | 19 | | Flash Driver Selection | Disabled | Disabled | +
    +

    (Go to Libcamhal Config)

    -### MIPI Camera Configuration for IPU8 +--- +### BIOS Settings for IPU8 -Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` +
    +MAX96724 CPHY + 8x ISX031 +

    (Back to Hardware Setup)

    -#### Connected to C-PHY of MAX96724 AIC (REV A) +>**Connection:**\ + Refer to [MAX96724 AIC (C-PHY) (REV A) Connection](#max96724-aic-c-phy-rev-a-connection) for hardware connection and jumper setup. -> **Note:** Refer to [MAX96724 AIC (REV A) Connection](#max96724-aic-rev-a-connection) for hardware connection. +> **Note:** No control logic or external clock is required. -
    -Click to expand BIOS camera link options +>**BIOS Camera Option 1 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 1` -> **Enabled** + +>**BIOS Camera Option 2 Path:**\ + `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Camera Configuration` -> `Camera Option 2` -> **Enabled** | | Camera1 Link options | Camera2 Link Options | |--- |--- | --- | @@ -385,58 +436,278 @@ Config path: `Intel Advanced Menu`->`System Agent (SA) Configuration`->`MIPI Cam
    -## Camera Configuration File Setup +

    (Go to Libcamhal Config)

    + +--- +## ACPI Setup - ASL Configuration + +

    IMPORTANT: Turn off the BIOS setting to use the ASL method.

    + +Using the ASL configuration exercises the [maxim-serdes](../../drivers/media/i2c/maxim-serdes/) drivers. + +To compile ASL and load an SSDT overlay image, refer to [acpi/kernelspace.md](../acpi/kernelspace.md#compile-and-load). + +--- +### ASL Configuration for IPU6EPMTL + +
    + MAX9296 DPHY + 4x D3 ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max9296_d3_isx031.asl](../../acpi/ipu6/max9296_d3_isx031.asl)\ + +
    + +
    + MAX9296 DPHY + 1x LI ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    -#### Setup for IPU6EP +>**ASL:** [max9296_li_isx031.asl](../../acpi/ipu6/max9296_li_isx031.asl)\ -Replace target system with recommended [ipu6ep](../../config/isx031/ipu6ep) setting +
    + +
    + MAX9296 DPHY + 1x Sensing ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    -> **Note:** Add config below only if using x1 GMSL sensor. +>**ASL:** [max9296_sensing_isx031.asl](../../acpi/ipu6/max9296_sensing_isx031.asl)\ + +
    + +
    + MAX96724 DPHY + 8x D3 ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_dphy_d3_isx031.asl](../../acpi/ipu6/max96724_dphy_d3_isx031.asl)\ + +
    + +

    (Go to Pipeline Configuration)

    + +--- +### ASL Configuration for IPU75XA + +
    + MAX96724 CPHY + 8x D3 ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_d3_isx031.asl](../../acpi/ipu7/max96724_d3_isx031.asl)\ + +
    + +
    + MAX96724 CPHY + 1x LI ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_li_isx031.asl](../../acpi/ipu7/max96724_li_isx031.asl)\ + +
    + +
    + MAX96724 CPHY + 1x Sensing ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_sensing_isx031.asl](../../acpi/ipu7/max96724_sensing_isx031.asl)\ + +
    + +
    + MAX96724 DPHY + 6x D3 ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_dphy_d3_isx031.asl](../../acpi/ipu7/max96724_dphy_d3_isx031.asl)\ + +
    + +

    (Go to Pipeline Configuration)

    + +--- +### ASL Configuration for IPU8 + +
    + MAX96724 CPHY + 8x D3 ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_d3_isx031.asl](../../acpi/ipu8/max96724_d3_isx031.asl)\ + +
    + +
    + MAX96724 CPHY + 1x LI ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_li_isx031.asl](../../acpi/ipu8/max96724_li_isx031.asl)\ + +
    + +
    + MAX96724 CPHY + 1x Sensing ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_sensing_isx031.asl](../../acpi/ipu8/max96724_sensing_isx031.asl)\ + +
    + +
    + MAX96724 DPHY + 6x D3 ISX031 GMSL sensor use case +

    (Back to Hardware Setup)

    + +>**ASL:** [max96724_dphy_d3_isx031.asl](../../acpi/ipu8/max96724_dphy_d3_isx031.asl)\ + +
    + +

    (Go to Pipeline Configuration)

    + +--- +## Libcamhal Configuration File Setup (BIOS-Configured Systems) + +**Note:**\ +For ASL-configured systems, refer to [acpi/userspace-gmsl.md](../acpi/userspace-gmsl.md#stream-verification) for the libcamhal configuration file setup. + +#### Libcamhal Config for IPU6EP + +
    +1x GMSL sensor use case + +Please use recommended config from [ipu6ep](../../config/isx031/ipu6ep). + +> **Note:** Add config below only if using 1x GMSL sensor. sudo cp -r ../../config/isx031/ipu6ep /etc/camera sudo sed -i '/availableSensors/c\ ' /etc/camera/ipu6ep/libcamhal_profile.xml -> **Note:** Add config below only if using x4 GMSL sensors. +
    + +
    +4x GMSL sensor use case + +**BIOS configuration:** [IPU6EP 4x GMSL sensor use case](#bios-settings-for-ipu6ep) Please use config from [VTG ipu6ep](https://github.com/intel/ipu6-camera-hal/tree/iotg_ipu6/config/linux/ipu6ep). +
    -#### Setup for IPU6EPMTL +

    (Go to Sensor Verification)

    +

    (Go to Stream Verification)

    -Replace target system with recommended [ipu6epmtl](../../config/isx031/ipu6epmtl) setting +--- +#### Libcamhal Config for IPU6EPMTL -> **Note:** Add config below only if using x1 GMSL sensor. +
    +1x GMSL sensor use case + +Please use config from [ipu6epmtl](../../config/isx031/ipu6epmtl). + +> **Note:** Add config below only if using 1x GMSL sensor. sudo cp -r ../../config/isx031/ipu6epmtl /etc/camera sudo sed -i '/availableSensors/c\ ' /etc/camera/ipu6epmtl/libcamhal_profile.xml -> **Note:** Add config below only if using x4 GMSL sensors. +
    + +
    +4x GMSL sensor use case Please use config from [VTG ipu6epmtl](https://github.com/intel/ipu6-camera-hal/tree/iotg_ipu6/config/linux/ipu6epmtl). -> **Note:** Add config below only if using x8 GMSL sensors. +
    + +
    +8x GMSL sensor use case + +Please use config from [ipu6epmtl](../../config/isx031/ipu6epmtl). + +> **Note:** Add config below only if using 8x GMSL sensors. sudo cp -r ../../config/isx031/ipu6epmtl /etc/camera sudo sed -i '/availableSensors/c\ ' /etc/camera/ipu6epmtl/libcamhal_profile.xml +
    -#### Setup for IPU75XA +

    (Go to Sensor Verification)

    +

    (Go to Stream Verification)

    -Replace target system with recommended [ipu75xa](../../config/isx031/ipu75xa) setting +--- +#### Libcamhal Config for IPU75XA -> **Note:** Add config below only if using x1 GMSL sensor. +
    +1x GMSL sensor use case + +Please use recommended config from [ipu75xa](../../config/isx031/ipu75xa). + +> **Note:** Add config below only if using 1x GMSL sensor. sudo cp -r ../../config/isx031/ipu75xa /etc/camera sudo sed -i '/"availableSensors"/c\ "availableSensors": ["isx031-1-0"],' /etc/camera/ipu75xa/libcamhal_configs.json -> **Note:** Add config below only if using x8 GMSL sensors. +
    + +
    +8x GMSL sensor use case + +> **Note:** Add config below only if using 8x GMSL sensors. Please use config from [VTG ipu75xa](https://github.com/intel/ipu7-camera-hal/tree/main/config/linux/ipu75xa). - sudo sed -i '/"availableSensors"/c\ "availableSensors": ["isx031-1-0","isx031-2-0","isx031-3-0","isx031-4-0","isx031-5-2","isx031-6-2","isx031-7-2","isx031-8-2",' /etc/camera/ipu75xa/libcamhal_configs.json +
    + +

    (Go to Sensor Verification)

    +

    (Go to Stream Verification)

    + +--- +#### Libcamhal Config for IPU8 -## Environment Setup +
    +1x GMSL sensor use case + +Please use recommended config from [ipu8](../../config/isx031/ipu8). + +> **Note:** Add config below only if using 1x GMSL sensor. + + sudo cp -r ../../config/isx031/ipu8 /etc/camera + sudo sed -i '/"availableSensors"/c\ "availableSensors": ["isx031-1-0"],' /etc/camera/ipu8/libcamhal_configs.json + +
    + +
    +8x GMSL sensor use case + +> **Note:** Add config below only if using 8x GMSL sensors. + +Please use config from [VTG ipu8](https://github.com/intel/ipu7-camera-hal/tree/main/config/linux/ipu8). + +
    + +

    (Go to Sensor Verification)

    +

    (Go to Stream Verification)

    + +--- +## Sensor Verification + +After completing the setup, verify that the sensor is probed and registered with the V4L2 framework: + + media-ctl -p + +When using the BIOS configuration, the output for a single camera should look like the example below. +![media-ctl output](img-entity-isx031-gmsl.png) + +--- +## Supported Configurations + +| Format | Resolution | Frame Rate | +|---|---|---| +| UYVY8_1X16 | 1920x1536 | 60, 30 | +| | 1920x1080 | 60, 30 (default) | +| | 1280x720 | 30 | -Export environment variables below +--- +## Stream Verification + +> **Note:** \ +> For an ASL-configured system, refer to [acpi/userspace-gmsl.md](../acpi/userspace-gmsl.md) for pipeline setup and stream verification commands. + +### Environment Setup + +Export the environment variables below: unset XDG_RUNTIME_DIR export DISPLAY=:0; xhost + @@ -450,82 +721,98 @@ Export environment variables below export logSink=terminal rm -rf ~/.cache/gstreamer-1.0 -(Required for IPU6 only) Configure isys_freq value +For IPU6 only, configure the `isys_freq` value: sudo bash -c 'echo "options intel-ipu6 isys_freq_override=475" >> /etc/modprobe.d/ipu.conf' -## Sensor Verification - -Upon setup completion, verify sensor with: +--- +### Stream with GStreamer icamerasrc - media-ctl -p +#### device-name Selection -![media-ctl output](img-entity-isx031-gmsl.png) - -## Sample Userspace Command - -#### Sensor Device Selection - -| Sensor Number | Command Pipeline | -|---|---| -| 1 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| AIC Link | Supported Device-name | Command Pipeline | +|:--------:|:---------------------:|------------------| +| A | isx031-1 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| B | isx031-2 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| C | isx031-3 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-3 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| D | isx031-4 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-4 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| E | isx031-5 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-5 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| F | isx031-6 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-6 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| G | isx031-7 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-7 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| H | isx031-8 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-8 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -> **Note**: Refer to icamerasrc device-name property for more sensor details. +> **Note:** Link ports E, F, G, and H apply only to the MAX96724 AIC. -##### How to relate Sensor Number with AIC Link Port +> **Note:** Refer to the icamerasrc `device-name` property for more sensor details. -| AIC Link Port | Sensor Number | -|--- |--- | -| A | 1 | -| B | 2 | -| C | 3 | -| D | 4 | - -> **Note:** Link ports C and D are only applicable for MAX96724 AIC. - -Refer to [MAX9296 (REV B) Connection](#max9296-rev-b-connection) or [MAX96724 AIC (REV A) Connection](#max96724-aic-rev-a-connection) under Hardware Connection for the physical link port layout. - -#### Frame Buffer Memory Type (IO Mode) Selection +--- +#### io-mode Selection | IO Mode | Command Pipeline | |---|---| | MMAP | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=mmap ! 'video/x-raw,format=UYVY,width=1920,height=1536' ! glimagesink sync=false | | DMA MODE | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -> **Note**: Refer to icamerasrc io-mode property for more sensor details. +> **Note:** Refer to the icamerasrc `io-mode` property for more sensor details. +--- #### Sensor Resolution Selection | Resolution | Command Pipeline | |---|---| | 1920x1536 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| 1920x1080 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1080' ! glimagesink sync=false | +| 1280x720 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1280,height=720' ! glimagesink sync=false | +

    (Back to Supported Configurations)

    +--- #### Sensor Format Selection | Format | Command Pipeline | |---|---| | UYVY | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | -#### Number of Stream (Single Stream / Multi Stream) Selection +

    (Back to Supported Configurations)

    + +--- +#### Number of Streams (Single-Stream / Multi-Stream) Selection | Number of Stream | Command Pipeline | |---|---| | x1 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=1 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | | x2 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=2 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=2 scene-mode=normal device-name=isx031-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | | x4 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=4 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=4 scene-mode=normal device-name=isx031-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=4 scene-mode=normal device-name=isx031-3 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=4 scene-mode=normal device-name=isx031-4 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | +| x6 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=6 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=6 scene-mode=normal device-name=isx031-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=6 scene-mode=normal device-name=isx031-3 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=6 scene-mode=normal device-name=isx031-4 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=6 scene-mode=normal device-name=isx031-5 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=6 scene-mode=normal device-name=isx031-6 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | | x8 | gst-launch-1.0 icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-1 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-2 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-3 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-4 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-5 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-6 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-7 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false icamerasrc num-buffers=-1 num-vc=8 scene-mode=normal device-name=isx031-8 printfps=true io-mode=dma_mode ! 'video/x-raw(memory:DMABuf),drm-format=UYVY,width=1920,height=1536' ! glimagesink sync=false | ## Streaming Result -| Number of Stream | IO Mode | FPS Result | -|--- |--- |--- | -| x1 | MMAP | 30 | -| x2 | MMAP | 30 | -| x4 | MMAP | 30 | -| x8 | MMAP | 30 | -| x1 | DMA MODE | 30 | -| x2 | DMA MODE | 30 | -| x4 | DMA MODE | 30 | -| x8 | DMA MODE | 30 | - -> **Note:** Please ensure your system enable support for specified number of stream before test. +| Number of Stream | IO Mode | FPS Result |ipu6ep|ipu6epmtl|ipu75xa|ipu8| +|:----------------:|:--------:|:----------:|:----:|:-------:|:-----:|:--:| +| x1 | MMAP | 30 |✅|✅|✅|✅| +| x2 | MMAP | 30 |✅|✅|✅|✅| +| x4 | MMAP | 30 |✅|✅|✅|✅| +| x6 | DMA MODE | 30 |❌|❌|✅|✅| +| x8 | MMAP | 30 |❌|✅|✅|✅| +| x1 | DMA MODE | 30 |✅|✅|✅|✅| +| x2 | DMA MODE | 30 |✅|✅|✅|✅| +| x4 | DMA MODE | 30 |✅|✅|✅|✅| +| x6 | DMA MODE | 30 |❌|❌|✅|✅| +| x8 | DMA MODE | 30 |❌|✅|✅|✅| + +> **Note:** Ensure that the system supports the specified number of streams before testing. + +### Highest Bandwidth Configuration + +The highest-bandwidth configurations tested are listed below. + + 1. DPHY 4-lane per MIPI Port + - 4x 1920x1536 @ 30fps (default) + - 3x 1920x1536 @ 60fps + + 2. DPHY 2-lane per MIPI Port + - 2x 1920x1536 @ 30fps (default) + + 3. CPHY 2-trio per MIPI Port + - 4x 1920x1536 @ 30fps (default) + - 3x 1920x1536 @ 60fps