Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ echoframe/cpp/out/
*.h5

*.mat
/Perrot et al. - 2021 - So you think you can DAS A viewpoint on delay-and-sum beamforming.pdf

__pycache__/

Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,6 @@
path = echoframe/cpp/libs/GSL
url = https://github.com/microsoft/GSL.git

[submodule "echoframe/cpp/libs/ffdas"]
path = echoframe/cpp/libs/ffdas
url = https://github.com/BrainEchoLab/ffdas.git
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,13 +183,17 @@ or:

## Quick Start<a name="quick-start"></a>

### 🔧 Submodules
### 🔧 Submodules<a name="submodules"></a>
Before continuing, make sure that the submodules included in the project are properly initialized.
```bash
# make sure you are in the EchoFrame directory
git submodule init
git submodule update
```
The [`ffdas`](echoframe/cpp/libs/README.md) submodule is **required**: it provides the
delay-and-sum (DAS) beamformer, which is built by default. If it is missing, CMake
stops with an error telling you to run the command above. Only GSL and ffdas are
needed; the rest of the build has no submodule dependencies.
You can continue with the rest of the build instructions if you have no errors.
Below you can find a set of commands to setup the project for both Linux and Windows.

Expand Down Expand Up @@ -290,7 +294,7 @@ This section is meant to clarify what changes you can make to the CMake build co
| `EF_BUILD_PYTHON` | **ON** | build the Python `echoframe` module |
| `EF_BUILD_CLI` | **ON** | build the EchoFrame `cli` (command line) module |
| `ENABLE_CUDA_TIMING` | **OFF** | print per-frame CUDA kernel timings from `process()` to the console |
| `CMAKE_CUDA_ARCHITECTURES` | set from the CUDA version: `61;75;86;89;90` on CUDA 12.4+, `75;80;86;89;90;100;103;121` on CUDA 13+ | compute capability list. `CMakeLists.txt` sets it after `project()`, so a value passed on the command line is replaced — edit the list there to narrow it |
| `CMAKE_CUDA_ARCHITECTURES` | set from the CUDA version: `61;75;86;89;90` on CUDA 12.4–12.7, `61;75;86;89;90;120` on CUDA 12.8–12.9, `75;80;86;89;90;100;103;120;121` on CUDA 13+ | compute capability list. `sm_120` covers consumer Blackwell (RTX 50 series) and needs CUDA 12.8 or newer; without it those GPUs JIT the `sm_90` PTX on every kernel launch. `CMakeLists.txt` sets it after `project()`, so a value passed on the command line is replaced — edit the list there to narrow it |

Example: disabling Python in Linux:

Expand Down
23 changes: 19 additions & 4 deletions build_scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ it, set up its dependencies, or check that a build works.
| Script | What it does |
|---|---|
| `build_echoframe.bat` | Automated Windows build. Walks the CUDA / MATLAB / Visual Studio combinations and reports what succeeded. The clearest reference for how a working Windows build is configured. Builds into `build_<CONFIG>/` at the repository root; set `EF_BUILD_ROOT` to put the build trees somewhere else. |
| `build_echoframe.sh` | Single-configuration Linux build — the counterpart of the `.bat`, minus the matrix: one configure, one build. Flags: `--build-dir`, `--build-type`, `--matlab-root`, `--no-mex`, `--no-python`, `--no-cli`, `--clean`, `-j N`. Checks for the GSL submodule and `VCPKG_ROOT` up front, and picks `CMAKE_CUDA_ARCHITECTURES` from the detected toolkit the same way `CMakeLists.txt` does. |
| `build_echoframe.sh` | Single-configuration Linux/macOS build — the counterpart of the `.bat`, minus the matrix: one configure, one build. This is the Linux build implementation: the `justfile`'s `configure` recipe delegates here, so the flag list and the preflight checks live in one place. Flags: `--build-dir`, `--build-type`, `--matlab-root`, `--no-mex`, `--no-python`, `--cli`, `--configure-only`, `--clean`, `-j N`. Checks the GSL and ffdas submodules, MATLAB and vcpkg up front, and picks `CMAKE_CUDA_ARCHITECTURES` from the detected toolkit the same way `CMakeLists.txt` does. |
| `pack_build.ps1` | Packages a `Release/` output directory as `binaries/build_<CONFIG>/build.zip`, with the `build/Release/<file>` layout the README documents. Called by `build_echoframe.bat`; can be run on its own to repackage without rebuilding. |
| `setup_vcpkg.ps1` | Clones and bootstraps vcpkg, then installs the ports EchoFrame needs. Defaults to `C:\vcpkg`; override with `-VcpkgRoot`. |
| `verify_builds.ps1` | Build verification harness. Copies the tree to a short path to dodge Windows `MAX_PATH`, then runs the MEX, CLI and Python builds. Non-zero exit on failure. Flags: `-SkipMex`, `-SkipCli`, `-Keep`, `-Python`, `-WorkDir`. |
Expand All @@ -19,11 +19,26 @@ Run them from the repository root:
pwsh -File build_scripts/verify_builds.ps1
```

On Linux:
On Linux and macOS, use the `justfile` at the repository root. `just configure` calls
`build_echoframe.sh`, so the flags below reach the same script the justfile uses:

```bash
build_scripts/build_echoframe.sh # everything
build_scripts/build_echoframe.sh --no-mex # no MATLAB installed
just configure # preflight, then configure
just build # build every target
just report # list artefacts with sizes
just build-target echoframe # build a single target

EF_BUILD_MEX=OFF just configure # no MATLAB on this host
EF_BUILD_PYTHON=OFF just configure # skip the Python extension
EF_BUILD_CLI=ON just configure # opt in to the CLI (needs vcpkg)
```

The CLI is off by default because it pulls Matio, HDF5 and ZLIB through vcpkg. The
script can also be run directly, which is what the justfile does:

```bash
build_scripts/build_echoframe.sh # everything
build_scripts/build_echoframe.sh --no-mex --configure-only
```

Each resolves the repository root from its own location, so they also work when
Expand Down
26 changes: 21 additions & 5 deletions build_scripts/build_echoframe.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
# $HOME/MATLAB/R2024a)
# --no-mex skip the MATLAB MEX layer
# --no-python skip the Python extension module
# --no-cli skip the CLI (this is what needs vcpkg)
# --cli also build the CLI (off by default: needs vcpkg)
# --configure-only configure the tree, then stop without building
# --clean delete the build tree first
# -j N parallel jobs (default: nproc)
# -h, --help this message
Expand All @@ -31,7 +32,8 @@ BUILD_TYPE="Release"
MATLAB_ROOT="${Matlab_ROOT_DIR:-${HOME}/MATLAB/R2024a}"
BUILD_MEX="ON"
BUILD_PYTHON="ON"
BUILD_CLI="ON"
BUILD_CLI="OFF"
CONFIGURE_ONLY=0
CLEAN=0
JOBS="$(nproc)"

Expand All @@ -54,7 +56,8 @@ while [ $# -gt 0 ]; do
--matlab-root) MATLAB_ROOT="$2"; shift 2 ;;
--no-mex) BUILD_MEX="OFF"; shift ;;
--no-python) BUILD_PYTHON="OFF"; shift ;;
--no-cli) BUILD_CLI="OFF"; shift ;;
--cli) BUILD_CLI="ON"; shift ;;
--configure-only) CONFIGURE_ONLY=1; shift ;;
--clean) CLEAN=1; shift ;;
-j) JOBS="$2"; shift 2 ;;
-h|--help) sed -n '2,24p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0 ;;
Expand All @@ -78,8 +81,14 @@ if [ ! -e "${REPO_ROOT}/echoframe/cpp/libs/GSL/include/gsl/gsl" ]; then
die "GSL submodule is empty. Run: git submodule update --init --recursive"
fi

# ffdas is a gitlink too, and it is not optional: CMake configures it in
# unconditionally now, so an empty checkout stops the configure step.
if [ ! -e "${REPO_ROOT}/echoframe/cpp/libs/ffdas/CMakeLists.txt" ]; then
die "ffdas submodule is empty. Run: git submodule update --init --recursive"
fi

if [ "${BUILD_CLI}" = "ON" ] && [ -z "${VCPKG_ROOT:-}" ]; then
die "EF_BUILD_CLI is ON but VCPKG_ROOT is unset. Set it, or pass --no-cli.
die "EF_BUILD_CLI is ON but VCPKG_ROOT is unset. Set it, or drop --cli.
Dependencies come from echoframe/cpp/src/vcpkg.json in manifest mode;
no manual 'vcpkg install' is needed."
fi
Expand All @@ -95,7 +104,10 @@ CUDA_VER="$(nvcc --version | sed -n 's/.*release \([0-9][0-9]*\.[0-9][0-9]*\).*/
CUDA_MAJOR="${CUDA_VER%%.*}"
if [ -z "${CMAKE_CUDA_ARCHITECTURES:-}" ]; then
if [ "${CUDA_MAJOR}" -ge 13 ]; then
CMAKE_CUDA_ARCHITECTURES="75;80;86;89;90;100;103;121"
CMAKE_CUDA_ARCHITECTURES="75;80;86;89;90;100;103;120;121"
elif [ "${CUDA_VER}" = "12.8" ] || [ "${CUDA_VER}" = "12.9" ]; then
# sm_120 (consumer Blackwell) needs CUDA 12.8 or newer.
CMAKE_CUDA_ARCHITECTURES="61;75;86;89;90;120"
else
CMAKE_CUDA_ARCHITECTURES="61;75;86;89;90"
fi
Expand Down Expand Up @@ -123,6 +135,10 @@ CONFIGURE_ARGS=(
command -v ninja >/dev/null && CONFIGURE_ARGS+=(-G Ninja)

cmake "${CONFIGURE_ARGS[@]}"
if [ "${CONFIGURE_ONLY}" = "1" ]; then
note "configured only; not building (--configure-only)"
exit 0
fi
cmake --build "${BUILD_DIR}" -j "${JOBS}"

# --- report ---------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion docker/ubuntu22.04-cuda12.8.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ ENV PATH="${VCPKG_ROOT}:${CARGO_HOME}/bin:${PATH}"
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
cmake \
curl \
fontconfig \
git \
Expand Down Expand Up @@ -74,6 +73,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
unzip \
xauth \
zip \
&& python3 -m pip install --no-cache-dir "cmake>=3.26,<4" \
&& rm -rf /var/lib/apt/lists/*

# Shell tooling for distrobox sessions that inherit host dotfiles. fzf comes from
Expand Down
14 changes: 10 additions & 4 deletions echoframe/cpp/libs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,22 @@ across the core for safer array views and pointer contracts. Header-only.
Note: this is Microsoft's **Guidelines** Support Library, not the GNU Scientific
Library.

## ffdas

A submodule (`github.com/BrainEchoLab/ffdas`) providing the CUDA
DAS beamformer backend. EchoFrame always builds it, so the submodule must
be initialised before configuring; a missing submodule is a hard CMake
error rather than a silently reduced build.

## Storage

An in-tree library (`Storage/`, not a submodule) that manages writing and reading
real-time RF, BF and PDI data to disk.

## Updating the GSL submodule
## Updating submodules

GSL is the only submodule here (Storage is a plain in-tree directory). To
initialise or update it from the repository root:
To initialise or update external submodules from the repository root:

```bash
git submodule update --init echoframe/cpp/libs/GSL
git submodule update --init echoframe/cpp/libs/GSL echoframe/cpp/libs/ffdas
```
1 change: 1 addition & 0 deletions echoframe/cpp/libs/ffdas
Submodule ffdas added at ea3a25
64 changes: 62 additions & 2 deletions echoframe/cpp/src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ project(EchoFrame VERSION 1.0 LANGUAGES CXX CUDA)
# find_package(matio MODULE ...) below.
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules")

# Keeps the fallback ffdas build from exporting its statically linked C++ runtime
# (see cmake/ffdas_exports.map for what goes wrong when it does).
include("${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules/ffdas_exports.cmake")


# --------------------------------------------------------------------------
# Global compile/architectures options
Expand Down Expand Up @@ -102,7 +106,14 @@ endif()
if(CUDAToolkit_VERSION VERSION_GREATER_EQUAL 13)
message(STATUS "CUDA Toolkit ${CUDAToolkit_VERSION} - using the CUDA 13 architecture list")
add_compile_definitions(GGML_CUDA_USE_GRAPHS) # try to enable cuda graphs on cu13 build
set(CMAKE_CUDA_ARCHITECTURES "75;80;86;89;90;100;103;121") # lowest CUDA 13 standard
set(CMAKE_CUDA_ARCHITECTURES "75;80;86;89;90;100;103;120;121") # lowest CUDA 13 standard
elseif(CUDAToolkit_VERSION VERSION_GREATER_EQUAL 12.8)
# sm_120 (consumer Blackwell, e.g. RTX 50 series) needs CUDA 12.8 or newer,
# so it cannot go in the 12.4 list below. Without it those GPUs fall back to
# JIT-compiling the sm_90 PTX on every kernel launch.
message(STATUS "CUDA Toolkit ${CUDAToolkit_VERSION} - using the CUDA 12.8 architecture list")
add_compile_definitions(GGML_CUDA_USE_GRAPHS) # try to enable cuda graphs on cu12 build
set(CMAKE_CUDA_ARCHITECTURES "61;75;86;89;90;120") # lowest CUDA 12 standard + consumer Blackwell
elseif(CUDAToolkit_VERSION VERSION_GREATER_EQUAL 12.4)
message(STATUS "CUDA Toolkit ${CUDAToolkit_VERSION} - using the CUDA 12 architecture list")
add_compile_definitions(GGML_CUDA_USE_GRAPHS) # try to enable cuda graphs on cu12 build
Expand All @@ -116,6 +127,51 @@ option(EF_BUILD_MEX "Build the MATLAB MEX interface" ON)
option(EF_BUILD_PYTHON "Build the Python extension module" ON)

option(ENABLE_CUDA_TIMING "Enable CUDA kernel timing instrumentation" OFF)
set(EF_FFDAS_CUDA_ARCHITECTURES "" CACHE STRING
"CUDA architectures for the ffdas submodule fallback build")

find_package(ffdas CONFIG QUIET)
if(NOT TARGET ffdas::ffdas AND NOT TARGET ffdas)
set(EF_FFDAS_SOURCE "${CMAKE_CURRENT_SOURCE_DIR}/../libs/ffdas")
if(NOT EXISTS "${EF_FFDAS_SOURCE}/CMakeLists.txt")
message(FATAL_ERROR
"The ffdas-backed DAS beamformer needs an installed ffdas package or the submodule at ${EF_FFDAS_SOURCE}. Run: git submodule update --init echoframe/cpp/libs/ffdas")
endif()
if(CMAKE_VERSION VERSION_LESS 3.26)
message(FATAL_ERROR
"Building the ffdas submodule needs CMake >= 3.26. Update the dev image or install CMake from PyPI/Kitware.")
endif()
set(EF_SAVED_CUDA_ARCHITECTURES "${CMAKE_CUDA_ARCHITECTURES}")
set(EF_SAVED_CUDA_SEPARABLE_COMPILATION
"${CMAKE_CUDA_SEPARABLE_COMPILATION}")
if(NOT EF_FFDAS_CUDA_ARCHITECTURES)
if(CUDAToolkit_VERSION VERSION_GREATER_EQUAL 13)
set(EF_FFDAS_CUDA_ARCHITECTURES "80;86;89;90;100;103;120;121")
else()
set(EF_FFDAS_CUDA_ARCHITECTURES "${CMAKE_CUDA_ARCHITECTURES}")
endif()
endif()
set(CMAKE_CUDA_ARCHITECTURES "${EF_FFDAS_CUDA_ARCHITECTURES}")
set(CMAKE_CUDA_SEPARABLE_COMPILATION OFF)
add_subdirectory("${EF_FFDAS_SOURCE}" "${CMAKE_BINARY_DIR}/ffdas")
if(TARGET ffdas)
set_property(TARGET ffdas PROPERTY
CUDA_SEPARABLE_COMPILATION OFF)
efconfine_ffdas_exports(ffdas)
endif()
set(CMAKE_CUDA_ARCHITECTURES "${EF_SAVED_CUDA_ARCHITECTURES}")
set(CMAKE_CUDA_SEPARABLE_COMPILATION
"${EF_SAVED_CUDA_SEPARABLE_COMPILATION}")
endif()
if(TARGET ffdas::ffdas)
set(EF_FFDAS_TARGET ffdas::ffdas)
# A prebuilt ffdas was linked before this project could reach it, so the
# version script cannot be added here. Check what it exports instead of
# leaving the banner truncation to be diagnosed from a Python session.
efreport_ffdas_exports(ffdas::ffdas)
else()
set(EF_FFDAS_TARGET ffdas)
endif()
# --------------------------------------------------------------------------
# Common include directories
# --------------------------------------------------------------------------
Expand Down Expand Up @@ -177,8 +233,12 @@ target_include_directories(echoframe_core

target_link_libraries(echoframe_core
PRIVATE
CUDA::cudart
CUDA::cudart
)
# PUBLIC, not PRIVATE: ffdas_beamformer.h includes <ffdas.h> and the out-of-line
# definitions call ffdas_*, so every target that includes that header needs the
# headers and the library, not just echoframe_core.
target_link_libraries(echoframe_core PUBLIC ${EF_FFDAS_TARGET})
target_compile_definitions(echoframe_core
PRIVATE
NOMINMAX
Expand Down
32 changes: 32 additions & 0 deletions echoframe/cpp/src/beamformer/das/das_recon_spec.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/**
* @file das_recon_spec.h
* @brief Delay-and-sum reconstruction parameters.
*/
#pragma once

#include <cstdint>

namespace Beamform {

struct DASReconSpec {
float *channelPositions{}; ///< Host pointer, (nActiveChannels, 3).
float *d_channelPositions{}; ///< Device pointer, (nActiveChannels, 3).
float *voxelPositions{}; ///< Host pointer, (totalSize, 3), z fastest.
float *d_voxelPositions{}; ///< Device pointer, (totalSize, 3).
float *offsets{}; ///< Host pointer, (nTX, totalSize).
float *d_offsets{}; ///< Device pointer, (nTX, totalSize).
float *weights{}; ///< Host pointer, (nTX, totalSize).
float *d_weights{}; ///< Device pointer, (nTX, totalSize).
float *tgcVector{}; ///< Host pointer, (nSamplesIQ).
float *d_tgcVector{}; ///< Device pointer, (nSamplesIQ).
float
*sourceDirections{}; ///< Optional host pointer, (nActiveChannels, 4).
float *d_sourceDirections{}; ///< Optional device pointer,
///< (nActiveChannels, 4).
float wavenum{0.0f}; ///< IQ phase rotation, -2*pi*Fc/Fs.
int32_t algorithm{1}; ///< ffdas algorithm: 1=ALG1, 2=ALG2, 4=ALG4.
int32_t computeType{0}; ///< ffdas compute type: 0=default.
bool useDirectivity{false};
};

} // namespace Beamform
49 changes: 49 additions & 0 deletions echoframe/cpp/src/beamformer/das/ffdas_beamformer.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/**
* @file ffdas_beamformer.h
* @brief ffdas-backed delay-and-sum beamformer.
*/
#pragma once

#include <cstdint>

#include <ffdas.h>

#include "../beamformer.h"
#include "das_recon_spec.h"

namespace Beamform {

template <typename bfType_t>
class FFDASBeamformer : public Beamformer<bfType_t> {
private:
DASReconSpec dasReconSpec;

ffdas_handle_t handle{};
ffdas_tensor_desc_t xDesc{};
ffdas_tensor_desc_t outDesc{};

bool mInitialized{false};

public:
FFDASBeamformer() = default;

FFDASBeamformer(ReceiveSpec pReceiveSpec, ReconSpec pReconSpec,
DASReconSpec pDASReconSpec, bfType_t *d_pRF)
: Beamformer<bfType_t>(std::move(pReceiveSpec), pReconSpec, d_pRF),
dasReconSpec(pDASReconSpec) {
initialize();
mInitialized = true;
}

~FFDASBeamformer();

FFDASBeamformer(FFDASBeamformer &&x) noexcept = delete;
FFDASBeamformer &operator=(FFDASBeamformer &&x) noexcept = delete;

void initialize();
void initGPU();
void clearGPU();
void process();
};

} // namespace Beamform
Loading
Loading