TensorRT CSharp API v4.0

TensorRT and CUDA bindings for C# and .NET, with managed APIs, project-owned bridge packages, runnable vision samples, and a TensorRtExec desktop workflow.

English | 简体中文

TensorRT CSharp API v4.0

TensorRT CSharp API v4.0 provides a .NET API for TensorRT inference, CUDA runtime compilation, memory, streams, callbacks, and the TensorRtExec desktop workflow. The stable 4.0.0 release is public; development now focuses on package-consuming examples, applications, and complete technical articles.

📖 Introduction

The managed API keeps the public namespace roots stable:

• JYPPX.TensorRtSharp is the main TensorRT surface and the default home for shared types.
• JYPPX.CudaSharp is the CUDA surface.
• Native bridge loading is explicit through jyppxtrtbridge and the versioned bridge package.

CUDA, cuDNN, TensorRT, and NVRTC are user-installed prerequisites. NVIDIA runtime redistribution is retired: this repository publishes managed source/packages and project-owned bridge-only packages, never CUDA/cuDNN/TensorRT vendor archives.

✨ Release Highlights

• C# bindings and source are grouped by module under src.
• TensorRT engine building, execution contexts, bindings, dynamic-shape profiles, allocators, logging, profiling, progress monitoring, streams, events, CUDA graphs, and CUDA RTC are covered by the managed API.
• Runtime package roles are explicit: the .Bridge packages with split_package_roles=bridge contain only the project bridge for a fixed CUDA/TensorRT line.
• NuGet branding is fixed: nuget/logo.jpg is the package logo and the English root README is embedded as the package README.
• The release license is Apache-2.0.

📢 Latest Update: 4.0.0

• Finalizes the stable JYPPX.TensorRtSharp and JYPPX.CudaSharp namespace roots and the 4.0 managed API surface.
• Publishes one managed package plus 6 Windows and 12 Linux project-owned Bridge packages; consumers install the matching NVIDIA runtime themselves.
• Keeps CUDA, cuDNN, TensorRT, NVRTC, sample applications, and model binaries outside all NuGet packages.
• The compatible managed-package maintenance revision 4.0.0.1 is available for current C# core changes; the Bridge package matrix remains on 4.0.0.

Read the detailed 4.0.0 notes, the 4.0.0.1 managed-package notes, or browse the complete version index.

🚀 Get Started In 30 Seconds

Create a console project, reference the managed package, and install the bridge package that matches the CUDA/TensorRT installation on the target machine:

dotnet new console -n TrtQuickstart
cd TrtQuickstart
dotnet add package JYPPX.TensorRT.CSharp.API --version 4.0.0
dotnet add package JYPPX.TensorRT.CSharp.API.Runtime.win-x64.trt10.11.cuda12.9.cudnn9.22.Bridge --version 4.0.0

The exact 4.0.0 version keeps restores reproducible without selecting API-incompatible historical 4.x packages such as 4.0.6170. Replace the Bridge package ID with the RID and NVIDIA-runtime matrix installed on the target machine.

Then create a runtime, load an engine, bind input/output tensors, execute, and read the result. The bridge package is not a replacement for the user-installed NVIDIA runtime. See the inference bindings tutorial and Windows installation guide.

📦 Package Layout

Package	Contents
JYPPX.TensorRT.CSharp.API	Managed TensorRT and CUDA-facing C# API
JYPPX.TensorRT.CSharp.API.Runtime.*.Bridge	Project-owned native bridge only; select one by RID/TensorRT/CUDA/cuDNN

Classification at samples/ComputerVision/01.Classification and YoloVision at applications/YoloVision are runnable examples. They consume the public 4-series managed package and are deliberately excluded from all public package feeds and Release assets.

🧪 Example Series

Series	Project	Focus
CUDA	Cuda/01.RuntimeCompilation	CUDA RTC compilation, module loading, launch, and readback
Inference	Inference/01.Bindings, Inference/02.DynamicShapes	Bindings, memory ownership, and dynamic profiles
Performance	Performance/01.MultiStream	CUDA streams, events, and ordering
Computer vision	Classification	Image preprocessing, Top-K output, JSON, and annotated results
Applications	YoloVision, OnnxToEngine, TensorRtExec	Complete multi-step workflows and advanced usage

See the sample series and applications for runnable commands and matching articles.

CUDA RTC roadmap: English | 简体中文 | technical article

🌐 Public Packages And Release Assets

The stable 4.0.0 release is available through GitHub Releases, NuGet.org, and GitHub Packages. The compatible managed-package maintenance revision is 4.0.0.1; the Bridge matrix remains at 4.0.0. Package README content is the English root README, package branding uses nuget/logo.jpg, and the core managed package uses the Apache-2.0 SPDX license expression.

Package	Version	NuGet.org	GitHub Packages	Purpose
JYPPX.TensorRT.CSharp.API		Gallery	Package feed	Core managed TensorRT/CUDA API

Release channel	Link	Assets
GitHub Release	v4.0.0	Source archive, core managed .nupkg, and project-owned Bridge .nupkg files
GitHub Packages	NuGet package feed	Core managed package and the published project-owned Bridge matrix
NuGet.org	Core package 4.0.0	Core managed package and all 18 project-owned Bridge packages

🧩 Bridge package matrix

Every published Bridge package is listed below. CUDA, cuDNN, and TensorRT are prerequisites installed by the consumer; the .Bridge package contains only jyppxtrtbridge. The Version column is a live NuGet.org badge. published-4.0.0 means the exact 4.0.0 package is available in GitHub Packages, the GitHub Release, and NuGet.org.

Package ID	Version	Runtime key	CUDA	cuDNN	TensorRT	Publication state
JYPPX.TensorRT.CSharp.API.Runtime.win-x64.trt8.6.cuda11.8.cudnn8.9.Bridge		win-x64-trt8.6-cuda11.8-cudnn8.9	11.8	8.9	8.6	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.win-x64.trt8.6.cuda12.1.cudnn8.9.Bridge		win-x64-trt8.6-cuda12.1-cudnn8.9	12.1	8.9	8.6	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.win-x64.trt10.11.cuda11.8.cudnn8.9.Bridge		win-x64-trt10.11-cuda11.8-cudnn8.9	11.8	8.9	10.11	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.win-x64.trt10.11.cuda12.9.cudnn9.22.Bridge		win-x64-trt10.11-cuda12.9-cudnn9.22	12.9	9.22	10.11	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.win-x64.trt11.0.cuda12.9.cudnn9.22.Bridge		win-x64-trt11.0-cuda12.9-cudnn9.22	12.9	9.22	11.0	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.win-x64.trt11.0.cuda13.2.cudnn9.22.Bridge		win-x64-trt11.0-cuda13.2-cudnn9.22	13.2	9.22	11.0	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu20.04.trt8.6.cuda11.8.cudnn8.9.Bridge		linux-x64-ubuntu20.04-trt8.6-cuda11.8-cudnn8.9	11.8	8.9	8.6	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu20.04.trt8.6.cuda12.1.cudnn8.9.Bridge		linux-x64-ubuntu20.04-trt8.6-cuda12.1-cudnn8.9	12.1	8.9	8.6	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu20.04.trt10.11.cuda11.8.cudnn8.9.Bridge		linux-x64-ubuntu20.04-trt10.11-cuda11.8-cudnn8.9	11.8	8.9	10.11	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu22.04.trt8.6.cuda11.8.cudnn8.9.Bridge		linux-x64-ubuntu22.04-trt8.6-cuda11.8-cudnn8.9	11.8	8.9	8.6	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu22.04.trt8.6.cuda12.1.cudnn8.9.Bridge		linux-x64-ubuntu22.04-trt8.6-cuda12.1-cudnn8.9	12.1	8.9	8.6	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu22.04.trt10.11.cuda11.8.cudnn8.9.Bridge		linux-x64-ubuntu22.04-trt10.11-cuda11.8-cudnn8.9	11.8	8.9	10.11	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu22.04.trt10.11.cuda12.9.cudnn9.22.Bridge		linux-x64-ubuntu22.04-trt10.11-cuda12.9-cudnn9.22	12.9	9.22	10.11	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu22.04.trt11.0.cuda12.9.cudnn9.22.Bridge		linux-x64-ubuntu22.04-trt11.0-cuda12.9-cudnn9.22	12.9	9.22	11.0	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu22.04.trt11.0.cuda13.2.cudnn9.22.Bridge		linux-x64-ubuntu22.04-trt11.0-cuda13.2-cudnn9.22	13.2	9.22	11.0	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu24.04.trt10.11.cuda12.9.cudnn9.22.Bridge		linux-x64-ubuntu24.04-trt10.11-cuda12.9-cudnn9.22	12.9	9.22	10.11	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu24.04.trt11.0.cuda12.9.cudnn9.22.Bridge		linux-x64-ubuntu24.04-trt11.0-cuda12.9-cudnn9.22	12.9	9.22	11.0	published-4.0.0
JYPPX.TensorRT.CSharp.API.Runtime.linux-x64.ubuntu24.04.trt11.0.cuda13.2.cudnn9.22.Bridge		linux-x64-ubuntu24.04-trt11.0-cuda13.2-cudnn9.22	13.2	9.22	11.0	published-4.0.0

Runtime packages do not bundle NVIDIA libraries. For local source builds, use the scripts in eng only through the documented entry points; most exporter and owner-proof scripts are internal engineering tools.

🛠️ Native Dependencies

Install the matching NVIDIA stack before running a bridge package:

Example line	Expected user installation
Windows x64 TRT 10.11 / CUDA 12.9	TensorRT 10.11, CUDA 12.9, cuDNN 9.22
Linux x64 TRT 11.0 / CUDA 13.2	TensorRT 11.0, CUDA 13.2, cuDNN 9.22

Use TENSORRT_PATH, JYPPX_TENSORRT_ROOT, and the platform loader path appropriate for your machine. The repository does not upload or package these vendor runtimes.

🧠 Models And ONNX Conversion

Demo models are staged outside Git in the sibling models directory and are not included in source archives or packages. Each article records the official acquisition URL, pinned revision, license, conversion command, input/output contract, and SHA256.

Demo	Official source and conversion
MNIST	Project-generated digits; export with the sample PyTorch/ONNX script, then build a TensorRT engine with trtexec.
ResNet18	torchvision official weights; export with torch.onnx.export using NCHW 224x224 and ImageNet normalization.
YOLOv8n detection/classification/segmentation/pose/OBB	Ultralytics official checkpoints; export with the pinned Ultralytics command and validate output names/shapes before TensorRT build.
YOLOv10n	THU-MIG official checkpoint; export with the repository reference script and preserve end-to-end output contract.
YOLOX-S	Megvii official checkpoint; export through the pinned YOLOX/ONNX path and validate decode metadata.
LRASPP MobileNetV3 Large	torchvision v0.25.0 official weights; export to [1,21,320,320] with ImageNet mean/std and compare argmax maps.

See the demo model inventory, acquisition and conversion guide, and eng/Sync-DemoOnnxModels.ps1. Model files stay in the external model store until ModelZoo is available.

📚 Documentation

• English documentation
• Chinese public article entrypoint
• Releases
• Sample series
• Applications
• API usage
• Installation and runtime environment
• Source build
• Project background and other topics
• 4.0.0 release article
• Windows installation
• Managed and Bridge package selection
• Sample series overview
• Project overview
• Source organization
• Model acquisition and ONNX conversion
• Inference bindings
• TensorRtExec GUI
• YOLOVision model matrix
• TensorRtExec feature matrix
• ONNX-to-engine parity matrix

🔨 Build From Source

dotnet restore TensorRtSharp.sln
dotnet build TensorRtSharp.sln -c Release
dotnet test tests/JYPPX.ProjectQuality.Tests/JYPPX.ProjectQuality.Tests.csproj -c Release --no-restore

For a future release, set JYPPXPackageVersion to the approved 4-series version. Inspect every generated nupkg before upload; it must contain the package README and logo.jpg and must not contain CUDA, cuDNN, or TensorRT vendor binaries.

🚢 Release And Action Policy

Workflows are manual-only to conserve Actions quota. The grape-yan repository is validation-only and never publishes. Run local restore, build, focused tests, package inspection, and clean-consumer checks first; dispatch a remote validation or formal release only after the Owner approves it.

NuGet publication requires the core package permission plus package-scoped push permission for each project-owned .Bridge ID. JYPPX.TensorRT.CSharp.API.YoloVision and JYPPX.TensorRT.CSharp.API.Classification are sample-only IDs and must never be uploaded. A nuget.org 403 is an authorization failure, not a retryable build failure.

🗂️ Repository Layout

• src: managed interfaces grouped by CUDA, TensorRT, runtime, memory, and shared modules.
• native: project bridge source and ABI exports.
• samples: runnable C# demonstrations and model metadata.
• applications/TensorRtExec: desktop engine builder and runner.
• pack: managed and bridge-only package definitions.
• docs: DocFX site and technical articles.
• eng: build, acquisition, validation, and release engineering scripts.

⚖️ Open Source and Usage Notice

1. Open Source License Notice

All open source project code by the author is released under the Apache License 2.0.

Special note: This project integrates several third-party libraries. If the license of any third-party library conflicts with or differs from the Apache License 2.0, the original license of that third-party library shall prevail. This project neither includes nor represents the licensing statements of those third-party libraries. Before use, you must read and comply with the applicable licenses of all third-party libraries.

2. Code Development and Quality Notice

• AI-assisted development: Artificial intelligence (AI) was used to assist in generating and optimizing this code during development; the code was not written entirely by hand, line by line.
• Security commitment: The author solemnly declares that this code contains no intentionally introduced backdoors, viruses, Trojan horses, or malicious code intended to damage user devices or steal data.
• Technical limitations: Because of the limits of the author's individual technical knowledge and ability, the code may contain basic issues caused by imperfect logic, insufficient optimization, or lack of experience, including but not limited to memory leaks, occasional crashes, and unreleased resources. Such issues result solely from limited ability and are not intentional.
• Testing scope: Because the author's time and resources are limited, this software has not undergone comprehensive testing covering every edge case.

3. Disclaimer (Important)

Before applying this code to any real-world project, especially a commercial, industrial, or mission-critical environment, you must conduct thorough and rigorous testing and validation yourself. In view of the possible code defects and limited test coverage described above, the author assumes no liability for any direct or indirect loss resulting from the use of this code, including but not limited to equipment failure, data loss, system outage, or loss of profit. By using this code, you acknowledge these risks and agree to bear all resulting consequences; the author is not responsible for related issues.

4. Scope of Open Source Code

This project commits to making its core logic code fully open source. However, the binary files, source code, and related resources of the aforementioned "third-party libraries" are outside this project's open source obligations; obtain them according to their respective instructions.

5. Community and Feedback

Despite the limitations described above, everyone is welcome to download and use the project, submit Issues, or participate in testing so that we can improve it together. If you discover a bug, memory overflow, or an opportunity for improvement, please contact the author through the channels provided on the project homepage. We will do our best to assist within the time available.

🤝 Contact and Sponsorship

When reporting a problem, include the package version, CUDA/cuDNN/TensorRT versions, GPU, operating system, and failing command. Do not upload model weights or NVIDIA runtime files whose licenses prohibit redistribution.