PhyAI (pronounced "phi") is a latency-first serving engine for Physical AI. It is designed first for latency critical workloads, such as policy and action models that run in interactive systems.
- [2026/10] Support PhyAI gateway for multi-host serving and LeRobot clients.
- [2026/09] π Support Qwen-Image 2.1! doc.
- [2026/09] π€ Support PI0.5 RL rollout for PPO. RLinf integration is in progress via RLinf PR #1587.
- [2026/07] π Day 0 support for MiniCPM-Robotic blog.
- [2026/07] π Introducing PhyAI, a latency-first serving engine for Physical AI. Read the Blog.
- [2026/07] Support PI0.
- [2026/07] Support Cosmos3-Super with TP and CFG parallelism in the unified Cosmos3 generation path.
- [2026/06] Support Pi0.5 and Cosmos3-Nano's policy mode and generation mode.
- π Runs on NVIDIA Jetson edge devices
- π Scales to GPU clusters with DP, TP, and CFG parallelism
- π Uses high-performance kernels from FlashInfer and Humming
- π€ Supports W4A8 (NVFP4, MXFP4, INT4), W8A8, and W8A16 quantization (PR under review)
| VLA | Ο0, Ο0.5(w/ DP), GR00T N1.7, MiniCPM-Robot |
|---|---|
| WAM | Cosmos3-Nano-Policy-DROID(w/ TP, CFG Parallel) |
| Foundation Model | Cosmos3-Nano(w/ TP, CFG Parallel), Cosmos3-Super(w/ TP, CFG Parallel), Qwen3.5, Qwen3-VL |
See the PhyAI installation guide for the latest source and nightly package instructions.
From source:
git clone https://github.com/mingti-org/phyai
cd phyai
uv syncNightly build:
uv pip install phyai phyai-ext \
--extra-index-url https://mingti-org.github.io/phyai/simple/ \
--prerelease=allowYou can run model servers on separate machines and connect them through
phyai-gateway. On each model host, prepare pi05.yaml with that host's
checkpoint, tokenizer, and robot settings. You can start from the
PI0.5 example, which serves pi05 on port 50063.
Check the configuration, then start the server:
phyai server pi05.yamlOn the gateway host, register both servers at startup. Replace the IP addresses
below with your server hosts; pi05 must match server.model_name in their YAML
files. Repeat --backend MODEL HOST:PORT to add more servers:
phyai-gateway \
--backend pi05 10.0.0.11:50063 \
--backend pi05 10.0.0.12:50063In another terminal on the gateway host, check that the backends report
healthy: true:
curl http://127.0.0.1:30000/v1/backendsClients use the gateway host's HTTP port 30000 or gRPC port 50111. The gateway
distributes stateless requests across healthy replicas. See the
server and gateway setup guide for client
configuration and adding or removing servers while the gateway is running.
We thank the contributors below and welcome more developers to join us in building PhyAI.
PhyAI is a latency-first, open-source serving engine for Physical AI. It is being adopted by companies working across AI infrastructure and robotics, including Mingti and ModelBest.
We are actively seeking partnerships with compute providers, chip vendors, and robotics companies. If you are interested in working with us, please contact us.
|
|
If you use PhyAI in research or production work, please cite the project:
@software{phyai2026,
title = {PhyAI: Latency-First Serving Engine for Physical AI},
author = {{PhyAI Team}},
year = {2026},
url = {https://github.com/mingti-org/phyai}
}We welcome PhD and master's students who want to help build Physical AGI, especially those interested in systems infrastructure. We also want to work with chip and compute companies, as well as robotics companies that plan to deploy models with PhyAI.
- PhyAI: Maintainer
- Mengwei Xu: mwx@bupt.edu.cn
- Daliang Xu: xudaliang@bupt.edu.cn
PhyAI is released under the MIT License. It uses FlashInfer, Humming, and FLA. We have also learned a great deal from SGLang, vLLM, and TokenSpeed. We thank the maintainers and contributors of all these projects.