Skip to content

Home page: link step-ledger from the boundary-faults section #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tactino merged 1 commit into from
Oct 27, 2026
Merged

Home page: link step-ledger from the boundary-faults section #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,14 @@ trainer's own log. Such faults happen outside experiments too: under
first reward of every episode after an environment's first, until 1.4
([E51](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e51-episode-accounting)).

The step ledger is now a package of its own,
[step-ledger](https://github.com/PlugRL/step-ledger), for Gymnasium and
Stable-Baselines3 with or without PlugRL. Wrap each environment in
`LedgerEnv` and the vector env the trainer sees in `LedgerVectorEnv` (or
`LedgerVecEnv` for Stable-Baselines3); after the run, `stepledger compare`
names the environment, episode, step and field of the first difference. Each
side adds a few microseconds per environment step.

This is the paper *Detecting Silent Faults at the Environment-Trainer
Boundary in Reinforcement Learning*, accepted to the CoRL 2026 Workshop on
Everything Beneath the Policy ([PDF](/assets/silent-faults-webp2026.pdf);
Expand Down
6 changes: 6 additions & 0 deletions docs/index.zh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,12 @@ Hopper 和 Pusher 见 E56、E58）。在环境端记的回合记录，抓到了
环境从第二个回合起都会丢掉回合的第一个奖励，直到 1.4 才修好
（[E51](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e51-episode-accounting)）。

步账本现在单独做成了一个包 [step-ledger](https://github.com/PlugRL/step-ledger)，Gymnasium
和 Stable-Baselines3 都能用，用不用 PlugRL 都行。给每个环境包一层 `LedgerEnv`，给训练端看到的
向量环境包一层 `LedgerVectorEnv`（Stable-Baselines3 用 `LedgerVecEnv`）；跑完后
`stepledger compare` 会指出第一处不一致在哪个环境、哪个回合、哪一步、哪个字段。每一侧每个环境步
只多几微秒。

这部分内容是论文 *Detecting Silent Faults at the Environment-Trainer Boundary in
Reinforcement Learning*，已被 CoRL 2026 Workshop on Everything Beneath the Policy 录用
（[PDF](/assets/silent-faults-webp2026.pdf)；[引用方式](#citing)）。
Expand Down
Loading