Skip to content

fix(rec): handle groups with a single experience - #638

Open
flyingdoubleG wants to merge 1 commit into
agentscope-ai:mainfrom
flyingdoubleG:fix/rec-singleton-group
Open

flyingdoubleG wants to merge 1 commit into
agentscope-ai:mainfrom
flyingdoubleG:fix/rec-singleton-group

Conversation

@flyingdoubleG

Copy link
Copy Markdown

Description

RECGroupedAdvantage.calculate_group_advantage (trinity/algorithm/advantage_fn/rec_advantage.py) builds rewards only in the len(exps) > 1 branch, then uses it for every group:

if len(exps) == 1:
    group_reward_mean = torch.tensor(0.0)
    group_reward_std = torch.tensor(1.0)
else:
    rewards = torch.tensor([exp.reward for exp in exps], dtype=torch.float32)
    ...
is_pos = rewards >= group_reward_mean   # UnboundLocalError for a group of one

So the rec advantage crashes with UnboundLocalError: cannot access local variable 'rewards' whenever a task group has a single experience (repeat_times: 1, or a partial group).

The fix builds rewards before the branch. A single experience keeps the existing convention of the len(exps) == 1 branch (mean 0, std 1, as in GRPO), so its advantage is its reward.

Test: tests/algorithm/advantage_fn_test.py::test_rec_advantage_single_experience_group covers the default args, std_normalize=True and drop="balance". It fails before the change; the file passes after (9 tests, CPU).

Checklist

  • Code has passed all tests
  • Docstrings have been added/updated in Google Style
  • Documentation has been updated
  • Code is ready for review

🤖 Generated with Claude Code

RECGroupedAdvantage.calculate_group_advantage only builds `rewards` when
the group has more than one experience, but uses it right after for
both branches, so a group of one (repeat_times=1, or a partial group)
raises UnboundLocalError. Build `rewards` before the branch; a single
experience keeps the existing no-baseline convention (mean 0, std 1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant