Bug in GRPO outcome advantage: torch.std(torch.tensor([id2score[idx]])) incorrectly computes group std.

[Hoyant-Su]

In compute_grpo_outcome_advantage, the implementation currently computes the standard deviation of group scores with:
id2std[idx] = torch.std(torch.tensor([id2score[idx]]))
This is incorrect because wrapping id2score[idx] in an extra list creates a tensor of shape (1, N) instead of (N,), leading to wrong std computation.
This bug affects the calculation of group-relative advantages in GRPO outcome supervision and may produce unstable training signals, especially when there are multiple rollouts per prompt.

[langfengQ]

@Hoyant-Su thank you for pointing this out.

This part was inherited from verl's original GRPO implementation. I previously double-checked the logic and confirmed the outputs are correct (or I was wrong and I would appreciate if you could provide your debug screenshot)

But I agree the implementation is non-standard. we noticed verl has updated their code recently, so we'll incorporate those changes in the next release to keep everything clean.