reverse_sums
reverse_sums
¶
ReverseSumsEval()
¶
Bases: RolloutEvaluation
Exact-match rollout evaluation at the reverse-sums <mid> boundary.
Given the dataset sequence::
<bos> 1 2 + 3 4 + 0 5 + 4 9 = <mid> 1 0 0 | 0 5 1 | 0 4 6 | 0 1 2 <eos>
get returns the numeric IDs through <mid> as the prompt and the
reversed prefix-sum IDs as the label. RolloutEvaluation removes token
0 (<eos>) before exact comparison, leaving matching integer strings.