[Question] Design question: where does a per-episode perturbation spec belong? #1139

The Project Status table lists "easy-to-configure sensitivity analysis" as planned but not yet implemented. Before anyone writes code against that, it would help to know the shape you have in mind, so I am asking rather than proposing.

The concrete case I keep running into: I want a subset of parallel envs to receive a modification to the instruction string or the camera observation, with the rest left clean, and I want the metric writer to know which arm each env is in so the two can be compared as matched pairs rather than as two separate runs.

Two places that could live:

1. In the evaluation metrics layer, as a per-env tag the metric writer reads.
2. As a new composable building block that wraps an env and declares its own arm.

With heterogeneous parallel evaluations in the picture, option 2 seems to compose better, but it also means every metric consumer has to learn about arms. Option 1 keeps the building blocks unaware and puts the burden on one place.

Which of those is closer to what you intend, or is it neither? Happy to write the PR once the shape is settled. I maintain a small external tool that does this outside Arena today, so I have opinions but no attachment to them.