Hi, thanks for sharing this excellent work and the code! While reading the implementation of aggregation_RPEA_joint_level, we had a question about the root translation used in RPEA.
It seems that the GT root is obtained from: gt_root = gt_3D[:, ..., 0, :] and then added to the predicted root-relative 3D hypotheses before computing the 2D reprojection error for the subsequent Top-K selection and weighting. Our understanding is that, under perspective projection, the GT root depth (Z) affects the 2D reprojection of each hypothesis. Therefore, RPEA seems to use the GT root depth to construct the reprojection criterion and then select the hypotheses that are most consistent with the observed 2D pose. Since this observed-2D consistency is itself evaluated conditioned on the GT root depth, the GT root depth also appears to affect the hypothesis selection and the final aggregation result. We may have missed some details in the implementation, so we would really appreciate your clarification on the motivation behind this design. Thanks again for the great work!
Hi, thanks for sharing this excellent work and the code! While reading the implementation of
aggregation_RPEA_joint_level, we had a question about the root translation used in RPEA.It seems that the GT root is obtained from: gt_root = gt_3D[:, ..., 0, :] and then added to the predicted root-relative 3D hypotheses before computing the 2D reprojection error for the subsequent Top-K selection and weighting. Our understanding is that, under perspective projection, the GT root depth (Z) affects the 2D reprojection of each hypothesis. Therefore, RPEA seems to use the GT root depth to construct the reprojection criterion and then select the hypotheses that are most consistent with the observed 2D pose. Since this observed-2D consistency is itself evaluated conditioned on the GT root depth, the GT root depth also appears to affect the hypothesis selection and the final aggregation result. We may have missed some details in the implementation, so we would really appreciate your clarification on the motivation behind this design. Thanks again for the great work!