Trajectory Error (ATE RMSE) in your paper evaluated only on keyFrames?

[Justzhongjj]

Dear Author,

I have reviewed the code you provided regarding the pose evaluation section.

MonoGS\utils\eval_utils.py：

def eval_ate(frames, kf_ids, save_dir, iterations, final=False, monocular=False):
    trj_data = dict()
    latest_frame_idx = kf_ids[-1] + 2 if final else kf_ids[-1] + 1
    trj_id, trj_est, trj_gt = [], [], []
    trj_est_np, trj_gt_np = [], []

    def gen_pose_matrix(R, T):
        pose = np.eye(4)
        pose[0:3, 0:3] = R.cpu().numpy()
        pose[0:3, 3] = T.cpu().numpy()
        return pose

    **for kf_id in kf_ids:
        kf = frames[kf_id]
        pose_est = np.linalg.inv(gen_pose_matrix(kf.R, kf.T))
        pose_gt = np.linalg.inv(gen_pose_matrix(kf.R_gt, kf.T_gt))

        trj_id.append(frames[kf_id].uid)
        trj_est.append(pose_est.tolist())
        trj_gt.append(pose_gt.tolist())

        trj_est_np.append(pose_est)
        trj_gt_np.append(pose_gt)**

    trj_data["trj_id"] = trj_id
    trj_data["trj_est"] = trj_est
    trj_data["trj_gt"] = trj_gt

    plot_dir = os.path.join(save_dir, "plot")
    mkdir_p(plot_dir)

    label_evo = "final" if final else "{:04}".format(iterations)
    with open(
        os.path.join(plot_dir, f"trj_{label_evo}.json"), "w", encoding="utf-8"
    ) as f:
        json.dump(trj_data, f, indent=4)

    ate = evaluate_evo(
        poses_gt=trj_gt_np,
        poses_est=trj_est_np,
        plot_dir=plot_dir,
        label=label_evo,
        monocular=monocular,
    )
    wandb.log({"frame_idx": latest_frame_idx, "ate": ate})
    return ate

Could you please clarify if the trajectory error (ATE RMSE) in your paper is derived from re-evaluating all estimated camera poses?

[CBY-9527]

Dear Author,

I have reviewed the code you provided regarding the pose evaluation section.

MonoGS\utils\eval_utils.py：

def eval_ate(frames, kf_ids, save_dir, iterations, final=False, monocular=False):
    trj_data = dict()
    latest_frame_idx = kf_ids[-1] + 2 if final else kf_ids[-1] + 1
    trj_id, trj_est, trj_gt = [], [], []
    trj_est_np, trj_gt_np = [], []

    def gen_pose_matrix(R, T):
        pose = np.eye(4)
        pose[0:3, 0:3] = R.cpu().numpy()
        pose[0:3, 3] = T.cpu().numpy()
        return pose

    **for kf_id in kf_ids:
        kf = frames[kf_id]
        pose_est = np.linalg.inv(gen_pose_matrix(kf.R, kf.T))
        pose_gt = np.linalg.inv(gen_pose_matrix(kf.R_gt, kf.T_gt))

        trj_id.append(frames[kf_id].uid)
        trj_est.append(pose_est.tolist())
        trj_gt.append(pose_gt.tolist())

        trj_est_np.append(pose_est)
        trj_gt_np.append(pose_gt)**

    trj_data["trj_id"] = trj_id
    trj_data["trj_est"] = trj_est
    trj_data["trj_gt"] = trj_gt

    plot_dir = os.path.join(save_dir, "plot")
    mkdir_p(plot_dir)

    label_evo = "final" if final else "{:04}".format(iterations)
    with open(
        os.path.join(plot_dir, f"trj_{label_evo}.json"), "w", encoding="utf-8"
    ) as f:
        json.dump(trj_data, f, indent=4)

    ate = evaluate_evo(
        poses_gt=trj_gt_np,
        poses_est=trj_est_np,
        plot_dir=plot_dir,
        label=label_evo,
        monocular=monocular,
    )
    wandb.log({"frame_idx": latest_frame_idx, "ate": ate})
    return ate

Could you please clarify if the trajectory error (ATE RMSE) in your paper is derived from re-evaluating all estimated camera poses?

Hello, I have the same confusion as you. If it is convenient, can you add WeChat ( ycb2468 ) for communication?

[Justzhongjj]

你可直接改成对所有帧进行评估，我的结果是tum数据集确实是比论文里的指标差的，replica是差不多的 @CBY-9527

[CBY-9527]

你可直接改成对所有帧进行评估，我的结果是tum数据集确实是比论文里的指标差的，replica是差不多的 @CBY-9527

还有个问题就是论文用图像评价指标是优化前还是后的呢，我看优化前的指标比论文低很多，但优化后的指标比比论文里面高很多

[Justzhongjj]

@CBY-9527 我感觉是color_refinement后的，如果是优化之前的话，那差的实在是太多了，优化之后的话，我这边的平均数据是跟原文数据的差不多，你可以算一下平均值，你的微信号好像不对，可以给个Q号吗？

[suvzzc]

@CBY-9527 我感觉是color_refinement后的，如果是优化之前的话，那差的实在是太多了，优化之后的话，我这边的平均数据是跟原文数据的差不多，你可以算一下平均值，你的微信号好像不对，可以给个Q号吗？
我也有这个疑惑，我想问一下您的平均是指将优化之前和优化之后的数据进行平均吗？

[CBY-9527]

@CBY-9527 我感觉是color_refinement后的，如果是优化之前的话，那差的实在是太多了，优化之后的话，我这边的平均数据是跟原文数据的差不多，你可以算一下平均值，你的微信号好像不对，可以给个Q号吗？

不好意思，之前微信验证设置里面关闭了微信号添加，现在可以用微信号添加了（ycb2468）。