e2e: yaw, pitch, roll swapped

[ivantishchenko]

Thank you for open-sourcing this library!

I'm confused about the arguments h_deg, v_deg, roll for e2e.

Using this example of e2e:

       img_hdr = e2e(torch.from_numpy(img_hdr),
                      h_deg=h_deg,
                      v_deg=v_deg,
                      roll=roll,
                      mode='bilinear',
                      channels_first=False).numpy()

Judging from the naming of arguments and also from the doc stub, I would expect that h_deg shift the image left/right, v_deg would shift the image up/down, and that roll would rotate clockwise/counter-clockwise.

However, that mapping I got while experimenting is actually:

• shifting image horizontally is the parameter v_deg.
• shifting image vertically is the parameter roll
• rolling the image is h_deg.

For example using this snippet:

        img_hdr = e2e(torch.from_numpy(img_hdr),
                      h_deg=90,
                      v_deg=0,
                      roll=0,
                      mode='bilinear',
                      channels_first=False).numpy()

I would expect that the image would be shifted horizontally by 90 degrees. However, in reality it's being rotated counter-clockwise. It seems that the order of rotations is not consistent with the arguments and the documentation.

    # Apply rotations: first roll, then pitch, then yaw
    xyz_rot = xyz @ Rx @ Ry @ Rz

Which coordinate system is being used? Here is how I understood it:

[ProGamerGov]

@ivantishchenko I see what you mean. Does swapping the two rotation matrices resolve some of the issues?

def e2e(...

    Ry = rotation_matrix(
        yaw_rad, torch.tensor([0.0, 0.0, 1.0], device=e_img.device, dtype=e_img.dtype)
    )
    Rz = rotation_matrix(
        pitch_rad, torch.tensor([0.0, 1.0, 0.0], device=e_img.device, dtype=e_img.dtype)
    )

With matrix variable names swapped:

def e2e(...

    Ry = rotation_matrix(
        pitch_rad, torch.tensor([1.0, 0.0, 0.0], device=e_img.device, dtype=e_img.dtype)
    )
    Rz = rotation_matrix(
        yaw_rad, torch.tensor([0.0, 1.0, 0.0], device=e_img.device, dtype=e_img.dtype)
    )

h_deg should control left/right movement, and v_deg should be up/down.

You are also right that yaw should be negative.

    yaw = -h_deg

[ivantishchenko]

@ProGamerGov I've tested pytorch360convert==0.2.3 with the fix inside and can confirm that the issue is fixed. Thank you for that! 🎉 ❤️

Now the mapping is correct:

                  h_deg=yaw,
                  v_deg=pitch,
                  roll=roll,

However, I've noticed that the arguments rotate the observer and not the image as in some other libraries. In other words yaw of +90 degrees would shift the image 90 degrees to the left and yaw of -90 degrees would shift the image to the right. In my opinion, it's fine the way it is, as one will figure out such details during the experimentation anyways. Maybe it could help to mention it in the documentation.

[baldassarreFe]

Thanks for the fix!