fastdev.xform ============= .. py:module:: fastdev.xform Submodules ---------- .. toctree:: :maxdepth: 1 /api/fastdev/xform/rotation/index /api/fastdev/xform/transforms/index /api/fastdev/xform/utils/index /api/fastdev/xform/warp_rotation/index /api/fastdev/xform/warp_transforms/index Package Contents ---------------- .. py:function:: axis_angle_vector_to_matrix(axis_angle: torch.Tensor) -> torch.Tensor Convert rotations given as axis/angle to rotation matrices. :param axis_angle: Rotations given as a vector in axis angle form, as a tensor of shape (..., 3), where the magnitude is the angle turned anticlockwise in radians around the vector's direction. :returns: Rotation matrices as tensor of shape (..., 3, 3). .. py:function:: axis_angle_vector_to_quaternion(axis_angle) Convert rotations given as axis/angle to quaternions. :param axis_angle: Rotations given as a vector in axis angle form, as a tensor of shape (..., 3), where the magnitude is the angle turned anticlockwise in radians around the vector's direction. :returns: quaternions with real part first, as tensor of shape (..., 4). Reference: https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation#Unit_quaternions .. py:function:: compose_axis_angle_vector(axis, angle) .. py:function:: matrix_to_axis_angle_vector(matrix: torch.Tensor) -> torch.Tensor Convert rotations given as rotation matrices to axis/angle. :param matrix: Rotation matrices as tensor of shape (..., 3, 3). :returns: Rotations given as a vector in axis angle form, as a tensor of shape (..., 3), where the magnitude is the angle turned anticlockwise in radians around the vector's direction. .. py:function:: matrix_to_euler_angles(matrix: torch.Tensor, convention: str = 'xyz') -> torch.Tensor Convert rotations given as rotation matrices to Euler angles in radians. :param matrix: Rotation matrices with shape (..., 3, 3). :param convention: Convention string of 3/4 letters, e.g. "xyz", "sxyz", "rxyz", "exyz". If the length is 3, the extrinsic rotation is assumed. If the length is 4, the first character is "r/i" (rotating/intrinsic), or "s/e" (static / extrinsic). The remaining characters are the axis "x, y, z" in the order. :returns: Euler angles in radians with shape (..., 3). .. py:function:: matrix_to_rotation_6d(matrix: torch.Tensor) -> torch.Tensor Converts rotation matrices to 6D rotation representation by Zhou et al. [1] by dropping the last row. Note that 6D representation is not unique. :param matrix: batch of rotation matrices of size [..., 3, 3] :returns: 6D rotation representation, of shape [..., 6] [1] Zhou, Y., Barnes, C., Lu, J., Yang, J., & Li, H. On the Continuity of Rotation Representations in Neural Networks. CVPR 2019. arxiv_ .. _arxiv: https://arxiv.org/pdf/1812.07035 .. py:function:: quaternion_invert(quaternion: torch.Tensor) -> torch.Tensor Given a quaternion representing rotation, get the quaternion representing its inverse. :param quaternion: Quaternions as tensor of shape (..., 4), with real part first, which must be versors (unit quaternions). :returns: The inverse, a tensor of quaternions of shape (..., 4). .. py:function:: quaternion_multiply(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor Multiply two quaternions representing rotations, returning the quaternion representing their composition, i.e. the versor with nonnegative real part. Usual torch rules for broadcasting apply. :param a: Quaternions as tensor of shape (..., 4), real part first. :param b: Quaternions as tensor of shape (..., 4), real part first. :returns: The product of a and b, a tensor of quaternions of shape (..., 4). .. py:function:: quaternion_real_to_first(quaternions) .. py:function:: quaternion_real_to_last(quaternions) .. py:function:: quaternion_to_axis_angle_vector(quaternions) Convert rotations given as quaternions to axis/angle. :param quaternions: quaternions with real part first, as tensor of shape (..., 4). :returns: Rotations given as a vector in axis angle form, as a tensor of shape (..., 3), where the magnitude is the angle turned anticlockwise in radians around the vector's direction. Reference: https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation#Unit_quaternions .. py:function:: quaternion_to_matrix(quaternions: torch.Tensor) -> torch.Tensor Convert rotations given as quaternions to rotation matrices. :param quaternions: quaternions with real part first with shape (..., 4). :type quaternions: Tensor :returns: Rotation matrices as tensor of shape (..., 3, 3). :rtype: Tensor Reference: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation .. py:function:: random_rotation_matrix(num: Optional[int] = None, random_state: Optional[Union[int, numpy.random.Generator, numpy.random.RandomState]] = None, return_tensors: Literal['np', 'pt'] = 'np') .. py:function:: rotation_6d_to_matrix(d6: torch.Tensor) -> torch.Tensor Converts 6D rotation representation by Zhou et al. [1] to rotation matrix using Gram--Schmidt orthogonalization per Section B of [1]. :param d6: 6D rotation representation of shape [..., 6] :type d6: Tensor :returns: Rotation matrices of shape [..., 3, 3] :rtype: Tensor [1] Zhou, Y., Barnes, C., Lu, J., Yang, J., & Li, H. On the Continuity of Rotation Representations in Neural Networks. CVPR 2019. arxiv_ `pytorch3d implementation`_ .. _arxiv: https://arxiv.org/pdf/1812.07035 .. _`pytorch3d implementation`: https://github.com/facebookresearch/pytorch3d/blob/bd52f4a408b29dc6b4357b70c93fd7a9749ca820/pytorch3d/transforms/rotation_conversions.py#L558 .. py:function:: split_axis_angle_vector(axis_angle) .. py:function:: standardize_quaternion(quaternions: torch.Tensor) -> torch.Tensor Convert a unit quaternion to a standard form: one in which the real part is non negative. :param quaternions: Quaternions with real part first, as tensor of shape (..., 4). :returns: Standardized quaternions as tensor of shape (..., 4). .. py:function:: expand_tf_mat(tf_mat: jaxtyping.Float[torch.Tensor, ... 3 4]) -> jaxtyping.Float[torch.Tensor, ... 4 4] Expand transformation matrix of shape [... 3 4] to shape [... 4 4]. :param tf_mat: Transformation matrix in shape [... 3 4] or [... 4 4]. :type tf_mat: torch.Tensor :returns: Expanded transformation matrix in shape [... 4 4]. :rtype: torch.Tensor .. rubric:: Examples >>> tf_mat = torch.tensor([[0, 1, 0, 1], [0, 0, 1, 2], [1, 0, 0, 3]], dtype=torch.float32) >>> expand_tf_mat(tf_mat) tensor([[0., 1., 0., 1.], [0., 0., 1., 2.], [1., 0., 0., 3.], [0., 0., 0., 1.]]) .. py:function:: inverse_tf_mat(rot_or_tf_mat: torch.Tensor) -> torch.Tensor Inverse a rotation matrix or a transformation matrix. Reference_ :param rot_or_tf_mat: Rotation matrix (in shape [..., 3, 3]) or transformation matrix (in shape [..., 4, 4]). :type rot_or_tf_mat: torch.Tensor :returns: Inversed matrix. :rtype: torch.Tensor .. rubric:: Examples >>> tf_mat = torch.tensor([[0, 1, 0, 1], [0, 0, 1, 2], [1, 0, 0, 3], [0, 0, 0, 1]], dtype=torch.float32) >>> torch.allclose(inverse_tf_mat(tf_mat) @ tf_mat, torch.eye(4)) True >>> rot_mat = torch.tensor([[0, 1, 0], [0, 0, 1], [1, 0, 0]], dtype=torch.float32) >>> torch.allclose(inverse_tf_mat(rot_mat) @ rot_mat, torch.eye(3)) True .. _Reference: https://math.stackexchange.com/a/1315407/757569 .. py:function:: project_points(pts: torch.Tensor, intr_mat: torch.Tensor, return_depth: Literal[False] = False) -> torch.Tensor project_points(pts: torch.Tensor, intr_mat: torch.Tensor, return_depth: Literal[True]) -> Tuple[torch.Tensor, torch.Tensor] Project 3D points in the camera space to the image plane. :param pts: 3D points, could be Nx3 or BxNx3. :param intr_mat: Intrinsic matrix, could be 3x3 or Bx3x3. :returns: the order is uv other than xy. depth (if return_depth): depth in the camera space. :rtype: pixels .. py:function:: rot_tl_to_tf_mat(rot_mat: Optional[jaxtyping.Float[torch.Tensor, ... 3 3]] = None, tl: Optional[jaxtyping.Float[torch.Tensor, ... 3]] = None) -> jaxtyping.Float[torch.Tensor, ... 4 4] Build transformation matrix with rotation matrix and translation vector. :param rot_mat: Rotation matrix in shape [... 3 3]. Defaults to None. :type rot_mat: torch.Tensor, optional :param tl: Translation vector in shape [... 3]. Defaults to None. :type tl: torch.Tensor, optional :returns: Transformation matrix in shape [... 4 4]. :rtype: torch.Tensor .. rubric:: Examples >>> rot_mat = torch.tensor([[0, 1, 0], [0, 0, 1], [1, 0, 0]], dtype=torch.float32) >>> tl = torch.tensor([1, 2, 3], dtype=torch.float32) >>> rot_tl_to_tf_mat(rot_mat, tl) tensor([[0., 1., 0., 1.], [0., 0., 1., 2.], [1., 0., 0., 3.], [0., 0., 0., 1.]]) >>> rot_tl_to_tf_mat(tl=tl) tensor([[1., 0., 0., 1.], [0., 1., 0., 2.], [0., 0., 1., 3.], [0., 0., 0., 1.]]) >>> rot_tl_to_tf_mat(rot_mat=rot_mat) tensor([[0., 1., 0., 0.], [0., 0., 1., 0.], [1., 0., 0., 0.], [0., 0., 0., 1.]]) .. py:function:: swap_major(rot_or_tf_mat: torch.Tensor) -> torch.Tensor Swap the major of a rotation matrix or a transformation matrix. Reference_ :param rot_or_tf_mat: Rotation or transformation matrix in shape [..., 3, 3] or [..., 4, 4]. :type rot_or_tf_mat: torch.Tensor :returns: Matrix with swapped major. :rtype: torch.Tensor .. _Reference: https://www.scratchapixel.com/lessons/mathematics-physics-for-computer-graphics/geometry/row-major-vs-column-major-vector # noqa .. py:function:: to_homo(pts_3d: jaxtyping.Float[torch.Tensor, ... 3]) -> jaxtyping.Float[torch.Tensor, ... 4] Convert Cartesian 3D points to Homogeneous 4D points. :param pts_3d: Cartesian 3D points in shape [... 3]. :type pts_3d: torch.Tensor :returns: Homogeneous 4D points in shape [... 4]. :rtype: torch.Tensor .. rubric:: Examples >>> pts = torch.tensor([[0, 1, 0], [0, 0, 1], [1, 0, 0]], dtype=torch.float32) >>> to_homo(pts) tensor([[0., 1., 0., 1.], [0., 0., 1., 1.], [1., 0., 0., 1.]]) .. py:function:: unproject_points(pixels, depth, intr_mat) Unproject pixels in the image plane to 3D points in the camera space. :param pixels: Pixels in the image plane, could be Nx2 or BxNx2. The order is uv rather than xy. :param depth: Depth in the camera space, could be N, Nx1, BxN or BxNx1. :param intr_mat: Intrinsic matrix, could be 3x3 or Bx3x3. :returns: 3D points, Nx3 or BxNx3. :rtype: pts .. py:function:: camera_position_from_spherical_angles(distance: float, elevation: float, azimuth: float, degrees: bool = True, device: Device = 'cpu') -> torch.Tensor Calculate the location of the camera based on the distance away from the target point, the elevation and azimuth angles. :param distance: distance of the camera from the object. :param elevation: angles. The inputs distance, elevation and azimuth can be one of the following - Python scalar - Torch scalar - Torch tensor of shape (N) or (1) :param azimuth: angles. The inputs distance, elevation and azimuth can be one of the following - Python scalar - Torch scalar - Torch tensor of shape (N) or (1) :param degrees: bool, whether the angles are specified in degrees or radians. :param device: str or torch.device, device for new tensors to be placed on. The vectors are broadcast against each other so they all have shape (N, 1). :returns: (N, 3) xyz location of the camera. :rtype: camera_position .. py:function:: compose_intr_mat(fu: float, fv: float, cu: float, cv: float, skew: float = 0.0) -> numpy.ndarray :param fu: horizontal focal length (width) :param fv: vertical focal length (height) :param cu: horizontal principal point (width) :param cv: vertical principal point (height) :param skew: skew coefficient, default to 0 .. py:function:: coord_conversion(src_spec: str, dst_spec: str, check_handness: bool = True, return_tensors: Literal['pt'] = 'pt') -> jaxtyping.Float[torch.Tensor, 3 3] coord_conversion(src_spec: str, dst_spec: str, check_handness: bool = True, return_tensors: Literal['np'] = 'np') -> jaxtyping.Float[numpy.ndarray, 3 3] Construct a rotation matrix based on given source and destination coordinate specifications. :param src_spec: Source coordinate specification, e.g., "x: right, y: down, z: front" or "opencv". :param dst_spec: Destination coordinate specification, e.g., "x: right, y: up, z: back" or "opengl". :param check_handness: If True, checks if the rotation matrix preserves right-handedness. :param return_tensors: Return type of the rotation matrix, either "np" for NumPy array or "pt" for PyTorch tensor. :returns: A 3x3 rotation matrix converting coordinates from the source to the destination specification. .. rubric:: Examples >>> coord_conversion("opencv", "opengl") array([[ 1., 0., 0.], [ 0., -1., 0.], [ 0., 0., -1.]], dtype=float32) >>> coord_conversion("x: front, y: left, z: up", "x: left, y: up, z: front") array([[0., 1., 0.], [0., 0., 1.], [1., 0., 0.]], dtype=float32) >>> coord_conversion("x: right, y: down, z: front", "x: left, y: up, z: front") array([[-1., 0., 0.], [ 0., -1., 0.], [ 0., 0., 1.]], dtype=float32) >>> coord_conversion("x: left, y: up, z: front", "x: front, y: left, z: up", return_tensors="pt") tensor([[0., 0., 1.], [1., 0., 0.], [0., 1., 0.]]) .. py:function:: look_at_rotation(camera_position: jaxtyping.Float[torch.Tensor, *batch 3], at: jaxtyping.Float[torch.Tensor, *batch 3], up: jaxtyping.Float[torch.Tensor, *batch 3]) -> jaxtyping.Float[torch.Tensor, *batch 3 3] This function takes a vector `camera_position` which specifies the location of the camera in world coordinates and two vectors `at` and `up` which indicate the position of the object and the up directions of the world coordinate system respectively. The output is a rotation matrix representing the rotation from camera coordinates to world coordinates. We use the OpenGL coordinate in this function, i.e. x -> right, y -> up, z -> backward. Hence, z_axis: pos - at, x_axis: cross(up, z_axis), y_axis: cross(z_axis, x_axis) Note that our implementation differs from pytorch3d. 1. our matrix is in the OpenGL coordinate 2. our matrix is column-major 3. our matrix is the camera-to-world transformation :param camera_position: position of the camera in world coordinates :param at: position of the object in world coordinates :param up: vector specifying the up direction in the world coordinate frame. :returns: rotation matrices of shape [..., 3, 3] :rtype: R .. py:function:: axis_angle_to_matrix(axis: jaxtyping.Float[torch.Tensor, ... 3], angle: jaxtyping.Float[torch.Tensor, ...]) -> jaxtyping.Float[torch.Tensor, ... 3 3] Converts axis angles to rotation matrices using Rodrigues formula. :param axis: axis, the shape could be [..., 3]. :type axis: torch.Tensor :param angle: angle, the shape could be [...]. :type angle: torch.Tensor :returns: Rotation matrices [..., 3, 3]. :rtype: torch.Tensor .. rubric:: Example >>> axis = torch.tensor([1.0, 0.0, 0.0]) >>> angle = torch.tensor(0.5) >>> axis_angle_to_matrix(axis, angle) tensor([[ 1.0000, 0.0000, 0.0000], [ 0.0000, 0.8776, -0.4794], [ 0.0000, 0.4794, 0.8776]]) .. py:function:: euler_angles_to_matrix(euler_angles: jaxtyping.Float[torch.Tensor, ... 3], axes: _AXES = 'sxyz') -> jaxtyping.Float[torch.Tensor, ... 3 3] Converts Euler angles to rotation matrices. :param euler_angles: Tensor of Euler angles with shape [..., 3]. :type euler_angles: torch.Tensor :param axes: Axis specification string, one of 24 possible sequences (e.g., "sxyz"). If only 3 characters are provided, "s" will be prefixed. :type axes: str :returns: Rotation matrices with shape [..., 3, 3]. :rtype: torch.Tensor .. rubric:: Example >>> euler_angles = torch.tensor([1.0, 0.5, 2.0]) >>> euler_angles_to_matrix(euler_angles, axes="sxyz") tensor([[-0.3652, -0.6592, 0.6574], [ 0.7980, 0.1420, 0.5857], [-0.4794, 0.7385, 0.4742]]) >>> euler_angles_to_matrix(euler_angles, axes="rxyz") tensor([[-0.3652, -0.7980, 0.4794], [ 0.3234, -0.5917, -0.7385], [ 0.8729, -0.1146, 0.4742]]) .. py:function:: matrix_to_quaternion(rot_mat: jaxtyping.Float[torch.Tensor, ... 3 3], scalar_first: bool = True, canonical: bool = True) -> jaxtyping.Float[torch.Tensor, ... 4] Converts rotation matrices to quaternions. :param rot_mat: Rotation matrices with shape [..., 3, 3]. :type rot_mat: torch.Tensor :returns: Quaternions with shape [..., 4]. :rtype: torch.Tensor .. rubric:: Example >>> rot_mat = torch.tensor([[-0.2533, -0.6075, 0.7529], ... [ 0.8445, -0.5185, -0.1343], ... [ 0.4720, 0.6017, 0.6443]]) >>> matrix_to_quaternion(rot_mat) tensor([0.4671, 0.3940, 0.1503, 0.7772]) .. note:: The gradient of this function differs from the pytorch3d implementation, but it should be okay for most use cases. Ref_ .. _Ref: https://github.com/facebookresearch/pytorch3d/issues/503#issuecomment-755493515 .. py:function:: rotate_points(pts: jaxtyping.Float[torch.Tensor, ... n 3], tf_mat: jaxtyping.Float[torch.Tensor, ... 3 3]) -> jaxtyping.Float[torch.Tensor, ... n 3] Apply a rotation matrix on a set of 3D points. :param pts: 3D points in shape [... n 3]. :type pts: torch.Tensor :param rot_mat: Rotation matrix in shape [... 3 3]. :type rot_mat: torch.Tensor :returns: Rotated points in shape [... n 3]. :rtype: torch.Tensor .. py:function:: transform_points(pts: jaxtyping.Float[torch.Tensor, ... n 3], tf_mat: jaxtyping.Float[torch.Tensor, ... 4 4]) -> jaxtyping.Float[torch.Tensor, ... n 3] Apply a transformation matrix on a set of 3D points. :param pts: 3D points, could be [... n 3] :type pts: torch.Tensor :param tf_mat: Transformation matrix, could be [... 4 4] :type tf_mat: torch.Tensor :returns: Transformed pts in shape of [... n 3] .. rubric:: Examples >>> pts = torch.tensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) >>> tf_mat = torch.tensor([[0.0, 1.0, 0.0, 1.0], [0.0, 0.0, 1.0, 2.0], [1.0, 0.0, 0.0, 3.0], [0.0, 0.0, 0.0, 1.0]]) >>> transform_points(pts, tf_mat) tensor([[3., 5., 4.], [6., 8., 7.]]) .. note:: The dimension number of `pts` and `tf_mat` should be the same. The batch dimensions (...) are broadcasted_ (and thus must be broadcastable). We don't adopt the shapes [... 3] and [... 4 4] because there is no real broadcasted vector-matrix multiplication in pytorch. [... 3] and [... 4 4] will be converted to [... 1 3] and [... 4 4] and apply a broadcasted matrix-matrix multiplication. .. _broadcasted: https://pytorch.org/docs/stable/notes/broadcasting.html