fastdev.nn.point_transformer_v3 =============================== .. py:module:: fastdev.nn.point_transformer_v3 .. autoapi-nested-parse:: Point Transformer - V3 Mode1 Adapted from: https://github.com/Pointcept/Pointcept This module requires the installation of the following packages: - addict: pip install addict - spconv: https://github.com/traveller59/spconv?tab=readme-ov-file#spconv-spatially-sparse-convolution-library - torch-scatter: https://github.com/rusty1s/pytorch_scatter?tab=readme-ov-file#installation - flash-attention: https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#installation-and-features Original Author: Xiaoyang Wu (xiaoyang.wu.cs@gmail.com) Please cite their work if you use the following code in your research paper. Module Contents --------------- .. py:data:: flash_attn :value: None .. py:data:: ocnn :value: None .. py:function:: drop_path(x, drop_prob: float = 0.0, training: bool = False, scale_by_keep: bool = True) Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. .. py:class:: DropPath(drop_prob: float = 0.0, scale_by_keep: bool = True) Bases: :py:obj:`torch.nn.Module` Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). .. py:attribute:: drop_prob :value: 0.0 .. py:attribute:: scale_by_keep :value: True .. py:method:: forward(x) .. py:method:: extra_repr() .. py:class:: KeyLUT .. py:method:: encode_lut(device=torch.device('cpu')) .. py:method:: decode_lut(device=torch.device('cpu')) .. py:method:: xyz2key(x, y, z, depth) .. py:method:: key2xyz(key, depth) .. py:function:: xyz2key(x: torch.Tensor, y: torch.Tensor, z: torch.Tensor, b: Optional[Union[torch.Tensor, int]] = None, depth: int = 16) Encodes :attr:`x`, :attr:`y`, :attr:`z` coordinates to the shuffled keys based on pre-computed look up tables. The speed of this function is much faster than the method based on for-loop. :param x: The x coordinate. :type x: torch.Tensor :param y: The y coordinate. :type y: torch.Tensor :param z: The z coordinate. :type z: torch.Tensor :param b: The batch index of the coordinates, and should be smaller than 32768. If :attr:`b` is :obj:`torch.Tensor`, the size of :attr:`b` must be the same as :attr:`x`, :attr:`y`, and :attr:`z`. :type b: torch.Tensor or int :param depth: The depth of the shuffled key, and must be smaller than 17 (< 17). :type depth: int .. py:function:: right_shift(binary, k=1, axis=-1) Right shift an array of binary values. Parameters: ----------- binary: An ndarray of binary values. k: The number of bits to shift. Default 1. axis: The axis along which to shift. Default -1. Returns: -------- Returns an ndarray with zero prepended and the ends truncated, along whatever axis was specified. .. py:function:: gray2binary(gray, axis=-1) Convert an array of Gray codes back into binary values. Parameters: ----------- gray: An ndarray of gray codes. axis: The axis along which to perform Gray decoding. Default=-1. Returns: -------- Returns an ndarray of binary values. .. py:function:: hilbert_encode_(locs, num_dims, num_bits) Decode an array of locations in a hypercube into a Hilbert integer. This is a vectorized-ish version of the Hilbert curve implementation by John Skilling as described in: Skilling, J. (2004, April). Programming the Hilbert curve. In AIP Conference Proceedings (Vol. 707, No. 1, pp. 381-387). American Institute of Physics. Params: ------- locs - An ndarray of locations in a hypercube of num_dims dimensions, in which each dimension runs from 0 to 2**num_bits-1. The shape can be arbitrary, as long as the last dimension of the same has size num_dims. num_dims - The dimensionality of the hypercube. Integer. num_bits - The number of bits for each dimension. Integer. Returns: -------- The output is an ndarray of uint64 integers with the same shape as the input, excluding the last dimension, which needs to be num_dims. .. py:function:: encode(grid_coord, batch=None, depth=16, order='z') .. py:function:: z_order_encode(grid_coord: torch.Tensor, depth: int = 16) .. py:function:: hilbert_encode(grid_coord: torch.Tensor, depth: int = 16) .. py:function:: offset2bincount(offset) .. py:function:: offset2batch(offset) .. py:function:: batch2offset(batch) .. py:class:: Point(*args, **kwargs) Bases: :py:obj:`addict.Dict` Point Structure of Pointcept A Point (point cloud) in Pointcept is a dictionary that contains various properties of a batched point cloud. The property with the following names have a specific definition as follows: - "coord": original coordinate of point cloud; - "grid_coord": grid coordinate for specific grid size (related to GridSampling); Point also support the following optional attributes: - "offset": if not exist, initialized as batch size is 1; - "batch": if not exist, initialized as batch size is 1; - "feat": feature of point cloud, default input of model; - "grid_size": Grid size of point cloud (related to GridSampling); (related to Serialization) - "serialized_depth": depth of serialization, 2 ** depth * grid_size describe the maximum of point cloud range; - "serialized_code": a list of serialization codes; - "serialized_order": a list of serialization order determined by code; - "serialized_inverse": a list of inverse mapping determined by code; (related to Sparsify: SpConv) - "sparse_shape": Sparse shape for Sparse Conv Tensor; - "sparse_conv_feat": SparseConvTensor init with information provide by Point; .. py:method:: serialization(order='z', depth=None, shuffle_orders=False) Point Cloud Serialization relay on ["grid_coord" or "coord" + "grid_size", "batch", "feat"] .. py:method:: sparsify(pad=96) Point Cloud Serialization Point cloud is sparse, here we use "sparsify" to specifically refer to preparing "spconv.SparseConvTensor" for SpConv. relay on ["grid_coord" or "coord" + "grid_size", "batch", "feat"] pad: padding sparse for sparse shape. .. py:method:: octreetization(depth=None, full_depth=None) Point Cloud Octreelization Generate octree with OCNN relay on ["grid_coord", "batch", "feat"] .. py:class:: PointModule(*args, **kwargs) Bases: :py:obj:`torch.nn.Module` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:class:: PointSequential(*args, **kwargs) Bases: :py:obj:`PointModule` A sequential container. Modules will be added to it in the order they are passed in the constructor. Alternatively, an ordered dict of modules can also be passed in. .. py:method:: __getitem__(idx) .. py:method:: __len__() .. py:method:: add(module, name=None) .. py:method:: forward(input) .. py:class:: PDNorm(num_features, norm_layer, context_channels=256, conditions=('ScanNet', 'S3DIS', 'Structured3D'), decouple=True, adaptive=False) Bases: :py:obj:`PointModule` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:attribute:: conditions :value: ('ScanNet', 'S3DIS', 'Structured3D') .. py:attribute:: decouple :value: True .. py:attribute:: adaptive :value: False .. py:method:: forward(point) .. py:class:: RPE(patch_size, num_heads) Bases: :py:obj:`torch.nn.Module` .. py:attribute:: patch_size .. py:attribute:: num_heads .. py:attribute:: pos_bnd .. py:attribute:: rpe_num .. py:attribute:: rpe_table .. py:method:: forward(coord) .. py:class:: SerializedAttention(channels, num_heads, patch_size, qkv_bias=True, qk_scale=None, attn_drop=0.0, proj_drop=0.0, order_index=0, enable_rpe=False, enable_flash=True, upcast_attention=True, upcast_softmax=True) Bases: :py:obj:`PointModule` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:attribute:: channels .. py:attribute:: num_heads .. py:attribute:: scale .. py:attribute:: order_index :value: 0 .. py:attribute:: upcast_attention :value: True .. py:attribute:: upcast_softmax :value: True .. py:attribute:: enable_rpe :value: False .. py:attribute:: enable_flash :value: True .. py:attribute:: qkv .. py:attribute:: proj .. py:attribute:: proj_drop .. py:attribute:: softmax .. py:attribute:: rpe :value: None .. py:method:: get_rel_pos(point, order) .. py:method:: get_padding_and_inverse(point) .. py:method:: forward(point) .. py:class:: MLP(in_channels, hidden_channels=None, out_channels=None, act_layer=nn.GELU, drop=0.0) Bases: :py:obj:`torch.nn.Module` .. py:attribute:: fc1 .. py:attribute:: act .. py:attribute:: fc2 .. py:attribute:: drop .. py:method:: forward(x) .. py:class:: Block(channels, num_heads, patch_size=48, mlp_ratio=4.0, qkv_bias=True, qk_scale=None, attn_drop=0.0, proj_drop=0.0, drop_path=0.0, norm_layer=nn.LayerNorm, act_layer=nn.GELU, pre_norm=True, order_index=0, cpe_indice_key=None, enable_rpe=False, enable_flash=True, upcast_attention=True, upcast_softmax=True) Bases: :py:obj:`PointModule` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:attribute:: channels .. py:attribute:: pre_norm :value: True .. py:attribute:: cpe .. py:attribute:: norm1 .. py:attribute:: attn .. py:attribute:: norm2 .. py:attribute:: mlp .. py:attribute:: drop_path .. py:method:: forward(point: Point) .. py:class:: SerializedPooling(in_channels, out_channels, stride=2, norm_layer=None, act_layer=None, reduce='max', shuffle_orders=True, traceable=True) Bases: :py:obj:`PointModule` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:attribute:: in_channels .. py:attribute:: out_channels .. py:attribute:: stride :value: 2 .. py:attribute:: reduce :value: 'max' .. py:attribute:: shuffle_orders :value: True .. py:attribute:: traceable :value: True .. py:attribute:: proj .. py:method:: forward(point: Point) .. py:class:: SerializedUnpooling(in_channels, skip_channels, out_channels, norm_layer=None, act_layer=None, traceable=False) Bases: :py:obj:`PointModule` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:attribute:: proj .. py:attribute:: proj_skip .. py:attribute:: traceable :value: False .. py:method:: forward(point) .. py:class:: Embedding(in_channels, embed_channels, norm_layer=None, act_layer=None) Bases: :py:obj:`PointModule` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:attribute:: in_channels .. py:attribute:: embed_channels .. py:attribute:: stem .. py:method:: forward(point: Point) .. py:class:: PointTransformerV3(in_channels=6, order=('z', 'z-trans'), stride=(2, 2, 2, 2), enc_depths=(2, 2, 2, 6, 2), enc_channels=(32, 64, 128, 256, 512), enc_num_head=(2, 4, 8, 16, 32), enc_patch_size=(48, 48, 48, 48, 48), dec_depths=(2, 2, 2, 2), dec_channels=(64, 64, 128, 256), dec_num_head=(4, 4, 8, 16), dec_patch_size=(48, 48, 48, 48), mlp_ratio=4, qkv_bias=True, qk_scale=None, attn_drop=0.0, proj_drop=0.0, drop_path=0.3, pre_norm=True, shuffle_orders=True, enable_rpe=False, enable_flash=True, upcast_attention=False, upcast_softmax=False, cls_mode=False, pdnorm_bn=False, pdnorm_ln=False, pdnorm_decouple=True, pdnorm_adaptive=False, pdnorm_affine=True, pdnorm_conditions=('ScanNet', 'S3DIS', 'Structured3D')) Bases: :py:obj:`PointModule` PointModule placeholder, all module subclass from this will take Point in PointSequential. .. py:attribute:: num_stages .. py:attribute:: order .. py:attribute:: cls_mode :value: False .. py:attribute:: shuffle_orders :value: True .. py:attribute:: embedding .. py:attribute:: enc .. py:method:: forward(data_dict)