Learning3D: A Modern Library for Deep Learning on 3D Point Clouds Data.

Learning3D is an open-source library that supports the development of deep learning algorithms that deal with 3D data. The Learning3D exposes a set of state of art deep neural networks in python. A modular code has been provided for further development. We welcome contributions from the open-source community.

Latest News:

[28 Feb, 2025]: CurveNet is now a part of learning3d library.
[7 Apr, 2024]: Now, learning3d is available as pypi package.
[24 Oct, 2023]: MaskNet++ is now a part of learning3d library.
[12 May, 2022]: ChamferDistance loss function is incorporated in learning3d. This is a purely pytorch based loss function.
[24 Dec. 2020]: MaskNet is now ready to enhance the performance of registration algorithms in learning3d for occluded point clouds.
[24 Dec. 2020]: Loss based on the predicted and ground truth correspondences is added in learning3d after consideration of Correspondence Matrices are Underrated paper.
[24 Dec. 2020]: PointConv, latent feature estimation using convolutions on point clouds is now available in learning3d.
[16 Oct. 2020]: DeepGMR, registration using gaussian mixture models is now available in learning3d
[14 Oct. 2020]: Now, use your own data in learning3d. (Check out UserData functionality!)

PyPI package setup

Setup from pypi server

pip install learning3d

Setup using code

git clone https://github.com/vinits5/learning3d.git
cd learning3d
git checkout pypi_v0.1.0
python3 -m pip install .

Available Computer Vision Algorithms in Learning3D

| Sr. No. | Tasks | Algorithms | |:-------------:|:----------:|:-----| | 1 | Classification | PointNet, DGCNN, PPFNet, PointConv, CurveNet | | 2 | Segmentation | PointNet, DGCNN | | 3 | Reconstruction | Point Completion Network (PCN) | | 4 | Registration | PointNetLK, PCRNet, DCP, PRNet, RPM-Net, DeepGMR | | 5 | Flow Estimation | FlowNet3D | | 6 | Inlier Estimation | MaskNet, MaskNet++ |

Available Pretrained Models

PointNet
PCN
PointNetLK
PCRNet
DCP
PRNet
FlowNet3D
RPM-Net (clean-trained.pth, noisy-trained.pth, partial-pretrained.pth)
DeepGMR
PointConv (Download from this link)
MaskNet
MaskNet++ / MaskNet2
CurveNet

Available Datasets

ModelNet40

Available Loss Functions

Classification Loss (Cross Entropy)
Registration Losses (FrobeniusNormLoss, RMSEFeaturesLoss)
Distance Losses (Chamfer Distance, Earth Mover's Distance)
Correspondence Loss (based on this paper)

Technical Details

Supported OS

Ubuntu 16.04
Ubuntu 18.04
Ubuntu 20.04.6
Linux Mint
macOS Sequoia 15.3.1

Requirements

CUDA 10.0 or higher
Pytorch 1.3 or higher
Python 3.8

How to use this library?

Important Note: Clone this repository in your project. Please don't add your codes in "learning3d" folder.

All networks are defined in the module "models".
All loss functions are defined in the module "losses".
Data loaders are pre-defined in data_utils/dataloaders.py file.
All pretrained models are provided in learning3d/pretrained folder.

Documentation

B: Batch Size, N: No. of points and C: Channels.

Use of Point Embedding Networks:

from learning3d.models import PointNet, DGCNN, PPFNet
pn = PointNet(emb_dims=1024, input_shape='bnc', use_bn=False)
dgcnn = DGCNN(emb_dims=1024, input_shape='bnc')
ppf = PPFNet(features=['ppf', 'dxyz', 'xyz'], emb_dims=96, radius='0.3', num_neighbours=64)

Use of Classification / Segmentation Network:

from learning3d.models import Classifier, PointNet, Segmentation
classifier = Classifier(feature_model=PointNet(), num_classes=40)
seg = Segmentation(feature_model=PointNet(), num_classes=40)

| Sr. No. | Variable | Data type | Shape | Choices | Use | |:---:|:---:|:---:|:---:|:---:|:---:| | 1. | feature_model | Object | - | PointNet / DGCNN | Point cloud embedding network | | 2. | num_classes | Integer | Scalar | 10, 40 | Number of object categories to be classified | | 3. | output | tensor | Classification: Bx40, Segmentation: BxNx40 | 10, 40 | Probabilities of each category or each point |

Use of Registration Networks:

from learning3d.models import PointNet, PointNetLK, DCP, iPCRNet, PRNet, PPFNet, RPMNet
pnlk = PointNetLK(feature_model=PointNet(), delta=1e-02, xtol=1e-07, p0_zero_mean=True, p1_zero_mean=True, pooling='max')
dcp = DCP(feature_model=PointNet(), pointer_='transformer', head='svd')
pcrnet = iPCRNet(feature_moodel=PointNet(), pooling='max')
rpmnet = RPMNet(feature_model=PPFNet())
deepgmr = DeepGMR(use_rri=True, feature_model=PointNet(), nearest_neighbors=20)

Use of Inlier Estimation Network (MaskNet):

from learning3d.models import MaskNet, PointNet, MaskNet2
masknet = MaskNet(feature_model=PointNet(), is_training=True) masknet2 = MaskNet2(feature_model=PointNet(), is_training=True)

Use of Point Completion Network:

from learning3d.models import PCN
pcn = PCN(emb_dims=1024, input_shape='bnc', num_coarse=1024, grid_size=4, detailed_output=True)

Use of PointConv:

Use the following to create pretrained model provided by authors.

from learning3d.models import create_pointconv
PointConv = create_pointconv(classifier=True, pretrained='path of checkpoint')
ptconv = PointConv(emb_dims=1024, input_shape='bnc', input_channel_dim=6, classifier=True)

OR
Use the following to create your own PointConv model.

PointConv = create_pointconv(classifier=False, pretrained=None)
ptconv = PointConv(emb_dims=1024, input_shape='bnc', input_channel_dim=3, classifier=True)

PointConv variable is a class. Users can use it to create a sub-class to override create_classifier and create_structure methods in order to change PointConv's network architecture.

| Sr. No. | Variable | Data type | Choices | Use | |:---:|:---:|:---:|

Learning3d

Install / Use

README