Krishna Murthy Jatavallabhula, Ganesh Iyer, Liam Paull ·

gradSLAM - Dense SLAM meets Automatic Differentiation

Abstract

The question of “representation” is central in the context of dense simultaneous localization and mapping (SLAM). Newer learning-based approaches have the potential to leverage data or task performance to directly inform the choice of representation. However, learning representations for SLAM has been an open question, because traditional SLAM systems are not end-to-end differentiable. In this work, we present gradSLAM, a differentiable computational graph take on SLAM. Leveraging the automatic differentiation capabilities of computational graphs, gradSLAM enables the design of SLAM systems that allow for gradient-based learning across each of their components, or the system as a whole. This is achieved by creating differentiable alternatives for each non-differentiable component in a typical dense SLAM system. Specifically, we demonstrate how to design differentiable trust-region optimizers, surface measurement and fusion schemes, as well as differentiate over rays, without sacrificing performance. This amalgamation of dense SLAM with computational graphs enables us to backprop all the way from 3D maps to 2D pixels, opening up new possibilities in gradient-based learning for SLAM.

Explanatory video

Click on the image below, to watch a video that explains what gradSLAM is, and showcases our primary results.

Click to watch the explanatory video.

About gradSLAM

gradSLAM is a fully differentiable dense SLAM framework. It provides a repository of differentiable building blocks for a dense SLAM system, such as differentiable nonlinear least squares solvers, differentiable ICP (iterative closest point) techniques, differentiable raycasting modules, and differentiable mapping/fusion blocks. One can use these blocks to construct SLAM systems that allow gradients to flow all the way from the outputs of the system (map, trajectory) to the inputs (raw color/depth images, parameters, calibration, etc.).

Specifically, we implement differentiable versions three classical dense SLAM systems using the gradSLAM framework: KinectFusion, PointFusion, ICP-SLAM. We choose these because

  • Each of these approaches sparked a new line-of-research in dense SLAM
  • The approaches themselves are fairly simple from an algorithmic standpoint
  • We aim to provide differentiable SLAM solutions for a wide variety of 3D representations (voxels, surfels, points).

Qualitative results

The differentiable SLAM systems perform quite similarly to the non-differentiable counterparts, while allowing for gradients to flow right through. This makes gradSLAM attractively poised to be used in gradient-based learning systems.

Here’s a comparative analysis of the three differentiable SLAM systems we provide.

More qualitative results, this time, on the ScanNet dataset.

Further, we have evaluated gradSLAM on the TUM RGB-D benchmark, as well as on an in-house sequence captured from an Intel RealSense D435 camera.

Paper

A preprint is currently available on arXiv, at the following link. Click on the thumbnail below, to access.

Click to access the preprint.

Citing us

If you would like to cite us, you could use the following BibTeX entry.

@article{gradslam,
    author  = { Krishna Murthy, J. and Iyer, Ganesh and Paull, Liam },
    title   = { gradSLAM: Dense SLAM meets Automatic Differentiation },
    journal = { arXiv },
    year    = { 2019 },
}

Code

gradSLAM will be released as an open-source SLAM framework for PyTorch. As the manuscript is currently under review, we are eyeing a release date in the latter half of January 2020.

To be notified when code is released, one can track this GitHub repository.

Correspondence

If you need to discuss any further, or seek clarifications on implementation detail, correspondence is encouraged to Krishna Murthy, Ganesh Iyer, or Liam Paull. Also, we would like to hear more from anyone working on similar ideas.

More qualitative results

Stay tuned for more gifs!!!

Acknowledgements

The authors are grateful to the wonderful help from several people, including, but not limited to, Gunshi Gupta, Ankur Handa, Bhairav Mehta, Mark Van der Merwe, Aaditya Saraiya, Parv Parkhiya, Akshit Gandhi, Shubham Garg, Tejas Khot, Gautham Swaminathan, Zeeshan Zia, Ronald Clark, and Sajad Saeedi.

Twitter, Facebook