book

3D Deep Learning with Python

Name: 3D Deep Learning with Python
ISBN: 9781803247823

by Xudong Ma, David Farrugia, Vishakh Hegde, Lilit Yolyan

October 2022

Intermediate to advanced

236 pages

5h 13m

English

Packt Publishing

Read now

Unlock full access

3D Deep Learning with Python
ContributorsAbout the authorAbout the reviewer
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
PART 1: 3D Data Processing Basics
Chapter 1: Introducing 3D Data Processing
Technical requirementsSetting up a development environment3D data representationUnderstanding point cloud representationUnderstanding mesh representationUnderstanding voxel representation3D data file format – Ply files3D data file format – OBJ filesUnderstanding 3D coordination systemsUnderstanding camera modelsCoding for camera models and coordination systemsSummary
Chapter 2: Introducing 3D Computer Vision and Geometry
Technical requirementsExploring the basic concepts of rendering, rasterization, and shadingUnderstanding barycentric coordinatesLight source modelsUnderstanding the Lambertian shading modelUnderstanding the Phong lighting modelCoding exercises for 3D renderingUsing PyTorch3D heterogeneous batches and PyTorch optimizersA coding exercise for a heterogeneous mini-batch Understanding transformations and rotationsA coding exercise for transformation and rotationSummary
PART 2: 3D Deep Learning Using PyTorch3D
Chapter 3: Fitting Deformable Mesh Models to Raw Point Clouds
Technical requirementsFitting meshes to point clouds – the problemFormulating a deformable mesh fitting problem into an optimization problemLoss functions for regularizationMesh Laplacian smoothing lossMesh normal consistency lossMesh edge lossImplementing the mesh fitting with PyTorch3DThe experiment of not using any regularization loss functionsThe experiment of using only the mesh edge lossSummary
Chapter 4: Learning Object Pose Detection and Tracking by Differentiable Rendering
Technical requirementsWhy we want to have differentiable renderingHow to make rendering differentiableWhat problems can be solved by using differentiable renderingThe object pose estimation problemHow it is codedAn example of object pose estimation for both silhouette fitting and texture fittingSummary
Chapter 5: Understanding Differentiable Volumetric Rendering
Technical requirementsOverview of volumetric renderingUnderstanding ray samplingUsing volume samplingExploring the ray marcherDifferentiable volumetric renderingReconstructing 3D models from multi-view imagesSummary
Chapter 6: Exploring Neural Radiance Fields (NeRF)
Technical requirementsUnderstanding NeRFWhat is a radiance field?Representing radiance fields with neural networksTraining a NeRF modelUnderstanding the NeRF model architectureUnderstanding volume rendering with radiance fieldsProjecting rays into the sceneAccumulating the color of a raySummary

PART 3: State-of-the-art 3D Deep Learning Using PyTorch3D
Chapter 7: Exploring Controllable Neural Feature Fields
Technical requirementsUnderstanding GAN-based image synthesisIntroducing compositional 3D-aware image synthesisGenerating feature fieldsMapping feature fields to imagesExploring controllable scene generationExploring controllable car generationExploring controllable face generationTraining the GIRAFFE modelFrechet Inception DistanceTraining the modelSummary
Chapter 8: Modeling the Human Body in 3D
Technical requirementsFormulating the 3D modeling problemDefining a good representationUnderstanding the Linear Blend Skinning techniqueUnderstanding the SMPL modelDefining the SMPL modelUsing the SMPL modelEstimating 3D human pose and shape using SMPLifyDefining the optimization objective functionExploring SMPLifyRunning the codeExploring the codeSummary
Chapter 9: Performing End-to-End View Synthesis with SynSin
Technical requirementsOverview of view synthesisSynSin network architectureSpatial feature and depth networksNeural point cloud rendererRefinement module and discriminatorHands-on model training and testingSummary
Chapter 10: Mesh R-CNN
Technical requirementsOverview of meshes and voxelsMesh R-CNN architectureGraph convolutionsMesh predictorDemo of Mesh R-CNN with PyTorchDemoSummary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from 3D Deep Learning with Python

10 Mesh R-CNN

This chapter is dedicated to a state-of-the-art model called Mesh R-CNN, which aims to combine two different but important tasks into one end-to-end model. It is a combination of the well-known image segmentation model Mask R-CNN and a new 3D structure prediction model. These two tasks were researched a lot separately.

Mask R-CNN is an object detection and instance segmentation algorithm that got the highest precision scores in benchmark datasets. It belongs to the R-CNN family and is a two-stage end-to-end object detection model.

Mesh R-CNN goes beyond the 2D object detection problem and outputs a 3D mesh of detected objects as well. If we think of the world, people see in 3D, which means the objects are 3D. So, why not have a ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Deep Learning with Python, Second Edition

Publisher Resources

ISBN: 9781803247823

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

3D Deep Learning with Python

by Xudong Ma, David Farrugia, Vishakh Hegde, Lilit Yolyan

10

Mesh R-CNN

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.