book

3D Data Science with Python

by Florent Poux

April 2025

Intermediate to advanced

690 pages

18h 19m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Should Read This Book?What Will You Learn?Why This Book?A Word from the AuthorNavigating This BookPrerequisitesCode, Data, and ResourcesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
3D Data Science in BriefDimensions and 3D Data ScienceSpatial AI: From Reality to Virtuality3D Data: Fundamental Building BlocksGeometry, Topology, and SemanticsIntegrating Geometry, Topology, and SemanticsIntroduction to 3D Point CloudsThe 3D Data Science Modular WorkflowData AcquisitionPreprocessingRegistration3D Data Classification (Semantic Injection)Structuration/Modeling3D Data Analysis3D Data VisualizationApplication (Software) DevelopmentThe Case for AutomationWorkflow Challenges in 3D Data Science3D Data Science in the IndustrySummary
Fundamental ResourcesMathematicsComputer Science3D Data ExpertiseArtificial Intelligence for 3DHardware Recommendations for 3DLocal 3D DevelopmentCloud ComputingEssential Software and Tools for 3D3D Reconstruction Software3D Data Processing Software3D Visualization SoftwareSummary
3D Python Setup and LibrariesChoice of OSEnvironment SetupBase Python Libraries3D Python LibrariesThe Python IDECreating a 3D Python ProgramImporting 3D Data in PythonExtracting Specific AttributesConducting Attribute-based Data Analysis3D Data Visualization and Export3D Reconstruction MethodsReal-World 3D Reconstruction (Sensor-Based)Creative 3D Reconstruction3D Dataset: Curation3D Data from Image-based ReconstructionMultimodal Web ScrapingSummary
3D Data Representations3D Point CloudsImage-based RepresentationsVolumetric (Voxel) ModelsHigh-level 3D Data Representation3D Surface Models3D Data Canonical LinkMesh to Point CloudVoxel to Point CloudRaster to Point Cloud3D Data Structures: k-d Trees, Octrees, BVHk-d TreesOctreesFile OrganizationSummary
3D Python and Code Setup3D Data Curation3D Data PreparationInitial Profiling3D Data DownsamplingData Preprocessing3D Data VisualizationMultimodal 3D ExperiencePoint of Interest QueryManual Boundary SelectionFind High and Low PointsPoint Cloud VoxelizationBuilt Coverage ExtractionSummary
FundamentalsInitial PreprocessingFeature Extraction FundamentalsStrategies for Point Cloud Feature ExtractionGlobal Feature ExtractionLocal Feature ExtractionPrincipal Component AnalysisPython and Data PreparationCluster Identification with pandas3D Data NormalizationExtracting the Principal Components3D Visualization of PCA3D Data Registration: Unifying Perspectives3D Data Registration FundamentalsRegistration InitializationCoarse RegistrationIterative Closest PointFine Registration: ICPSummary
3D Project Environment PreparationGathering DatasetsPython and Environment Setup3D Data Fundamentals with PyVista3D Data Structure Creation (KDTree)Covariance Matrix, Eigenvalues, and EigenvectorsPlanarity, Linearity, Omnivariance, Verticality, NormalsNeighborhood Definition and SelectionAutomation and ScalingInteractive Thresholding3D Data Results ExportSummary
Types of 3D Data Analysis3D Descriptive Data Analysis3D Exploratory Data Analysis3D Predictive Data Analysis3D Prescriptive Data AnalysisAdditional Considerations3D Data Analytical ToolsEnvironment and Data PreparationMetadata Analysis and Data ProfilingGeometry and Shape AnalysisStatistical AnalysisAttribute Analysis3D Diagnostic Tools3D Deviation Analysis: Planar Case3D Deviation Analysis: Mesh CaseSummary

RANSAC from Scratch: 3D Planar Shape RecognitionRANSACData and Environment SetupGeometric Model Selection3D Shape FittingIteration and Function DefinitionApplication 1: RANSAC for Segmentation TasksApplication 2: RANSAC for Analytical TasksApplication 3: RANSAC for Modeling TasksRegion Growing for 3D Shape DetectionRegion Growing PrinciplesRegion Growing: Real-World SetupRegion Growing: ImplementationA Hybrid Approach: RANSAC and Region GrowingSummary
High-Fidelity MeshingGeneral Overview of High-Fidelity 3D MeshesThe MissionData PreparationChoose a Meshing StrategyOther 3D Meshing Strategies3D Meshing with PythonLevels of Detail CreationVisualization and Software3D Voxels and VoxelizationPython Environment InitializationLoading the DataCreating the Voxel GridGenerating the Voxel Cubes (3D Meshes)Export the Mesh Object (.ply or .obj)Parametric ModelingCadQuery and Environment SetupI/O for Parametric Models: 2D (DXF) and 3D (STL)Parametric Modeling TechniquesThe Boolean OperationsModeling Various PiecesConclusionMonocular Image-based 3D Modeling: Depth Estimation and ReconstructionSetting Up the Environment and Installing the LibrariesGathering a DatasetImage Preprocessing and Model SetupDepth Estimation Predictions from the ModelPoint Cloud GenerationDefining the Camera Intrinsics3D Modeling: 3D Point Cloud to MeshSummary
Phase 1: 3D Python SetupProject Environment SetupProject Notebook SetupPhase 2: Data PreparationAerial LiDAR Data CurationAerial LiDAR Data PreprocessingPhase 3: ExperimentsUnsupervised Point Cloud Segmentation3D House Segment Isolation2D Building Footprint ExtractionSemantic and Attribute Extraction2D to 3D Vectors3D Model Creation: MeshPhase 4: Automation and ScalingSummary
Clustering for Unsupervised SegmentationClustering FundamentalsClustering RepresentativityTypes of Clustering Algorithmsk-Means Clusteringk-Means: Workflow Definition3D Python Context DefinitionLiDAR Data Preprocessingk-Means ImplementationDBSCAN for Unsupervised SegmentationDBSCAN PrinciplesThe StrategyExperimental Setup3D Planar Shape Recognition with RANSACDBSCAN for 3D Point Cloud SegmentationThe Multi-RANSAC FrameworkMulti-RANSAC Refinement with DBSCANDBSCAN RefinementDBSCAN Versus k-MeansSummary
Connectivity-based ClusteringThe Mission BriefCore PrinciplesStep 1: Environment SetupStep 2: Graph Theory for 3D ClusteringStep 3: Graph AnalyticsStep 4: Plotting Graphs (Optional)Step 5: Connected Components for Point CloudsStep 6: Euclidean Clustering for 3D Point CloudsDiscussion and PerspectivesThe Segment Anything ModelThe Mission3D Project SetupSegment Anything Model Core Concepts3D Point Cloud to Image ProjectionsUnsupervised Segmentation with SAMSummary
From Unsupervised to Supervised LearningSupervised Learning ConceptsSupervised Learning Classification3D Semantic Segmentation Example3D Point Cloud Semantic Segmentation3D Python and Data SetupFeature Selection and PreparationMetrics and ModelsInference and GeneralizationSpecializing 3D Machine Learning with 3D Deep LearningSummary
3D Deep Learning BackboneNetwork ArchitectureData PreparationAI Model TrainingServing a Trained ModelImplementation with PyTorchInstalling PyTorch (with CUDA)Tensors: The Building BlocksNeural Network ModulesDefining a 3D Neural NetworkHyperparameter DefinitionOptimizer and Loss FunctionsPyTorch DataLoaderPyTorch Training LoopPyTorch Inference3D Deep Learning: The Architectures3D Convolutional Neural Networks: Voxels3D Graph Neural NetworksPoint-based Architectures: PointNet and Point CloudsMultiview CNNs3D Machine Learning Versus 3D Deep LearningFine-Tuning, Transfer Learning, and 3D Data AugmentationTransfer LearningFine-Tuning3D Data Augmentation: Expanding the DatasetSummary
PointNet: A Point-based 3D Deep Learning Architecture3D Object Classification3D Object Classification FundamentalsEnvironment SetupDataset CurationPointNet: Dataset PreparationPointNet Architecture DefinitionPointNet Loss DefinitionPointNet TrainingPointNet Metrics and EvaluationPointNet Real-World InferenceLarge-Scale Semantic Segmentation ConsiderationsSummary
3D Data Acquisition3D Data Preparation and EngineeringNoise RemovalSubsamplingFeature Extraction3D Data Modeling3D Mesh ReconstructionVoxelization of 3D Digital Environmentsk-d TreesOctreesSemantic ExtractionClustering and Unsupervised SegmentationSemantic Segmentation3D Object Classification3D Data Visualization and Analysis3D Shape Recognition3D Data Analytical Tools3D Multimodal Python ViewerSummary
Advanced 3D ProjectsGenerative AI for 3D Reconstruction3D Deep Point Cloud Registration3D Semantic Modeling3D Semantic Extraction with Transformers3D Gaussian Splatting for 3D VisualizationSpatial AI: The Future of 3D Experiences3D Scene Understanding with Open Vocabularies3D Spatial AI ReasoningConclusion

Overview

Our physical world is grounded in three dimensions. To create technology that can reason about and interact with it, our data must be 3D too. This practical guide offers data scientists, engineers, and researchers a hands-on approach to working with 3D data using Python. From 3D reconstruction to 3D deep learning techniques, you'll learn how to extract valuable insights from massive datasets, including point clouds, voxels, 3D CAD models, meshes, images, and more.

Dr. Florent Poux helps you leverage the potential of cutting-edge algorithms and spatial AI models to develop production-ready systems with a focus on automation. You'll get the 3D data science knowledge and code to:

Understand core concepts and representations of 3D data
Load, manipulate, analyze, and visualize 3D data using powerful Python libraries
Apply advanced AI algorithms for 3D pattern recognition (supervised and unsupervised)
Use 3D reconstruction techniques to generate 3D datasets
Implement automated 3D modeling and generative AI workflows
Explore practical applications in areas like computer vision/graphics, geospatial intelligence, scientific computing, robotics, and autonomous driving
Build accurate digital environments that spatial AI solutions can leverage

Florent Poux is an esteemed authority in the field of 3D data science who teaches and conducts research for top European universities. He's also head professor at the 3D Geodata Academy and innovation director for French Tech 120 companies.