book

3D Data Science with Python

by Florent Poux

April 2025

Intermediate to advanced

690 pages

18h 19m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Should Read This Book?What Will You Learn?Why This Book?A Word from the AuthorNavigating This BookPrerequisitesCode, Data, and ResourcesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
3D Data Science in BriefDimensions and 3D Data ScienceSpatial AI: From Reality to Virtuality3D Data: Fundamental Building BlocksGeometry, Topology, and SemanticsIntegrating Geometry, Topology, and SemanticsIntroduction to 3D Point CloudsThe 3D Data Science Modular WorkflowData AcquisitionPreprocessingRegistration3D Data Classification (Semantic Injection)Structuration/Modeling3D Data Analysis3D Data VisualizationApplication (Software) DevelopmentThe Case for AutomationWorkflow Challenges in 3D Data Science3D Data Science in the IndustrySummary
Fundamental ResourcesMathematicsComputer Science3D Data ExpertiseArtificial Intelligence for 3DHardware Recommendations for 3DLocal 3D DevelopmentCloud ComputingEssential Software and Tools for 3D3D Reconstruction Software3D Data Processing Software3D Visualization SoftwareSummary
3D Python Setup and LibrariesChoice of OSEnvironment SetupBase Python Libraries3D Python LibrariesThe Python IDECreating a 3D Python ProgramImporting 3D Data in PythonExtracting Specific AttributesConducting Attribute-based Data Analysis3D Data Visualization and Export3D Reconstruction MethodsReal-World 3D Reconstruction (Sensor-Based)Creative 3D Reconstruction3D Dataset: Curation3D Data from Image-based ReconstructionMultimodal Web ScrapingSummary
3D Data Representations3D Point CloudsImage-based RepresentationsVolumetric (Voxel) ModelsHigh-level 3D Data Representation3D Surface Models3D Data Canonical LinkMesh to Point CloudVoxel to Point CloudRaster to Point Cloud3D Data Structures: k-d Trees, Octrees, BVHk-d TreesOctreesFile OrganizationSummary
3D Python and Code Setup3D Data Curation3D Data PreparationInitial Profiling3D Data DownsamplingData Preprocessing3D Data VisualizationMultimodal 3D ExperiencePoint of Interest QueryManual Boundary SelectionFind High and Low PointsPoint Cloud VoxelizationBuilt Coverage ExtractionSummary
FundamentalsInitial PreprocessingFeature Extraction FundamentalsStrategies for Point Cloud Feature ExtractionGlobal Feature ExtractionLocal Feature ExtractionPrincipal Component AnalysisPython and Data PreparationCluster Identification with pandas3D Data NormalizationExtracting the Principal Components3D Visualization of PCA3D Data Registration: Unifying Perspectives3D Data Registration FundamentalsRegistration InitializationCoarse RegistrationIterative Closest PointFine Registration: ICPSummary
3D Project Environment PreparationGathering DatasetsPython and Environment Setup3D Data Fundamentals with PyVista3D Data Structure Creation (KDTree)Covariance Matrix, Eigenvalues, and EigenvectorsPlanarity, Linearity, Omnivariance, Verticality, NormalsNeighborhood Definition and SelectionAutomation and ScalingInteractive Thresholding3D Data Results ExportSummary
Types of 3D Data Analysis3D Descriptive Data Analysis3D Exploratory Data Analysis3D Predictive Data Analysis3D Prescriptive Data AnalysisAdditional Considerations3D Data Analytical ToolsEnvironment and Data PreparationMetadata Analysis and Data ProfilingGeometry and Shape AnalysisStatistical AnalysisAttribute Analysis3D Diagnostic Tools3D Deviation Analysis: Planar Case3D Deviation Analysis: Mesh CaseSummary

RANSAC from Scratch: 3D Planar Shape RecognitionRANSACData and Environment SetupGeometric Model Selection3D Shape FittingIteration and Function DefinitionApplication 1: RANSAC for Segmentation TasksApplication 2: RANSAC for Analytical TasksApplication 3: RANSAC for Modeling TasksRegion Growing for 3D Shape DetectionRegion Growing PrinciplesRegion Growing: Real-World SetupRegion Growing: ImplementationA Hybrid Approach: RANSAC and Region GrowingSummary
High-Fidelity MeshingGeneral Overview of High-Fidelity 3D MeshesThe MissionData PreparationChoose a Meshing StrategyOther 3D Meshing Strategies3D Meshing with PythonLevels of Detail CreationVisualization and Software3D Voxels and VoxelizationPython Environment InitializationLoading the DataCreating the Voxel GridGenerating the Voxel Cubes (3D Meshes)Export the Mesh Object (.ply or .obj)Parametric ModelingCadQuery and Environment SetupI/O for Parametric Models: 2D (DXF) and 3D (STL)Parametric Modeling TechniquesThe Boolean OperationsModeling Various PiecesConclusionMonocular Image-based 3D Modeling: Depth Estimation and ReconstructionSetting Up the Environment and Installing the LibrariesGathering a DatasetImage Preprocessing and Model SetupDepth Estimation Predictions from the ModelPoint Cloud GenerationDefining the Camera Intrinsics3D Modeling: 3D Point Cloud to MeshSummary
Phase 1: 3D Python SetupProject Environment SetupProject Notebook SetupPhase 2: Data PreparationAerial LiDAR Data CurationAerial LiDAR Data PreprocessingPhase 3: ExperimentsUnsupervised Point Cloud Segmentation3D House Segment Isolation2D Building Footprint ExtractionSemantic and Attribute Extraction2D to 3D Vectors3D Model Creation: MeshPhase 4: Automation and ScalingSummary
Clustering for Unsupervised SegmentationClustering FundamentalsClustering RepresentativityTypes of Clustering Algorithmsk-Means Clusteringk-Means: Workflow Definition3D Python Context DefinitionLiDAR Data Preprocessingk-Means ImplementationDBSCAN for Unsupervised SegmentationDBSCAN PrinciplesThe StrategyExperimental Setup3D Planar Shape Recognition with RANSACDBSCAN for 3D Point Cloud SegmentationThe Multi-RANSAC FrameworkMulti-RANSAC Refinement with DBSCANDBSCAN RefinementDBSCAN Versus k-MeansSummary
Connectivity-based ClusteringThe Mission BriefCore PrinciplesStep 1: Environment SetupStep 2: Graph Theory for 3D ClusteringStep 3: Graph AnalyticsStep 4: Plotting Graphs (Optional)Step 5: Connected Components for Point CloudsStep 6: Euclidean Clustering for 3D Point CloudsDiscussion and PerspectivesThe Segment Anything ModelThe Mission3D Project SetupSegment Anything Model Core Concepts3D Point Cloud to Image ProjectionsUnsupervised Segmentation with SAMSummary
From Unsupervised to Supervised LearningSupervised Learning ConceptsSupervised Learning Classification3D Semantic Segmentation Example3D Point Cloud Semantic Segmentation3D Python and Data SetupFeature Selection and PreparationMetrics and ModelsInference and GeneralizationSpecializing 3D Machine Learning with 3D Deep LearningSummary
3D Deep Learning BackboneNetwork ArchitectureData PreparationAI Model TrainingServing a Trained ModelImplementation with PyTorchInstalling PyTorch (with CUDA)Tensors: The Building BlocksNeural Network ModulesDefining a 3D Neural NetworkHyperparameter DefinitionOptimizer and Loss FunctionsPyTorch DataLoaderPyTorch Training LoopPyTorch Inference3D Deep Learning: The Architectures3D Convolutional Neural Networks: Voxels3D Graph Neural NetworksPoint-based Architectures: PointNet and Point CloudsMultiview CNNs3D Machine Learning Versus 3D Deep LearningFine-Tuning, Transfer Learning, and 3D Data AugmentationTransfer LearningFine-Tuning3D Data Augmentation: Expanding the DatasetSummary
PointNet: A Point-based 3D Deep Learning Architecture3D Object Classification3D Object Classification FundamentalsEnvironment SetupDataset CurationPointNet: Dataset PreparationPointNet Architecture DefinitionPointNet Loss DefinitionPointNet TrainingPointNet Metrics and EvaluationPointNet Real-World InferenceLarge-Scale Semantic Segmentation ConsiderationsSummary
3D Data Acquisition3D Data Preparation and EngineeringNoise RemovalSubsamplingFeature Extraction3D Data Modeling3D Mesh ReconstructionVoxelization of 3D Digital Environmentsk-d TreesOctreesSemantic ExtractionClustering and Unsupervised SegmentationSemantic Segmentation3D Object Classification3D Data Visualization and Analysis3D Shape Recognition3D Data Analytical Tools3D Multimodal Python ViewerSummary
Advanced 3D ProjectsGenerative AI for 3D Reconstruction3D Deep Point Cloud Registration3D Semantic Modeling3D Semantic Extraction with Transformers3D Gaussian Splatting for 3D VisualizationSpatial AI: The Future of 3D Experiences3D Scene Understanding with Open Vocabularies3D Spatial AI ReasoningConclusion

Content preview from 3D Data Science with Python

Chapter 9. 3D Shape Recognition

You may remember the introductory words from the start of the book: manmade environments are predominantly geometric assemblies of known shapes (planes, spheres, cylinders, etc.). Naturally, we ask ourselves if we can leverage this characteristic as it has the potential to benefit 3D scene understanding tasks.

What if we design a system capable of identifying such 3D shapes with precision? We could help automate manufacturing quality control processes by rapidly inspecting products for defects. We could enable autonomous vehicles to perceive their surroundings accurately, enhancing safety and efficiency. We could help reconstruct ancient artifacts for archeological simulations.

Giving machines 3D shape recognition capabilities enables them to perceive and interact with the physical world. As a first stage, it helps to understand the environment. By recognizing objects and their spatial relationships, systems can navigate complex environments and make informed decisions.

Furthermore, 3D shape recognition allows machines to perform tasks autonomously. Automated systems can execute functions like object manipulation, assembly, and inspection with remarkable precision and efficiency. This technology also aids in detecting abnormalities and analyzing geometrical structures in a given 3D dataset. And suppose you reason with enough of a low-level view. In that case, you can extract a scene decomposition without going into explicit functional definition ...