book

Elegant SciPy

by Juan Nunez-Iglesias, Stéfan van der Walt, Harriet Dashnow

August 2017

Intermediate to advanced

280 pages

6h 19m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Is This Book For?Why SciPy?What Is the SciPy Ecosystem?The Great Cataclysm: Python 2 Versus Python 3SciPy Ecosystem and CommunityFree and Open Source Software (FOSS)GitHub: Taking Coding SocialMake Your Mark on the SciPy EcosystemA Touch of Whimsy with Your PyGetting HelpInstalling PythonAccessing the Book MaterialsDiving InConventions Used in This BookUse of ColorUsing Code ExamplesO’Reilly SafariHow to Contact UsAcknowledgments
Introduction to the Data: What Is Gene Expression?NumPy N-Dimensional ArraysWhy Use ndarrays Instead of Python Lists?VectorizationBroadcastingExploring a Gene Expression DatasetReading in the Data with pandasNormalizationBetween SamplesBetween GenesNormalizing Over Samples and Genes: RPKMTaking Stock
Getting the DataGene Expression Distribution Differences Between IndividualsBiclustering the Counts DataVisualizing ClustersPredicting SurvivalFurther Work: Using the TCGA’s Patient ClustersFurther Work: Reproducing the TCGA’s clusters
Images Are Just NumPy ArraysExercise: Adding a Grid OverlayFilters in Signal ProcessingFiltering Images (2D Filters)Generic Filters: Arbitrary Functions of Neighborhood ValuesExercise: Conway’s Game of LifeExercise: Sobel Gradient MagnitudeGraphs and the NetworkX libraryExercise: Curve Fitting with SciPyRegion Adjacency GraphsElegant ndimage: How to Build Graphs from Image RegionsPutting It All Together: Mean Color Segmentation
Introducing FrequencyIllustration: A Birdsong SpectrogramHistoryImplementationChoosing the Length of the DFTMore DFT ConceptsFrequencies and Their OrderingWindowingReal-World Application: Analyzing Radar DataSignal Properties in the Frequency DomainWindowing, AppliedRadar ImagesFurther Applications of the FFTFurther ReadingExercise: Image Convolution
Contingency TablesExercise: Computational Complexity of Confusion MatricesExercise: Alternative Algorithm to Compute the Confusion MatrixExercise: Multiclass Confusion Matrixscipy.sparse Data FormatsCOO FormatExercise: COO RepresentationCompressed Sparse Row FormatApplications of Sparse Matrices: Image TransformationsExercise: Image RotationBack to Contingency TablesExercise: Reducing the Memory FootprintContingency Tables in SegmentationInformation Theory in BriefExercise: Computing Conditional EntropyInformation Theory in Segmentation: Variation of InformationConverting NumPy Array Code to Use Sparse MatricesUsing Variation of InformationFurther Work: Segmentation in Practice
Linear Algebra BasicsLaplacian Matrix of a GraphExercise: Rotation MatrixLaplacians with Brain DataExercise: Showing the Affinity ViewExercise Challenge: Linear Algebra with Sparse MatricesPageRank: Linear Algebra for Reputation and ImportanceExercise: Dealing with Dangling NodesExercise: Equivalence of Different Eigenvector MethodsConcluding Remarks
Optimization in SciPy: scipy.optimizeAn Example: Computing Optimal Image ShiftImage Registration with OptimizeAvoiding Local Minima with Basin HoppingExercise: Modify the align Function“What Is Best?”: Choosing the Right Objective Function
Streaming with yieldIntroducing the Toolz Streaming Libraryk-mer Counting and Error CorrectionCurrying: The Spice of StreamingBack to Counting k-mersExercise: PCA of Streaming DataMarkov Model from a Full GenomeExercise: Online Unzip
Where to Next?Mailing ListsGitHubConferencesBeyond SciPyContributing to This BookUntil Next Time...

Solution: Adding a Grid OverlaySolution: Conway’s Game of LifeSolution: Sobel Gradient MagnitudeSolution: Curve Fitting with SciPySolution: Image ConvolutionSolution: Computational Complexity of Confusion MatricesSolution: Alternative Confusion Matrix ComputingSolution: Computing the Confusion MatrixSolution: COO RepresentationSolution: Image RotationSolution: Reducing the Memory FootprintSolution: Computing Conditional EntropySolution: Rotation MatrixSolution: Showing the Affinity ViewChallenge Accepted: Linear Algebra with Sparse MatricesSolution: Dealing with Dangling NodesSolution: Verify MethodsSolution: Modify the align FunctionSolution: scikit-learn LibrarySolution: Add a Step to the Start of the Pipe

Content preview from Elegant SciPy

Epilogue

Quality means doing it right when no one is looking.

Henry Ford

Our main goal with this book was to promote elegant uses of the NumPy and SciPy libraries. While teaching you how to do effective scientific analysis with SciPy, we hope to have inspired in you the feeling that quality code is something worth striving for.

Where to Next?

Now that you know enough SciPy to analyze whatever data gets thrown your way, how do you move forward? We said when we started that we couldn’t hope to cover all there is to know about the library and all its offshoots. Before we part ways, we want to point you to the many resources available to help you.

Mailing Lists

We mentioned in the preface that SciPy is a community. A great way to continue learning is to subscribe to the main mailing lists for NumPy, SciPy, pandas, Matplotlib, scikit-image, and other libraries you might be interested in, and read them regularly.

And when you do get stuck in your own work, don’t be afraid to seek help there! We are a friendly bunch! The main requirement when seeking help is to show that you’ve tried a bit of problem solving yourself, and to provide others with a minimal script and enough sample data to demonstrate your problem and how you’ve tried to fix it.

No: “I need to generate a big array of random Gaussians. Can someone help?”
No: “I have this huge library at https://github.com/ron_obvious. If you look in the statistics library, there’s a part that really needs random Gaussians. Can someone ...