book

Julia for Data Analysis

Name: Julia for Data Analysis
Author: Bogumil Kaminski
ISBN: 9781633439368

by Bogumil Kaminski

January 2023

Beginner to intermediate

472 pages

13h 21m

English

Manning Publications

Audiobook available

Read now

Unlock full access

inside front cover
Julia for Data Analysis
Copyright
contents
front matter
forewordprefaceacknowledgmentsabout this bookWho should read this bookHow this book is organized: A roadmapAbout the codeliveBook discussion forumOther online resourcesabout the authorabout the cover illustration
1 Introduction
1.1 What is Julia and why is it useful?1.2 Key features of Julia from a data scientist’s perspective1.2.1 Julia is fast because it is a compiled language1.2.2 Julia provides full support for interactive workflows1.2.3 Julia programs are highly reusable and easy to compose together1.2.4 Julia has a built-in state-of-the-art package manager1.2.5 It is easy to integrate existing code with Julia1.3 Usage scenarios of tools presented in the book1.4 Julia’s drawbacks1.5 What data analysis skills will you learn?1.6 How can Julia be used for data analysis?Summary
Part 1 Essential Julia skills
2 Getting started with Julia
2.1 Representing values2.2 Defining variables2.3 Using the most important control-flow constructs2.3.1 Computations depending on a Boolean condition2.3.2 Loops2.3.3 Compound expressions2.3.4 A first approach to calculating the winsorized mean2.4 Defining functions2.4.1 Defining functions using the function keyword2.4.2 Positional and keyword arguments of functions2.4.3 Rules for passing arguments to functions2.4.4 Short syntax for defining simple functions2.4.5 Anonymous functions2.4.6 Do blocks2.4.7 Function-naming convention in Julia2.4.8 A simplified definition of a function computing the winsorized mean2.5 Understanding variable scoping rulesSummary
3 Julia’s support for scaling projects
3.1 Understanding Julia’s type system3.1.1 A single function in Julia may have multiple methods3.1.2 Types in Julia are arranged in a hierarchy3.1.3 Finding all supertypes of a type3.1.4 Finding all subtypes of a type3.1.5 Union of types3.1.6 Deciding what type restrictions to put in method signature3.2 Using multiple dispatch in Julia3.2.1 Rules for defining methods of a function3.2.2 Method ambiguity problem3.2.3 Improved implementation of winsorized mean3.3 Working with packages and modules3.3.1 What is a module in Julia?3.3.2 How can packages be used in Julia?3.3.3 Using StatsBase.jl to compute the winsorized mean3.4 Using macrosSummary
4 Working with collections in Julia
4.1 Working with arrays4.1.1 Getting the data into a matrix4.1.2 Computing basic statistics of the data stored in a matrix4.1.3 Indexing into arrays4.1.4 Performance considerations of copying vs. making a view4.1.5 Calculating correlations between variables4.1.6 Fitting a linear regression4.1.7 Plotting the Anscombe’s quartet data4.2 Mapping key-value pairs with dictionaries4.3 Structuring your data by using named tuples4.3.1 Defining named tuples and accessing their contents4.3.2 Analyzing Anscombe’s quartet data stored in a named tuple4.3.3 Understanding composite types and mutability of values in JuliaSummary

5 Advanced topics on handling collections
5.1 Vectorizing your code using broadcasting5.1.1 Understanding syntax and meaning of broadcasting in Julia5.1.2 Expanding length-1 dimensions in broadcasting5.1.3 Protecting collections from being broadcasted over5.1.4 Analyzing Anscombe’s quartet data using broadcasting5.2 Defining methods with parametric types5.2.1 Most collection types in Julia are parametric5.2.2 Rules for subtyping of parametric types5.2.3 Using subtyping rules to define the covariance function5.3 Integrating with Python5.3.1 Preparing data for dimensionality reduction using t-SNE5.3.2 Calling Python from Julia5.3.3 Visualizing the results of the t-SNE algorithmSummary
6 Working with strings
6.1 Getting and inspecting the data6.1.1 Downloading files from the web6.1.2 Using common techniques of string construction6.1.3 Reading the contents of a file6.2 Splitting strings6.3 Using regular expressions to work with strings6.3.1 Working with regular expressions6.3.2 Writing a parser of a single line of movies.dat file6.4 Extracting a subset from a string with indexing6.4.1 UTF-8 encoding of strings in Julia6.4.2 Character vs. byte indexing of strings6.4.3 ASCII strings6.4.4 The Char type6.5 Analyzing genre frequency in movies.dat6.5.1 Finding common movie genres6.5.2 Understanding genre popularity evolution over the years6.6 Introducing symbols6.6.1 Creating symbols6.6.2 Using symbols6.7 Using fixed-width string types to improve performance6.7.1 Available fixed-width strings6.7.2 Performance of fixed-width strings6.8 Compressing vectors of strings with PooledArrays.jl6.8.1 Creating a file containing flower names6.8.2 Reading in the data to a vector and compressing it6.8.3 Understanding the internal design of PooledArray6.9 Choosing appropriate storage for collections of stringsSummary
7 Handling time-series data and missing values
7.1 Understanding the NBP Web API7.1.1 Getting the data via a web browser7.1.2 Getting the data by using Julia7.1.3 Handling cases when an NBP Web API query fails7.2 Working with missing data in Julia7.2.1 Definition of the missing value7.2.2 Working with missing values7.3 Getting time-series data from the NBP Web API7.3.1 Working with dates7.3.2 Fetching data from the NBP Web API for a range of dates7.4 Analyzing data fetched from the NBP Web API7.4.1 Computing summary statistics7.4.2 Finding which days of the week have the most missing values7.4.3 Plotting the PLN/USD exchange rateSummary
Part 2 Toolbox for data analysis
8 First steps with data frames
8.1 Fetching, unpacking, and inspecting the data8.1.1 Downloading the file from the web8.1.2 Working with bzip2 archives8.1.3 Inspecting the CSV file8.2 Loading the data to a data frame8.2.1 Reading a CSV file into a data frame8.2.2 Inspecting the contents of a data frame8.2.3 Saving a data frame to a CSV file8.3 Getting a column out of a data frame8.3.1 Understanding the data frame’s storage model8.3.2 Treating a data frame column as a property8.3.3 Getting a column by using data frame indexing8.3.4 Visualizing data stored in columns of a data frame8.4 Reading and writing data frames using different formats8.4.1 Apache Arrow8.4.2 SQLiteSummary
9 Getting data from a data frame
9.1 Advanced data frame indexing9.1.1 Getting a reduced puzzles data frame9.1.2 Overview of allowed column selectors9.1.3 Overview of allowed row-subsetting values9.1.4 Making views of data frame objects9.2 Analyzing the relationship between puzzle difficulty and popularity9.2.1 Calculating mean puzzle popularity by its rating9.2.2 Fitting LOESS regressionSummary
10 Creating data frame objects
10.1 Reviewing the most important ways to create a data frame10.1.1 Creating a data frame from a matrix10.1.2 Creating a data frame from vectors10.1.3 Creating a data frame using a Tables.jl interface10.1.4 Plotting a correlation matrix of data stored in a data frame10.2 Creating data frames incrementally10.2.1 Vertically concatenating data frames10.2.2 Appending a table to a data frame10.2.3 Adding a new row to an existing data frame10.2.4 Storing simulation results in a data frameSummary
11 Converting and grouping data frames
11.1 Converting a data frame to other value types11.1.1 Conversion to a matrix11.1.2 Conversion to a named tuple of vectors11.1.3 Other common conversions11.2 Grouping data frame objects11.2.1 Preparing the source data frame11.2.2 Grouping a data frame11.2.3 Getting group keys of a grouped data frame11.2.4 Indexing a grouped data frame with a single value11.2.5 Comparing performance of indexing methods11.2.6 Indexing a grouped data frame with multiple values11.2.7 Iterating a grouped data frameSummary
12 Mutating and transforming data frames
12.1 Getting and loading the GitHub developers data set12.1.1 Understanding graphs12.1.2 Fetching GitHub developer data from the web12.1.3 Implementing a function that extracts data from a ZIP file12.1.4 Reading the GitHub developer data into a data frame12.2 Computing additional node features12.2.1 Creating a SimpleGraph object12.2.2 Computing features of nodes by using the Graphs.jl package12.2.3 Counting a node’s web and machine learning neighbors12.3 Using the split-apply-combine approach to predict the developer’s type12.3.1 Computing summary statistics of web and machine learning developer features12.3.2 Visualizing the relationship between the number of web and machine learning neighbors of a node12.3.3 Fitting a logistic regression model predicting developer type12.4 Reviewing data frame mutation operations12.4.1 Performing low-level API operations12.4.2 Using the insertcols! function to mutate a data frameSummary
13 Advanced transformations of data frames
13.1 Getting and preprocessing the police stop data set13.1.1 Loading all required packages13.1.2 Introducing the @chain macro13.1.3 Getting the police stop data set13.1.4 Comparing functions that perform operations on columns13.1.5 Using short forms of operation specification syntax13.2 Investigating the violation column13.2.1 Finding the most frequent violations13.2.2 Vectorizing functions by using the ByRow wrapper13.2.3 Flattening data frames13.2.4 Using convenience syntax to get the number of rows of a data frame13.2.5 Sorting data frames13.2.6 Using advanced functionalities of DataFramesMeta.jl13.3 Preparing data for making predictions13.3.1 Performing initial transformation of the data13.3.2 Working with categorical data13.3.3 Joining data frames13.3.4 Reshaping data frames13.3.5 Dropping rows of a data frame that hold missing values13.4 Building a predictive model of arrest probability13.4.1 Splitting the data into train and test data sets13.4.2 Fitting a logistic regression model13.4.3 Evaluating the quality of a model’s predictions13.5 Reviewing functionalities provided by DataFrames.jlSummary
14 Creating web services for sharing data analysis results
14.1 Pricing financial options by using a Monte Carlo simulation14.1.1 Calculating the payoff of an Asian option definition14.1.2 Computing the value of an Asian option14.1.3 Understanding GBM14.1.4 Using a numerical approach to computing the Asian option value14.2 Implementing the option pricing simulator14.2.1 Starting Julia with multiple-thread support14.2.2 Computing the option payoff for a single sample of stock prices14.2.3 Computing the option value14.3 Creating a web service serving the Asian option valuation14.3.1 A general approach to building a web service14.3.2 Creating a web service using Genie.jl14.3.3 Running the web service14.4 Using the Asian option pricing web service14.4.1 Sending a single request to the web service14.4.2 Collecting responses to multiple requests from a web service in a data frame14.4.3 Unnesting a column of a data frame14.4.4 Plotting the results of Asian option pricingSummary
Appendix A First steps with Julia
A.1 Installing and setting up JuliaA.2 Getting help in and about JuliaA.3 Managing packages in JuliaA.3.1 Project environmentsA.3.2 Activating project environmentsA.3.3 Potential issues with installing packagesA.3.4 Managing packagesA.3.5 Setting up integration with PythonA.3.6 Setting up integration with RA.4 Reviewing standard ways to work with JuliaA.4.1 Using a terminalA.4.2 Using Visual Studio CodeA.4.3 Using Jupyter NotebookA.4.4 Using Pluto notebooks
Appendix B Solutions to exercises
Appendix C Julia packages for data science
C.1 Plotting ecosystems in JuliaC.2 Scaling computing with JuliaC.3 Working with databases and data storage formatsC.4 Using data science methodsSummary
index
inside back cover

Content preview from Julia for Data Analysis

9 Getting data from a data frame

This chapter covers

Subsetting rows of a data frame
Selecting columns of a data frame
Creating local linear regression (LOESS) models
Visualizing LOESS predictions

In chapter 8, you learned the basic principles of working with data frames in Julia provided by the DataFrames.jl package, and we started to analyze the Lichess chess puzzle data. Recall that our objective was to identify the relationship between puzzle difficulty and popularity.

In section 8.3, we stopped our investigation by concluding that we would like to clean the original data before performing its final analysis (in figure 9.1, I reproduce the histograms we used in chapter 8 to conclude that the original data is significantly skewed). The simplest ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Interactive Visualization and Plotting with Julia

Publisher Resources

ISBN: 9781633439368Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Julia for Data Analysis

by Bogumil Kaminski

9 Getting data from a data frame

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.