Book description
Statistics, big data, and machine learning for Clojure programmers
About This Book
- Write code using Clojure to harness the power of your data
- Discover the libraries and frameworks that will help you succeed
- A practical guide to understanding how the Clojure programming language can be used to derive insights from data
Who This Book Is For
This book is aimed at developers who are already productive in Clojure but who are overwhelmed by the breadth and depth of understanding required to be effective in the field of data science. Whether you're tasked with delivering a specific analytics project or simply suspect that you could be deriving more value from your data, this book will inspire you with the opportunities?and inform you of the risks?that exist in data of all shapes and sizes.
What You Will Learn
- Perform hypothesis testing and understand feature selection and statistical significance to interpret your results with confidence
- Implement the core machine learning techniques of regression, classification, clustering and recommendation
- Understand the importance of the value of simple statistics and distributions in exploratory data analysis
- Scale algorithms to web-sized datasets efficiently using distributed programming models on Hadoop and Spark
- Apply suitable analytic approaches for text, graph, and time series data
- Interpret the terminology that you will encounter in technical papers
- Import libraries from other JVM languages such as Java and Scala
- Communicate your findings clearly and convincingly to nontechnical colleagues
In Detail
The term ?data science? has been widely used to define this new profession that is expected to interpret vast datasets and translate them to improved decision-making and performance. Clojure is a powerful language that combines the interactivity of a scripting language with the speed of a compiled language. Together with its rich ecosystem of native libraries and an extremely simple and consistent functional approach to data manipulation, which maps closely to mathematical formula, it is an ideal, practical, and flexible language to meet a data scientist's diverse needs.
Taking you on a journey from simple summary statistics to sophisticated machine learning algorithms, this book shows how the Clojure programming language can be used to derive insights from data. Data scientists often forge a novel path, and you'll see how to make use of Clojure's Java interoperability capabilities to access libraries such as Mahout and Mllib for which Clojure wrappers don't yet exist. Even seasoned Clojure developers will develop a deeper appreciation for their language's flexibility!
You'll learn how to apply statistical thinking to your own data and use Clojure to explore, analyze, and visualize it in a technically and statistically robust way. You can also use Incanter for local data processing and ClojureScript to present interactive visualisations and understand how distributed platforms such as Hadoop sand Spark's MapReduce and GraphX's BSP solve the challenges of data analysis at scale, and how to explain algorithms using those programming models.
Above all, by following the explanations in this book, you'll learn not just how to be effective using the current state-of-the-art methods in data science, but why such methods work so that you can continue to be productive as the field evolves into the future.
Style and approach
This is a practical guide to data science that teaches theory by example through the libraries and frameworks accessible from the Clojure programming language.
Table of contents
-
Clojure for Data Science
- Table of Contents
- Clojure for Data Science
- Credits
- About the Author
- Acknowledgments
- About the Reviewer
- www.PacktPub.com
- Preface
-
1. Statistics
- Downloading the sample code
- Running the examples
- Downloading the data
- Inspecting the data
- Data scrubbing
- Descriptive statistics
- Variance
- Quantiles
- Binning data
- Histograms
- The normal distribution
- Poincaré's baker
- Skewness
- Comparative visualizations
- The importance of visualizations
- Adding columns
- Comparative visualizations of electorate data
- Visualizing the Russian election data
- Comparative visualizations
- Summary
-
2. Inference
- Introducing AcmeContent
- Download the sample code
- Load and inspect the data
- Visualizing the dwell times
- The exponential distribution
- The central limit theorem
- Standard error
- Samples and populations
- Confidence intervals
- Visualizing different populations
- Hypothesis testing
- Testing a new site design
- The t-statistic
- Performing the t-test
- One-sample t-test
- Resampling
- Testing multiple designs
- Multiple comparisons
- The browser simulation
- jStat
- B1
- Plotting probability densities
- State and Reagent
- Simulating multiple tests
- The Bonferroni correction
- Analysis of variance
- The F-distribution
- The F-statistic
- The F-test
- Effect size
- Summary
-
3. Correlation
- About the data
- Inspecting the data
- Visualizing the data
- The log-normal distribution
- Covariance
- Pearson's correlation
- Hypothesis testing
- Confidence intervals
- Regression
- Ordinary least squares
- Goodness-of-fit and R-square
- Multiple linear regression
- Matrices
- The normal equation
- Multiple R-squared
- Adjusted R-squared
- Collinearity
- Prediction
- Summary
-
4. Classification
- About the data
- Inspecting the data
- Comparisons with relative risk and odds
- The standard error of a proportion
- The binomial distribution
- Significance testing proportions
- Chi-squared multiple significance testing
- Classification with logistic regression
- Implementing logistic regression with Incanter
- Probability
- Naive Bayes classification
- Decision trees
- Classification with clj-ml
- Bias and variance
- Ensemble learning and random forests
- Saving the classifier to a file
- Summary
- 5. Big Data
-
6. Clustering
- Downloading the data
- Extracting the data
- Inspecting the data
- Clustering text
- Creating term frequency vectors
- Clustering with k-means and Incanter
- Better clustering with TF-IDF
- Large-scale clustering with Mahout
- Running k-means clustering with Mahout
- Cluster evaluation measures
- The drawbacks of k-means
- The curse of dimensionality
- Summary
-
7. Recommender Systems
- Download the code and data
- Inspect the data
- Parse the data
- Types of recommender systems
- Item-based and user-based recommenders
- Slope One recommenders
- Building a user-based recommender with Mahout
- k-nearest neighbors
- Recommender evaluation with Mahout
- Probabilistic methods for large sets
- Jaccard similarity for large sets with MinHash
- Dimensionality reduction
- Large-scale machine learning with Apache Spark and MLlib
- Machine learning on Spark with MLlib
- Summary
-
8. Network Analysis
- Download the data
- Graph traversal with Loom
- Breadth-first and depth-first search
- Finding the shortest path
- Whole-graph analysis
- Scale-free networks
-
Distributed graph computation with GraphX
- Creating RDGs with Glittering
- Measuring graph density with triangle counting
- Running the built-in triangle counting algorithm
- Implement triangle counting with Glittering
- Running the custom triangle counting algorithm
- The Pregel API
- Connected components with the Pregel API
- Running connected components
- Calculating the size of the largest connected component
- Detecting communities with label propagation
- Running label propagation
- Measuring community influence using PageRank
- The flow formulation
- Running PageRank to determine community influencers
- Summary
-
9. Time Series
- About the data
- Fitting curves with a linear model
- Time series decomposition
- Discrete time models
- Maximum likelihood estimation
- Time series forecasting
- Summary
- 10. Visualization
- Index
Product information
- Title: Clojure for Data Science
- Author(s):
- Release date: September 2015
- Publisher(s): Packt Publishing
- ISBN: 9781784397180
You might also like
book
Clojure Data Structures and Algorithms Cookbook
25 recipes to deeply understand and implement advanced algorithms in Clojure About This Book Explore various …
book
Clojure Applied
Think in the Clojure way! Once you're familiar with Clojure, take the next step with extended …
video
Clojure Inside Out
Why is Clojure the most interesting new language on the horizon? With this video, you’ll not …
book
Clojure Programming
Clojure is a practical, general-purpose language that offers expressivity rivaling other dynamic languages like Ruby and …