book

Data Science with Java

by Michael R. Brzustowicz

June 2017

Beginner to intermediate

233 pages

5h 57m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Should Read This BookWhy I Wrote This BookA Word on Data Science TodayNavigating This BookConventions Used in This BookUsing Code ExamplesO’Reilly SafariHow to Contact UsAcknowledgments
What Is Data, Anyway?Data ModelsUnivariate ArraysMultivariate ArraysData ObjectsMatrices and VectorsJSONDealing with Real DataNullsBlank SpacesParse ErrorsOutliersManaging Data FilesUnderstanding File Contents FirstReading from a Text FileReading from a JSON FileReading from an Image FileWriting to a Text FileMastering Database OperationsCommand-Line ClientsStructured Query LanguageJava Database ConnectivityVisualizing Data with PlotsCreating Simple PlotsPlotting Mixed Chart TypesSaving a Plot to a File
Building Vectors and MatricesArray StorageBlock StorageMap StorageAccessing ElementsWorking with SubmatricesRandomizationOperating on Vectors and MatricesScalingTransposingAddition and SubtractionLengthDistancesMultiplicationInner ProductOuter ProductEntrywise ProductCompound OperationsAffine TransformationMapping a FunctionDecomposing MatricesCholesky DecompositionLU DecompositionQR DecompositionSingular Value DecompositionEigen DecompositionDeterminantInverseSolving Linear Systems
The Probabilistic Origins of DataProbability DensityCumulative ProbabilityStatistical MomentsEntropyContinuous DistributionsDiscrete DistributionsCharacterizing DatasetsCalculating MomentsDescriptive StatisticsMultivariate StatisticsCovariance and CorrelationRegressionWorking with Large DatasetsAccumulating StatisticsMerging StatisticsRegressionUsing Built-in Database Functions
Transforming Text DataExtracting Tokens from a DocumentUtilizing DictionariesVectorizing a DocumentScaling and Regularizing Numeric DataScaling ColumnsScaling RowsMatrix Scaling OperatorReducing Data to Principal ComponentsCovariance MethodSVD MethodCreating Training, Validation, and Test SetsIndex-Based ResamplingList-Based ResamplingMini-BatchesEncoding LabelsA Generic EncoderOne-Hot Encoding
Learning AlgorithmsIterative Learning ProcedureGradient Descent OptimizerEvaluating Learning ProcessesMinimizing a Loss FunctionMinimizing the Sum of VariancesSilhouette CoefficientLog-LikelihoodClassifier AccuracyUnsupervised Learningk-Means ClusteringDBSCANGaussian MixturesSupervised LearningNaive BayesLinear ModelsDeep Networks
Hadoop Distributed File SystemMapReduce ArchitectureWriting MapReduce ApplicationsAnatomy of a MapReduce JobHadoop Data TypesMappersReducersThe Simplicity of a JSON String as TextDeployment WizardryMapReduce ExamplesWord CountCustom Word CountSparse Linear Algebra
Anscombe’s QuartetSentimentGaussian MixturesIrisMNIST

Content preview from Data Science with Java

Chapter 3. Statistics

Applying the basic principles of statistics to data science provides vital insight into our data. Statistics is a powerful tool. Used correctly, it enables us to be sure of our decision-making process. However, it is easy to use statistics incorrectly. One example is Anscombe’s quartet (Figure 3-1), which demonstrates how four distinct datasets can have nearly identical statistics. In many cases, a simple plot of the data can alert us right away to what is really going on with the data. In the case of Anscombe’s quartet, we can instantly pick out these features: in the upper-left panel, and appear to be linear, but noisy. In the upper-right panel, we see that and form a peaked relationship that is nonlinear. In the lower-left panel, and are precisely linear, except for one outlier. The lower-right panel shows that is statistically distributed for and that there ...