book

Big Data Analytics Methods

Name: Big Data Analytics Methods
Author: Peter Ghavami
ISBN: 9781547401581

by Peter Ghavami

December 2019

Intermediate to advanced

254 pages

7h 54m

English

De Gruyter

Read now

Unlock full access

Acknowledgments
About the Author
Introduction
Part I: Big Data Analytics
Chapter 1 Data Analytics Overview1.1 Data Analytics Definition1.2 The Distinction between BI and Analytics1.3 Why Advanced Data Analytics?1.4 Analytics Platform Framework1.5 Data Connection Layer1.6 Data Management Layer1.7 Analytics Layer1.8 Presentation Layer1.9 Data Analytics ProcessChapter 2 Basic Data Analysis2.1 KPIs, Analytics and Business Optimization2.2 Key Considerations in Data Analytics Reports2.3 The Four Pillars of a Real World Data Analytics Program2.4 The Eight Axioms of Big Data Analytics2.5 Basic Models2.6 Complexity of Data Analytics2.7 Introduction to Data Analytics Methods2.8 Statistical Models2.9 Predictive Analytics2.10 Advanced Analytics MethodsChapter 3 Data Analytics Process3.1 A Survey of Data Analytics Process3.2 KDD—Knowledge Discovery Databases3.3 CRISP-DM Process Model3.4 The SEMMA Process Model3.5 Microsoft TDSP Framework3.6 Data Analytics Process Example—Predictive Modeling Case Study
Part II: Advanced Analytics Methods
Chapter 4 Natural Language Processing4.1 Natural Language Processing (NLP)4.2 NLP Capability Maturity Model4.3 Introduction to Natural Language Processing4.4 NLP Techniques—Topic Modeling4.5 NLP—Names Entity Recognition (NER)4.6 NLP—Part of Speech (POS) Tagging4.7 NLP—Probabilistic Context-Free Grammars (PCFG)4.8 NLP Learning Method4.9 Word Embedding and Neural Networks4.10 Semantic Modeling Using Graph Analysis Technique4.11 Putting It All TogetherChapter 5 Quantitative Analysis—Prediction and Prognostics5.1 Probabilities and Odds Ratio5.2 Additive Interaction of Predictive Variables5.3 Prognostics and Prediction5.4 Framework for Prognostics, Prediction and Accuracy5.5 Significance of Predictive Analytics5.6 Prognostics in Literature5.7 Control Theoretic Approach to Prognostics5.8 Artificial Neural NetworksChapter 6 Advanced Analytics and Predictive Modeling6.1 History of Predictive Methods and Prognostics6.2 Model Viability and Validation Methods6.3 Classification Methods6.4 Traditional Analysis Methods vs. Advanced Analytics Methods6.5 Traditional Analysis Overview: Quantitative Methods6.6 Regression Analysis Overview6.7 Cox Hazard Model6.8 Correlation Analysis6.9 Non-linear Correlation6.10 Kaplan-Meier Estimate of Survival Function6.11 Handling Dirty, Noisy and Missing Data6.12 Data Cleansing Techniques6.13 Analysis of Variance (ANOVA) and MANOVA6.14 Advanced Analytics Methods At-a-Glance6.15 LASSO, L1 and L2 Norm Methods6.16 Kalman Filtering6.17 Trajectory Tracking6.18 N-point Correlation6.19 Bi-partite Matching6.20 Mean Shift and K-means Algorithm6.21 Gaussian Graphical Model6.22 Parametric vs. Non-parametric Methods6.23 Non-parametric Bayesian Classifier6.24 Machine Learning6.25 Geo-spatial Analysis6.26 Logistic Regression or Logit6.27 Predictive Modeling Approaches6.28 Alternate Conditional Expectation (ACE)6.29 Clustering vs. Classification6.30 K-means Clustering Method6.31 Classification Using Neural Networks6.32 Principal Component Analysis6.33 Stratification Method6.34 Propensity Score Matching Approach6.35 Adherence Analysis Method6.36 Meta-analysis Methods6.37 Stochastic Models—Markov Chain Analysis6.38 Handling Noisy Data—Kalman Filters6.39 Tree-based Analysis6.40 Random Forest Techniques6.41 Hierarchical Clustering Analysis (HCA) Method6.42 Outlier Detection by Robust Estimation Method6.43 Feature Selection Techniques6.44 Bridging Studies6.45 Signal Boosting and Bagging Methods6.46 Generalized Estimating Equation (GEE) Method6.47 Q-Q Plots6.48 Reduction in Variance (RIV) —Intergroup Variation6.49 Coefficient of Variation (CV)—Intra Group VariationChapter 7 Ensemble of Models: Data Analytics Prediction Framework7.1 Ensemble of Models7.2 Artificial Neural Network Models7.3 Analytic Model Comparison and EvaluationChapter 8 Machine Learning, Deep Learning—Artificial Neural Networks8.1 Introduction to ANNs8.2 A Simple Example8.3 A Simplified Mathematical Example8.4 Activation Functions8.5 Why Artificial Neural Network Algorithms8.6 Deep Learning8.7 Mathematical Foundations of Artificial Neural Networks8.8 Gradient Descent Methods8.9 Neural Network Learning Processes8.10 Selected Analytics Models8.11 Probabilistic Neural Networks8.12 Support Vector Machine (SVM) Networks8.13 General Feed-forward Neural Network8.14 MLP with Levenberg-Marquardt (LM) AlgorithmChapter 9 Model Accuracy and Optimization9.1 Accuracy Measures9.2 Accuracy and Validation9.3 Vote-based Schema9.4 Accuracy-based Ensemble Schema9.5 Diversity-based Schema9.6 Optimization-based Schema
Part III: Case Study—Prediction and Advanced Analytics in Practice
Chapter 10 Ensemble of Models—Medical Prediction Case Study: Data Types, Data Requirements and Data Pre-Processing10.1 How Much Data Is Needed for Machine Learning?10.2 Learning Despite Noisy Data10.3 Pre-processing and Data Scaling10.4 Data Acquisition for ANN Models10.5 Ensemble Models Case StudyAppendices
Appendix A: Prognostics Methods
Reliability and Predictive Analytics
Appendix B: A Neural Network Example
Appendix C: Back Propagation Algorithm Derivation
Appendix D: The Oracle Program

References
Index

Content preview from Big Data Analytics Methods

Appendix D: The Oracle Program

The oracle program is the overseer module that selects the ensemble with the highest value of AUC. The oracle program was written in R, an open source statistical package (Wang 2012) and was tested on the 1,073 data sets and all ensembles. The source code is listed below. The program computes the AUC for each ANN algorithm and ensemble compared to the actual truth (presence or absence of disease) for prior data. The first column is the actual truth and the subsequent columns are the data for ANN algorithms and ensembles.

 ###################################### ###### R code for computing AUC ###### ###### Peter Ghavami – Adapted ###### ###### From Wang 2012 `###### ######################################  ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781547401581

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Big Data Analytics Methods

by Peter Ghavami

Appendix D: The Oracle Program

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.