book

Deep Learning: Practical Neural Networks with Java

Name: Deep Learning: Practical Neural Networks with Java
ISBN: 9781788470315

by Yusuke Sugomori, Boštjan Kaluža, Fábio M. Soares, Alan M. F. Souza

June 2017

Intermediate to advanced

744 pages

16h 41m

English

Packt Publishing

Read now

Unlock full access

Deep Learning: Practical Neural Networks with Java
Table of Contents
Deep Learning: Practical Neural Networks with Java
Deep Learning: Practical Neural Networks with Java
Credits
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions

1. Java Deep Learning Essentials
1. Deep Learning Overview
Transition of AIDefinition of AIAI booms in the pastMachine learning evolvesWhat even machine learning cannot do
Things dividing a machine and human
AI and deep learning
Summary
2. Algorithms for Machine Learning – Preparing for Deep Learning
Getting started
The need for training in machine learning
Supervised and unsupervised learning
Support Vector Machine (SVM)Hidden Markov Model (HMM)Neural networksLogistic regressionReinforcement learning
Machine learning application flow
Theories and algorithms of neural networks
Perceptrons (single-layer neural networks)Logistic regressionMulti-class logistic regressionMulti-layer perceptrons (multi-layer neural networks)
Summary
3. Deep Belief Nets and Stacked Denoising Autoencoders
Neural networks fall
Neural networks' revenge
Deep learning's evolution – what was the breakthrough?Deep learning with pre-training
Deep learning algorithms
Restricted Boltzmann machinesDeep Belief Nets (DBNs)Denoising AutoencodersStacked Denoising Autoencoders (SDA)
Summary
4. Dropout and Convolutional Neural Networks
Deep learning algorithms without pre-training
Dropout
Convolutional neural networks
ConvolutionPoolingEquations and implementations
Summary
5. Exploring Java Deep Learning Libraries – DL4J, ND4J, and More
Implementing from scratch versus a library/framework
Introducing DL4J and ND4J
Implementations with ND4J
Implementations with DL4J
SetupBuildDBNIrisExample.javaCSVExample.javaCNNMnistExample.java/LenetMnistExample.javaLearning rate optimization
Summary
6. Approaches to Practical Applications – Recurrent Neural Networks and More
Fields where deep learning is activeImage recognitionNatural language processingFeed-forward neural networks for NLPDeep learning for NLPRecurrent neural networksLong short term memory networks
The difficulties of deep learning
The approaches to maximizing deep learning possibilities and abilities
Field-oriented approachMedicineAutomobilesAdvert technologiesProfession or practiceSportsBreakdown-oriented approachOutput-oriented approach
Summary
7. Other Important Deep Learning Libraries
Theano
TensorFlow
Caffe
Summary
8. What's Next?
Breaking news about deep learning
Expected next actions
Useful news sources for deep learning
Summary
2. Machine Learning in Java
1. Applied Machine Learning Quick Start
Machine learning and data scienceWhat kind of problems can machine learning solve?Applied machine learning workflow
Data and problem definition
Measurement scales
Data collection
Find or observe dataGenerate dataSampling traps
Data pre-processing
Data cleaningFill missing valuesRemove outliersData transformationData reduction
Unsupervised learning
Find similar itemsEuclidean distancesNon-Euclidean distancesThe curse of dimensionalityClustering
Supervised learning
ClassificationDecision tree learningProbabilistic classifiersKernel methodsArtificial neural networksEnsemble learningEvaluating classificationPrecision and recallRoc curvesRegressionLinear regressionEvaluating regressionMean squared errorMean absolute errorCorrelation coefficient
Generalization and evaluation
Underfitting and overfittingTrain and test setsCross-validationLeave-one-out validationStratification
Summary
2. Java Libraries and Platforms for Machine Learning
The need for Java
Machine learning libraries
WekaJava machine learningApache MahoutApache SparkDeeplearning4jMALLETComparing libraries
Building a machine learning application
Traditional machine learning architectureDealing with big dataBig data application architecture
Summary
3. Basic Algorithms – Classification, Regression, and Clustering
Before you start
Classification
DataLoading dataFeature selectionLearning algorithmsClassify new dataEvaluation and prediction error metricsConfusion matrixChoosing a classification algorithm
Regression
Loading the dataAnalyzing attributesBuilding and evaluating regression modelLinear regressionRegression treesTips to avoid common regression problems
Clustering
Clustering algorithmsEvaluation
Summary
4. Customer Relationship Prediction with Ensembles
Customer relationship databaseChallengeDatasetEvaluation
Basic naive Bayes classifier baseline
Getting the dataLoading the data
Basic modeling
Evaluating modelsImplementing naive Bayes baseline
Advanced modeling with ensembles
Before we startData pre-processingAttribute selectionModel selectionPerformance evaluation
Summary
5. Affinity Analysis
Market basket analysisAffinity analysis
Association rule learning
Basic conceptsDatabase of transactionsItemset and ruleSupportConfidenceApriori algorithmFP-growth algorithm
The supermarket dataset
Discover patterns
AprioriFP-growth
Other applications in various areas
Medical diagnosisProtein sequencesCensus dataCustomer relationship managementIT Operations Analytics
Summary
6. Recommendation Engine with Apache Mahout
Basic conceptsKey conceptsUser-based and item-based analysisApproaches to calculate similarityCollaborative filteringContent-based filteringHybrid approachExploitation versus exploration
Getting Apache Mahout
Configuring Mahout in Eclipse with the Maven plugin
Building a recommendation engine
Book ratings datasetLoading the dataLoading data from fileLoading data from databaseIn-memory databaseCollaborative filteringUser-based filteringItem-based filteringAdding custom rules to recommendationsEvaluationOnline learning engine
Content-based filtering
Summary
7. Fraud and Anomaly Detection
Suspicious and anomalous behavior detectionUnknown-unknowns
Suspicious pattern detection
Anomalous pattern detection
Analysis typesPattern analysisTransaction analysisPlan recognition
Fraud detection of insurance claims
DatasetModeling suspicious patternsVanilla approachDataset rebalancing
Anomaly detection in website traffic
DatasetAnomaly detection in time series dataHistogram-based anomaly detectionLoading the dataCreating histogramsDensity based k-nearest neighbors
Summary
8. Image Recognition with Deeplearning4j
Introducing image recognitionNeural networksPerceptronFeedforward neural networksAutoencoderRestricted Boltzmann machineDeep convolutional networks
Image classification
Deeplearning4jGetting DL4JMNIST datasetLoading the dataBuilding modelsBuilding a single-layer regression modelBuilding a deep belief networkBuild a Multilayer Convolutional Network
Summary
9. Activity Recognition with Mobile Phone Sensors
Introducing activity recognitionMobile phone sensorsActivity recognition pipelineThe plan
Collecting data from a mobile phone
Installing Android StudioLoading the data collectorFeature extractionCollecting training data
Building a classifier
Reducing spurious transitionsPlugging the classifier into a mobile app
Summary
10. Text Mining with Mallet – Topic Modeling and Spam Detection
Introducing text miningTopic modelingText classification
Installing Mallet
Working with text data
Importing dataImporting from directoryImporting from filePre-processing text data
Topic modeling for BBC news
BBC datasetModelingEvaluating a modelReusing a modelSaving a modelRestoring a model
E-mail spam detection
E-mail spam datasetFeature generationTraining and testingModel performance
Summary
11. What is Next?
Machine learning in real lifeNoisy dataClass unbalanceFeature selection is hardModel chainingImportance of evaluationGetting models into productionModel maintenance
Standards and markup languages
CRISP-DMSEMMA methodologyPredictive Model Markup Language
Machine learning in the cloud
Machine learning as a service
Web resources and competitions
DatasetsOnline coursesCompetitionsWebsites and blogsVenues and conferences
Summary
A. References
3. Neural Network Programming with Java, Second Edition
1. Getting Started with Neural Networks
Discovering neural networks
Why artificial neural networks?
How neural networks are arrangedThe very basic element – artificial neuronGiving life to neurons – activation functionThe flexible values – weightsAn extra parameter – biasThe parts forming the whole – layersLearning about neural network architecturesMonolayer networksMultilayer networksFeedforward networksFeedback networks
From ignorance to knowledge – learning process
Let the coding begin! Neural networks in practice
The neuron class
The NeuralLayer class
The ActivationFunction interface
The neural network class
Time to play!
Summary
2. Getting Neural Networks to Learn
Learning ability in neural networksHow learning helps solving problems
Learning paradigms
Supervised learningUnsupervised learning
The learning process
The cost function finding the way down to the optimumLearning in progress - weight updateCalculating the cost functionGeneral error and overall errorCan the neural network learn forever? When is it good to stop?
Examples of learning algorithms
The delta ruleThe learning rateImplementing the delta ruleThe core of the delta rule learning - train and calcNewWeight methodsAnother learning algorithm - Hebbian learningAdaline
Time to see the learning in practice!
Teaching the neural network – the training dataset
Amazing, it learned! Or, did it really? A further step – testing
Overfitting and overtraining
Summary
3. Perceptrons and Supervised Learning
Supervised learning – teaching the neural netClassification – finding the appropriate classRegression – mapping real inputs to outputs
A basic neural architecture – perceptrons
Applications and limitationsLinear separationThe XOR case
Multi-layer perceptrons
MLP propertiesMLP weightsRecurrent MLPCoding an MLP
Learning in MLPs
Backpropagation algorithmThe momentumCoding the backpropagationLevenberg-Marquardt algorithmCoding the Levenberg-Marquardt with matrix algebraExtreme learning machines
Practical example 1 – the XOR case with delta rule and backpropagation
Practical example 2 – predicting enrolment status
Summary
4. Self-Organizing Maps
Neural networks unsupervised learning
Unsupervised learning algorithms
Competitive learningCompetitive layer
Kohonen self-organizing maps
Extending the neural network code to KohonenZero-dimensional SOMOne-dimensional SOMTwo-dimensional SOM2D competitive layerSOM learning algorithmEffect of neighboring neurons – the neighborhood functionThe learning rateA new class for competitive learningVisualizing the SOMsPlotting 2D training datasets and neuron weightsTesting Kohonen learning
Summary
5. Forecasting Weather
Neural networks for regression problems
Loading/selecting data
Building auxiliary classesGetting a dataset from a CSV fileBuilding time seriesDropping NaNsGetting weather dataWeather variables
Choosing input and output variables
Preprocessing
NormalizationAdapting NeuralDataSet to handle normalizationAdapting the learning algorithm to normalizationJava implementation of weather forecastingCollecting weather dataDelaying variablesLoading the data and beginning to play!Let's perform a correlation analysisCreating neural networksTraining and testTraining the neural networkPlotting the errorViewing the neural network output
Empirical design of neural networks
Designing experimentsResults and simulations
Summary
6. Classifying Disease Diagnosis
Foundations of classification problemsCategorical dataWorking with categorical data
Logistic regression
Multiple classes versus binary classesConfusion matrixSensitivity and specificityImplementing a confusion matrix
Neural networks for classification
Disease diagnosis with neural networks
Breast cancerDiabetes
Summary
7. Clustering Customer Profiles
Clustering tasksCluster analysisCluster evaluation and validationImplementationExternal validation
Applied unsupervised learning
Kohonen neural network
Profiling
Pre-processingImplementation in JavaCard – credit analysis for customer profilingProduct profilingHow many clusters?
Summary
8. Text Recognition
Pattern recognitionDefined classesUndefined classes
Neural networks in pattern recognition
Data pre-processingText recognition (optical character recognition)Digit recognitionDigit representationImplementation in JavaGenerating dataNeural architectureExperimentsResults
Summary
9. Optimizing and Adapting Neural Networks
Common issues in neural network implementations
Input selection
Data correlationTransforming dataDimensionality reductionData filteringCross-validationStructure selection
Online retraining
Stochastic online learningImplementationApplication
Adaptive neural networks
Adaptive resonance theoryImplementation
Summary
10. Current Trends in Neural Networks
Deep learning
Deep architectures
How to implement deep learning in JavaHybrid systemsNeuro-fuzzyNeuro-genetic
Implementing a hybrid neural network
Summary
A. References
Chapter 1: Getting Started with Neural Networks
Chapter 2: Getting Neural Networks to Learn
Chapter 3: Perceptrons and Supervised Learning
Chapter 4: Self-Organizing Maps
Chapter 5: Forecasting Weather
Chapter 6: Classifying Disease Diagnosis
Chapter 7: Clustering Customer Profiles
Chapter 8: Text Recognition
Chapter 9: Optimizing and Adapting Neural Networks
Chapter 10: Current Trends in Neural Networks
Bibliography
Index

Content preview from Deep Learning: Practical Neural Networks with Java

Dropout

If there's a problem with the network being tied densely, just force it to be sparse. Then the vanishing gradient problem won't occur and learning can be done properly. The algorithm based on such an idea is the dropout algorithm. Dropout for deep neural networks was introduced in Improving neural networks by preventing co adaptation of feature detectors (Hinton, et. al. 2012, http://arxiv.org/pdf/1207.0580.pdf) and refined in Dropout: A Simple Way to Prevent Neural Networks from Overfitting (Srivastava, et. al. 2014, https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf). In dropout, some of the units are, literally, forcibly dropped while training. What does this mean? Let's look at the following figures—firstly, neural networks: ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Deep Learning for Natural Language Processing: Creating Neural Networks with Python

Publisher Resources

ISBN: 9781788470315Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills