book

Deep Learning with TensorFlow 2 and Keras - Second Edition

by Antonio Gulli, Dr. Amita Kapoor, Sujit Pal

December 2019

Intermediate to advanced

646 pages

15h 11m

English

Packt Publishing

Read now

Unlock full access

MissionMachine learning, artificial intelligence, and the deep learning Cambrian explosionWho this book is forWhat this book coversWhat you need for this bookDownloading the example codeDownload the color imagesConventionsGet in touchReviewsReferences
What is TensorFlow (TF)?What is Keras?What are the most important changes in TensorFlow 2.0?Introduction to neural networksPerceptronA first example of TensorFlow 2.0 codeMulti-layer perceptron – our first example of a networkProblems in training the perceptron and their solutionsActivation function – sigmoidActivation function – tanhActivation function – ReLUTwo additional activation functions – ELU and LeakyReLUActivation functionsIn short – what are neural networks after all?A real example – recognizing handwritten digitsOne-hot encoding (OHE)Defining a simple neural network in TensorFlow 2.0Running a simple TensorFlow 2.0 net and establishing a baselineImproving the simple net in TensorFlow 2.0 with hidden layersFurther improving the simple net in TensorFlow with DropoutTesting different optimizers in TensorFlow 2.0Increasing the number of epochsControlling the optimizer learning rateIncreasing the number of internal hidden neuronsIncreasing the size of batch computationSummarizing experiments run for recognizing handwritten chartsRegularizationAdopting regularization to avoid overfittingUnderstanding BatchNormalizationPlaying with Google Colab – CPUs, GPUs, and TPUsSentiment analysisHyperparameter tuning and AutoMLPredicting outputA practical overview of backpropagation What have we learned so far?Towards a deep learning approachReferences
Understanding TensorFlow 1.xTensorFlow 1.x computational graph program structureComputational graphsWorking with constants, variables, and placeholdersExamples of operationsConstantsSequencesRandom tensorsVariablesAn example of TensorFlow 1.x in TensorFlow 2.xUnderstanding TensorFlow 2.xEager executionAutoGraphKeras APIs – three programming modelsSequential APIFunctional APIModel subclassingCallbacksSaving a model and weightsTraining from tf.data.datasetstf.keras or Estimators?Ragged tensorsCustom trainingDistributed training in TensorFlow 2.xMultiple GPUsMultiWorkerMirroredStrategyTPUStrategyParameterServerStrategyChanges in namespacesConverting from 1.x to 2.xUsing TensorFlow 2.x effectivelyThe TensorFlow 2.x ecosystemLanguage bindingsKeras or tf.keras?Summary
What is regression?Prediction using linear regressionSimple linear regressionMultiple linear regressionMultivariate linear regressionTensorFlow EstimatorsFeature columnsInput functionsMNIST using TensorFlow Estimator APIPredicting house price using linear regressionClassification tasks and decision boundariesLogistic regressionLogistic regression on the MNIST datasetSummaryReferences
Deep Convolutional Neural Network (DCNN)Local receptive fieldsShared weights and biasA mathematical exampleConvNets in TensorFlow 2.xPooling layersMax poolingAverage poolingConvNets summaryAn example of DCNN ‒ LeNetLeNet code in TensorFlow 2.0Understanding the power of deep learningRecognizing CIFAR-10 images with deep learning Improving the CIFAR-10 performance with a deeper networkImproving the CIFAR-10 performance with data augmentationPredicting with CIFAR-10Very deep convolutional networks for large-scale image recognitionRecognizing cats with a VGG16 NetUtilizing tf.keras built-in VGG16 Net moduleRecycling prebuilt deep learning models for extracting featuresSummaryReferences
Computer visionComposing CNNs for complex tasksClassification and localizationSemantic segmentationObject detectionInstance segmentationClassifying Fashion-MNIST with a tf.keras - estimator modelRun Fashion-MNIST the tf.keras - estimator model on GPUsDeep Inception-v3 Net used for transfer learningTransfer learning for classifying horses and humansApplication Zoos with tf.keras and TensorFlow HubKeras applicationsTensorFlow HubOther CNN architecturesAlexNetResidual networksHighwayNets and DenseNetsXceptionAnswering questions about images (VQA)Style transferContent distanceStyle distanceCreating a DeepDream networkInspecting what a network has learned VideoClassifying videos with pretrained nets in six different waysTextual documentsUsing a CNN for sentiment analysisAudio and musicDilated ConvNets, WaveNet, and NSynthA summary of convolution operationsBasic convolutional neural networks (CNN or ConvNet)Dilated convolutionTransposed convolutionSeparable convolutionDepthwise convolutionDepthwise separable convolutionCapsule networksSo what is the problem with CNNs?So what is new with Capsule networks?SummaryReferences
What is a GAN?MNIST using GAN in TensorFlowDeep convolutional GAN (DCGAN)DCGAN for MNIST digitsSome interesting GAN architecturesSRGANCycleGANInfoGANCool applications of GANsCycleGAN in TensorFlow 2.0SummaryReferences
Word embedding ‒ origins and fundamentalsDistributed representationsStatic embeddingsWord2VecGloVeCreating your own embedding using gensimExploring the embedding space with gensimUsing word embeddings for spam detectionGetting the dataMaking the data ready for useBuilding the embedding matrixDefine the spam classifierTrain and evaluate the modelRunning the spam detectorNeural embeddings – not just for wordsItem2Vecnode2vecCharacter and subword embeddingsDynamic embeddingsSentence and paragraph embeddingsLanguage model-based embeddingsUsing BERT as a feature extractorFine-tuning BERTClassifying with BERT ‒ command lineUsing BERT as part of your own networkSummaryReferences
The basic RNN cellBackpropagation through time (BPTT)Vanishing and exploding gradientsRNN cell variantsLong short-term memory (LSTM)Gated recurrent unit (GRU)Peephole LSTMRNN variantsBidirectional RNNsStateful RNNsRNN topologiesExample ‒ One-to-Many – learning to generate textExample ‒ Many-to-One – Sentiment AnalysisExample ‒ Many-to-Many – POS taggingEncoder-Decoder architecture – seq2seqExample ‒ seq2seq without attention for machine translationAttention mechanismExample ‒ seq2seq with attention for machine translationTransformer architectureSummaryReferences
Introduction to autoencodersVanilla autoencodersTensorFlow Keras layers ‒ defining custom layersReconstructing handwritten digits using an autoencoderSparse autoencoderDenoising autoencodersClearing images using a Denoising autoencoderStacked autoencoderConvolutional autoencoder for removing noise from imagesKeras autoencoder example ‒ sentence vectorsSummaryReferences

Principal component analysisPCA on the MNIST datasetTensorFlow Embedding APIK-means clusteringK-means in TensorFlow 2.0Variations in k-meansSelf-organizing mapsColour mapping using SOMRestricted Boltzmann machinesReconstructing images using RBMDeep belief networksVariational AutoencodersSummaryReferences
IntroductionRL lingoDeep reinforcement learning algorithmsReinforcement success in recent yearsIntroduction to OpenAI GymRandom agent playing BreakoutDeep Q-NetworksDQN for CartPoleDQN to play a game of AtariDQN variantsDouble DQNDueling DQNRainbowDeep deterministic policy gradientSummaryReferences
Deep learning on cloudMicrosoft AzureAmazon Web Services (AWS)Google Cloud Platform (GCP)IBM CloudVirtual machines on cloudEC2 on AmazonCompute Instance on GCPVirtual machine on Microsoft AzureJupyter Notebooks on cloudSageMakerGoogle ColaboratoryMicrosoft Azure NotebooksTensorFlow Extended for productionTFX PipelinesTFX pipeline componentsTFX librariesTensorFlow EnterpriseSummaryReferences
TensorFlow MobileTensorFlow LiteQuantizationFlatBuffersMobile converterMobile optimized interpreterSupported platformsArchitectureUsing TensorFlow LiteA generic example of applicationUsing GPUs and acceleratorsAn example of applicationPretrained models in TensorFlow LiteImage classificationObject detectionPose estimationSmart replySegmentationStyle transferText classificationQuestion and answeringA note about using mobile GPUsAn overview of federated learning at the edgeTensorFlow FL APIsTensorFlow.jsVanilla TensorFlow.jsConverting modelsPretrained modelsNode.jsSummaryReferences
What is AutoML?Achieving AutoMLAutomatic data preparationAutomatic feature engineeringAutomatic model generationAutoKerasGoogle Cloud AutoMLUsing Cloud AutoML ‒ Tables solutionUsing Cloud AutoML ‒ Vision solutionUsing Cloud AutoML ‒ Text Classification solutionUsing Cloud AutoML ‒ Translation solutionUsing Cloud AutoML ‒ Video Intelligence Classification solutionCostBringing Google AutoML to KaggleSummaryReferences
HistorySome mathematical toolsDerivatives and gradients everywhereGradient descentChain ruleA few differentiation rulesMatrix operationsActivation functionsDerivative of the sigmoidDerivative of tanhDerivative of ReLUBackpropagationForward stepBackstepCase 1 – From hidden layer to output layerCase 2 ‒ From hidden layer to hidden layerLimit of backpropagationCross entropy and its derivativeBatch gradient descent, stochastic gradient descent, and mini-batchBatch Gradient Descent (BGD)Stochastic Gradient Descent (SGD)Mini-Batch Gradient Descent (MBGD)Thinking about backpropagation and convnetsThinking about backpropagation and RNNsA note on TensorFlow and automatic differentiationSummaryReferences
C/G/T processing unitsCPUs and GPUsTPUsThree generations of TPUs and Edge TPUFirst-generation TPUSecond-generation TPUThird-generation TPUEdge TPUTPU performanceHow to use TPUs with ColabChecking whether TPUs are availableLoading data with tf.dataBuilding a model and loading it into the TPUUsing pretrained TPU modelsUsing TensorFlow 2.1 and nightly buildSummaryReferences

Content preview from Deep Learning with TensorFlow 2 and Keras - Second Edition

12 TensorFlow and Cloud

AI algorithms require extensive computing resources. With the availability of a large number of cloud platforms offering their services at competitive prices, cloud computing offers a cost-effective solution. In this chapter, we will talk about three main cloud platform providers that occupy the majority of the market share: Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform. Moreover, once you have trained your model on cloud, you can use TensorFlow Extended (TFX) to move your model to production. The chapter will cover:

Creating and using virtual machines on cloud
Creating and training directly on Jupyter Notebook on cloud
Deploying the model on cloud
Using TFX for production
TensorFlow Enterpris ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Advanced Deep Learning with TensorFlow 2 and Keras - Second Edition

Publisher Resources

ISBN: 9781838823412Supplemental Content

Deep Learning with TensorFlow 2 and Keras - Second Edition

by Antonio Gulli, Dr. Amita Kapoor, Sujit Pal

12

TensorFlow and Cloud

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

Advanced Deep Learning with TensorFlow 2 and Keras - Second Edition

Machine Learning with TensorFlow, Second Edition

Deep Learning with Keras

Deep Learning Projects Using TensorFlow 2: Neural Network Development with Python and Keras

Publisher Resources

12

TensorFlow and Cloud

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Advanced Deep Learning with TensorFlow 2 and Keras - Second Edition

Machine Learning with TensorFlow, Second Edition

Deep Learning with Keras

Deep Learning Projects Using TensorFlow 2: Neural Network Development with Python and Keras

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.