Implementing MLOps in the Enterprise

Chapter 8. Building Scalable Deep Learning and Large Language Model Projects

Deep learning (DL) is a machine learning subdomain inspired by the human brain’s structure and functioning. In deep learning, neural networks consisting of interconnected layers of artificial neurons process data hierarchically and can capture complex patterns in data. Each layer learns and transforms the input data, gradually capturing higher-level features and abstractions.

The DL training process involves feeding labeled data to the neural network and adjusting the weights and biases of the neurons iteratively. It can reduce the dependency on manual feature engineering and achieve impressive results in various domains such as computer vision, natural language processing, speech recognition, and reinforcement learning.

DL technologies are transforming the world with innovations such as transformers, generative AI, ChatGPT, and more. In addition, larger and more intelligent foundation models can perform human-like tasks, generate and understand content, and more.

Working and developing deep learning models introduce additional operational complexities and scaling challenges. This is where MLOps comes in to help simplify and abstract complexities and operationalize the process of developing and using complex models.

There are multiple deep learning frameworks. The major ones are:

TensorFlow

Developed by Google, TensorFlow is one of the most widely used deep learning frameworks. TensorFlow is open source ...