Data Engineering Best Practices

16

Machine Learning Pipeline Best Practices and Processes

At this point in the book, you are equipped with a fine understanding of how to produce a data factory pipeline and render data into releasable sets in a consumable area. After making them clearly available as a knowledge base with transparent metadata and lineage services, you can provide analysis capabilities in your analytics workbench. What we have not yet covered is the iterative cycles needed to tease out insights from your quality information via machine learning algorithms. This involves minimizing the technical effort, implementing a high degree of objective quality, organizing flows and optimized models, while integrating cutting-edge technologies. All this must take place to ...