Chapter 42. Documenting Your Data Preparation
Jobs are no longer for life; these days, people look for variety and challenges in their careers. Data roles are no different, as the skills are highly transferable between companies and industries. Therefore, the data sets you prepare are likely to be passed along to others. To ensure the work continues to be understandable, up-to-date, and valuable, it needs to be well documented so you can walk other preppers through your logic once you move on or get promoted. In this chapter, we will cover the fundamentals of documenting your data preparation work, Prep Builder’s built-in functionality to aid basic documentation, and the considerations to keep in mind for each data preparation step in Prep Builder.
Basic Documentation
No matter what tool you use to prepare your data sets, there are a number of aspects you should consider.
Folder Structure
All documentation within the data preparation file is useless if you can’t find the flow file in the first place. Keeping a folder that is available to you and your team for all the flows will help tremendously. My organization uses Google Drive, as it has the benefit of not just controlled sharing but also being available on any computer I log in to. Setting up a standardized structure for those files is also key. In large organizations with more complex data flows, you should consider setting up the following folders:
- In production
- The holy grail of folders, this file should be under strict ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access