97 Things Every Data Engineer Should Know

Chapter 26. Develop Communities, Not Just Code

Emily Riederer

As a data engineer, you may think of your core job responsibility as building a data pipeline. In fact, that’s probably what was laid out in your job description and how your value is measured by your manager. However, you’re uniquely positioned to exercise a much greater impact on your organization if you think about developing a data culture along with your data products.

The data you produce is likely consumed by countless data scientists, data analysts, and users spread across disparate lines of business. In large organizations, these individuals may be fragmented and unaware of similar questions they are asking and answering with the same core data assets. In this sense, your data is posed to be a shared platform that can connect a network of users with common objectives. Empowered users not only will derive more value from your data, but also may be able to self-service or crowdsource more of their data needs instead of continuing to rely on a data-engineering team for ad hoc requests.

Such networks and communities won’t arise without concerted effort. Downstream users are typically focused on obtaining answers to specific business and strategy questions and see data as a means to an end—not a central, binding force in an analytical ecosystem. Additionally, the data literacy and proficiency of data consumers across ...