Chapter 37. Data Security Code and Tests
Diogo Miyake
Securing data code and tests is essential to protect data, maintain data integrity, prevent cyberattacks, ensure business continuity, uphold trust, and comply with legal and regulatory standards. One way to do this is to secure data pipelines. The importance of securing data pipelines lies in some steps and processes that must be taken before simply trying to solve the problem. For example, making a security assessment in data pipelines helps to understand risks and vulnerabilities to mitigate them before deploying data pipelines.
A data pipeline is an automated method in which data processes are used to fetch, transform, or make data available, either via API, frameworks, or in-house created systems; the sources can be diverse, such as SQL, NoSQL, files, and videos.
To secure data pipelines, consider asking some questions before simply delivering a certain code that performs a certain operation. For example, if it performs data ingestion or data transformation, in order to deliver it is needed to check what the business needs, to deliver more quality and security. The following are examples of questions:
-
What is the demand of the business area?
-
Do we have a sensible default describing the best practices and processes for the technology area?
-
Do we have an SOC area? If yes, how can we create a product that is in conformity ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access