June 2021
Intermediate to advanced
264 pages
7h 19m
English
Content preview from 97 Things Every Data Engineer Should Know
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Chapter 34. Fundamental Knowledge
Pedro Marcelino
Knowledge is growing exponentially. It is hard to measure the growth of knowledge, especially over a long period of time. However, if we use the number of academic publications as a proxy for knowledge, we can observe that knowledge is doubling every nine years. If you start working today, 20 years from now, when you are in the middle of your career, the amount of knowledge involved in your professional life will be approximately four times more than it is today.
This conclusion would be similar (or even more shocking) if we instead consider the number of books published, the number of scientists, or the amount of data created each day as a proxy for knowledge. And that’s scary.
All knowledge workers, such as data engineers, therefore face a challenge. On the one hand, the growth of knowledge leads to an unmanageable need to keep up-to-date with several new concepts, technologies, and frameworks. On the other hand, knowledge is becoming more transient, and what we learn today may be obsolete tomorrow. Since it is hard to believe that in the near future knowledge growth rates will decrease, all data engineers suffer from this particular curse of knowledge.
Thus, the most important question is, how do we deal with this growth of knowledge?
One possible answer is to focus on the fundamentals. In a world of constant change, the fundamentals ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.