Chapter 8. Getting Data in One Place
Once you collect all your data, you have to have an environment where you can process it and produce results. In this chapter, I provide some notes on an architecture to facilitate the rapid development and operational deployment of security analysis software (analytics1).
There are a number of ways to implement this; the version youâll see in Figure 8-1 is a high-level diagram for a basic environment. In general, these environments should have the following attributes:
-
Robust, universal access to all sensor data. The term âuniversalâ here is used in lieu of âcentralizedââitâs not critical that the data be in one place, but it is critical that anyone implementing analytical code have uniform access to all the data.
-
Access to a Turing-complete language. This differentiates an analysis environment from the classic security console. Complex analytics require access to a general-purpose programming language and the ability to build constructs that rely on in-place memory manipulationâso, Python good, R good, SQL bad.
-
Performance. Any analytic system will have to deal with resource contention; it is better to overprovision for multiple simultaneous queries early on rather than have your analysts fighting to get results in a crisis.
Get Network Security Through Data Analysis, 2nd Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
