Hands-On Artificial Intelligence for Cybersecurity

Turning service logs into datasets

One of the problems related to network anomaly detection is how to collect sufficient and reliable data to perform algorithm analysis and training. There are hundreds of datasets freely available on the internet, and there are different datasets on which to carry out our analysis; however, it is also possible to use our own network devices to accumulate data that is more representative of our specific reality.

To this end, we can use the following:

• Network devices, such as routers or network sensors, using tools such as tcpdump for data collection
• Services logs and system logs

Within the operating system, services logs and system logs can be stored in various places. In the case of a Unix-like system, ...