In this recipe, we will look at how to parse logs and quickly find errors. There are job logs, which are aggregated on HDFS, logs which include daemon logs, system logs, and so on.
We will look at some keywords and commands to find the errors in logs.
To complete the recipe, the user must have a running Hadoop cluster, must have completed the Setting up multi-node HBase cluster recipe in Chapter 9, HBase Administration, and know Bash or Perl/Python scripting basics.
edge1.cyrus.comnode in the cluster and switch to user
hadoop. However, we can connect to any node in the cluster from which we can access the logs.