The convergence plots were obtained by parsing the Amazon ML model logs to extract the data into a CSV file that could be used later on to create plots. The process is simple and mostly based on command line scripting using the grep and the sed commands. We want to extract and parse the following lines from the log file:
16/12/25 13:54:03 INFO: learner-id=4202 model-performance: accuracy=0.6562 recall=0.5000 precision=0.5909 f1-score=0.5417 auc=0.7095
And convert them into a CSV format as follows:
| iteration | alpha | learner | accuracy | recall | precision | f1 | auc |
| 1 | 0.01 | 1050 | 0.5937 | 0.56 | 0.4828 | 0.5185 | 0.6015 |
The first step is to extract the right lines from the log file. We notice that they all contain the string ...
