Hands-On Artificial Intelligence on Amazon Web Services

Evaluating model performance

In this section, we will evaluate the performance of the three trained models. Our hypothesis is that the first model, which was trained on clicks from Monday and Tuesday, is less predictive of app downloads on the later part of Thursday compared to the second and third models. Similarly, the performance of the second model, which was trained on clicks from Monday through Wednesday, should be less than that of the third model, which was trained on clicks from Monday through the majority of Thursday.

We will begin by analyzing the features that are deemed important for each of the models, as shown in the following code:

exp_lst = ['exp1', 'exp2', 'exp3']for exp in exp_lst: model_file = os.path.join(sm_output_loc, ...