Here are the steps to evaluate a feature engineering procedure:
- Obtain a baseline performance of the machine learning model before applying any feature engineering procedures
- Apply feature engineering and combinations of feature engineering procedures
- For each application of feature engineering, obtain a performance measure and compare it to our baseline performance
- If the delta (change in) performance precedes a threshold (usually defined by the human), we deem that procedure helpful and apply it to our machine learning pipeline
- This change in performance will usually be measured as a percentage (if the baseline went from 40% accuracy to 76% accuracy, that is a 90% improvement)
In terms ...