Analytics for the Internet of Things (IoT)

Test set

It is best practice to keep a percentage of your training data aside, until you have found what you believe to be the optimal performing model or set of optimized models (from different techniques).

This percentage of data held out, usually called the test set, will serve as your final test of the models that have been optimized through the validation process. If you are hiring a consulting company to perform the modeling, keep a test set to yourself and do not provide it to them. Use it to confirm the accuracy of their resulting optimized model. You should get similar results to what they reported. If not, you have something to discuss with them before you pay their fees.