Test Set

A test set is a portion of a data set used in data mining to assess the likely future performance of a single prediction or classification model that has been selected from among competing models, based on its performance with the validation set. While the validation set provides a useful guide to selecting the best-performing model, that best-performing model may still do better with the validation data than it will with unseen future data. This is particularly the case when many different models are assessed – in such cases there is a greater chance that the “winning” model emerges on top because it just happens (by chance) to fit some particularities of the validation set quite well.