Ensemble Methods

In predictive modeling, ensemble methods refer to the practice of taking multiple models and averaging their predictions. In the case of classification models, the average can be that of a probability score attached to the classification. Models can differ with respect to algorithms used (e.g. neural net, logistic regression), settings used to configure algorithms (e.g. number of hidden layers in a neural net), variables used, and case weights assigned. Ensemble predictions often outperform all of the constituent models that make up the ensemble.