Regularization

Regularization refers to a wide variety of techniques used to bring structure to statistical models in the face of data size, complexity and sparseness. Advances in digital processing, storage and retrieval have led to huge and growing data sets (“Big Data”). Regularization is used to allow models to usefully model such data without overfitting. A very simple example is linear regression; other examples are smoothing techniques.