Bandits

Bandits refers to a class of algorithms in which users or subjects make repeated choices among, or decisions in reaction to, multiple alternatives. For example, a web retailer might have a set of N ways of presenting an offer. The task of the algorithm is to efficiently and accurately select the optimum alternative — in the case of the retailer, it might be the offer than generates the most sales. The algorithm must balance the temptation to rapidly move to an apparent winner (paying too much attention to the signal) against the loss of time and sales if it waits too long to anoint a winner.