Online Randomized Controlled Experiments
Web sites can run randomized controlled experiments to create a feedback loop. Key differences between offline experimentation and online experiments include:
- Logging: user interactions can be logged reliably.
- Number of users: large sites, such as Amazon, Bing/Microsoft, and Google run experiments, each with over a million users.
- Number of concurrent experiments: large sites run tens to hundreds of overlapping, or concurrent, experiments.
- Robots, whether web crawlers from valid sources or malicious internet bots.
- Ability to ramp-up experiments from low percentages to higher percentages.
- Ability to use the pre-experiment period as an A/A test to reduce variance.
Read more about this topic: Randomized Experiment
Famous quotes containing the words controlled and/or experiments:
“You enter a state of controlled passivity, you relax your grip and accept that even if your declared intention is to justify the ways of God to man, you might end up interesting your readers rather more in Satan.”
—Ian McEwan (b. 1938)
“The man who invented Eskimo Pie made a million dollars, so one is told, but E.E. Cummings, whose verse has been appearing off and on for three years now, and whose experiments should not be more appalling to those interested in poetry than the experiment of surrounding ice-cream with a layer of chocolate was to those interested in soda fountains, has hardly made a dent in the doughy minds of our so-called poetry lovers.”
—John Dos Passos (1896–1970)