Statistically Sound Associations
One limitation of the standard approach to discovering associations is that by searching massive numbers of possible associations to look for collections of items that appear to be associated, there is a large risk of finding many spurious associations. These are collections of items that co-occur with unexpected frequency in the data, but only do so by chance. For example, suppose we are considering a collection of 10,000 items and looking for rules containing two items in the left-hand-side and 1 item in the right-hand-side. There are approximately 1,000,000,000,000 such rules. If we apply a statistical test for independence with a significance level of 0.05 it means there is only a 5% chance of accepting a rule if there is no association. If we assume there are no associations, we should nonetheless expect to find 50,000,000,000 rules. Statistically sound association discovery controls this risk, in most cases reducing the risk of finding any spurious associations to a user-specified significance level.
Read more about this topic: Association Rule Learning
Famous quotes containing the words sound and/or associations:
“This is of the loon—I do not mean its laugh, but its looning,—is a long-drawn call, as it were, sometimes singularly human to my ear,—hoo-hoo-ooooo, like the hallooing of a man on a very high key, having thrown his voice into his head. I have heard a sound exactly like it when breathing heavily through my own nostrils, half awake at ten at night, suggesting my affinity to the loon; as if its language were but a dialect of my own, after all.”
—Henry David Thoreau (1817–1862)
“Hardly a man in the world has an opinion upon morals, politics or religion which he got otherwise than through his associations and sympathies. Broadly speaking, there are none but corn-pone opinions. And broadly speaking, Corn-Pone stands for Self- Approval. Self-approval is acquired mainly from the approval of other people. The result is Conformity.”
—Mark Twain [Samuel Langhorne Clemens] (1835–1910)