Idea
The basic idea behind LMS filter is to approach the optimum filter weights, by updating the filter weights in a manner to converge to the optimum filter weight. The algorithm starts by assuming a small weights (zero in most cases), and at each step, by finding the gradient of the mean square error, the weights are updated. That is, if the MSE-gradient is positive, it implies, the error would keep increasing positively, if the same weight is used for further iterations, which means we need to reduce the weights. In the same way, if the gradient is negative, we need to increase the weights. So, the basic weight update equation is :
,
where represents the mean-square error. The negative sign indicates that, we need to change the weights in a direction opposite to that of the gradient slope.
The mean-square error, as a function of filter weights is a quadratic function which means it has only one extrema, that minimises the mean-square error, which is the optimal weight. The LMS thus, approaches towards this optimal weights by ascending/descending down the mean-square-error vs filter weight curve.
Read more about this topic: Least Mean Squares Filter
Famous quotes containing the word idea:
“See, in the Navy, during the war, I got used to the idea that something might happen to me, I might not make it. Well, I also got used to the idea that my wife and children were safe at home, they’d be all right no matter what. But what I didn’t reckon with was that in this, this kind of a monstrous war, something might happen to them, and not to me. Well it did, and I can’t, I can’t cope with it.”
—John Paxton (1911–1985)
“To me, literature is a calling, even a kind of salvation. It connects me with an enterprise that is over 2,000 years old. What do we have from the past? Art and thought. That’s what lasts. That’s what continues to feed people and given them an idea of something better. A better state of one’s feelings or simply the idea of a silence in one’s self that allows one to think or to feel. Which to me is the same.”
—Susan Sontag (b. 1933)
“No great idea in its beginning can ever be within the law. How can it be within the law? The law is stationary. The law is fixed. The law is a chariot wheel which binds us all regardless of conditions or place or time.”
—Emma Goldman (1869–1940)