Idea
The basic idea behind LMS filter is to approach the optimum filter weights, by updating the filter weights in a manner to converge to the optimum filter weight. The algorithm starts by assuming a small weights (zero in most cases), and at each step, by finding the gradient of the mean square error, the weights are updated. That is, if the MSE-gradient is positive, it implies, the error would keep increasing positively, if the same weight is used for further iterations, which means we need to reduce the weights. In the same way, if the gradient is negative, we need to increase the weights. So, the basic weight update equation is :
,
where represents the mean-square error. The negative sign indicates that, we need to change the weights in a direction opposite to that of the gradient slope.
The mean-square error, as a function of filter weights is a quadratic function which means it has only one extrema, that minimises the mean-square error, which is the optimal weight. The LMS thus, approaches towards this optimal weights by ascending/descending down the mean-square-error vs filter weight curve.
Read more about this topic: Least Mean Squares Filter
Famous quotes containing the word idea:
“I don’t think he has any idea he’s a dog, not really. Of course, he thinks he has a rather odd figure for a man.”
—Dodie Smith, and Lewis Allen. Pamela Fitzgerald (Ruth Hussey)
“It is very iniquitous to make me pay my debts—you have no idea of the pain it gives one.”
—George Gordon Noel Byron (1788–1824)
“A cow does not know how much milk it has until the milkman starts working on it. Then it looks round in surprise and sees the pail full to the brim. In the same way a writer has no idea how much he has to say till his pen draws it out of him. Thoughts will then appear on the paper that he is amazed to find that he possessed. “How brilliant!” he says to himself. “I had no idea I was so intelligent.” But the reader may not be so im pressed.”
—Gerald Branan (1894–1987)