Idea
The basic idea behind LMS filter is to approach the optimum filter weights, by updating the filter weights in a manner to converge to the optimum filter weight. The algorithm starts by assuming a small weights (zero in most cases), and at each step, by finding the gradient of the mean square error, the weights are updated. That is, if the MSE-gradient is positive, it implies, the error would keep increasing positively, if the same weight is used for further iterations, which means we need to reduce the weights. In the same way, if the gradient is negative, we need to increase the weights. So, the basic weight update equation is :
,
where represents the mean-square error. The negative sign indicates that, we need to change the weights in a direction opposite to that of the gradient slope.
The mean-square error, as a function of filter weights is a quadratic function which means it has only one extrema, that minimises the mean-square error, which is the optimal weight. The LMS thus, approaches towards this optimal weights by ascending/descending down the mean-square-error vs filter weight curve.
Read more about this topic: Least Mean Squares Filter
Famous quotes containing the word idea:
“If in the opinion of the Tsars authors were to be the servants of the state, in the opinion of the radical critics writers were to be the servants of the masses. The two lines of thought were bound to meet and join forces when at last, in our times, a new kind of regime the synthesis of a Hegelian triad, combined the idea of the masses with the idea of the state.”
—Vladimir Nabokov (1899–1977)
“I think of you more often than of anyone else in this part of the world. I’d have liked to have you for a sweetheart, or a wife, or my mother or my sister—anything that a woman can be to a man. The idea of you is a part of my mind; you influence my likes and dislikes, all my tastes, hundreds of times when I don’t realize it. You really are a part of me.”
—Willa Cather (1873–1947)
“Why inspire in us a horror of our being?... To look upon the universe as a prison cell and all men as criminals about to be executed is the idea of a fanatic.”
—Voltaire [François Marie Arouet] (1694–1778)