The Chain Rule, Forward and Reverse Accumulation
Fundamental to AD is the decomposition of differentials provided by the chain rule. For the simple composition the chain rule gives
Usually, two distinct modes of AD are presented, forward accumulation (or forward mode) and reverse accumulation (or reverse mode). Forward accumulation specifies that one traverses the chain rule from right to left (that is, first one computes and then ), while reverse accumulation has the traversal from left to right.
Read more about this topic: Automatic Differentiation
Famous quotes containing the words the chain, chain, reverse and/or accumulation:
“I was
the girl of the chain letter,
the girl full of talk of coffins and keyholes,
the one of the telephone bills,
the wrinkled photo and the lost connections....”
—Anne Sexton (1928–1974)
“To avoid tripping on the chain of the past, you have to pick it up and wind it about you.”
—Mason Cooley (b. 1927)
“During the late war [the American Revolution] I had an infallible rule for deciding what [Great Britain] would do on every occasion. It was, to consider what they ought to do, and to take the reverse of that as what they would assuredly do, and I can say with truth that I was never deceived.”
—Thomas Jefferson (1743–1826)
“Every political system is an accumulation of habits, customs, prejudices, and principles that have survived a long process of trial and error and of ceaseless response to changing circumstances. If the system works well on the whole, it is a lucky accident—the luckiest, indeed, that can befall a society.”
—Edward C. Banfield (b. 1916)