The Chain Rule, Forward and Reverse Accumulation
Fundamental to AD is the decomposition of differentials provided by the chain rule. For the simple composition the chain rule gives
Usually, two distinct modes of AD are presented, forward accumulation (or forward mode) and reverse accumulation (or reverse mode). Forward accumulation specifies that one traverses the chain rule from right to left (that is, first one computes and then ), while reverse accumulation has the traversal from left to right.
Read more about this topic: Automatic Differentiation
Famous quotes containing the words chain, reverse and/or accumulation:
“It could not have come down to us so far,
Through the interstices of things ajar
On the long bead chain of repeated birth,
To be a bird while we are men on earth,”
—Robert Frost (1874–1963)
“We came home from the ridotto so late, or rather so early, that it was not possible for me to write. Indeed we did not go ... till past eleven o’clock: but nobody does. A terrible reverse of the order of nature! We sleep with the sun, and wake with the moon.”
—Frances Burney (1752–1840)
“Every political system is an accumulation of habits, customs, prejudices, and principles that have survived a long process of trial and error and of ceaseless response to changing circumstances. If the system works well on the whole, it is a lucky accident—the luckiest, indeed, that can befall a society.”
—Edward C. Banfield (b. 1916)