The Chain Rule, Forward and Reverse Accumulation
Fundamental to AD is the decomposition of differentials provided by the chain rule. For the simple composition the chain rule gives
Usually, two distinct modes of AD are presented, forward accumulation (or forward mode) and reverse accumulation (or reverse mode). Forward accumulation specifies that one traverses the chain rule from right to left (that is, first one computes and then ), while reverse accumulation has the traversal from left to right.
Read more about this topic: Automatic Differentiation
Famous quotes containing the words chain, reverse and/or accumulation:
“The conclusion suggested by these arguments might be called the paradox of theorizing. It asserts that if the terms and the general principles of a scientific theory serve their purpose, i. e., if they establish the definite connections among observable phenomena, then they can be dispensed with since any chain of laws and interpretive statements establishing such a connection should then be replaceable by a law which directly links observational antecedents to observational consequents.”
—C.G. (Carl Gustav)
“We came home from the ridotto so late, or rather so early, that it was not possible for me to write. Indeed we did not go ... till past eleven o’clock: but nobody does. A terrible reverse of the order of nature! We sleep with the sun, and wake with the moon.”
—Frances Burney (1752–1840)
“In societies where modern conditions of production prevail, all of life presents itself as an immense accumulation of spectacles. Everything that was directly lived has moved away into a representation.”
—Guy Debord (b. 1931)