The Chain Rule, Forward and Reverse Accumulation
Fundamental to AD is the decomposition of differentials provided by the chain rule. For the simple composition the chain rule gives
Usually, two distinct modes of AD are presented, forward accumulation (or forward mode) and reverse accumulation (or reverse mode). Forward accumulation specifies that one traverses the chain rule from right to left (that is, first one computes and then ), while reverse accumulation has the traversal from left to right.
Read more about this topic: Automatic Differentiation
Famous quotes containing the words chain, reverse and/or accumulation:
“The conclusion suggested by these arguments might be called the paradox of theorizing. It asserts that if the terms and the general principles of a scientific theory serve their purpose, i. e., if they establish the definite connections among observable phenomena, then they can be dispensed with since any chain of laws and interpretive statements establishing such a connection should then be replaceable by a law which directly links observational antecedents to observational consequents.”
—C.G. (Carl Gustav)
“Now listen, buddy, there are a few corny ideas you got to get out of your head if you’re going to fly an airplane. Most things are just the reverse from what people think. The higher you are the safer you are. The Earth down there, that, that’s your enemy because once you hit that, boy, you splatter.”
—Dalton Trumbo (1905–1976)
“Every political system is an accumulation of habits, customs, prejudices, and principles that have survived a long process of trial and error and of ceaseless response to changing circumstances. If the system works well on the whole, it is a lucky accident—the luckiest, indeed, that can befall a society.”
—Edward C. Banfield (b. 1916)