The Chain Rule, Forward and Reverse Accumulation
Fundamental to AD is the decomposition of differentials provided by the chain rule. For the simple composition the chain rule gives
Usually, two distinct modes of AD are presented, forward accumulation (or forward mode) and reverse accumulation (or reverse mode). Forward accumulation specifies that one traverses the chain rule from right to left (that is, first one computes and then ), while reverse accumulation has the traversal from left to right.
Read more about this topic: Automatic Differentiation
Famous quotes containing the words chain, reverse and/or accumulation:
“Loyalty to petrified opinions never yet broke a chain or freed a human soul in this world—and never will.”
—Mark Twain [Samuel Langhorne Clemens] (1835–1910)
“During the late war [the American Revolution] I had an infallible rule for deciding what [Great Britain] would do on every occasion. It was, to consider what they ought to do, and to take the reverse of that as what they would assuredly do, and I can say with truth that I was never deceived.”
—Thomas Jefferson (1743–1826)
“Creation destroys as it goes, throws down one tree for the rise of another. But ideal mankind would abolish death, multiply itself million upon million, rear up city upon city, save every parasite alive, until the accumulation of mere existence is swollen to a horror.”
—D.H. (David Herbert)