Automatic Differentiation Using Dual Numbers
Forward mode automatic differentiation is accomplished by augmenting the algebra of real numbers and obtaining a new arithmetic. An additional component is added to every number which will represent the derivative of a function at the number, and all arithmetic operators are extended for the augmented algebra. The augmented algebra is the algebra of dual numbers. Computer programs often implement this using the complex number representation.
Replace every number with the number, where is a real number, but is nothing but a symbol with the property . Using only this, we get for the regular arithmetic
and likewise for subtraction and division.
Now, we may calculate polynomials in this augmented arithmetic. If, then
where denotes the derivative of with respect to its first argument, and, called a seed, can be chosen arbitrarily.
The new arithmetic consists of ordered pairs, elements written, with ordinary arithmetics on the first component, and first order differentiation arithmetic on the second component, as described above. Extending the above results on polynomials to analytic functions we obtain a list of the basic arithmetic and some standard functions for the new arithmetic:
and in general for the primitive function ,
where and are the derivatives of with respect to its first and second arguments, respectively.
When a binary basic arithmetic operation is applied to mixed arguments — the pair and the real number — the real number is first lifted to . The derivative of a function at the point is now found by calculating using the above arithmetic, which gives as the result.
Read more about this topic: Automatic Differentiation
Famous quotes containing the words automatic, dual and/or numbers:
“Predictions of the future are never anything but projections of present automatic processes and procedures, that is, of occurrences that are likely to come to pass if men do not act and if nothing unexpected happens; every action, for better or worse, and every accident necessarily destroys the whole pattern in whose frame the prediction moves and where it finds its evidence.”
—Hannah Arendt (1906–1975)
“Thee for my recitative,
Thee in the driving storm even as now, the snow, the winter-day
declining,
Thee in thy panoply, thy measur’d dual throbbing and thy beat
convulsive,
Thy black cylindric body, golden brass and silvery steel,”
—Walt Whitman (1819–1892)
“The principle of majority rule is the mildest form in which the force of numbers can be exercised. It is a pacific substitute for civil war in which the opposing armies are counted and the victory is awarded to the larger before any blood is shed. Except in the sacred tests of democracy and in the incantations of the orators, we hardly take the trouble to pretend that the rule of the majority is not at bottom a rule of force.”
—Walter Lippmann (1889–1974)