Solution of A Non-linear System
Gradient descent can also be used to solve a system of nonlinear equations. Below is an example that shows how to use the gradient descent to solve for three unknown variables, x1, x2, and x3. This example shows one iteration of the gradient descent.
Consider a nonlinear system of equations:
suppose we have the function
where
and the objective function
With initial guess
We know that
where
The Jacobian matrix
Then evaluating these terms at
and
So that
and
Now a suitable must be found such that . This can be done with any of a variety of line search algorithms. One might also simply guess which gives
evaluating at this value,
The decrease from to the next step's value of is a sizable decrease in the objective function. Further steps would reduce its value until a solution to the system was found.
Read more about this topic: Gradient Descent
Famous quotes containing the words solution of, solution and/or system:
“Give a scientist a problem and he will probably provide a solution; historians and sociologists, by contrast, can offer only opinions. Ask a dozen chemists the composition of an organic compound such as methane, and within a short time all twelve will have come up with the same solution of CH4. Ask, however, a dozen economists or sociologists to provide policies to reduce unemployment or the level of crime and twelve widely differing opinions are likely to be offered.”
—Derek Gjertsen, British scientist, author. Science and Philosophy: Past and Present, ch. 3, Penguin (1989)
“What is history? Its beginning is that of the centuries of systematic work devoted to the solution of the enigma of death, so that death itself may eventually be overcome. That is why people write symphonies, and why they discover mathematical infinity and electromagnetic waves.”
—Boris Pasternak (18901960)
“An avant-garde man is like an enemy inside a city he is bent on destroying, against which he rebels; for like any system of government, an established form of expression is also a form of oppression. The avant-garde man is the opponent of an existing system.”
—Eugène Ionesco (b. 1912)










