Solution of A Non-linear System
Gradient descent can also be used to solve a system of nonlinear equations. Below is an example that shows how to use the gradient descent to solve for three unknown variables, x1, x2, and x3. This example shows one iteration of the gradient descent.
Consider a nonlinear system of equations:
suppose we have the function
where
and the objective function
With initial guess
We know that
where
The Jacobian matrix
Then evaluating these terms at
and
So that
and
Now a suitable must be found such that . This can be done with any of a variety of line search algorithms. One might also simply guess which gives
evaluating at this value,
The decrease from to the next step's value of is a sizable decrease in the objective function. Further steps would reduce its value until a solution to the system was found.
Read more about this topic: Gradient Descent
Famous quotes containing the words solution of, solution and/or system:
“What is history? Its beginning is that of the centuries of systematic work devoted to the solution of the enigma of death, so that death itself may eventually be overcome. That is why people write symphonies, and why they discover mathematical infinity and electromagnetic waves.”
—Boris Pasternak (18901960)
“Who shall forbid a wise skepticism, seeing that there is no practical question on which any thing more than an approximate solution can be had? Is not marriage an open question, when it is alleged, from the beginning of the world, that such as are in the institution wish to get out, and such as are out wish to get in?”
—Ralph Waldo Emerson (18031882)
“The system was breaking down. The one who had wandered alone past so many happenings and events began to feel, backing up along the primal vein that led to his center, the beginning of hiccup that would, if left to gather, explode the center to the extremities of life, the suburbs through which one makes ones way to where the country is.”
—John Ashbery (b. 1927)










