Solution of A Non-linear System
Gradient descent can also be used to solve a system of nonlinear equations. Below is an example that shows how to use the gradient descent to solve for three unknown variables, x1, x2, and x3. This example shows one iteration of the gradient descent.
Consider a nonlinear system of equations:
suppose we have the function
where
and the objective function
With initial guess
We know that
where
The Jacobian matrix
Then evaluating these terms at
and
So that
and
Now a suitable must be found such that . This can be done with any of a variety of line search algorithms. One might also simply guess which gives
evaluating at this value,
The decrease from to the next step's value of is a sizable decrease in the objective function. Further steps would reduce its value until a solution to the system was found.
Read more about this topic: Gradient Descent
Famous quotes containing the words solution of, solution and/or system:
“The truth of the thoughts that are here set forth seems to me unassailable and definitive. I therefore believe myself to have found, on all essential points, the final solution of the problems. And if I am not mistaken in this belief, then the second thing in which the value of this work consists is that it shows how little is achieved when these problems are solved.”
—Ludwig Wittgenstein (18891951)
“Theres one solution that ends all lifes problems.”
—Chinese proverb.
“Social and scientific progress are assured, sir, once our great system of postpossession payments is in operation, not the installment plan, no sir, but a system of small postpossession payments that clinch the investment. No possible rational human wish unfulfilled. A man with a salary of fifty dollars a week can start payments on a Rolls-Royce, the Waldorf-Astoria, or a troupe of trained seals if he so desires.”
—John Dos Passos (18961970)