History
The first artificial neuron was the Threshold Logic Unit (TLU) first proposed by Warren McCulloch and Walter Pitts in 1943. As a transfer function, it employed a threshold, equivalent to using the Heaviside step function. Initially, only a simple model was considered, with binary inputs and outputs, some restrictions on the possible weights, and a more flexible threshold value. Since the beginning it was already noticed that any boolean function could be implemented by networks of such devices, what is easily seen from the fact that one can implement the AND and OR functions, and use them in the disjunctive or the conjunctive normal form.
Researchers also soon realized that cyclic networks, with feedbacks through neurons, could define dynamical systems with memory, but most of the research concentrated (and still does) on strictly feed-forward networks because of the smaller difficulty they present.
One important and pioneering artificial neural network that used the linear threshold function was the perceptron, developed by Frank Rosenblatt. This model already considered more flexible weight values in the neurons, and was used in machines with adaptive capabilities. The representation of the threshold values as a bias term was introduced by Widrow in 1960.
In the late 1980s, when research on neural networks regained strength, neurons with more continuous shapes started to be considered. The possibility of differentiating the activation function allows the direct use of the gradient descent and other optimization algorithms for the adjustment of the weights. Neural networks also started to be used as a general function approximation model.
Read more about this topic: Artificial Neuron
Famous quotes containing the word history:
“What would we not give for some great poem to read now, which would be in harmony with the scenery,—for if men read aright, methinks they would never read anything but poems. No history nor philosophy can supply their place.”
—Henry David Thoreau (1817–1862)
“They are a sort of post-house,where the Fates
Change horses, making history change its tune,
Then spur away o’er empires and o’er states,
Leaving at last not much besides chronology,
Excepting the post-obits of theology.”
—George Gordon Noel Byron (1788–1824)
“Yet poetry, though the last and finest result, is a natural fruit. As naturally as the oak bears an acorn, and the vine a gourd, man bears a poem, either spoken or done. It is the chief and most memorable success, for history is but a prose narrative of poetic deeds.”
—Henry David Thoreau (1817–1862)