Radial Basis Function Network - Network Architecture

Network Architecture

Radial basis function (RBF) networks typically have three layers: an input layer, a hidden layer with a non-linear RBF activation function and a linear output layer. The input can be modeled as a vector of real numbers . The output of the network is then a scalar function of the input vector, and is given by

where N is the number of neurons in the hidden layer, is the center vector for neuron i, and is the weight of neuron i in the linear output neuron. Functions that depend only on the distance from a center vector are radially symmetric about that vector, hence the name radial basis function. In the basic form all inputs are connected to each hidden neuron. The norm is typically taken to be the Euclidean distance (although the Mahalanobis distance appears to perform better in general) and the radial basis function is commonly taken to be Gaussian

.

The Gaussian basis functions are local to the center vector in the sense that

i.e. changing parameters of one neuron has only a small effect for input values that are far away from the center of that neuron.

Given certain mild conditions on the shape of the activation function, RBF networks are universal approximators on a compact subset of . This means that an RBF network with enough hidden neurons can approximate any continuous function with arbitrary precision.

The parameters, and are determined in a manner that optimizes the fit between and the data.