Meta Learning (computer Science) - Different Views On Meta Learning

Different Views On Meta Learning

These are some of the views on (and approaches to) meta learning, please note that there exist many variations on these general approaches:

• Discovering meta-knowledge works by inducing knowledge (e.g. rules) that expresses how each learning method will perform on different learning problems. The meta-data is formed by characteristics of the data (general, statistical, information-theoretic,... ) in the learning problem, and characteristics of the learning algorithm (type, parameter settings, performance measures,...). Another learning algorithm then learns how the data characteristics relate to the algorithm characteristics. Given a new learning problem, the data characteristics are measured, and the performance of different learning algorithms can be predicted. Hence, one can select the algorithms best suited for the new problem, at least if the induced relationship holds.

• Stacked generalisation works by combining a number of (different) learning algorithms. The meta-data is formed by the predictions of those different algorithms. Then another learning algorithm learns from this meta-data to predict which combinations of algorithms give generally good results. Given a new learning problem, the predictions of the selected set of algorithms are combined (e.g. by (weighted) voting) to provide the final prediction. Since each algorithm is deemed to work on a subset of problems, a combination is hoped to be more flexible and still able to make good predictions.

• Boosting is related to stacked generalisation, but uses the same algorithm multiple times, where the examples in the training data get different weights over each run. This yields different predictions, each focused on rightly predicting a subset of the data, and combining those predictions leads to better (but more expensive) results.

• Dynamic bias selection works by altering the inductive bias of a learning algorithm to match the given problem. This is done by altering key aspects of the learning algorithm, such as the hypothesis representation, heuristic formulae, or parameters. Many different approaches exist.

• Inductive transfer also called learning to learn, studies how the learning process can be improved over time. Meta-data consists of knowledge about previous learning episodes, and is used to efficiently develop an effective hypothesis for a new task. A related approach is called learning to learn, in which the goal is to use acquired knowledge from one domain to help learning in other domains.

• Other approaches using meta-data to improve automatic learning are learning classifier systems, case-based reasoning and constraint satisfaction.