Machine learning


 * For the journal, see Machine Learning (journal).

As a broad subfield of artificial intelligence, machine learning is concerned with the design and development of algorithms and techniques that allow computers to "learn". At a general level, there are two types of learning: inductive, and deductive. Inductive machine learning methods extract rules and patterns out of massive data sets.

The major focus of machine learning research is to extract information from data automatically, by computational and statistical methods. Hence, machine learning is closely related to data mining and statistics but also theoretical computer science.

Machine learning has a wide spectrum of applications including natural language processing, syntactic pattern recognition, search engines, medical diagnosis, bioinformatics and cheminformatics, detecting credit card fraud, stock market analysis, classifying DNA sequences, speech and handwriting recognition, object recognition in computer vision, game playing and robot locomotion.

Human interaction
Some machine learning systems attempt to eliminate the need for human intuition in the analysis of the data, while others adopt a collaborative approach between human and machine. Human intuition cannot be entirely eliminated since the designer of the system must specify how the data are to be represented and what mechanisms will be used to search for a characterization of the data. Machine learning can be viewed as an attempt to automate parts of the scientific method.

Some statistical machine learning researchers create methods within the framework of Bayesian statistics.

Algorithm types
Machine learning algorithms are organized into a taxonomy, based on the desired outcome of the algorithm. Common algorithm types include:


 * Supervised learning &mdash; in which the algorithm generates a function that maps inputs to desired outputs. One standard formulation of the supervised learning task is the classification problem: the learner is required to learn (to approximate) the behavior of a function which maps a vector $$[X_1, X_2, \ldots X_N]\,$$ into one of several classes by looking at several input-output examples of the function.
 * Unsupervised learning &mdash; which models a set of inputs: labeled examples are not available.
 * Semi-supervised learning &mdash; which combines both labeled and unlabeled examples to generate an appropriate function or classifier.
 * Reinforcement learning &mdash; in which the algorithm learns a policy of how to act given an observation of the world. Every action has some impact in the environment, and the environment provides feedback that guides the learning algorithm.
 * Transduction &mdash; similar to supervised learning, but does not explicitly construct a function: instead, tries to predict new outputs based on training inputs, training outputs, and test inputs which are available while training.
 * Learning to learn &mdash; in which the algorithm learns its own inductive bias based on previous experience.

The computational analysis of machine learning algorithms and their performance is a branch of theoretical computer science known as computational learning theory.

Machine learning topics

 * This list represents the topics covered on a typical machine learning course.


 * Prerequisites
 * Bayesian theory


 * Modeling conditional probability density functions : regression and classification
 * Artificial neural networks
 * Decision trees
 * Gene expression programming
 * Genetic algorithms
 * Genetic programming
 * Inductive Logic Programming
 * Gaussian process regression
 * Linear discriminant analysis
 * K-nearest neighbor
 * Minimum message length
 * Perceptron
 * Quadratic classifier
 * Radial basis function networks
 * Support vector machines


 * Algorithms for estimating model parameters:
 * Dynamic programming
 * Expectation-maximization algorithm


 * Modeling probability density functions through generative models:
 * Graphical models including Bayesian networks and Markov Random Fields
 * Generative Topographic Mapping


 * Approximate inference techniques
 * Monte Carlo methods
 * Variational Bayes
 * Variable-order Markov models
 * Variable-order Bayesian networks
 * Loopy belief propagation


 * Optimization
 * Most of methods listed above either use optimization or are instances of optimization algorithms


 * Meta-learning (ensemble methods)
 * Boosting
 * Bootstrap aggregating
 * Random forest
 * Weighted majority algorithm


 * Inductive transfer and learning to learn
 * Inductive transfer
 * Reinforcement learning
 * Temporal difference
 * Monte-Carlo method