Statistical classification is a statistical procedure in which individual items are placed into groups based on quantitative information on one or more characteristics inherent in the items (referred to as traits, variables, characters, etc) and based on a training set of previously labeled items.

Formally, the problem can be stated as follows: given training data $\backslash \{(\backslash mathbf\{x\_1\},y),\backslash dots,(\backslash mathbf\{x\_n\},\; y)\backslash \}$ produce a classifier $h:\backslash mathcal\{X\}\backslash rightarrow\backslash mathcal\{Y\}$ which maps an object $\backslash mathbf\{x\}\; \backslash in\; \backslash mathcal\{X\}$ to its classification label $y\; \backslash in\; \backslash mathcal\{Y\}$. For example, if the problem is filtering spam, then $\backslash mathbf\{x\_i\}$ is some representation of an email and $y$ is either "Spam" or "Non-Spam".

Statistical classification algorithms are typically used in pattern recognition systems.

Note: in community ecology, the term "classification" is synonymous with what is commonly known (in machine learning) as clustering. See that article for more information about purely unsupervised techniques.

Statistical classification techniques

---

While there are many methods for classification, they are solving one of three related mathematical problems.

The first is to find a map of a feature space (which is typically a multi-dimensional vector space) to a set of labels. This is equivalent to partitioning the feature space into regions, then assigning a label to each region. Such algorithms (e.g., the nearest neighbour algorithm) typically do not yield confidence or class probabilities, unless post-processing is applied. Another set of algorithms to solve this problem first apply unsupervised clustering to the feature space, then attempt to label each of the clusters or regions.

The second problem is to consider classification as an estimation problem, where the goal is to estimate a function of the form

$P(\{\backslash rm\; class\}|\{\backslash vec\; x\})\; =\; f\backslash left(\backslash vec\; x;\backslash vec\; \backslash theta\backslash right)$

where the feature vector input is $\backslash vec\; x$, and the function f is typically parameterized by some parameters $\backslash vec\; \backslash theta$. In the Bayesian approach to this problem, instead of choosing a single parameter vector $\backslash vec\; \backslash theta$, the result is integrated over all possible thetas, with weighted by how likely they are given the training data D:

$P(\{\backslash rm\; class\}|\{\backslash vec\; x\})\; =\; \backslash int\; f\backslash left(\backslash vec\; x;\backslash vec\; \backslash theta\backslash right)P(\backslash vec\; \backslash theta|D)\; d\backslash vec\; \backslash theta$

The third problem is related to the second, but the problem is to estimate the class-conditional probabilities $P(\backslash vec\; x|\{\backslash rm\; class\})$ and then use Bayes' rule to produce the class probability as in the second problem.

Examples of classification algorithms include:

• Linear classifiers
  • Fisher's linear discriminant
  • Logistic regression
  • Naive Bayes classifier
  • Perceptron
• k-nearest neighbor
• Boosting
• Decision trees
• Neural networks
• Bayesian networks
• Support vector machines
• Hidden Markov models

Application domains

---

• Computer vision
  • Medical Imaging and Medical Image Analysis
  • Optical character recognition
• Geostatistics
• Speech recognition
• Handwriting recognition
• Biometric indentification
• Document classification
• Internet search engines
• Credit scoring

External links

---

• Classifier showdown A practical comparison of classification algorithms.

See also

---

• Data mining
• Fuzzy logic
• Information retrieval

Machine learning | Classification algorithms

تصنيف إحصائي | Phân loại bằng thống kê