A simple way of defining machine learning is that it either predicts an instance’s class, which is known as classification, or it predicts a number or value, which is known as regression. We’ll look at classification first. In technical terms, classification predictive modelling is the task of approximating a mapping function (f) from input variables (X) to discrete output variables (y). 

Successful machine learning produces accurate predictions; unsuccessful machine learning produces misclassification. 

An example of a classification problem is a model predicting whether or not an instance is junk mail. Another would be a model predicting whether next summer is likely to be hot and dry or mild and rainy. A model predicting tomorrow’s temperature would be an example of regression, not classification. 

Some algorithms can be used for both classification and regression with small modifications, such as decision trees and artificial neural networks. We’ll be looking at decision trees later in this unit and artificial neural networks in Unit 5. This is not always the case, and some algorithms are not normally used both for classification and regression problems. Examples include linear regression for regression predictive modelling and (somewhat confusingly from the name) logistic regression for classification predictive modelling.

The evaluation of classification and regression predictions does not overlap. For example:

  • Classification predictions can be evaluated using accuracy, whereas regression predictions cannot.

  • Regression predictions can be evaluated using root mean squared error, whereas classification predictions cannot.

Next, we’ll be looking specifically at the classification algorithms below:

  • K-Nearest Neighbors

  • Decision Trees

  • Logistic Regression

  • Support Vector Machine