First exams 2027 · HL only

A4.3.2 · Machine learning approaches (HL only)

How Classification Techniques Predict Categorical Outcomes

A previous statement covered predicting a number that could fall anywhere along a range. Here, the question is different: which one of a fixed set of categories does this new case belong to? K-Nearest Neighbours and decision trees are two supervised learning techniques built to answer exactly that.

HL only Typically 4–6 marks · AO2 Command term: explain

Explainer

Df

A discrete categorical outcome is one of a fixed, limited set of labels, not a value that could fall anywhere on a scale. "Sparrow", "spam", "benign" and "malignant" are all discrete categorical outcomes; a temperature or a price is not.

K-Nearest Neighbours (K-NN) and decision trees are both used to categorize a new, unlabelled data point, based on patterns learned from existing labelled data, but they arrive at a category in genuinely different ways: K-NN looks at which already-labelled cases the new one most closely resembles, while a decision tree asks a fixed sequence of yes/no questions about the new case's features until it lands on an answer.

A birdwatching club has been logging sightings for years: for each bird, two measurements (wing length in centimetres, beak length in millimetres) and its confirmed species. A new photo comes in of a bird nobody has identified yet, with both measurements taken from the photo. Can a model suggest which species it most likely is?

Worked example

K-Nearest Neighbours and decision trees

Click each technique to see how it would classify the club's new bird, and where each one turns up in the real world.

Try it yourself

Classifying a bird with numpy

This runs K-NN's own logic from scratch: measure the new bird's distance to every already-identified bird, find the k closest ones, and let them vote on the species.

main.py
Loading Python engine…
Output
Press Run to see the output here.

Running in offline fallback mode. This example needs numpy, which only the full Python engine (Pyodide) provides, so it isn't available here. If Pyodide can't load, usually a blocked school network or no internet connection, pressing Run will show a clear message explaining why, rather than a confusing crash.

With k = 3, the new bird's three nearest neighbours are two Sparrows and one Robin, so it's classified as a Sparrow. Try changing k to 5 or 1 and re-running to see whether the answer changes, and try moving new_bird closer to the Blackbird cluster (say, [14, 19]) to see the vote flip entirely.

Worked example

Classifying with scikit-learn's decision tree

The same nine birds, this time classified with a decision tree instead. Rather than measuring distance to every point, the tree learns a sequence of yes/no splits on the features themselves.

classify_tree.pyRun locally, e.g. in VS Code
import numpy as np
import matplotlib.pyplot as plt
from sklearn.tree import DecisionTreeClassifier, plot_tree

# The same nine already-identified birds used in the K-NN example above
wing_beak = np.array([
    [11, 15], [12, 16], [10, 14],   # Robin
    [8, 10], [7, 9], [9, 11],       # Sparrow
    [15, 20], [16, 22], [14, 19],   # Blackbird
])
species = ["Robin", "Robin", "Robin",
           "Sparrow", "Sparrow", "Sparrow",
           "Blackbird", "Blackbird", "Blackbird"]

tree = DecisionTreeClassifier(random_state=0)
tree.fit(wing_beak, species)

# Classify the same unidentified bird used in the K-NN example above
new_bird = np.array([[9.5, 12]])
print("Predicted species:", tree.predict(new_bird)[0])

plt.figure(figsize=(10, 6))
plot_tree(tree, feature_names=["Wing length (cm)", "Beak length (mm)"],
          class_names=tree.classes_, filled=True)
plt.show()

Running this also predicts Sparrow, the same answer as the K-NN example, a reassuring sign when two genuinely different techniques agree. The plot window then shows the tree itself: a sequence of wing-length and beak-length splits, coloured by predicted species at each leaf. This isn't wired into this page's own in-browser console, for the same reason as the earlier statement's scikit-learn example: too much supporting library to download just for one page, so try this one in your own Python setup instead (pip install scikit-learn numpy matplotlib).

Check your understanding

A birdwatching club has exactly two possible species left to distinguish between (Robin or Sparrow) and is choosing a value of k for K-NN. Which choice risks an unbreakable tie in the majority vote?

Common mistake

A recurring assumption flagged directly for decision trees

RecurringA genuinely documented pattern: candidates assume a deeper, more detailed decision tree is automatically a better one. A deeper tree fits its own training data more closely, but past a certain depth it starts memorising quirks of the specific birds it was trained on rather than learning the genuine differences between species, and performs worse, not better, on birds it hasn't seen before.

✕

Treats depth as a free improvement: "The decision tree was made deeper so it could classify the birds more accurately." Assumes more splits can only help, when a tree deep enough to perfectly separate every single training bird is likely to have started splitting on coincidences rather than real wing- and beak-length differences between species.

✓

States the actual trade-off: "Making the tree deeper lets it fit the training birds more closely, but past a certain point it starts overfitting to quirks specific to those birds, so its depth needs to be limited or the tree pruned back to keep it accurate on birds it hasn't seen before." Treats depth as something to balance, not something to maximise.

Revision questions

Past-paper style. Write an answer, then reveal the marking points and mark yourself.

Question 1 · 2 marks

[2]State one real-world application of K-NN and one real-world application of decision trees.

Question 2 · 4 marks

[4]Explain how K-NN classifies a new, unlabelled data point.

Question 3 · 6 marks

[6]A veterinary clinic wants to classify new patient records as one of three risk categories using measurements already recorded for thousands of past patients. Compare K-NN and decision trees as options for this task, referring to how each one would use the existing labelled records to classify a new patient.

Flashcards

Active recall for this statement’s key terms.