IGCSE Computer Science 0478: Machine Learning vs a Fixed Rule (Apple-Sorting Case Study)

FreshPick Orchards has two sorting machines on its packing line, each fitted with a camera that photographs every apple as it passes.

Machine 1 is new. Before it was switched on, an employee spent several weeks looking at thousands of stored apple photographs and marking each one "Grade A" (sold in shops) or "Reject" (sent for juicing) by hand. Machine 1's software studied this large collection of already-marked photographs, and every week the employee adds that week's newly marked photographs to the collection it studies. Over several months, the proportion of apples Machine 1 sorts correctly has been getting steadily higher.

Machine 2 is older. Its software was given a single rule by a programmer when it was installed: measure the percentage of each apple's skin covered by brown marks in the photograph, and reject the apple if this is above 5%. Machine 2's accuracy at sorting apples correctly has stayed exactly the same since the day it was installed, no matter how many more apples have passed under its camera.

(a) Explain why the employee needed to mark thousands of stored apple photographs as "Grade A" or "Reject" before Machine 1's software could begin sorting apples using machine learning. [2]

(b) Explain why Machine 1's accuracy is expected to keep improving as more of each week's newly marked photographs are added to the collection it studies, whereas Machine 2's accuracy stays exactly the same however many more apples pass under its camera. [3]

(c) State what would specifically have to happen to Machine 2's software if FreshPick wanted it to become more accurate. [1]

(d) Give one disadvantage to FreshPick of using machine learning, rather than a fixed pre-programmed rule, to sort the apples. [1]

Part (a): Why the photographs had to be marked first

Machine-learning software does not start with any built-in knowledge of what a “Grade A” or “Reject” apple looks like. Instead, it learns by studying a large collection of examples where the correct answer is already known. This is exactly what marking (labelling) each stored photograph “Grade A” or “Reject” provides.

By studying thousands of these already-correctly-marked photographs, Machine 1’s software could start finding its own patterns connecting features of a photograph (such as colour and blemishes) to the correct grade, and use these patterns to make its own automatic sorting decisions. Without any already-marked examples, the software would have nothing to learn from and no way of knowing which grade was correct. [2 marks]: [1] for identifying that machine-learning software needs examples where the correct answer is already known, [1] for explaining that marking the photographs is what supplied these known-correct examples for the software to learn from.

Part (b): Why Machine 1 keeps improving but Machine 2 does not

Machine 1 (machine learning): its software automatically adapts its own process as it is given more data. Every week’s newly marked photographs are added to the collection it studies, giving it more examples, including rarer or more unusual apples, from which to refine the patterns it uses to decide a grade. Because the software itself changes as more labelled data becomes available, its sorting tends to become more accurate over time, without a programmer rewriting any code.

Machine 2 (fixed pre-programmed rule): its software carries out exactly the same calculation on every photograph, measuring the percentage of brown marks and comparing it with the fixed 5% threshold. This rule was written once and does not store, study or adapt to past decisions in any way, so processing more apples changes nothing about how it calculates its answer. Its accuracy is fixed at whatever the original rule achieves, whether it has sorted a hundred apples or a hundred thousand.

[3 marks]: [1] for explaining that Machine 1’s software adapts its own process using the growing labelled data, [1] for explaining that more labelled examples let it refine its patterns and typically become more accurate, [1] for contrasting this with Machine 2’s rule always performing the identical fixed calculation regardless of how much data it processes.

Part (c): What Machine 2 would need for its accuracy to improve

Because Machine 2 uses a fixed pre-programmed rule, it cannot improve itself. A programmer would have to open the program and manually rewrite or edit the rule (for example, changing the 5% brown-mark threshold or adding further conditions) before its accuracy could change. [1 mark]

Part (d): A disadvantage of using machine learning here

Training Machine 1’s software required an employee to correctly mark thousands of stored photographs by hand, a process that took several weeks and is far more time-consuming and costly than Machine 2’s rule, which only had to be written once by a programmer. A related risk is that if any of the marked photographs were labelled incorrectly, Machine 1’s software could learn an inaccurate pattern from them and go on to sort future apples wrongly, without this being obvious until a person checks its results. [1 mark] for any one suitable disadvantage.

Final answers

(a) Marking the photographs gave the software examples with an already-known correct answer, so it had something to learn patterns from.
(b) Machine 1’s software adapts itself using the growing collection of labelled data, so it tends to improve; Machine 2’s fixed rule performs the same calculation every time and never changes by itself.
(c) A programmer must manually rewrite or edit Machine 2’s rule; it cannot improve on its own.
(d) Any suitable disadvantage, e.g. the time and cost of labelling thousands of training photographs, or the risk of learning an inaccurate pattern from mislabelled photographs.