Leukemia Cell Subtyping from Blood Smears

A computer-vision classifier that sorts a single white blood cell into a benign look-alike or one of three stages of B-cell acute lymphoblastic leukemia — and a case study in why a 99.8% test score can be the least trustworthy figure in this portfolio.

Try it in your browser →
Classifies a real cell as leukemic or normal on your device — nothing is uploaded.

Summary

The four classes

Three of the classes — Early Pre-B, Pre-B, and Pro-B — are maturation stages of the same B-cell acute lymphoblastic leukemia, while the fourth, Benign, is a non-cancerous look-alike (hematogones) that can deceive the eye. Distinguishing them is genuinely subtle work, which is what makes it a substantive computer-vision problem.

Training progression

The model learns the four cell types rapidly and cleanly, and validation accuracy reaches 100%. On most projects this would be the headline result. Here it functions as the opposite — a signal to examine the data split more critically.

Why 99.8% is a warning rather than a result

The images derive from a real set of patients, but this public release does not include patient identifiers. Consequently, a random train/test split almost certainly places cells from the same patient on both sides of the partition. Cells from a single slide share staining, lighting, focus, and background, and a high-capacity model can score near-perfectly by recognizing those cues — having learned considerably less about leukemia than the metric implies.

This is the same failure mode the EEG project avoids through subject-level splitting. It cannot be avoided here because the identifiers are simply absent from the data, and reporting the figure without this caveat would be misleading. Accordingly, the 99.8% is presented with a caveat commensurate with the result itself.

v2 — the honest patient-level result

This iteration moves to the C-NMC 2019 dataset, which encodes a patient ID in every filename. That makes it possible to hold out entire patients — the same protocol as the EEG project — so the score measures what the model learned about leukemia, not which patient a cell came from. C-NMC labels cells as leukemic or normal, so this is a binary task: a different, harder problem than v1's four-class subtyping, not a like-for-like rematch.

The data is split across 73 patients with no patient on both sides, and the test set is held out as 11 patients the model never saw (1,014 cells). On that test set:

• 80.8% accuracy and ROC-AUC 0.857 per image — a large, honest drop from v1's leakage-inflated 99.8%.
• 93.6% sensitivity: the model rarely misses a malignant cell.
• 53.4% specificity: but it over-calls malignant, flagging many normal cells — an honest weakness, driven by a training set with far more leukemic patients than normal ones.

The fall from 99.8% to about 81% is the whole point: the near-perfect v1 figure was largely leakage, and this is what the model can actually do on people it has never seen. The test set is small — only 11 patients, and just four of them normal — so the per-patient view is noisy (it flagged all seven leukemia patients but misread three of the four normal ones), which is why the per-image figures are the more stable read. A lower, honestly-measured number is worth far more than an inflated one.

v3 — does it survive a different microscope?

v2 answered "does it work on unseen patients?" (81%). The harder, real-world question — and the single biggest failure mode of medical imaging AI — is whether it works on a different lab's microscope: another hospital's staining, scanner and lighting. So I ran the unchanged v2 model on two independent datasets it had never seen — Aria (Tehran) and Acevedo (Barcelona, 17,092 images of normal cells).

It failed — badly, and honestly. On Barcelona's healthy cells the v2 model flagged 80% of them as leukemic (89% of the lymphocytes — the cell a lymphoblast most resembles). On Tehran's images it was near-random. And a telling detail: on raw images it scored worse than a coin flip (ROC-AUC 0.40), but on the same cells re-segmented onto a black background — matching C-NMC's style — it recovered toward 0.51. That gap is the smoking gun: even the honest 81% model had learned a lot about C-NMC's particular background and staining, not just leukemia. It's the v1 leakage worry, one level deeper.

The fix: train on more than one lab

Augmentation can't rescue this — C-NMC contains only black-background cells, so the model has nothing to learn real backgrounds from. The real fix is more than one source. v3 trains on two labs at once — C-NMC (Delhi) plus Aria (Tehran), including both the real-background and segmented versions — so it sees two staining pipelines and both background types. Acevedo (Barcelona) is held out entirely as an unseen third lab, to test whether this genuinely generalizes.

Training on a second lab more than tripled specificity on a completely unseen third lab (20% → 67%): the false-alarm rate on healthy Barcelona cells fell from 80% to 33%, and on the hard-negative lymphocytes from 89% to 36% — while also nudging within-source accuracy up. That's the whole project's lesson in one number: a single-source model, even one evaluated correctly, is still a single-source model. Robustness comes from diversity in the training data, not only discipline in the split.

Honest limits: 67% specificity still means a third of normal cells get flagged — improved, not solved. Aria is now a training source, so its near-perfect v3 numbers are in-domain, not a generalization claim; the honest proof is Barcelona, which the model never saw. And a truly held-out external blast source would need a fourth dataset. But the direction is unambiguous — and it's the right one.