Cockroach or Date: What an Image Classifier Actually Sees

A machine-learning classifier can tell a cockroach from a date — a problem that sounds like a joke until you realize the two genuinely look alike when cropped. The part I kept getting wrong: the model never learns what a cockroach or a date is. It learns a boundary between patterns of pixels, and everything confusing about these systems falls out of that one distinction.

The way of thinking I settled on: an image classifier is a function that maps a grid of numbers to a small set of scores, and "learning" means carving a decision surface through the space those numbers live in. The model has no idea one side is an insect and the other is a fruit.

Why this pairing works as a lesson

A cockroach and a date share a lot at first glance: both are small, dark, and roughly oval, with a segmented or wrinkled surface. Cropped tight and lit badly, a person can hesitate. That's precisely why the pairing is a good toy problem — it forces the question of what the model is even looking at. If the classifier can separate them, it isn't because one is obviously an insect; it's because the two have measurably different pixel patterns, and the model found them.

From pixels to a score

The input to any image classifier is a fixed-size grid of numbers — for a color image, three values (red, green, blue) per pixel [1]. The output is one score per class, often run through a softmax (a normalizing step that rescales the scores) so they sum to 1 and read like probabilities. In between is a stack of functions with millions of adjustable weights.

Training is the part that matters conceptually: you show the network many labeled examples and nudge the weights so correct labels get higher scores [2]. The weights are just numbers; there is no slot where "cockroachness" gets stored. What gets stored is a shape in feature space.

The boundary is the whole model

The mental image I keep returning to is the decision boundary. Take the network's second-to-last layer — a vector of, say, 1,024 numbers for each image — and picture each image as a point in that space. Images of dates cluster together, images of cockroaches cluster together, and training carves the surface between them. "Classifying" a new image is just asking which side of the surface is this point on? [3]

Two consequences follow, and they explain most of the weirdness:

• Confidence is distance, not certainty. A score of 0.92 means the point sits deep on the cockroach side — not that the model has 92% of a person's understanding. An image near the boundary gets an honest 0.51, which is the model confessing it can't tell.
• The model sees patterns, not objects. It never learns "six legs" or "a pit and a skin." It learns whatever features separate the training clusters — which can be the right thing or a shortcut.

What a CNN actually computes

A convolutional network works in stages, and each stage answers a different question about the input. Early layers respond to edges and blobs of color; middle layers combine those into textures and simple shapes; later layers assemble them into object-like parts [4]. For cockroach-versus-date, the useful signal is probably mid-level texture — the difference between a segmented exoskeleton and the wrinkled, fibrous skin of a date — not a high-level semantic like "legs."

This is why the pairing is such a good reminder: a human leans on high-level meaning, the model leans on low- and mid-level texture, and those two strategies can disagree.

Where it breaks — and why that's the lesson

The failure modes are the most instructive part, because they follow directly from the boundary model:

• Texture bias. Networks trained on natural images lean hard on texture over shape, which is why a date re-skinned with cockroach-like speckle can flip the label, and why a silhouette swap can break a model that looked solid [5].
• Out-of-distribution input. Show something neither cockroach nor date — a small dark pebble, say — and the model still has to pick the least-wrong class. It has no "I don't know" unless that was trained in explicitly.
• Adversarial perturbation. Nudge the pixels by an amount invisible to the eye and the point crosses the boundary. Not because the model is fragile in some deep way, but because the boundary is always somewhere, and small pixel changes can step across it [6].

Each of these reads as "the AI is dumb," but under the boundary model they're all the same sentence: the point moved to the other side of a surface.

How I use this

The practical payoff is calibration. When I see a classifier's confidence number, I read it as distance from a boundary, not certainty about the world — so I ask what the training clusters looked like and whether my input is even from that distribution. The cockroach-versus-date pairing is the example I reach for to explain all of it at once: two things that look alike, separated only by a surface in a space of numbers, with no understanding anywhere in the machine.

References

[1] S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed. Pearson, 2020.

[2] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016. [Online]. Available: https://www.deeplearningbook.org/

[3] C. M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.

[4] M. D. Zeiler and R. Fergus, "Visualizing and Understanding Convolutional Networks," in Proc. European Conference on Computer Vision (ECCV), 2014. [Online]. Available: https://arxiv.org/abs/1311.2901

[5] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, "ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness," in Proc. International Conference on Learning Representations (ICLR), 2019. [Online]. Available: https://arxiv.org/abs/1811.12231

[6] I. J. Goodfellow, J. Shlens, and C. Szegedy, "Explaining and Harnessing Adversarial Examples," in Proc. International Conference on Learning Representations (ICLR), 2015. [Online]. Available: https://arxiv.org/abs/1412.6572

Q: What does an image classifier actually store about its classes?
- A boundary between patterns of pixels
- A list of facts about each object
correct: 0
explain: The model learns a decision surface in feature space, not semantic knowledge about cockroaches or dates.

Q: In a classifier's output, a high score mostly reflects…
- distance from the decision boundary
- the model's genuine certainty about the world
correct: 0
explain: Confidence is distance in feature space. A point deep on one side of the boundary gets a high score.

Q: Which features matter most for separating a cockroach from a date?
- Mid-level texture differences
- High-level meaning like "it has legs"
correct: 0
explain: CNNs lean on texture and shape cues. The exoskeleton vs. fibrous skin is the useful mid-level signal.

Q: Why can invisible pixel changes flip a correct classification?
- The point steps across the decision boundary
- The model suddenly forgets its training data
correct: 0
explain: A tiny perturbation moves the point in feature space across the boundary; no forgetting is involved.