Visual Inventory Counter

The exploration

Manual stock counts are slow, dull, and error-prone — which makes “point your phone at the shelf” an obvious pitch and a suspicious one. This exploration asks the narrow technical question honestly: under real conditions, how accurate can on-device counting get, at what speed, for which shelf types — and where does it stop being worth using? Everything runs on the phone: no cloud round-trip per shelf, no store imagery leaving the device, latency measured in camera frames.

Reality: lighting, occlusion, depth

Demo shelves are front-faced and evenly lit. Real shelves have products two rows deep, glare off refrigerator doors, half-turned labels, and near-identical SKUs distinguishable only by text a camera catches at an angle. The test matrix is built from these failure modes on purpose — canned goods (easy), bottled drinks behind glass (glare), folded textiles (deformable), small cosmetics (dense and occluded) — with per-category accuracy tracked separately, because a blended average would flatter the system exactly where it fails.

The correction UX: the human count wins

The design rule is absolute — a person’s correction always overrides the model, with no confirmation friction and no argument. The screen shows counted items as overlay marks; tapping adds or removes a mark in place, and disagreement is captured as labeled training data rather than treated as user error. Under test: whether per-item marks or zone subtotals produce faster verification, and a budget hypothesis — correcting a 90%-accurate count must cost under a quarter of the time of counting by hand, or the tool loses to the clipboard on the honest total.

When a barcode beats the camera

Part of the deliverable is a decision map, not a product pitch. Deep-stocked shelves where the camera sees one row, mixed SKUs the model can’t distinguish, backroom boxes: scan, don’t count. The prototype should recognize its own weak conditions — low confidence, known-hard category — and say “scan this section instead” rather than produce a number that erodes trust in every other number. A counting tool is only useful if its user knows when to believe it.

Evaluation

• Per-category count accuracy against manual ground truth across three lighting conditions; publish the failures alongside the wins
• End-to-end time per shelf section including corrections, versus clipboard and barcode-scanner baselines
• Confidence calibration: when the system claims high confidence, is it right — and does the “scan instead” fallback trigger where it should?