CV Inventory Tracking

Overview

CV Inventory Tracking is a computer vision-based inventory management system designed for 3D printing facilities. It uses YOLOv8 object detection to monitor filament spools and printer availability across physical zones, treating CV detections as signals rather than direct state commits. A human confirmation workflow handles uncertain detections, and QR codes provide a reliable ground-truth anchor for inventory state.

The architecture is event-driven: the vision module fires events, a reconciliation engine resolves them against current inventory state, and a confirmation manager escalates ambiguous cases to a human operator before any state change is committed.

Tech Stack

• FastAPI — Async REST API backend with Pydantic v2 for request/response validation.
• Uvicorn — ASGI server for production FastAPI deployment.
• YOLOv8 (Ultralytics) — Object detection model for identifying filament spools and printer states.
• PyTorch — Deep learning runtime backing the YOLO inference pipeline.
• OpenCV — Image preprocessing, frame capture, and annotation.
• scikit-learn — Data preprocessing utilities for the YOLO training pipeline.
• RabbitMQ (pika) — Message queue for decoupling the CV detection pipeline from the inventory API.
• qrcode / pyzbar — QR code generation for spool labelling and live decoding for inventory confirmation.
• Pillow — Image manipulation for QR overlay and annotation export.
• PyYAML — Dataset configuration for YOLOv8 training.
• labelImg — Manual annotation tool used during dataset construction.

Features

• Zone-based tracking — Inventory state is maintained per physical zone; object transfers between zones are detected as move events.
• Human confirmation workflow — Detections below a confidence threshold are flagged and held pending operator confirmation rather than being committed automatically.
• Event reconciliation engine — Resolves concurrent or conflicting detection events against the current state before applying changes.
• QR code integration — Each spool can be labelled with a generated QR code; live decoding provides a high-confidence confirmation signal independent of YOLO.
• Gap detection — The system identifies zone-switching gaps where an object leaves one zone without immediately appearing in another, triggering an alert.
• YOLO training pipeline — End-to-end pipeline from Roboflow annotation export through scikit-learn preprocessing to YOLOv8 fine-tuning on custom spool and printer datasets.

Key Challenges

CV as signal, not source of truth: A naive implementation would let YOLO detections directly update inventory state, causing flicker and false transitions. Instead, detections are published as events to RabbitMQ, and the reconciliation engine requires multi-frame consistency before committing a state change.

Uncertain detections: Low-confidence detections are routed to a confirmation manager that holds the proposed state change and exposes it via an API endpoint for human review. The operator can confirm or reject without needing access to the raw model output.

Zone-switching gap detection: Objects temporarily disappear from the camera frame during zone transfers. The system maintains a short-lived "in transit" state per object and only raises an alert if the object does not appear in the expected destination zone within a configurable time window.

Training data quality: Early models suffered from false positives caused by spool shadow and partial occlusion. The training pipeline was extended to include hard-negative mining and augmentation passes (brightness jitter, random crop) to improve robustness under facility lighting conditions.

Outcomes

The phased build approach delivered a working inventory truth model first, validated human workflows before adding CV automation, and produced a trained YOLO model capable of reliable spool detection under controlled facility lighting. QR confirmation provides a fallback path when optical detection confidence is low.