Object detection is the computer vision task of locating and classifying multiple objects within an image — drawing a bounding box around each one and labeling it. You've seen it in action on self-driving car dashboards and in airport security scanners, but today it's equally relevant for warehouse inventory counting, retail shelf analysis, agricultural yield estimation, and manufacturing defect detection. What's changed in the last few years is that building a custom object detection model for your specific use case no longer requires a large research team or millions of images. A focused dataset, a solid pretrained backbone, and a disciplined training process can get you to production-quality results in weeks, not months. Here's the full pipeline.
Defining the Problem Before You Touch Any Code
The most common mistake I see in detection projects is jumping to training before the problem is fully specified. Spend time on these questions first — they'll save you from expensive U-turns later:
- How many distinct object classes do you need to detect?
- What is the minimum acceptable size (in pixels) of the objects you need to detect?
- How many objects per image, on average and at maximum?
- What inference speed do you need? Real-time (30+ FPS) or batch-overnight?
- Where does inference run — a cloud API, a workstation, or an embedded edge device?
- What counts as "good enough" accuracy for this use case — and what's the cost of a false positive vs. a missed detection?
That last question matters more than people realize. For detecting overripe fruit on a sorting line, missing a bad piece (false negative) is worse than falsely flagging a good one. For a security system, the calculus reverses. Your precision-recall tradeoff should be set by the business consequence, not by what the model maximizes by default.
Dataset Collection: Volume, Variety, and Representativeness
There are no reliable universal minimums, because the required dataset size depends on class diversity, visual complexity, and how much the pretrained backbone already understands about your domain. A working rule of thumb for fine-tuning: 200–500 labeled images per class for simple, visually distinct objects in controlled environments; 800–2,000+ for complex classes with cluttered backgrounds or high visual variability. For small object detection or high-stakes accuracy requirements, more data almost always helps.
Data quality matters as much as volume. Images should cover the full range of conditions you'll encounter in production: different lighting, angles, distances, and occlusion scenarios. If all your training images are clean, well-lit photos with simple backgrounds, the model will surprise you badly in your actual environment. Build augmentation (random crops, flips, brightness/contrast shifts, Mosaic for small objects) into your training pipeline from the start using Albumentations — not as an afterthought.
Data Labeling: Annotation Tools and Quality Control
Bounding-box annotation records the class label and box coordinates (center x, center y, width, height as fractions of image dimensions in YOLO format; absolute pixels in COCO format). Main tooling options: Label Studio (open-source, self-hostable, good for privacy requirements), Roboflow (cloud-based, excellent UX, includes versioning and augmentation pipelines), CVAT (open-source, strong for video annotation), and Scale AI / Labelbox for managed labeling at volume.
Annotation quality control matters more than people expect. Establish clear labeling guidelines — with visual examples — before annotation starts. Define whether to include partially occluded objects and what minimum visibility threshold to annotate. Run an inter-annotator agreement check on 50–100 shared images. Inconsistent labels teach the model contradictions, and inconsistency hurts accuracy more than modest dataset size does.
Choosing a Model Architecture
For most custom object detection projects today, starting from the YOLO family makes sense: fast, well-documented, easy to fine-tune, and with strong community support. As of 2025–2026, YOLOv8 and YOLOv11 from Ultralytics are the practical go-to choices for new projects. RT-DETR is worth considering if you need transformer-based detection and have the compute budget.
| Architecture | Inference Speed | Accuracy (COCO mAP) | Edge-Friendly? | Best For |
|---|---|---|---|---|
| YOLOv8n / YOLOv8s | Very fast | Good | Yes | Real-time, resource-constrained edge |
| YOLOv8m / YOLOv8l | Fast | Very good | Partially | Balanced accuracy/speed on GPU |
| YOLOv8x / YOLOv11x | Moderate | Excellent | No | Maximum accuracy, server inference |
| RT-DETR | Moderate | Excellent | No | Complex scenes, research-grade accuracy |
| EfficientDet | Fast | Very good | Yes (D0-D2) | Mobile deployment, medical imaging |
Start with the nano or small variant and scale up only if accuracy is inadequate. Smaller models are faster to train, cheaper to run, and often good enough for single-domain tasks where the pretrained backbone already understands the visual domain well.
Training: A Practical Checklist
Before your first training run, have these in place: an 80/10/10 train/validation/test split (the test set stays completely held out, including from hyperparameter decisions); a baseline run at default settings to confirm your data pipeline is correct; experiment tracking via MLflow or Weights & Biases from run one; checkpoint saving by best validation mAP (not last epoch); and early stopping with 20–30 epoch patience to avoid wasting compute on plateaued runs.
After the baseline, the most impactful hyperparameters to tune are learning rate, input image resolution (larger inputs find small objects better but increase memory and training time), and augmentation strength (too aggressive augmentation can hurt on visually simple domains). Run experiments systematically, one change at a time.
Evaluation: Beyond the mAP Number
Mean Average Precision (mAP) is the standard metric, but don't let a single number signal readiness. Before calling training complete: review the precision-recall curve per class (rare classes are often outliers); visually inspect 50–100 predictions on the held-out test set for missed boxes, wrong class labels, and overlapping duplicates from loose NMS thresholds; test on deliberately challenging images (worst lighting, most cluttered backgrounds, smallest objects); and measure inference latency on the actual deployment hardware — a model at 45ms on an A100 may run at 280ms on a Jetson Nano.
Deployment: Cloud, On-Premise, and Edge
For server deployment, ONNX export plus ONNX Runtime is the most portable path — it decouples the model from the training framework and runs efficiently across CPU and GPU targets. For edge deployment on NVIDIA Jetson, Coral Dev Board, or Raspberry Pi 5, model quantization is usually necessary: INT8 quantization reduces model size and speeds inference at minimal accuracy cost when calibrated with representative data. Standard export targets: ONNX → TensorRT for Jetson, TFLite for Android/Coral, CoreML for Apple Silicon.
Wrap the model behind a FastAPI REST endpoint or gRPC service for higher throughput. Add a health endpoint, structured logging with inference timestamps and confidence scores, and Prometheus metrics from day one — these are far easier to build in upfront than to retrofit during a production incident. Teams at Mexilet Technologies typically containerize detection services and manage edge fleets with a lightweight OTA update system so model improvements reach distributed edge devices without requiring physical access — critical for multi-site manufacturing or retail installations.
Monitoring and Model Refresh
A deployed model that isn't monitored will degrade silently. Log a random sample of production images and their predictions, and review weekly for the first month. Watch for confidence score distributions shifting downward — a model whose average confidence is dropping is encountering something it wasn't trained on. Schedule model refresh when new class variations appear, the production environment changes, or accuracy on your monitored sample drops below your defined threshold. Treat the retraining pipeline like application code: version it, maintain it, and improve it over time.
Frequently Asked Questions
How long does it take to build a custom object detection model from scratch?
For a focused single-domain project with existing image data, data labeling typically takes 2–4 weeks, model training and iteration another 2–4 weeks, and deployment setup 1–2 weeks. A realistic end-to-end timeline from kickoff to production deployment is 6–10 weeks for a straightforward use case, 3–5 months for complex multi-class systems with edge deployment and system integration requirements.
How much labeled data do I actually need to get started?
You can start an initial experiment with as few as 100–200 annotated images per class if your classes are visually distinct and you're fine-tuning a pretrained model. This won't give you production-ready accuracy, but it will tell you whether the problem is tractable, which classes are harder to learn, and where more data will have the highest impact. Use this initial experiment to prioritize your labeling budget intelligently rather than labeling everything upfront.
What's the difference between object detection and image classification?
Image classification assigns a single label to the whole image ("this image contains a cat"). Object detection finds every instance of each class within the image and draws a bounding box around each one ("there's a cat in the top-left and a dog in the bottom-right"). If you need to count objects, locate them, or handle multiple objects per image, you need detection. If you only need to categorize an entire image and there's one main subject, classification is simpler, faster, and requires less training data.
Can I use a pre-trained general model like CLIP or SAM instead of training a custom detector?
Foundation models like CLIP (zero-shot classification) and SAM (segment anything) are genuinely impressive for general cases. For specialized domains — micro-defects in PCBs, specific product SKUs, fine-grained agricultural conditions — their zero-shot performance usually doesn't meet production accuracy requirements. They can, however, meaningfully accelerate your data preparation: SAM can generate initial segmentation masks that annotators refine rather than draw from scratch, cutting labeling time substantially. Using foundation models to assist data preparation and a fine-tuned detector for inference is often the best of both worlds.
Need a partner for this? Mexilet offers computer vision services and AI solutions.
If you're at the planning stage or have a dataset and aren't sure where to start, the Mexilet Technologies computer vision team works with clients across manufacturing, logistics, agriculture, and retail. Get in touch with us — we're happy to review your use case, give you an honest assessment of what's achievable, and map out a development approach that fits your timeline and budget.