Most organisations deploying computer vision in 2026 are running their inference in entirely the wrong place — and paying for it in ways that don't show up on a single line of any budget spreadsheet. The real cost is split across three ledgers simultaneously: bandwidth bills that quietly compound, latency that makes real-time use cases physically impossible at cloud distance, and privacy exposure that creates regulatory liability. The edge-vs-cloud question is not a preference; it's an engineering constraint that should be derived from your workload's actual requirements before you architect anything.
The Core Tradeoff Map
Edge AI means running inference on hardware physically co-located with the data source — a factory floor camera, an in-store retail terminal, a drone, a vehicle. Cloud AI means shipping data to a managed GPU instance (AWS, GCP, Azure or a specialist provider) and receiving results back. The latency difference alone can be an order of magnitude, but that's only one dimension.
| Dimension | Edge Inference | Cloud Inference |
|---|---|---|
| Round-trip latency | 1–20ms (local) | 50–300ms (network dependent) |
| Bandwidth cost | Near zero (raw frames stay local) | High if streaming full video |
| Hardware capex | $150–$2,000+ per device (GPU/NPU) | Low to zero upfront |
| Compute opex | Fixed after device purchase | Variable; scales with call volume |
| Model update agility | Requires OTA deployment to fleet | Instant; update once centrally |
| Privacy / data residency | Raw data never leaves premises | Data leaves the building; GDPR risk |
| Offline resilience | Continues without internet | Fails completely if connection drops |
| Model complexity ceiling | Constrained by device memory/TDP | Effectively unlimited (scale up) |
When Edge Is the Only Real Answer
Certain computer vision workloads are structurally incompatible with cloud inference, regardless of how fast your connection is.
Real-time safety and control systems
Autonomous machinery, robotic assembly lines, collision avoidance in vehicles — any application where the model output triggers a physical action cannot tolerate network jitter. A 200ms round-trip through a cloud API might be undetectable in a web app. In a conveyor-belt defect detection system running at 1,200 items per minute, it means you've missed 4 units before the signal arrives back. These workloads belong on an NVIDIA Jetson, Intel Neural Compute Stick, or similar edge accelerator, full stop.
High-bandwidth video feeds at scale
A single 1080p camera at 30fps produces roughly 3–8 Mbps of compressed stream. Run 50 cameras at a manufacturing site and you're looking at 150–400 Mbps of continuous uplink just to move raw frames to the cloud. That's before you pay for ingestion, storage and GPU compute. Streaming compressed keyframes or running inference locally and sending only metadata to the cloud drops the required bandwidth by 95–99%.
Sensitive biometric or healthcare data
GDPR, HIPAA, and an increasing number of national biometric data laws make it legally precarious to route facial images, medical scans, or patient data through third-party cloud infrastructure without explicit consent frameworks and data processing agreements. Edge inference where biometric data is classified locally and only anonymised attributes leave the device can sidestep this entirely — a structurally cleaner solution than building a privacy compliance layer around a cloud pipeline.
When Cloud Inference Makes More Sense
Edge hardware constraints are real. A quantised MobileNetV3 runs comfortably on a Raspberry Pi 5. A Vision Transformer processing high-resolution pathology slides does not. There are workloads where cloud is simply the pragmatic choice.
Non-real-time batch processing
Medical image analysis (CT scans, MRIs, histopathology), satellite imagery analysis, large-scale document digitisation — these workloads are not time-critical to milliseconds. Running them on cloud GPUs during off-peak hours (spot instances reduce cost by 60–90% vs on-demand) makes economic sense. You get the compute you need, pay only for what you use, and avoid device procurement entirely.
Intermittent or geographically distributed workloads
If you have 10 retail locations each processing 2 hours of meaningful camera footage per day, deploying a dedicated edge GPU to each is difficult to justify. Cloud inference lets you share a single compute pool across all locations, paying for the aggregate 20 hours per day rather than provisioning for each site's peak.
Rapid model iteration during development
During the research and experimentation phase, you want to try large models, iterate quickly, and avoid managing hardware. Cloud — specifically managed notebooks and GPU instances — accelerates this. Even if your final deployment target is edge, develop and validate in the cloud first.
The Hybrid Architecture: Where Most Real Deployments Land
The cleanest architecture for most production computer vision systems is a split between edge and cloud, with each layer doing what it's actually suited for.
A common pattern: a lightweight detection model runs on-device (object detection, anomaly flagging, face detection without recognition). When the local model flags something worth investigating, it sends a cropped, anonymised region or a compressed keyframe to the cloud for further analysis — a deeper classification model, anomaly severity scoring, or data aggregation across sites. Results come back asynchronously.
This approach keeps bandwidth and latency manageable, limits the raw sensitive data leaving the building, and reserves cloud compute for genuinely complex inference tasks. The on-device model acts as an intelligent pre-filter, not just a relay.
Choosing Your Edge Hardware
The edge hardware landscape has matured significantly. A practical comparison for the most common deployment tiers:
- NVIDIA Jetson Orin Nano (8GB): ~$500, up to 40 TOPS, suits real-time object detection, pose estimation, lightweight segmentation. The pragmatic choice for most industrial deployments.
- NVIDIA Jetson AGX Orin: ~$800–$2,000, up to 275 TOPS, handles multi-stream inference, transformer-based models. For high-throughput or multi-camera installations.
- Google Coral Dev Board / Edge TPU: ~$100–$150, optimised for TensorFlow Lite models quantised to INT8. Very low power, suited for battery-powered or embedded deployments.
- Raspberry Pi 5 + Hailo-8 AI HAT: ~$150–$200 combined, 26 TOPS via the Hailo accelerator. A cost-effective option for lower-throughput use cases.
- Intel OpenVINO on Core Ultra / Arc: Suited for edge deployments that use standard x86 hardware already present (retail terminals, industrial PCs).
Model format matters as much as hardware. Edge deployment typically requires model optimisation: quantisation (INT8 reduces model size and increases throughput with minimal accuracy loss), pruning, and format conversion to TensorRT (NVIDIA), TFLite, or OpenVINO IR. Expect to spend meaningful engineering time on this optimisation step — it's rarely plug-and-play.
Practical Cost Comparison
For a hypothetical 20-camera retail analytics deployment running 12 hours per day:
- Pure cloud: Streaming compressed video at ~2 Mbps per camera = ~40 Mbps upload, roughly 2.1TB/day. At typical egress and compute pricing, expect $800–$2,000/month depending on region and inference frequency.
- Pure edge (Jetson Orin Nano per 4 cameras): 5 devices at ~$500 each = $2,500 one-time capex. Monthly opex near zero beyond power (~$10–20). Break-even vs cloud in roughly 3–4 months.
- Hybrid (edge pre-filter, cloud for flagged events): 5 edge devices ($2,500 capex) + cloud inference on ~5% of frames = $50–$150/month. Best long-term economics for most mixed workloads.
Frequently Asked Questions
Can I use 5G to make cloud inference fast enough for real-time vision?
5G reduces latency to roughly 10–30ms in ideal conditions, which helps — but adds a carrier dependency, coverage limitations, and ongoing data costs for streaming video. For most real-time control applications, even 20ms of additional latency is problematic. 5G makes cloud inference viable for a wider range of use cases, but doesn't eliminate the edge advantage for safety-critical applications.
How do I keep edge models updated when I retrain in the cloud?
Over-the-air (OTA) model deployment is the standard approach. Tools like Azure IoT Hub, AWS IoT Greengrass, or open-source alternatives like Eclipse Hawkbit let you push model updates to a fleet of devices with version control and rollback capability. Always validate new model versions on a staged subset of devices before fleet-wide rollout.
Does edge AI require specialised expertise I won't find easily?
Edge AI combines three skills that don't always sit with the same person: embedded systems engineering, ML model optimisation, and MLOps for fleet management. Most teams either hire a specialist or work with an offshore development partner that has this combination. The optimisation step — quantising and converting a PyTorch model to TensorRT — typically takes 1–3 weeks for an experienced engineer, not months.
What's the right approach for a company building its first computer vision product?
Start in the cloud. Validate that your model solves the problem before optimising for deployment. Use a GPU instance to iterate on model architecture. Once accuracy meets your target, profile inference speed and decide which components need to move to edge. This prototype-in-cloud, deploy-to-edge pattern consistently outperforms trying to build edge-first from day one.
Mexilet Technologies supports teams on exactly this kind of work through our computer vision services and AI solutions.
Mexilet Technologies works as an offshore computer vision development partner for product companies and enterprises across the UK, USA, UAE, and Australia — covering everything from model training through edge deployment and fleet management. If you're deciding where to run your computer vision workloads and want a technical opinion grounded in real deployment experience, start a conversation with our team.