You have a model that works in the notebook. Now what? This is the question most ML startup teams hit around week six or eight of a project — after the proof-of-concept has impressed the investors or the product manager, and before anyone has seriously thought about how this thing runs reliably in production at 2 AM when the engineer who built it is asleep. MLOps for startups is not about building Google-scale infrastructure. It is about answering that question with the minimum viable set of tooling and process that will not wake you up at night.
The Gap Nobody Prepares You For
A Jupyter notebook that produces accurate predictions is not a product. It is a research artifact. The gap between that artifact and a production ML system involves: packaging the model so it can be called from your application, versioning it so you can roll back, logging predictions so you can audit behavior, monitoring inputs and outputs so you notice when the world changes, and retraining the model so it does not quietly degrade over time.
Established MLOps platforms try to address all of this at once, which is why they feel overwhelming for a team of two or three engineers who also need to ship features. The approach below is deliberately lean — it adds infrastructure only when there is a clear operational need.
Step One: Get the Model Out of the Notebook
The first thing to do is separate the training code from the inference code. These have very different operational requirements. Training runs infrequently (or on a schedule) and needs access to raw data and substantial compute. Inference needs to be fast, stateless, and continuously available.
Package your model as a REST API using FastAPI or Flask. Containerize it with Docker. This is not exciting advice, but it is the correct foundation — a containerized inference service can be deployed to almost any infrastructure, scaled independently of your main application, and replaced or rolled back without touching anything else in your stack. Keep the API contract simple: an input schema, an output schema, and a health check endpoint. Add versioning to the API path from day one (/v1/predict) so you can introduce a new model version without breaking the existing integration.
Step Two: Version Everything, Including the Data
When a model behaves unexpectedly in production, the first question is: which model, trained on which data? If you cannot answer that question precisely, debugging becomes archaeology. MLOps for startups does not require a fully-fledged feature store. It requires discipline about three things:
- Model artifacts: Tag every trained model with the training run ID, the dataset version it was trained on, the evaluation metrics at the time of promotion, and the git commit of the training code. MLflow and Weights and Biases both offer free tiers that handle this with minimal setup.
- Training datasets: Store each training dataset as an immutable snapshot with a timestamp and a hash. DVC (Data Version Control) works alongside git and adds minimal overhead.
- Configuration: Hyperparameters, preprocessing steps, and feature engineering decisions should be versioned alongside the model. A model trained with a different tokenizer or a different image normalization scheme is a different model, even if the architecture is identical.
Step Three: Build a Prediction Log Before You Need It
Log every prediction your model makes in production — the input features (or a hash/sample if privacy requires it), the model's output, the model version that produced it, and a timestamp. This log is the raw material for everything else in your MLOps stack: detecting drift, building evaluation datasets, debugging complaints, and constructing the training data for the next model version.
A simple database table or an append-only log file in object storage is sufficient to start. The important thing is that this logging exists from the first day the model is in production, not added later when you are already trying to diagnose a problem. Adding it retroactively always happens under pressure, which is the worst time to instrument observability.
Step Four: Monitor the Things That Actually Break
Model monitoring has a reputation for complexity that it does not entirely deserve. For a startup with one or two models in production, the monitoring that matters most is:
| Signal | What It Tells You | Simplest Implementation |
|---|---|---|
| Input feature distribution | Has the data coming into the model changed? | Compare mean/stddev of numeric features weekly against a baseline window |
| Prediction distribution | Is the model producing different outputs than usual? | Track class probability histograms; alert if class distribution shifts >15% |
| Inference latency | Is the model still responding within SLA? | P50/P95/P99 latency tracked via Prometheus or a hosted APM |
| Error rate | Are inputs causing exceptions? | Standard application error monitoring (Sentry is fine) |
| Business outcome (if available) | Is model accuracy translating to business results? | Track downstream conversion, fraud rate, churn rate — whatever the model is supposed to move |
Sophisticated drift detection libraries exist (Evidently, NannyML, WhyLabs), and they are worth adopting when your monitoring needs grow. At the start, simple statistical checks run as a daily cron job and surfaced in Slack will catch 80% of real problems at a fraction of the complexity.
Step Five: Define Your Retraining Trigger Before You Need to Retrain
Retraining a model because "it feels like it might be degrading" is not a process — it is a guess. Before deploying, agree on what will trigger retraining:
- Scheduled retraining on a fixed cadence (weekly, monthly) regardless of observed drift — simple and predictable
- Drift-triggered retraining when statistical tests detect significant distribution shift — more responsive, but requires monitoring to be live first
- Performance-triggered retraining when a labeled sample shows accuracy below a threshold — highest signal, highest operational cost
Pick one and implement it. The choice matters less than having a defined trigger at all. Undocumented retraining cadences produce models retrained reactively after customer complaints — the MLOps equivalent of changing the oil after the engine seizes.
What a Lean MLOps Stack Actually Looks Like
For a startup shipping its first model to production, a pragmatic stack might look like this:
- Model registry: MLflow (self-hosted on a $20/month VPS) or Weights and Biases free tier
- Data versioning: DVC with an S3 or GCS remote
- Inference serving: FastAPI inside Docker, deployed on Railway, Render, or a small Kubernetes cluster
- Prediction logging: PostgreSQL table or Parquet files in object storage
- Monitoring: A scheduled Python script that computes drift statistics and posts to a Slack webhook, plus Sentry for error tracking
- CI/CD for models: GitHub Actions pipeline that runs evaluation on a held-out test set before promoting a new model version to production
Total additional infrastructure cost: roughly $50–$150 per month. Total engineering time to set up: two to four days for a small team that has done it before. The ROI on that investment becomes clear the first time a data pipeline upstream of your model changes silently and you catch it via monitoring rather than via a customer complaint.
When to Bring In Outside Help
The tooling decisions above are not deeply controversial. Where startups typically need external expertise is in designing the training pipeline, debugging production behavior that does not match offline evaluation, and managing ML infrastructure overhead when the core team needs to focus on product. Mexilet Technologies works as a backend development partner for startups in exactly this position — building and maintaining ML infrastructure so the product team can stay focused on customer problems.
Frequently Asked Questions
Do I need a dedicated MLOps engineer, or can a senior ML engineer handle this?
At the scale of one or two models in production, a senior ML engineer who understands software engineering fundamentals can handle MLOps setup. The workflow described above does not require MLOps specialization — it requires engineering discipline. Dedicated MLOps roles become justified when you have many models in production, multiple teams contributing models, or when the retraining and evaluation pipelines are complex enough to be a full-time engineering concern. Most startups reach that point at three to five models in production with active retraining loops.
Should I use a managed ML platform like SageMaker, Vertex AI, or Azure ML?
Managed platforms reduce infrastructure management overhead at the cost of vendor lock-in and higher costs. For a startup, they make most sense when your team lacks infrastructure engineering capacity and the higher cost is acceptable, or when you are already deeply committed to one cloud provider's ecosystem. If you have engineering bandwidth, starting with the lean self-hosted approach described above gives you more flexibility and lower costs, with the option to migrate to a managed platform later when your requirements are better defined.
How do I handle model rollback if a new version performs worse in production?
Keep your model registry and deployment configuration version-controlled. Your inference service should be deployable via a configuration change (a different container image tag or a different model artifact path) that can be reverted in minutes. Blue-green deployment — running the old and new model simultaneously and gradually shifting traffic — lets you catch performance regressions with limited blast radius before fully cutting over. The discipline here is never deploying a new model version without a documented rollback path.
How long does it typically take to go from a notebook prototype to a production ML system?
With a clear scope and experienced engineers, the initial production deployment typically takes two to six weeks from a working prototype. The wide range reflects variation in data pipeline complexity, integration requirements, and the team's familiarity with the tooling. An initial deployment is not the same as a mature system — the monitoring, retraining pipeline, and operational runbooks typically evolve over the first two to three months of production operation as real-world edge cases surface.
If you'd rather not build it alone, see our computer vision services and AI solutions.
If you are trying to figure out what a production-ready ML system looks like for your specific product, schedule a free technical scoping call with Mexilet Technologies. We will walk through your current setup, identify the gaps, and give you a concrete roadmap — no commitment required.