MLOps is the set of practices and tools that take a machine learning model from experiment to reliable production, then keep it accurate, secure and auditable. For edge AI, the same lifecycle has to work across hundreds or thousands of devices with limited compute, patchy connectivity and physical exposure. In 2026 that means packaging models for on-device runtimes, rolling them out in stages, monitoring them where they run, and keeping records that regulators can check.
Updated October 2026
What changed in 2026
- AWS SageMaker Edge Manager is gone. It was discontinued on 26 April 2024; its APIs no longer work. AWS recommends ONNX Runtime for portable inference and AWS IoT Greengrass V2 for deployment and telemetry (AWS docs). Older guides that recommend it are out of date.
- Edge platforms consolidated. Azure IoT Operations, an edge data plane on Arc-enabled Kubernetes, has been generally available since November 2024 (Microsoft). Qualcomm acquired Edge Impulse in March 2025 and said it would keep supporting other vendors' hardware (SiliconANGLE).
- On-device runtimes matured. Meta and Arm made ExecuTorch 1.0 generally available in October 2025 for mobile, embedded and microcontroller targets (Embedded.com).
- Bigger models at the edge. NVIDIA announced Jetson T3000 (up to 865 FP4 TFLOPS) and T2000 in July 2026, with modules expected in Q1 2027 (eeNews Europe). Edge MLOps now has to handle small language and vision-language models, not only classifiers.
- LLMOps and AgentOps joined the stack. Teams now trace prompts, tool calls and agent steps alongside model metrics. The OpenTelemetry GenAI and agent conventions exist but are still in Development status (Greptime).
- EU AI Act dates moved. The Digital Omnibus moved high-risk obligations from 2 August 2026 to 2 December 2027 (Annex III uses) and 2 August 2028 (AI in regulated products such as machinery and vehicles) (Orrick). Transparency duties and penalty powers apply from 2 August 2026 (Compliance Hub).
What MLOps is, and how it differs from DevOps
DevOps ships code. MLOps ships code plus data plus a trained model, and the model can get worse without any code change because the world it sees changes (data drift). So MLOps adds three things DevOps does not need:
- Versioning of data and models, not just code, so any prediction can be traced back to the data and training run that produced it.
- Continuous training (CT): retraining when performance drops or new labelled data arrives.
- Model monitoring: tracking accuracy, drift and bias in production, not only uptime and latency.
The MLOps lifecycle, cloud vs. edge
| Stage | Cloud MLOps | Edge MLOps: what is different |
|---|---|---|
| Data | Central data lake | Data is created on devices; most is filtered or summarised locally; sampling for retraining must be planned |
| Training | Cloud GPUs | Still mostly cloud; federated or on-site fine-tuning for privacy or bandwidth reasons |
| Optimisation | Optional | Required: quantisation (INT8/FP4), pruning, compilation for the target chip |
| Packaging | Container image | Container for gateways; ONNX, LiteRT, ExecuTorch or TensorRT artefacts for devices and MCUs |
| Deployment | Blue/green on a cluster | Staged over-the-air rollout across a device fleet, with rollback if a device goes offline mid-update |
| Monitoring | Central logs and metrics | Devices report compact metrics; must buffer when offline; drift checked on summaries |
| Governance | Model registry, approvals | Plus a record of which model version runs on which device, where, since when |
Edge MLOps in practice: a rollout pattern
- Register the trained model with its data version, metrics and intended hardware.
- Optimise and compile for each target (for example Jetson, Qualcomm, Arm Cortex-M) and re-test accuracy after quantisation.
- Sign the artefact so devices only run approved models.
- Canary: push to 1–5% of devices or one site; compare against the previous model.
- Roll out in waves by site, region or hardware type; pause automatically on error spikes.
- Monitor latency, confidence, input drift and device health; sample hard cases back to the cloud.
- Retrain and repeat, keeping the previous version ready for one-click rollback.
Microcontroller-class devices (TinyML) follow the same pattern, with smaller artefacts, tighter memory budgets and less frequent telemetry.
Edge MLOps tools in 2026
| Tool | What it does | Notes |
|---|---|---|
| AWS IoT Greengrass V2 | Deploys components and models to edge devices; MQTT telemetry back to AWS | AWS's recommended path after Edge Manager |
| SageMaker Neo | Compiles models for specific edge chips | Still documented; check your target device is supported |
| Azure IoT Operations | Edge data plane on Arc-enabled Kubernetes: MQTT broker, connectors, data flows; runs offline up to 72 hours | Data platform rather than a model deployment tool; models run as Kubernetes workloads |
| Edge Impulse (Qualcomm) | Data, training and optimisation for MCUs, CPUs, GPUs and NPUs | Strong for sensor and TinyML use cases |
| ExecuTorch / LiteRT / ONNX Runtime | On-device inference runtimes | Choose by framework and target chip |
| MLflow, Kubeflow, cloud registries | Experiment tracking, pipelines, model registry | Cloud side of the loop |
MLOps vs ModelOps vs AIOps vs LLMOps
These terms overlap, and vendors use them loosely. A practical way to separate them:
| MLOps | ModelOps | AIOps | LLMOps / AgentOps | |
|---|---|---|---|---|
| What it manages | ML models through their lifecycle | All decision models (ML, rules, optimisation, LLMs) across the enterprise | IT operations, using AI | LLM apps and agents |
| Main users | ML engineers, data scientists | Risk, governance and business owners plus ML teams | IT ops and SRE teams | AI engineers, platform teams |
| Core tasks | Versioning, CI/CD/CT, deployment, monitoring | Inventory, approval, risk tiering, audit, retirement | Anomaly detection, event correlation, root-cause analysis | Prompt and model versioning, evaluation, tracing tool calls, cost and safety guardrails |
| Question it answers | "Is this model working in production?" | "Which models do we run, who approved them, and are they still fit?" | "Why is the system down?" | "Is this agent doing the right thing, safely, at what cost?" |
In short: MLOps is the engineering core; ModelOps is the governance layer above it; AIOps uses AI to run IT; LLMOps and AgentOps extend MLOps to language models and agents. At the edge you usually need MLOps plus a ModelOps-style inventory of what runs where.
The 7 layers of MLOps security, for edge deployments
| Layer | Main risk | Edge-specific control |
|---|---|---|
| 1. Data | Poisoned or leaked training data | Validate data provenance; strip personal data on device |
| 2. Data storage | Theft of datasets and features | Encrypt at rest; least-privilege access; local buffers encrypted |
| 3. Pipeline / orchestrator | Tampered training or build jobs | Locked-down CI/CD; signed pipeline steps; secrets in a vault |
| 4. Model | Model theft, backdoors, adversarial inputs | Sign and hash models; adversarial testing before release |
| 5. Model hosting | Compromised device or runtime | Secure boot, trusted execution where available, runtime attestation |
| 6. Exposure / APIs | Abuse of inference endpoints; prompt injection for LLMs | Authenticated APIs; rate limits; input filtering; tool permissions for agents |
| 7. Logging and monitoring | Attacks going unseen | Tamper-evident logs; alerts on drift and odd query patterns; offline buffering |
Physical access is the difference at the edge: an attacker can hold the device. Assume the model file can be copied, and protect what matters (keys, data, the update channel) accordingly.
What happened to SageMaker for edge MLOps
SageMaker remains a full cloud MLOps platform: Pipelines, Model Registry, Model Monitor, Experiments and Autopilot. What changed is the edge piece. SageMaker Edge Manager, which packaged models and managed device fleets, was discontinued on 26 April 2024. Leftover resources such as S3 packages and IoT things can still incur charges until you delete them (AWS docs).
A typical 2026 AWS pattern: train and register in SageMaker, compile with Neo or export to ONNX, deploy with Greengrass V2, and send telemetry back to SageMaker Model Monitor or S3. Teams that run multi-cloud or multi-vendor hardware often use a vendor-neutral edge platform instead.
Regulation: what edge MLOps teams must record
The EU AI Act does not use the word MLOps, but its high-risk duties (risk management, data governance, logging, human oversight, accuracy and robustness) map closely to MLOps practice. Under the 2026 Omnibus, they apply from 2 December 2027 for Annex III uses and 2 August 2028 for AI inside regulated products such as machinery (Orrick). Teams that already log model versions per device, training data versions and approvals will have little to retrofit.
Where it's heading: 2026 → 2030
| Area | Around 2020 | 2026 (now) | By 2028 (expected) | By 2030 (expected) |
|---|---|---|---|---|
| Edge deployment tools | Cloud-vendor edge managers launched (incl. SageMaker Edge Manager) | Edge Manager discontinued (Apr 2024); Greengrass, Azure IoT Operations, vendor-neutral platforms (AWS) | Kubernetes-based edge platforms as the default for gateways | — |
| On-device runtimes | TensorFlow Lite, framework-specific | ExecuTorch 1.0, LiteRT, ONNX Runtime (Embedded.com) | Fewer, more portable runtimes | — |
| Edge hardware | Small CNNs on Jetson Nano-class boards | Jetson Thor family; T3000/T2000 modules due Q1 2027 (eeNews Europe) | Small language and vision-language models common on gateways and robots | — |
| Observability | Model metrics only | GenAI and agent tracing conventions in Development (Greptime) | Stable shared conventions for model, LLM and agent telemetry | — |
| Regulation | None AI-specific | AI Act transparency and penalties apply; high-risk deferred (Compliance Hub) | Annex III high-risk from 2 Dec 2027; Annex I from 2 Aug 2028 (Orrick) | Public-authority deadline 2 Aug 2030 (Orrick) |
Cells without a source are editorial expectations, not sourced forecasts.
How NexaStack helps
NexaStack is an edge AI platform for running models on gateways, robots, vehicles and industrial PCs. It handles packaging, staged over-the-air rollout, rollback, on-device monitoring and a record of which model version runs on which device, across mixed hardware and clouds.
For a worked example of edge models in production, see our guide to predictive maintenance with IoT and AI.
Frequently asked questions
What is MLOps and why is it used?
MLOps is the practice of building, deploying and running ML models reliably in production. It is used because models degrade as real-world data changes, and without versioning, monitoring and retraining, nobody notices until results go wrong.
What is MLOps vs DevOps?
DevOps automates building and releasing software. MLOps adds data versioning, model training and retraining, and model performance monitoring, because a model can fail without any code change.
Does MLOps need coding?
Yes, mostly. Python and familiarity with containers, CI/CD and cloud services are standard. Managed platforms reduce the code needed for pipelines and deployment, but not to zero.
What is the difference between MLOps and ModelOps?
MLOps covers the engineering lifecycle of ML models. ModelOps is broader: it governs all decision models across an organisation, including approval, risk tiering and retirement.
Is SageMaker Edge Manager still available?
No. AWS discontinued it on 26 April 2024. AWS suggests ONNX Runtime plus AWS IoT Greengrass V2 for edge deployment and monitoring.
Run edge MLOps across your device fleet
See how NexaStack packages, signs, rolls out and monitors models across gateways, robots and vehicles, with a per-device record of what runs where. Book an edge MLOps fleet assessment →
