TinyML is machine learning that runs on microcontrollers: chips with kilobytes of memory, no operating system and power budgets in the milliwatt range. It lets a sensor detect a sound, a gesture or a fault on the device itself, without sending data to the cloud. MLOps is what keeps those models accurate once thousands of devices are in the field.
Updated October 2026
What changed in 2026
- Generative models reached microcontrollers. In September 2026, open-source projects reported running a compact diffusion model and a small language model on ST and NXP microcontroller boards, without Linux or a GPU (Open Source For You).
- PyTorch now reaches the MCU. ExecuTorch 1.0 (general availability, October 2025) runs on Arm Cortex-M with CMSIS-NN and on Ethos-U NPUs, with more than 100 models running end to end (Arm).
- NPU microcontrollers are the new default for vision. Chips such as the STM32N6 (Neural-ART NPU) and Ethos-U85-based MCUs support transformer operators (Derek Molloy).
- Firmware updates are now a compliance issue. From 11 September 2026 the EU Cyber Resilience Act requires connected-product makers to report actively exploited vulnerabilities within 24 hours (McCann FitzGerald).
What TinyML is used for
- Keyword spotting: "Hey…" wake words on earbuds and appliances.
- Predictive maintenance: vibration and acoustic anomaly detection on motors, pumps and fans.
- Agriculture: soil, pest and livestock sensors that run for months on a battery.
- Wearables and health devices: activity and fall detection.
- Smart buildings: occupancy and people counting without storing video.
- Industrial vision: simple pass/fail inspection on low-cost camera boards.
Why run models on a microcontroller
| Benefit | What it means in practice |
|---|---|
| Low power | Battery devices can run for months or years |
| Low latency | The decision happens on the chip, in milliseconds |
| Privacy | Raw audio or images never leave the device |
| Low cost | Boards cost a few dollars, so you can deploy thousands |
| Works offline | No dependency on connectivity |
Hardware and tools
- Boards: Arm Cortex-M class microcontrollers (for example Arduino Nano 33 BLE Sense, STM32), Espressif ESP32-S3, and camera boards such as OpenMV.
- Runtimes: LiteRT for Microcontrollers (formerly TensorFlow Lite Micro), plus vendor runtimes and compilers.
- Platforms: Edge Impulse for data collection, training and deployment; vendor toolchains such as STM32Cube.AI.
- Benchmarks: MLPerf Tiny from MLCommons, for comparing boards on the same tasks.
Can ESP32 run TinyML? Yes. The ESP32-S3 adds vector instructions, and Espressif maintains optimised kernels for LiteRT Micro. It is a common choice for keyword spotting and simple vision.
How to shrink a model to fit
- Quantization: convert 32-bit floats to 8-bit integers; usually about 4x smaller with small accuracy loss.
- Pruning: remove weights that contribute little.
- Knowledge distillation: train a small "student" model to copy a larger one.
- Architecture search: design networks for the memory limit from the start (the approach behind MIT's MCUNet work).
MLOps for TinyML: the part most guides skip
A model that works on one dev board is a demo. A model running on 10,000 devices is a product. That needs:
| Stage | What to put in place |
|---|---|
| Data | Versioned sensor datasets, labelled per device type and environment |
| Build | One pipeline that trains, quantizes and compiles for each target board |
| Test | Accuracy, latency, memory and power tests on real hardware before release |
| Deploy | Signed over-the-air (OTA) updates with staged rollouts and rollback |
| Monitor | On-device counters and sampled data sent back to detect drift |
| Govern | A record of which model version runs on which device, and who approved it |
This is where an edge AI platform earns its place. NexaStack manages model packaging, rollout and monitoring across mixed device fleets, so teams can treat microcontrollers like any other deployment target. See how to choose an edge computing platform and what is Physical AI.
Is TinyML worth learning?
Yes, if you work in embedded systems, IoT or industrial AI. Demand is growing as more devices ship with AI accelerators. Good free starting points are the Harvard TinyML course on edX and the Edge Impulse documentation.
Where it's heading: 2026 → 2030
| When | What it looks like |
|---|---|
| Around 2020 | Keyword spotting and simple sensor models on Cortex-M4 boards with TensorFlow Lite for Microcontrollers. |
| 2026 (now) | MCUs with built-in NPUs run vision and small transformer models; ExecuTorch and LiteRT both target microcontrollers; first generative demos on bare metal. |
| By 2028 (expected) | Most EU Cyber Resilience Act duties apply from 11 December 2027, so secure over-the-air model and firmware updates become a must-have of TinyML MLOps (McCann FitzGerald). |
| By 2030 (expected) | Gartner expects over two-thirds of enterprises to deploy edge AI by 2029, up from 10% in 2025, with the smallest devices handling local filtering and wake-up (Gartner via ZEDEDA). |
Frequently asked questions
What is TinyML used for?
Running small ML models on microcontrollers for wake words, anomaly detection, gesture recognition and simple vision, where power, cost or privacy rule out the cloud.
Can ESP32 run TinyML?
Yes. The ESP32-S3 is widely used with LiteRT for Microcontrollers and Edge Impulse.
What is the difference between TinyML and edge AI?
TinyML is the smallest end of edge AI: microcontrollers with kilobytes of memory. Edge AI also covers gateways, GPUs and edge servers.
What happened to TensorFlow Lite?
Google renamed it LiteRT in September 2024. The microcontroller version is LiteRT for Microcontrollers.
What are the latest TinyML trends in 2026?
Microcontrollers with built-in NPUs, PyTorch models deployed through ExecuTorch, small generative models running on bare-metal boards, and secure update pipelines driven by the EU Cyber Resilience Act.
Run TinyML across thousands of devices
Got a model working on one board and need it on a whole fleet? We will walk through packaging, over-the-air rollout and drift monitoring for your devices. Book a 30-minute edge MLOps session →
