Visual anomaly detection uses computer vision to find anything in an image or video that does not look normal, such as a scratch, a missing part, a leak or an unusual event, often after training only on normal examples. In 2026 the leading methods compare image features against a memory of "good" samples or use vision-language models for zero-shot checks, and many run directly on edge devices next to the camera.
Updated October 2026
What changed in 2026
- Benchmarks got harder and more realistic. MVTec AD 2 adds eight new industrial scenarios and more than 8,000 high-resolution images, with test images under lighting conditions not seen in training and a private test set evaluated on a public server.
- Open-source tooling matured. Anomalib 2.7.0 (October 2026) added MH-PatchCore, retrieval-based RAD and few-shot FoundAD, and most models export to OpenVINO for fast inference on Intel hardware.
- Zero-shot and few-shot methods became practical. CLIP-style approaches such as WinCLIP and newer vision-language models can flag anomalies from text prompts and a handful of good images.
- Generative synthetic defects are routine. Teams use generative models to create rare defect images for testing and supervised fine-tuning.
What is visual anomaly detection?
An anomaly is anything that differs from the expected pattern. In images, that can be a local defect (a crack on a casting), a logical error (a connector in the wrong slot) or a scene-level event (smoke in a warehouse aisle). The core challenge is that anomalies are rare and varied, so you can rarely collect enough labelled examples of every kind. That is why most methods learn what "normal" looks like and score how far a new image is from it.
Main approaches
| Approach | How it works | Examples | Best for |
|---|---|---|---|
| Classical image processing | Thresholds, templates, statistical rules | Blob analysis, golden-image diff | Fixed parts, controlled lighting |
| Supervised detection / segmentation | Train on labelled defect classes | YOLO-style detectors, U-Net | Known, frequent defect types |
| Feature-embedding (unsupervised) | Compare patch features from a pre-trained backbone to a memory of normal patches | PatchCore, PaDiM | Few or no defect samples |
| Knowledge distillation | A student network fails to mimic the teacher on anomalous regions | STFPM, EfficientAD | Fast inference on edge hardware |
| Reconstruction / generative | Reconstruct a normal version and measure the difference | Autoencoders, GAN and diffusion-based methods | Textures and surfaces |
| Zero- / few-shot VLM | Match image regions to text prompts like "a scratched surface" | WinCLIP, VLM-based detectors | New products, quick start |
Generative models also help create training data; see our guide to generative adversarial networks.
Use cases
- Manufacturing quality inspection. Surface defects, assembly errors, weld and solder faults, packaging and label checks. See machine vision applications.
- Infrastructure and energy. Drone and robot imagery of power lines, pipelines, wind blades and solar panels.
- Medical imaging research. Flagging scans that differ from normal anatomy for radiologist review.
- Security and safety. Unusual movement, abandoned objects, smoke or spills in video streams.
- Retail and logistics. Damaged parcels, empty shelves and mis-sorted items.
- Agriculture. Diseased plants or fruit on sorting lines.
How to build a visual anomaly detection system
- Fix the imaging first. Camera, lens, lighting and part position cause more failures than models do.
- Collect normal images. Start with a few hundred good samples covering normal variation in colour, position and lighting.
- Gather some real defects for testing. Even a small set lets you measure recall and set thresholds.
- Benchmark several methods. Try a memory-bank method (PatchCore), a fast distillation method and a zero-shot baseline. Compare image-level and pixel-level scores.
- Set thresholds with the business. Decide the cost of a missed defect versus a false reject, then tune.
- Optimise for the edge. Quantise, export to OpenVINO, TensorRT or ONNX, and test latency on the real device.
- Operate it. Monitor score drift, retrain when products or lighting change, and keep humans in the loop for borderline cases. Our guide to MLOps for edge AI covers this.
Challenges and fixes
- Changing "normal". New suppliers, materials or seasons shift appearance. Use continual updates of the normal memory bank and drift alerts.
- Imbalanced data. Defects are rare. Use unsupervised methods, synthetic defects and careful evaluation.
- Logical anomalies. A correct-looking part in the wrong place fools texture-based methods. Combine with object detection or VLM reasoning.
- Explainability. Show heatmaps and nearest normal examples so operators trust the decision.
- Real-time demands. Lines can run many parts per second; choose models by measured latency, not paper results.
Where it's heading: 2026 → 2030
| Period | Visual anomaly detection |
|---|---|
| 2020 | Rule-based machine vision plus early deep learning; MVTec AD becomes the standard benchmark. |
| 2026 | Memory-bank and distillation methods in production; zero-/few-shot VLMs for fast starts; harder benchmarks like MVTec AD 2. |
| 2028 (our view) | Multimodal inspection that fuses images with sensor and process data, and explains anomalies in plain language. |
| 2030 (our view) | Inspection agents that detect, explain, adjust process settings and log decisions automatically within quality policies. |
How NexaStack helps
NexaStack helps teams train, package and deploy anomaly detection models to cameras, industrial PCs and edge servers, then monitor scores, drift and false rejects across lines and plants. Results can feed production systems, as covered in our guide to enterprise manufacturing intelligence. Explore our Vision AI solutions and broader computer vision solutions.
FAQ
What is anomaly detection in images?
It is the task of identifying images, or regions within images, that differ from normal patterns, such as defects, damage or unusual objects.
Can anomaly detection work without defect samples?
Yes. Unsupervised methods like PatchCore learn only from normal images. You still need some real defects to test and set thresholds.
Which algorithm is best for visual anomaly detection?
There is no single best. PatchCore-style methods are strong baselines for accuracy; distillation methods suit fast edge inference; zero-shot VLMs help when you have almost no data.
How is visual anomaly detection used in manufacturing?
For automated quality inspection: finding surface defects, assembly errors and packaging faults at line speed, often on edge devices beside the camera.
What are the latest visual anomaly detection trends in 2026?
Harder benchmarks with lighting shifts, few-shot and retrieval-based methods, VLM-based zero-shot checks, synthetic defect generation, and optimised edge deployment.
Ready to catch defects your rules miss? Book a NexaStack demo of edge visual anomaly detection on your own line data.
