Defect detection with deep learning uses neural networks to find scratches, cracks, missing features and other flaws in product images. In 2026 teams pick from four model families: classification, object detection, segmentation and unsupervised anomaly detection, often adding synthetic defect images and foundation-model backbones to cut labelling effort.
Updated October 2026
What changed in 2026
- Benchmarks got harder and closer to real factories. MVTec AD 2 adds eight new scenarios and over 8,000 high-resolution images, with test lighting that may differ from training (MVTec).
- Synthetic defects went mainstream. NVIDIA's July 2026 Metropolis release includes a Defect Image Generation skill for rare defect data (IT Brief).
- Edge video pipelines added foundation-model tools. DeepStream 8.0 brought Blackwell and Jetson Thor support and a MaskTracker built on Segment Anything Model 2 (NVIDIA docs).
- Commercial vision vendors now manage models across plants. Cognex OneVision reached general availability in May 2026 (Cognex).
Four deep learning approaches to defect detection
| Approach | Output | Training data | Good for |
|---|---|---|---|
| Classification | Good or bad (or defect class) per image | Labelled images per class | Simple pass/fail, sorting |
| Object detection | Boxes around each defect | Boxes drawn on images | Counting and locating discrete defects |
| Segmentation | Pixel mask of each defect | Pixel-level masks | Measuring defect size and area |
| Anomaly detection | Anomaly score and heatmap | Mostly good parts only | Rare or unknown defects, new lines |
Anomaly detection: start here when defects are rare
Many plants have thousands of good images and only a handful of bad ones. Anomaly methods learn what "normal" looks like and flag anything else. PatchCore, a widely used method, compares image patches against a memory bank of normal features (Roth et al., arXiv). Open-source libraries such as Anomalib package PatchCore and similar models with export to edge runtimes.
Supervised models: use when defect classes are known
When you need to tell a scratch from a dent, or measure a crack, supervised detection or segmentation is the better fit. Modern backbones, including vision transformers, transfer well from general images to industrial ones. See our guide to vision transformers for backbone choices.
Data: where most projects win or lose
- Define a defect catalogue with photos, limits and severity agreed by quality.
- Capture on the line, with the same camera, lens and lighting as production.
- Label consistently. Two inspectors should label the same image the same way. Measure their agreement.
- Fill gaps with augmentation and synthetic defects, then test on real defects only.
- Keep a hold-out set from a different week or shift to test drift.
Metrics that matter on the shop floor
| Metric | Meaning | Who cares |
|---|---|---|
| Escape rate (false negatives) | Defects the model missed | Quality, customers |
| False reject rate (false positives) | Good parts flagged as bad | Production, finance |
| Inference time per part | Must fit the line cycle time | Automation engineers |
| Recall at fixed false reject rate | Fair way to compare models | Data scientists |
Accuracy alone hides the trade-off. Agree with quality and production what escape and false reject rates are acceptable, then tune the threshold to that point.
From model to line
- Optimise models with TensorRT or OpenVINO and run on smart cameras, Jetson modules or industrial GPU PCs.
- Connect results to the PLC reject gate and MES traceability. See vision inspection on the assembly line.
- Monitor score drift and retrain when products, suppliers or lighting change. See MLOps for edge AI.
- Feed results into wider AI quality control and root-cause work.
Where it's heading: 2026 → 2030
| Period | Defect detection with deep learning |
|---|---|
| 2019–2022 (history) | CNN classifiers, first public anomaly benchmark (MVTec AD), PatchCore-style methods. |
| 2023–2024 (history) | Foundation models such as SAM and DINOv2 reach industrial teams. Edge learning on smart cameras. |
| 2026 (now) | Harder benchmarks, synthetic defects, managed model fleets, Blackwell-class edge compute. |
| 2028 (expected) | Few-shot and zero-shot inspection from text descriptions of defects becomes practical for many parts (our view). |
| 2030 (our view) | Inspection models adapt continuously on the line under human-approved update policies. |
How NexaStack fits
NexaStack packages, deploys and monitors defect detection models on edge devices and plant servers, with version control, drift monitoring and rollback across stations. It supports open model families and your existing cameras. See our Vision AI solutions.
FAQ
Which deep learning model is best for defect detection?
Use anomaly detection when defects are rare or unknown, object detection to locate known defects, and segmentation when defect size matters.
How many defect images do I need?
Anomaly models can start with good images only. Supervised models typically need dozens to hundreds of labelled examples per defect class, depending on variation.
What is the MVTec AD dataset?
It is a public benchmark of industrial images with real defects, used to compare anomaly detection methods. MVTec AD 2 adds harder scenarios.
Can defect detection run on edge devices?
Yes. Optimised models run on smart cameras, Jetson modules and industrial PCs within typical line cycle times.
What are the latest defect detection trends in 2026?
Synthetic defect data, foundation-model backbones, harder real-world benchmarks and fleet-wide model management.
Test your defects in a 30-day pilot
Send us sample images of your hardest defect. NexaStack's 30-day vision inspection pilot compares anomaly and supervised models on your data and reports escape and false reject rates. Start your pilot.
