Trusted software development company since 2009. Custom DS/ML, AR, IoT solutions https://mobidev.biz
“Why should I care about a cool new technology until it’s solving any of my problems?” – this was the exact conversation I had with the executive of a water purification plant over a warm cup of coffee.
What started as idle chit chat ended up evolving into an insightful and meaningful discussion about AI technologies for defect detection and quality control. This conversation significantly changed my vision and approach.
The goal of this article is to describe how to implement AI-based visual inspection for defect detection. It’s not about the coolness of artificial intelligence—no, it’s about practical solutions to specific problems.
In this article, we will dicuss:
Artificial intelligence in manufacturing is a trendy term. When it comes to describing AI-based defect detection solutions, it’s often about some kind of visual inspection technology based on deep learning and computer vision.
WHAT IS DEEP LEARNING IN A VISUAL INSPECTION?
Deep learning is an aspect of machine learning technology that’s powered by artificial neural networks. The operating principle of deep learning technology is teaching machines to learn by example. By providing a neural network with labeled examples of specific types of data, it’s possible to extract common patterns between those examples, and then transform it into a math equation. This helps to classify future pieces of information.
With visual inspection technology, integration of deep learning algorithms allows differentiating parts, anomalies, and characters, which imitate a human visual inspection while running a computerized system.
So, what does this mean exactly? Let’s use an example.
If you were to create a visual inspection software for automotive manufacturing, you should develop a deep learning-based algorithm and train it with examples of defects it must detect. With enough data, the neural network will eventually detect defects without any additional instructions.
Deep learning-based visual inspection systems are good at detecting defects that are complex in nature. They not only address complex surfaces and cosmetic flaws—but also generalize and conceptualize the parts’ surfaces.
How to Integrate AI-Based Visual Inspection System:
Visual inspection development often starts with a business and technical analysis. The goal here is to determine what kind of defects the system should detect.
Other important questions to ask include:
Data science engineers choose the optimal technical solution and flow to proceed based on the answers they receive.
Data science engineers must gather and prepare data required to train a future model before deep learning model development starts. For manufacturing processes, it’s important to implement IoT data analytics. When talking about visual inspection models, the data is often video records, where images processed by a visual inspection model include video frames. There are several options for data gathering, but the most common are:
The most important parameters here are the video record’s quality. Higher quality data will lead to more accurate results.
Once we gather the data, we prepare it for modeling, clean it, check it for anomalies, and ensure its relevance.
The selection of a deep learning model development approach depends on the complexity of a task, required delivery time, and budget limitations. There are several approaches:
3.1 Using a deep learning model development service (e.g: Google Cloud ML Engine, Amazon ML, etc.)
This type of approach makes sense when requirements for defect detection features are in line with templates provided by a given service. These services can save both time and budget as there is no need to develop models from scratch. You just have to upload data and set model options according to the relevant tasks.
What’s the catch? These types of models are not customizable. Models’ capabilities are limited to options provided by a given service.
3.2 Using pre-trained models
A pre-trained model is an already created deep learning model that accomplishes tasks similar to the one we want to perform. We do not have to build a model from scratch as it uses a trained model based on our data.
A pre-trained model may not 100% comply with all of our tasks, but it offers significant time and cost savings. Using previously trained on large datasets models lets us customize these solutions according to our problem.
3.3 Deep Learning model development from scratch
This method is ideal for complex and secure visual inspection systems. The approach may be time and effort-intensive, but the results are worth it.
When developing custom visual inspection models, data scientists use one or several computer vision algorithms. These include image classification, object detection, and instance segmentation.
Many factors influence the choice of a deep learning algorithm(s). These include:
An example of defect categories:
Let’s say that we’re developing a visual inspection model for quality assessment in buildings. The main focus is to detect defects on the walls. An extensive dataset is necessary to obtain accurate visual inspection results, as the defect categories might be incredibly diverse, from peeling paint and mold to wall cracks. The optimal approach here would be to develop an instance segmentation-based model from scratch. A pre-trained model approach is also viable in some cases.
Another example is a visual inspection for pharmaceutical manufacturing, where you want to differentiate air bubbles from particles in products like highly viscous parental solutions. The presence of bubbles is the only defect category here, so the required dataset will not be as extensive as in the example above. The optimal deep learning model development approach might be to use a model development service over developing one from scratch.
The next step after developing the visual inspection model is to train it. In this stage, data scientists validate and evaluate the performance and result accuracy of the model. A test dataset is useful here. For a visual inspection system, it may be a set of video records that are either outdated or similar to ones we want to process after deployment.
When deploying a visual inspection model, it’s important to consider how software and hardware system architectures correspond to a model capacity.
The structure of visual inspection-powered software bases itself on the combination of web solutions for data transmission and a Python framework for neural network processing.
The key parameter here is data storage. There are three common ways to store data: local server, a cloud streaming service, or serverless architecture.
A visual inspection system involves the storage of video records. The choice of a data storage solution often depends on a deep learning model functionality. For example, if a visual inspection system uses a large dataset, the optimal selection may be a cloud streaming service.
Depending on the industry and automation processes, devices required to integrate visual inspection system may include:
Deep learning models are open to improvement after deployment. A deep learning approach can increase the accuracy of the neural network through the iterative gathering of new data and model re-training. The result is a “smarter” visual inspection model that learns through increasing the amount of data during operation.
In the fight against COVID-19, most airports and border crossings can now check passengers for signs of the disease.
Baidu, the large Chinese tech company, developed a large-scale visual inspection system based on AI. The system consists of computer vision-based cameras and infrared sensors that predict the temperatures of passengers. Now the technology, operational in Beijing’s Qinghe Railway Station, can screen up to 200 people per minute. The AI algorithm detects anyone who has a temperature above 37.3 degrees.
Another real-life case is the deep learning-based system developed by the Alibaba company. The system can detect the coronavirus in chest CT scans with 96% accuracy. With access to data from 5,000 COVID-19 cases, the system performs the test in 20 seconds. Moreover, it can differentiate between ordinary viral pneumonia and the coronavirus.
According to Boeing, 70% of the $2.6 trillion aerospace services market is now dedicated to quality and maintenance. In 2018, Airbus introduced a new automated, drone-based aircraft inspection system that accelerates and facilitates visual inspections. This development reduces aircraft downtime while simultaneously increasing the quality of inspection reports.
Toyota has recently agreed to a $1.3 billion settlement due to a defect that caused cars to accelerate even when drivers attempted to slow down, resulting in 6 deaths in the U.S. Using the cognitive capabilities of visual inspection systems like Cognex ViDi, automotive manufacturers can analyze and identify quality issues much more accurately and resolve them before they occur.
COMPUTER EQUIPMENT MANUFACTURING
The demand for smaller circuit board designs is growing. Fujitsu Laboratories has been spearheading the development of AI-enabled recognition systems for the electronics industry. They report significant progress in quality, cost, and delivery.
The implementation of automated visual inspection, along with a deep learning approach, can now detect issues of texture, weaving, stitching, and color matching.
For example, Datacolor’s AI system can consider historical data of past visual inspections to create custom tolerances that match more closely to the samples.
We will conclude with a quotation from the general manager we mentioned earlier: “It makes no difference to me whether the suggested technology is the best, but I do care how well it’s going to solve my problems.”
Written by Serhii Maksymenko, Data Science Solution Architect MobiDev. This article is based on our technology research and experience providing software development services.
Previously published at https://mobidev.biz/blog/ai-visual-inspection-deep-learning-computer-vision-defect-detection
Create your free account to unlock your custom reading experience.