Review on Machine Learning Based Welding Quality Improvement

3.1 Artificial Neural Networks

An Artificial Neural Network (ANN) is a weighted directed graph first devised by Warren McCulloch [13] in 1943. ANNs are composed of multiple layers and contain numerous nodes that mimic biological neurons in the human brain, such as Axons, Synapses, and Dendrites. It has evolved from the simplest artificial neural network proposed by Frank Rosenblatt in 1957 to a deep neural network (DNN) and Convolution neural network (CNN) [14].

Choudhury et al. [15] proposed microstructure evaluation of welding, parametric analysis, and ANN modeling. The results of the ANN-teach-learning-based optimization (ANN-TLBO) integrated modeling approach to optimize welding strength are as follows. a) Compared to multiple regression analysis (MRA), the ANN model confirmed 91.53% accuracy. b) As a result of parametric analysis, welding current was the most dominant factor, followed by welding speed. High heat input at low welding speed increased the penetration, improving the maximum tensile strength of welding. The proposed integrated ANN-TLBO optimization approach was optimal welding parameters I = 120 A, V = 180 mm/min, GFR (gas flow rate) = 121/min, and N = 2.24 mm, with a maximum tensile strength of 701.43 MPa.

Jacques et al. [16] proposed an ANN-based model to predict welding shapes and dimensions in laser-contact welding of galvanized steel. The model was constructed based on the fusion data provided by 3D finite element analysis. The possible relationship between welding parameters such as laser power, welding speed, diameter, and gap and the geometric properties of welding, especially penetration depth and bead width, was analyzed, and sensitivity to welding conditions was evaluated using relevant statistical tools. As a result, it was verified that the model has excellent performance and that the shape and dimensions of laser welding can be effectively predicted with an average prediction error of less than 10%.

Li et al. [17] developed an ANN model to predict bead shapes under welding conditions in which welding speed changes continuously. Experiments on several structures to optimize the hyperparameters of the model showed that the structure with 33-6-2 layers was the most suitable, and the result of the bead shape prediction model using it showed little deviation between the predicted value and the actual value. The highest error is 0.899 mm and the average bead width error is 0.223 mm. In addition, the highest error in bead reinforcement was 0.534 mm and the average error was 0.09 mm. It explained that this was caused by a lack of training data and an error due to the environment, and added that this precision is relatively acceptable in the welding process.

Thekkuden et al. [18] developed a voltage-based feed-forward backpropagation network for immediate inspection of V-groove joint welding quality by conducting bead-on-plate test welding using a gas metal arc welding process. Among the classification accuracy obtained using a total of nine techniques, the highest accuracy of ANN was 94.7%, and the voltage data corresponding to Bid-on-plate and V-groove welding were selected by Train and Test, respectively. The accuracy of Train, Validation, and Test was 92.3%, 100%, and 100%; the overall classification accuracy achieved was 94.7%.

Kamla Babu et al. [19] performed ANN modeling through Genetic Algorithm (GA) optimization for maximum tensile strength, microhardness, and corrosion resistance of AA2219 alloys. ANN established the relationship by providing L9 orthogonal array experimental input and output response, and optimization using GA was carried out in the experimental results. In the Friction Stir Wedding (FSW) joint results consisting of ANN-GA modeling optimization and feasible solutions, the Batch Back Propagation (BBP) model was measured at a minimum Root Mean Square Deviation (RMSE) value of 0.0089991 when seven hidden Layers were used.

Paven Kumar et al. [20] said that the process parameter is the penetration depth in the Cold Metal Transfer (CMT) pulse welding process using ANN. The effect on the weld pool width and bead height was investigated, and from experimental data, the neural network model developed in this study was used to control the process parameters to achieve the desired bead shape. In addition, it was observed that a correlation coefficient of 0.999 was obtained between the experimental results and the developed Back Propagation Network (BPN) model, and it was confirmed that the error was small for the desired bead shape. This confirmed that the developed neural network model can predict the weld bead dimensions with high accuracy.

He et al. [21] performed a pulsed tungsten inert gas (TIG) welding-brazing process to bond with aluminum-stainless steel dissimilar metals. The main parameters that have a large effect on the bonding characteristics are pulse peak current, base current, pulse on time, frequency, etc., which were verified through pre-experiments. The samples were set according to the central composite design and based on the samples, Response Surface Methodology (RSM) and ANN were used to separately predict the tensile strength of the joints. A mathematical model for joint strength prediction was derived using RSM, in which an algorithm consisting of one input layer with four neurons, one hidden layer with eight neurons, and one output layer with one neuron predicted strength, and compared to RSM, ANN's average state prediction error was about 10%, stable and more accurate.

Hamid Reza Fazli Shahri et al [22] predicted the Heat Affected Zone (HAZ) and temperature distribution of samples using ANN and GA. The HAZ of the material is prone to numerous mechanical defects such as residual stress and microcracks through a thermal process, and the temperature distribution of the sample is considered a key indicator that can determine the HAZ and Fusion Zone (FZ) regions. The model introduced in this paper optimized the three characteristics of backpropagation algorithms: number of neurons, learning volume, and learning speed in hidden layers using GA and developed to address the shortcomings of artificial neural networks. The combined model of GA-ANN was applicable to establish the relationship between HAZ and temperature distribution and three process parameters. As a result, it was confirmed that the experimental data and the prediction result were well-matched. We conclude that this model can predict physical and metallurgical results such as temperature distribution and HAZ for laser curing processes as well as other Gaussian heat source heat treatment processes such as EDM and laser welding.

Anand et al. [23] describe the use of a central composite design matrix to conduct experiments on friction welding of Incoloy 800 H. Tensile strength and burn-off length were measured using RSM and ANN, and the response was optimized and predicted by applying RSM first. After that, independent variables such as Heating Pressure (HP), Heating Time (HT), Upsetting Pressure (UP), and Upsetting Time (UT) were supplied as inputs to ANN, and the output of the network was tensile strength and burn-off length. A quick propagation algorithm was used to create different combinations of input/output patterns that further train multi-layer feed-forward networks and compared the results obtained through the RSM and ANN models to analyze which methods were more specific. Comparing the predictive capabilities of the two methodologies, it was determined that ANN (0.98031) provided a solution close to the actual value compared to RSM (2.167).

Olabi et al. [24] uses the L9 Taguchi approach and artificial neural networks to optimize the CO₂ butt welding process in the range of parameters investigated. They proposed Taguchi analysis as a solution to neural networks that only work within the range of parameters. Taguchi analysis allowed us to fill the available experimental data’s space and calculate the optimal penetration depth, fusion area width, and heat-affected area width. As a result, it was explained that the analysis of the data was performed quickly and intuitively, and the optimization process worked reasonably. They described that there was no problem investigating other keyhole laser welding applications.

Chaki [25] compared 65 different artificial neural network architectures to estimate the tensile strength of hybrid CO₂ laser metal inert gas (MIG) welded aluminum alloy plates. Sensitivity analysis was performed to study the effect of process parameters on tensile strength using Backpropagation Neural Networks (BPNNs), Raial basis function networks (RBFNs), and 65 ANN architectures. Based on the results, the following conclusions were drawn. (1) Bayesian regularization (BR) training and testing showed that the BPNN 3-1-1 network had the best prediction performance among various architectures with a welding strength prediction MSE of 3.24E-04. The 3-1-1 network trained using BPNN with batesian regularization (BR) once again confirmed the superiority of predictive ability with an average absolute error of 1.7%, a maximum error of 3.2%, and a minimum error of 0.3%. (2) As a result of the sensitivity analysis of the process, it was found that the welding strength increased as the laser power increased and the welding speed decreased as the welding speed increased. Therefore, welding speed and laser output were found to be the factors that had the greatest influence.

Chen et al. [26] applied the automatic detection and classification method of welding defects in the galvanized steel laser welding process. The authors compared ANN with the Support vector machine (SVM) and described that experiments showed that SVM and ANN were able to efficiently detect and classify defects such as voids, top bead depression, and bead separation. In SVM, the prediction rate of 48-dimensional data (98.08%) was higher than that of 16 dimensions (92.73%), and the prediction rate of ANN (97.27%) was higher than that of SVM (92.73%). Although the ANN algorithm performed well in terms of prediction rates because similar results were shown in cross-validation, we explained that the SVM has excellent generalization capabilities and is easy to adjust the parameters.

Sathiya et al. [27] performed laser beam welding to bond AISI 904L Super Austenitic Stainless Steel with three different shielding gases (Argon, Helium, Nitrogen). Laser welded butt joints were performed using a three-level full factorial design with three factors (beam output, moving speed and focal position) and three environments (argon, helium and nitrogen). Through this, welding bead shapes such as penetration depth, bead width, and tensile strength of joints were estimated, and the developed ANN model was used in conjunction with optimization algorithms such as GA to optimize welding parameters. Laser welding using optimized welding parameters showed better quality of joints, and the good agreement between theoretically predicted GA and experimentally obtained tensile strength, penetration depth, and bead width confirmed the applicability of calculation techniques to optimize parameters in the welding process.

Patil and Reddy [28] proposed an independent technique for detecting and classifying multi-type welding bonding defects, including cracks, undercuts, gas pores, pores, slags, wormholes, lack of penetration, and non-failure, in X-ray images through artificial neural networks and support vector machines. To implement this technology, a brightness histogram was built through image processing, boundaries were clarified through edge operation, and major analysis factors were selected to quickly detect and classify welding defects using ANN and SVM, respectively.

3.2 Deep Neural Networks

DNN is a neural network developed to solve problems in the ANN technique and is a learning method with two or more hidden layers in the model. DNN, also called deep learning, is a subfield of machine learning and has been continuously studied since 2010 due to the performance improvements of PCs and the spread of GPUs. As a result of the application of technology, it is actively expanded and applied to industrial sites because it is a learning method that can derive performance similar to or above people's level.

Oh et al. [29] conducted an eradication welding experiment to develop a bead width prediction model for the Gas Tungsten Arc (GTA) welding process, and analyzed the correlation between real-time process variables and bead width prediction models using the data obtained. The results are as follows. A total of 27 GTA welding experiments were conducted for each welding condition according to the complete factor experiment method, and current and voltage data were extracted in real-time using the welding monitoring system during the welding experiment. The welding quality was verified in real-time based on the extracted data, and a 3D scanner was used to check the trend of bead width according to the current-voltage data. To develop a DNN algorithm using a neural network based on the experimental results measured in the welding experiment, it was decided to minimize the error by setting the number of neurons in the hidden layer through a neural network structure. In addition, to verify the reliability of the DNN algorithm, the bead width obtained from the actual welding experiment was compared and analyzed, and the accuracy was evaluated using Predictive Ability of Model (PAM). As a result, it was confirmed that the bead width of the DNN algorithm was 100% predicted, and the exact bead width was predicted when the DNN algorithm was used.

Yang et al. [30] extracted 11 welding defect characteristics and proposed four factors based on the background and strong contrast. We also implement a cross-validation strategy, randomly dividing the entire dataset into five subsets, and then dividing 220 samples in each cross-validation interval into 176 Train data and 44 Test data. After applying it to the DNN algorithm using, the train set and test set were constructed by the Stacked Auto-Encoder (SAE). Finally, for comparison, welding defects were classified into the same dataset using SVM and DNN models. As a result, we confirm that SAE and Data augmentation via microrotation can achieve superior performance over existing models even with small datasets. This demonstrates superior performance with a 3.18% and 4.33% improvement in classification accuracy compared to regular DNN models, reaching 91.36%, and achieves good performance in welding defect classification.

Günther et al. [31] proposed an architecture for laser welding. It goes through the process of data observation, training, and control, which explains the combination of reinforcement learning and deep learning algorithms, and deep learning was used for the first time in laser welding and industrial production processes. The study was developed in line with laser welding and confirmed that DNN can derive meaningful features from laser welding settings, which can be used to review and control current process conditions. Furthermore, experiments with SVM demonstrated that DNN outperforms Principal Components Analysis (PCA). This structure learned how to predict photodiode data, and the learned behavior demonstrated the validity of the DNN function. It was also experimentally verified that adding a sensor does not increase the difficulty of calculation. Research shows that 52.6 seconds is used to simulate a laser welding system, whereas the overall simulation takes 66.3 seconds. Therefore, it is confirmed that single management takes 0.34 ms, which meets the process requirements for real-time process capability (1 ms).

Jin and Rhee [32] proposed a new DNN-based welding monitoring system and developed a LabVIEW software-based signal measurement, analysis, judgment, and control system by synchronizing robots and welders. Welding current and voltage signals were analyzed in the time and frequency domain, respectively, and a total of 24 feature variables were derived, and the proposed DNN model was used as input to train the model. In addition, signal preprocessing software has been developed to extract the corresponding feature variable in real-time. Notable results from the study are as follows. a) The proposed DNN-based gap detection model was trained based on feature variables extracted from welding current and voltage signals, showing a training accuracy of about 94.3%. b) When the verification results were performed with data not included in the training data offline and online, 93.7% and 87.7% accuracy were obtained, respectively. c) Based on the real-time welding gap detection results using this, the welding robot was controlled, and uniform external welding beads were achieved.

Shin et al. [33] developed a system to detect and predict porous defects occurring in the welding area using galvanized high-tension steel plates used in automobile parts in the GMA welding process. Through this, the following results were obtained. a) The welding current and arc voltage signals generated in the GMA welding process were measured in real-time, and characteristic variables were extracted through pretreatment. In addition, by analyzing the correlation between the shape variable and the porosity, a shape variable considered to be a defect signal was selected. b) Artificial intelligence techniques bonded to nonlinear arc welding process prediction were used, and a model was developed to detect and predict porosity by comparing DNN and ANN. c) We evaluated the predictive performance of ANN models and DNN models. The evaluation results show that the predictive performance of the DNN model is 15.2% higher on average than that of the ANN model. d) As a result of field application, all defects in the welding area were detected.

3.3 Convolution Neural Networks

Convolutional Neural Network (CNN) is a technique derived from DNN and is mainly used for image or image processing. For DNNs that use data in essentially one-dimensional form, training time and performance suffer because spatial and local information is lost within the image. Therefore, detection and classification are performed using the main information of the input image. Fig. 2 shows the general architecture of CNN.

Fig. 2

General architecture of CNN [34] (Adapted from Ref. 34 on the basis of OA)

Kumaresan et al. [35] proposed object detector-based casting defect detection for aluminum components used in automobiles. The object detector can classify defects and 14 types of defects, and the proposed defect classifier was used in object detectors by adjusting the key elements of existing histograms of oriented gradients (HOG) linear and SVM detectors. Key elements include a) image pyramid configuration, b) ROI extraction via sliding windows, and c) non-maxima suppression. The image begins with welding seam radiography. The sliding window is then run on images of all scales, ROI is detected, then passed through the weld defect classifier model, and the label is predicted after defect identification. Finally, the process ends when Non-maxima suppression deletes objects with low prediction accuracy and then visualizes radiographs. However, it may not match the performance of deep learning-based object detectors such as Fast R-CNN and You only look once (YOLO), because running a sliding window on multi-scale image data can slow the entire process for real-time detection, and end-to-end training is not possible.

Nomura et al. [36] predicted or estimated the welding quality by the CNN model, including changes in the difference between tapers or steps, using a molten pool image obtained by a camera during MAG (metal active gas) welding. The input image is a top image containing a molten pool obtained using a bandpass filter of 980 nm every 0.1 seconds during welding. As a result of the study, a) a burn-through prediction model was built with a classification problem to predict excessive penetration and burn-through in advance, but there was a problem that the prediction accuracy was poor because it was treated as a discontinuous phenomenon. Therefore, as a result of configuring the penetration depth estimation model as a regression problem, more than 95% were confirmed to have a 1 mm error, and more than 87% had a 0.5 mm error. Although this was an excellent estimation result, it was found that the estimation accuracy varies locally with the quality of the training data. On the other hand, when examining the effect of the image size used in the input image, it is confirmed that the estimation accuracy does not change significantly even if the image size is very different. As a result, it was experimentally verified that the calculation time was short enough to be used as a real-time monitoring tool.

Guo et al. [37] proposed a new defect detection method for resistance welding spots, down-sampling the size of the data image, and designating the number of classes with the help of experts. The loss function is cross-entropy and the optimization algorithm is ADADELTA (An Adaptive Learning Rate Method). Added Dropout layer to prevent overfitting. To find the optimal hyperparameters, each layer was experimentally identified, and as a result of comparing 5HL-CNN, 7HL-CNN, and 13HL-CNN, 13HL-CNN showed the best performance and the time per Epoch was not large at 5.58 s and 1.02 s, respectively. Defect detection of resistance welding spots using 13HL-CNN showed 99.01% performance.