Development of a Cable Damage Detection Deep Learning Method based on Acceleration Response of Cable-Stayed Bridge

The purpose of this thesis was to select a cable-stayed bridge to which external force may cause damage as the subject, to develop a damage detection deep learning method capable of detecting cable damage, and to test and verify the developed damage detection deep learning method. The damage detection method was developed as a system that utilizes the acceleration response of a structure measured for maintenance purposes. To extract information capable of identifying the damage locations from among the measured acceleration responses, a CNN ID was used to develop the damage detection deep learning method. The developed damage detection deep learning method was developed in a way not independently arranging 1 machine learning model per each measuring point and finally predicting the damage location based on the decision-making results collected from each machine learning model. The developed damage detection deep learning method performed the learning per each machine learning model by utilizing the acceleration response of a structure acquired based on the preliminary damage test. Finally, the damage detection deep learning method that completed the learning verified the cable damage location detection performance by utilizing the data acquired based on the cable-stayed bridge damage test. As a result, it was confirmed that the developed damage detection deep learning method predicted the damage location of a cable-stayed bridge at an average accuracy of 89%. In the current research, only the cable-stayed bridge of the Seohaegyo Bridge was studied, but in the improved study, the research will be conducted on other bridges and damage assessment will be conducted on all cables.


Introduction
A cable-stayed bridge is a bridge where its cables bear the load of the superstructure for elongation purposes, and any cable damage is closely related to bridge safety. As demonstrated through a recently occurred bridge collapse in Taiwan, a cable bridge may be damaged due to diverse reasons, and may collapse in a worstcase scenario. As of now, a long-span cable-stayed bridge is designed to last approximately 100 years and the cables used are designed to last 20 years, in general. Therefore, it is necessary to continuously manage the cables of a cable-stayed bridge [1][2][3][4][5][6]. Diverse structural heath monitoring(SHM) methods are being studied to prevent such damage and collapse from causing losses of both life and property in advance, and are being used for the maintenance of cable-stayed bridges. Structural health monitoring method is a method that not only utilizes the data acquired from the diverse sensors installed on a structure to analyze the action of the structure, but also utilizes the analyzed results to identify the damage and health statuses of the structure.
A number of researches were conducted on such SHM method in the past. For example, a research conducted to determine the bridge damage status through identifying the statistical damage resulting from the measured ambient vibratory response [7], and the SHM method studied applicable to a cable-stayed bridge through statistically analyzing the natural frequencies, the modal damping, the mode shapes of the bridge, and the correlation shared between temperature and natural frequency [8].
The final purpose of such studied SHM method is to analyze the response data of a structure, to predict the damage level and damage location of the structure, and to secure structural safety through providing prompt responses based on the predicted results. To evaluate the damage to a non-simple structure, a prediction model taking into consideration diverse factors such as structural data, action features and external environments is required. Due to such reasons, pre-existing statistics-based prediction models used complicated, non-linear classification methods, and a number of questions were raised regarding the applicability of such prediction models [9]. To solve problems relating to such statistics-based damage detection methods as well as to utilize the developed models, deep learning technology started being applied to the SHM field. The deep learning method used in the SHM field is being diversely studied based on IT technology which is making rapid progress.
From among the deep learning methods, artificial neural network(ANN) and convolutional neural network(CNN) are the representative deep learning methods applied to the SHM field. ANN is a computing system designed by emulating the human brain structure, and is a method that uses artificial neurons to construct a network and perform machine learning. As a representative research utilizing such ANN, a research conducted to utilize the ANN to predict and evaluate the tension created by the cables of a bridge [10]. However, the problem with deep learning, a data-based learning model, is that it requires a number of variables(cable length, axis and bending stiffness, cable mass per unit length, etc.) to which the bridge status is applied. Namely, an ANN-based SHM requires considerable time and equipment to measure a number of occurring variables. Convolutional neural network(CNN) is a deep learning technology that self-learns the process of extracting image features, and has high accuracy in terms of CNN-based prediction models [11]. In general, a CNN-based SHM method was utilized in the SHM field to extract the features of an image as well as to predict the damage level, damage location and damage progress of a complicated image. Through the use of faster-region based-CNN, a more prompt and more accurate image processing based prediction model was studied [12]. However, most CNN-based SHM methods are used to classify image-or video-based data. In general, as far as the image processing based SHM research is concerned, the overall image(or video) of a structure including the damage location must be secured to enhance data accuracy and reliability. In general, image(or video) data take up a larger capacity than text data such as acceleration and displacement data measured for maintenance purposes, and the maintenance and server extension costs required for long-term monitoring are extreme [13][14][15][16][17]. Actually, a number of sensors were installed on the 'Tsing Ma' bridge in Hong Kong. The installation cost more than 27,000 dollars per sensing channel, and, taking into consideration that a number of channels were installed on 1 bridge, the cost escalated exponentially.
In this thesis, a method capable of evaluating the cable damage location of a cable-stayed bridge to which structural deterioration or external force may cause damage was developed, and the developed method was experimentally verified. A deep learning method that made progress and attracted recently attention was utilized as the cable damage location detection method, and the CNN method that is mainly utilized in image(or video) data-based researches despite its highly accurate performance compared to other diverse deep learning methods was used to develop the damage evaluation method. The damage evaluation method utilizing the developed CNN method was developed based on the acceleration data generally acquired for structural maintenance purposes, and the data acquired based on the model bridge damage test were utilized to learn and verify the damage detection deep learning method. it was confirmed that the developed damage detection deep learning method predicted the damage location of a cable-stayed bridge at an average accuracy of 89%.

Development of an ML Model for Damage Location Detection 2.1. Machine Learning and CNN
Machine Learning(ML) is one field of artificial intelligence, and defined as a "field of study that gives computers the ability to learn without being explicitly programmed". To successfully complete ML, as people learn through experience, computers must be provided with data to be learned. In this research, the ANN(Artificial Neural Network) from among diverse learning algorithms was used to develop the ML for detecting the damage location of a cable-stayed bridge. ANN is a statistical learning algorithm configured in the shape of a biological neural network, and consists of artificial neurons(nodes) for data transfer. In addition, an artificial neural network changes the strength(weighted value) of the network consisting of artificial neurons to classify and learn the features of the target. As shown in Figure 1, a normal artificial neural network has a structure consisting of a number of layers(x, h, g and f) connected through nodes. In addition, the number of layers increases as the network gets more complicated, and the number of input neurons and of output neurons within the layers increases as well. 1 , Han Ju-Hyuck 2 , Kim Chung-Gil 3 , Heo Gwang-Hee 4 , Kim Yong-Suk* 5 640 Figure 1. Artificial Neural Network The layers of an artificial neural network are divided into input layers, hidden layers and output layers depending on the role, and, in Figure 1, x is the input layer, h and g are the hidden layers, and f is the output layer. The input layer is where the data input is made, the output layer is where the data output is made, and the hidden layers are where the inter-layer connection strength is updated according to the node value. In addition, 'Fully Connected(FC) Layer' is the layer where all the nodes forming each layer are connected. The ANN shown in Figure 1 can be mathematically defined as Formula 1. The neuron's network function f(x) is defined as a composition of other functions( (x)), and such functions can be defined as a composition of other functions to conveniently represent the dependency relationship shared among variables.
( ) = (∑ ( * ( ))) =0 (1) In Formula 1, x and g are the input layer and hidden layer, respectively, of a neural network, and w is the connection strength(weight) between fs. K refers to the activation function that counts as a neural network classification method, and is mainly used as a non-linear weighted sum. In this research, the CNN(Convolution Neural Network) from among diverse ANNs is selected as the learning method. The CNN method is one of the methods frequently used in deep learning, and its strength is that it enhances the performance of a deep learning model by extracting only the important features of input data to be used for learning and that it uses CNN 1D or CNN 2D depending on the type of data. CNN 1D is used in association with sequence or time series data such as change data according to sentence or time progress, and CNN 2D is frequently used in association with image data.  In Figures 2 and 3, kernel is a queue consisting of weighted values(weights), and 1 kernel outputs 1 result per sequence or image data, and the number of results(features) matching the number of designated kernels is extracted and used for learning. Since rather learning only the important features of input data than learning all the unnecessary parts of input data allows a model to provide more accurate predictions, the weighted values of a kernel change in a way allowing the important features to be extracted as learning progresses. more features can be extracted from data as the number of kernels increases, and the extracted features vary depending on the size of kernels. Since the number or size of kernels varies depending on the data to be learned, it is necessary to find the optimized value through using a pre-existing model that learns data similar to the data to be learned or through conducting a test.

Development of a Deep Learning Model for Cable Damage Evaluation
In this research, a deep learning model capable of evaluating the cable damage status of a cable-stayed bridge was realized to detect the cable damage location of a cable-stayed bridge. The 1D CNN method was used to have this deep learning model learn the bridge response data according to time progress, and this deep learning model was configured as shown in Figure 4. Such input is the bridge response data for learning, and the acceleration data acquired from the acceleration sensor attached to the bridge were used. Conv1D layer is a layer that extracts the important features for learning from the input data, and, as shown in Figure 4, Phases 0~2 were configured using the Conv1D layer. FC layer is a layer that learns and classifies the extracted features into 5 cases, and Phases 3~5 were configured using the FC layer. The result(output) learned through the 1D CNN method represents the percentage of each case. The size of kernels used for all the Conv1Ds was 3, and the depth(the output according to the number of kernels) of 0:Conv1D, of 1:Conv1D and of 3:Conv1D was 256, 128 and 64, respectively. And the output shape in the FC layer was 1024, 128 and 5, respectively. The values used here were the values found through a test, were suitable for the data used in this research, and may be changed in the future. The ReLu(Rectified Linear Unit) was used as the activation function. ReLu is one of the most frequently used activation functions, has a fast learning speed compared to other activation functions, and is capable of solving the gradient vanishing and exploding problems that occur as the weight value decreases or increases excessively during the learning process. In this mode, the ReLu was used for all the layers except for the output layer. The Conv1D layer used in the developed model was used to learn the vibration data according to time progress, and the process is as shown in Figure 5. The input data shown in Figure 5(1) are the input data applied to the Conv 1D layer for learning or prediction, and the overall time step length data are received, divided into feature/step sizes, and are applied to the Conv1D layer. The input data of the model realized in this research are the structure's response acceleration data acquired for 6.4 seconds from the sensor attached to the structure, and the total length of such data is 16,384 frames. Such data are divided into 64 time steps and 256 features/steps(ie.64,256) and are applied to the model. This is input size of 0:Conv1D to which the collected data are applied, and the output from the previous layer are applied to 1:Conv1D and 2:Conv1D. In Figure 5(2), the convolution operation is performed using the selected area of the input data through the kernel. The kernel-related variable value was differently set per each Conv1D layer(0:Conv1D~2:Conv1D), and was determined through an experimental test. As far as the convolution operation is concerned, the selected (a)Input Data area and (b)Kernel shown in Figure 9 were used, and the operation was performed as shown in Formula 2. (2) In Figure 6, n, h are the width and height, respectively, of (a) and (b), and must share the same value. In (a), x is the input data value and is the weight value, and such values are updated as the learning process progresses. as 1 kernel performs the convolution operation in all the timesteps of the input data, Conv 1D layer results are saved as shown in Figure 5(3). The next kernel repeats the same convolution operation, and the same convolution operation repeats as many times as the number(c) of set kernels. Once this process is completed, the operation of 1 Conv 1D layer is terminated.

Cable-stayed Bridge Cable Damage Location Prediction through Cable-stayed Bridge Cable Damage Detection Deep Learning Model
In this research, the realized cable damage detection deep learning model is used to conduct a damage location prediction research. The realized deep learning model learns the data collected from the sensor installed on a cable-stayed bridge on a random basis. In this research, a total of n sensors are used to collect data, and, accordingly, n cable damage detection deep learning models are used, and the data acquired from 1 sensor are learned by 1 model. Once each model learns the data collected from a separate sensor, to predict the final damage location, as shown in Formula 3, n cable damage detection deep learning models( ) predict the damage results( ) using the data( ) acquired from sensors. The results( ) acquired from models( ) are integrated as shown in Formula 5, and the damage possibility( ) per each cable damage location is acquired.
The cable damage location(m) having the highest damage possibility( ) is determined to be damaged.

Cable-stayed Bridge Cable Damage Location Prediction Using Accelerometer and Deep Learning 3.1. Data Collection for Learning
In this research, a damage test was conducted to evaluate the performance of the machine learning model developed for detecting the cable damage location of a cable-stayed bridge. As far as the damage evaluation test is concerned, a cable-stayed bridge model approximately 1/200 the size of actual Seohaedaegyo Bridge was manufactured and used. The subject model having an overall length of 4.2m, a distance from tower to tower of 2.2m, a width of 0.17m and a tower height of 0.70m was designed/manufactured as shown in Figure 7, and 20 pairs of cables were connected to each tower, allowing a total of 40 pairs of cables to bear the weight. The cable-stayed bridge model damage test targeted the cables connecting the middle span between 2 towers. The damage test subjects, damage scenario cases, and the sensor location information for structure response measurements are as shown in Figure 8.  Figure 8 shows the middle span of the bridge model under test. A total of 8 accelerometers were installed in the middle span at the same interval, the structure response was measured at intervals of 1/256 during the test, and the accelerometers manufactured by Dytran were used. As far as the excitation for test is concerned, as specified in Figure 6, external force was applied to the structure by performing the hammer excitation between Channel 4 and Channel 5. From among a total of 20 pairs of cables, 5 damage test cases of 2 pairs(4 cables) each were selected as shown in Table 1. Cables 19 and 20 Case 4 Cables 7 and 8 Case 5 All cables function properly.
From among a total of 5 cases specified in Table 1, Case 5 had all the cables properly connected, and Cases 1~4 had the cables disconnected to simulate damage(cable rupture). Each case was repeatedly tested 300 times to measure the structure response, and a consistent impact was applied to the center of the bridge to measure the data for 6.4 seconds starting from the time the impact is applied. Figure 9. Data Collection, Pre-processing and Learning Process Examples in Damage Case 3 Figure 9 shows a process applicable to Case 3, one of diverse damage test cases. The learning process was performed by individually pre-processing the data acquired from 8 acceleration sensors and applying them to the models per channel. The data acquired from the pre-processing process were normalized within a range of -1 ~ 1, and were re-shaped in a way matching the input size(64.256) of the models. The final damage location is determined by having the results acquired from the 8 cable damage evaluation models run through Formulas 3 ~ 5 described above 1 , Han Ju-Hyuck 2 , Kim Chung-Gil 3 , Heo Gwang-Hee 4 , Kim Yong-Suk* 5 644

Result discussions 4.1. Cable-stayed Bridge Cable Damage Location Prediction Using Accelerometer and Deep Learning
Whether or not the cable damage location prediction model that completed the learning process through the training set is capable of accurately detecting the damage location of the collected test set is confirmed, Figure 8 shows the damage location(Pos) results calculated based on the data randomly brought from each case of the test set. In Figure 10, since 1 has the highest Pos value in Case 1 of the test set, the involved data are determined to be Damage Case 1. The same procedures are used to determine the results regarding the remaining cases of the test set. The results that can be acquired from the bridge cable damage location prediction model based on the random test set case 1~5 data are shown in Figures 11 ~ 15. In Figure 11, the damage location is accurately detected on Channels 1, 7 and 8 in Test Set Case 1, but Channels 4, 5 and 6 show no significant difference between Case 1 and Case 3, and it failed to predict the damage location on Channels 2 and 3 in Case 3. However, it is confirmed that Case 1, the damage location of the actual data, can be predicted by calculating the damage location(Pos) based on such results.
In Figures 12 and 14, Test Set Cases 2 and 4 show a significantly high accuracy compared to other cases, and, in Figures 13 and 15, Test Set Cases 3 and 5 show inaccurate results acquired from several channels(which were similar to the results that Test Set Case 1 show), but it was confirmed that the damage location of the actual data can be predicted by calculating the damage location(Pos).   Table 2 shows the calculated sensitivity results of the 8 bridge cable damage location prediction models that independently performed the learning process. Such sensitivity results were calculated based on the results acquired from a total of 450 test sets(90 test sets per case). Sensitivity refers to the percentage of true values predicted by the model out of actual true values, and is calculated as shown in Formula 6. TruePositive is a case where the actual damage case is predicted as a damage case, and FalsePositive is a case where the actual damage case is predicted as a no-damage case. = + As shown in Table 2, except for Channels 4 and 5, the remaining channels show an average sensitivity of 89% ~ 99% in all the cases. Channels 4 and 5 show a low sensitivity in Cases 3 and 5, and Channel 4 showed a low Park Gi-Hun 1 , Han Ju-Hyuck 2 , Kim Chung-Gil 3 , Heo Gwang-Hee 4 , Kim Yong-Suk* 5 646 sensitivity in Case 1 as well.

Conclusions
In this thesis, for the SHM, a research utilizing accelerometer and deep learning to predict the cable damage location of a cable-stayed bridge was conducted. The pre-existing researches were mainly focused on analyzing the accurate status of each cable installed on a cable-stayed bridge, and this research was aimed at detecting the location of the damaged cables requiring precision inspection. A Seohaedaegyo Bridge cable-stayed bridge model was designed and selected as the subject, the damage cases were divided into 5 different cases depending on the cable damage location, 8 accelerometers(channels) were used to collect the response data, and the collected data were applied to the learning process. The response data acquired from the 8 acceleration sensors were independently learned by the cable-stayed bridge cable damage evaluation models, the prediction results acquired from the 8 cable-stayed bridge cable damage evaluation models after completing the learning process were integrated, and the location of the damaged cable was finally predicted based on the integrated results. As a result of calculating the sensitivity through the test set, it was confirmed that the damaged cable can be predicted at an average accuracy of 89%.