The recognition and detection of cracks in historical landscape buildings through deep learning technology

Бесплатный доступ

This paper added the spatial attention fusion module with an attention mechanism to the You Only Look Once version 5 algorithm to improve its building crack detection performance. Simulation experiments were conducted. Ablation experiments were conducted on the improved model to verify the role of the spatial attention fusion module module. It was then compared with three target detection algorithms: Faster-regional-based convolutional neural network, You Only Look Once version 4, and You Only Look Once version 6. It was found that after the introduction of the self-attentive feature map module, the precision of the improved You Only Look Once version 5 on the test set increased from 0.852 (the traditional You Only Look Once version 5) to 0.989, the recall rate increased from 0.843 to 0.988, the F1 value increased from 0.847 to 0.988, the mean average precision increased from 0.789 to 0.987, the intersection over union increased from 0.841 to 0.986, and the increase in each indicator was approximately 14.0%-19.8%. In comparisons with other algorithms, the mean average precision of the improved You Only Look Once version 5 was 0.253, 0.189, and 0.158 higher than Faster-regional-based convolutional neural network (0.734), You Only Look Once version 4 (0.798) and You Only Look Once version 6 (0.829) respectively. Its intersection over union was 0.245, 0.174, and 0.138 higher, respectively. These improvements mean that the model is more reliable in both crack identification and location: on the one hand, higher precision and recall rates can reduce missed detections and false detections, and lower the costs of manual rechecks and secondary inspections; on the other hand, a higher intersection over union indicates a more accurate fit of the crack location and range, which is conducive to quantitative analysis of parameters such as crack length and width, providing a more reliable basis for the preventive protection and scientific restoration of historical landscape buildings, and thus bringing practical benefits to heritage protection, tourism safety, and public life and property safety.

building crack \ deep learning \ crack identification \ You Only Look Once version 5

Короткий адрес: https://sciup.org/140316974

IDS: 140316974   |   DOI: 10.18287/COJ1864

Распознавание и обнаружение трещин в исторических ландшафтных сооружениях с помощью технологий глубокого обучения

В данной работе в алгоритм YOLOv5 (You Only Look Once, версия 5) был добавлен модуль пространственного внимания (spatial attention fusion module), что позволило повысить эффективность обнаружения трещин в строительных конструкциях. Были проведены имитационные эксперименты, а также серия экспериментов по методу абляции (удаления отдельных компонентов) для проверки роли модуля пространственного внимания в улучшенной модели. Кроме того, было выполнено сравнение с тремя другими алгоритмами обнаружения объектов: Faster R-CNN (Faster-regional-based convolutional neural network), YOLOv4 и YOLOv6. Установлено, что после внедрения модуля пространственного внимания точность (precision) улучшенной модели YOLOv5 на тестовом наборе данных возросла с 0,852 (в базовой версии) до 0,989; полнота (recall) увеличилась с 0,843 до 0,988; F1-мера — с 0,847 до 0,988; средняя точность (mAP) — с 0,789 до 0,987; а показатель пересечения объединения (IoU) — с 0,841 до 0,986. Прирост по каждому из показателей составил примерно 14,0–19,8%. При сравнении с другими алгоритмами средняя точность (mAP) улучшенной модели YOLOv5 оказалась выше, чем у Faster R-CNN (0,734), YOLOv4 (0,798) и YOLOv6 (0,829), на 0,253, 0,189 и 0,158 соответственно. Показатель IoU оказался выше на 0,245, 0,174 и 0,138 соответственно. Благодаря этим улучшениям модель стала более надежной как в плане обнаружения трещин, так и в плане определения их местоположения. С одной стороны, более высокие показатели точности (precision) и полноты (recall) позволяют сократить число пропусков и ложных срабатываний, а также снизить затраты на ручную перепроверку и повторные инспекции. С другой стороны, более высокий коэффициент пересечения объединений (Intersection over Union, IoU) свидетельствует о более точном определении расположения и протяженности трещин. Это создает благоприятные условия для количественного анализа таких параметров, как длина и ширина трещины, обеспечивая надежную основу для превентивной защиты и научно обоснованной реставрации исторических ландшафтных сооружений. Тем самым достигается практическая польза для сохранения культурного наследия, обеспечения безопасности в туристических зонах, а также защиты жизни и имущества граждан.

Текст научной статьи The recognition and detection of cracks in historical landscape buildings through deep learning technology

Historical landscape architecture is an important material carrier of human civilization, carrying rich cultural memory and artistic value [1]. These buildings not only reflect the technological level and aesthetic style of a specific historical period but also play an irreplaceable role in urban cultural identity, tourism development, and heritage protection [2]. However, over time, under the influence of factors such as natural environment erosion, climate change, human destruction, and material aging, historical landscape buildings suffer from varying degrees of structural damage, among which cracks are the most common and representative type of disease [3]. The appearance of building cracks not only affects the appearance of buildings but also indicates a decline in structural stability. In severe cases, it can threaten the safety of buildings and the lives and property of the public. Timely and accurate detection of cracks in historic landscape buildings is a key link in achieving health monitoring, preventive protection, and scientific restoration. Traditional manual inspection and visual identification have problems such as low efficiency, strong subjectivity, and difficulty in quantification, which are difficult to meet the needs of the protection of historical landscape buildings [4]. The development of computer vision, image processing technology, and artificial intelligence has demonstrated powerful capabilities in tasks such as image recognition and target detection, providing new ways for the identification of cracks in historical landscape buildings. Zhang et al. [5] introduced an optimized version of the You Only Look Once version 8 (YOLOv8) model, called MBE-YOLOv8, for detecting building cracks. The experiments found that the ability to detect building cracks was significantly enhanced compared to the previous YOLOv8 model. Sharma et al. [6] proposed a new convolutional and composite attention transformation network model and found through experiments that this method could accurately, efficiently, and reliably detect cracks in building structures. Lou et al. [7] presented a deep learningbased three-dimensional crack detection method that uses light detection and ranging point cloud data to detect building cracks. Golding et al. [8] proposed an autonomous crack detection method based on a convolutional neural network (CNN) technology and discovered that the deep learning-based crack detection did not rely on color. Krishnan et al. [9] filled the research gap and addressed the data shortage in the detection of brick wall cracks by developing a novel deep learning image processing technique. They also trained and validated multiple deep learning models to classify brick wall images into cracked or normal states. Katsigiannis et al. [10] proposed a deep learning method for crack detection on the exterior walls of brick buildings using limited labeled data. The results showed that this method was very effective in crack detection. After end-to-end training of the neural network, its accuracy and F1 value reached up to 100%. Ali et al. [11] developed a customized CNN for crack detection in concrete structures. This method was compared with four existing deep learning-based methods, and the results showed that this model performed better in crack detection and localization of concrete structures. The above-mentioned relevant studies have all analyzed the detection of building cracks. Some of them focus on the algorithms capable of identifying crack images, while others direct their research towards the data collection methods for building cracks. This paper focuses on the algorithms used to identify cracks in building images. This study employs YOLOv5 for the detection of building crack images. The selection of YOLOv5 is primarily due to its enhanced architecture compared to earlier versions (YOLOv1-YOLOv4), which significantly improves both detection speed and accuracy. Although YOLOv6-YOLOv8 achieve higher overall precision via structural refinements, these advancements come at the cost of increased training time, hindering efficient model iteration and optimization. In applications requiring rapid iteration and deployment, such delays can become a constraint. Moreover, detecting building cracks falls under the category of small-object detection, where moderate accuracy is often sufficient, and pursuing marginal gains in precision is unnecessary. In such cases, the more complex architectures of YOLOv6– YOLOv8, with improved accuracy, may introduce excessive computational overhead without providing significant benefits. YOLOv5 achieves a favorable balance between detection accuracy and inference speed, and its traditional architectural design has good potential for enhancement. Thus, this paper finally improves YOLOv5 by introducing the SAFM module with an attention mechanism. Then, a simulation experiment was carried out. The novelty of this paper lies in the introduction of the SAFM module with an attention mechanism into YOLOv5 for building crack recognition, enabling to reduce the computational load while maintaining the recognition performance.

1.    Building crack recognition based on deep learning

The technology for detecting cracks in historical landscape buildings must have a high enough recognition accuracy, be able to automate processing to reduce human intervention, and be able to quantitatively analyze parameters such as crack length and width. Traditional manual inspection relying on the experience of technicians is labor-intensive, and the inspection results are subject to subjective influence [12]. Infrared thermal imaging technology uses the differences in infrared images caused by temperature differences to identify targets but has limited detection effect on cracks because cracks do not have a significant impact on the heat of buildings. Deep learning technology can extract features and identify crack types from building crack images with its feature extraction capabilities and nonlinear modeling capabilities [13].

Backbone network

Neck network

Input

1 Conv

Norm

CBS module SiLU

CBS module

| Bottleneck p—|~CBS module |

-^ CBS module |—{~ Concat |

C3 module

| CBS module

CBS module

[ C3 module [~

| CBS module |

[ C3 module

| CBS module |

| C3 module |

CBS module

SPPF module

Concat ~|

Upsample"—|

[ CBS module

Concat

module

I | SAFM module

C3 module"!

Concat ~|

| CBS module

| Maxpoof

| Maxpoof

Concat

C3 module

Split

| CBS module

MFGU

GELU

Head j            network i—>|   Detect

|—> |   Detect

т—» [   Detect

Concat | —» |   Cony

SAFM module

Fig. 1. The architecture of the improved YOLOv5 model

Crack detection on building surfaces is an image target detection task. In this paper, the YOLOv5 model is chosen for the detection and recognition of building cracks. To improve its recognition performance, an attention mechanism is introduced into it. Fig. 1 shows the architecture of the improved YOLOv5 model, where the SAFM module in the virtual box is the introduced attention mechanism module [14]. The attention mechanism of this module can add weights to the feature channels of the feature images extracted by the backbone network. In this way, during training and usage, it can focus on important feature information and ignore secondary feature information, so as to jointly maintain the training and recognition effects and reduce the computational complexity. The process of detecting and identifying building cracks with the improved YOLOv5 model is as follows.

f(x,y) = (G(x,y) - m g ) CSg^C)CSf + bmf + (1 -b)mg,                                            (1)

where G(x, y) is the grayscale value of pixel (x, y) in the original image, f(x, y) is the grayscale value of (x, y) after the enhancement operation, m g is the mean grayscale value of pixels in the original image, m f is the mean grayscale value of image pixels after the enhancement operation, sg is the standard deviation of the grayscale value of image pixels in the original image, S f is the standard deviation of the grayscale value of image pixels after the enhancement operation, c is the contrast spread constant, and b indicates the luminance coefficient.

@ The preprocessed building surface image is input into the backbone network of the improved YOLOv5 for layer-by-layer calculation according to the structure shown in Fig. 1 [16].

@ The backbone network extracts image features of different scales, and they are fused in the neck network [17]. The SAFM module consists of segmentation layers, multi-feature gate unit (MFGU) layers, concatenation layers, convolutional layers, Gaussian error linear unit (GELU) layers, and element product layers. The calculation formula for the entire module is:

г       [Bo, B i , B2, B3] = Split(B)

B q = Conv 3x3 (Bo')

.       B t =ГР (Conv3 x 3(l^ (B t ))      ,                                                                           (2)

B? = Convlxl(Concat(Bo, Bl, B2, B3))

< B = GELU(B)®B where B is the input image features, SplitQ is the channel segmentation function, Bo , Br, B2, and B3 are four image features after segmentation, Bo is the convolution-processed Bo, Conv3x3( ) is a convolution operation that indicates a 3*3 convolution kernel, Bt is the pooled, convolved, and upsampled B^, Ip ( ) indicates pooling features to a size of 2'

2^ , T p ( ) indicates upsampling features to a size of p , Concat ( ) indicates a concatenation operation, Convlxl( ) indicates a convolution operation of a 1 * 1 convolution kernel, B is the feature obtained after concatenating and convolving Bo and Bi t , GELU () is the activation function, ® represents the element product operation, and B is an output feature of the SAFM module.

@ After the neck network fuses the extracted image features, the head network detects targets in the fused image features. Candidate boxes of different sizes are used for target detection of the image features output by the three SAFM modules of the neck network, identify the content in the candidate boxes respectively, determine whether there are cracks in the candidate boxes and the types of cracks, and finally perform regression calculation on the grid points of the candidate boxes to determine their coordinates in the original image [18].

2.    Simulation experiment 2.1.    Experimental data

The data used in the simulation experiments were partly from the building structure crack annotation dataset , and the remaining were from self-taken building surface images. There were totally 14,000 images of building surface cracks. All of them included brick and concrete structures. When marking the sample images, the target boxes fit the cracks as closely as possible. The types of building cracks in the sample set included fine cracks, adjacent edge-through fracture cracks, opposite edge-through fracture cracks, diagonal through cracks, defect cracks, perforated cracks, and no cracks (Fig. 2). Each type included 2,000 images, among which 1,000 images were set as the training set, 500 as the validation set, and 500 as the test set.

  • 2.2.    Experimental setup

The structure settings for the improved YOLOv5 model are shown in Figure 1. Some of the structure settings have been described in the previous process, and the relevant parameter settings for the remaining parts are shown in Table 1. The improvement of the YOLOv5 model lies in the introduction of the attention mechanism, i.e., the addition of the SAFM module reflected in the model structure. Therefore, to verify the effectiveness of the introduced mechanism, ablation experiments were conducted on the improved YOLOv5 model, and the same training and testing were carried out on the algorithm after the SAFM module was removed.

Table 1. Relevant parameters for the improved YOLOv5 model

Parameter

Setting

Parameter

Setting

Input layer

4,000×3,600

CBS module

A convolutional layer with 32 convolution kernels of a 3×3 specification, a moving step size of

2, and a SiLU activation function

SPPF module

A 3× pooling layer, maxpooling, a moving step size of 2

Number of training sessions

200

Batch

8

Optimizer

Adam [19]

Learning rate

0.01

Fig. 2. Crack examples of different types

Moreover, to further verify the target detection performance of the improved YOLOv5 model, it was compared with other algorithms, namely Faster-regional-based convolutional neural network (Faster-R-CNN), YOLOv4, and YOLOv6.

  • 2.3.    Evaluation indicators

Common indicators for evaluating target detection algorithms include precision, recall rate, and F1 value:

(P = -T—

TP+FP

R= -P^— , TP+FN ,

I F = — к P+R where TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives. The precision-recall rate curve of the target detection model can be plotted by precision and recall rate, and the area below the curve is the average precision (AP) of the detection model. If it is a multi-class detection model, the corresponding precision-recall rate curve can be plotted for each corresponding class to obtain the AP of the corresponding class. The mean average precision (mAP) value obtained by summing and averaging the AP of all classes is the detection accuracy of the multi-class detection model, which is between 0 and 1.

In addition to identifying the type of target, the target detection algorithm also needs to locate the target through the target box, so its evaluation indicator also includes the positioning accuracy of the target box:

IOU =

DRnGT

DRUGT’

where IOU represents the degree of overlap of the target box, DR represents the target box predicted by the algorithm, and GT represents the actual target box in the image.

  • 2.4.    Experimental results

To test the effect of the introduced attention mechanism, ablation experiments were conducted on the improved YOLOv5 model with the SAFM module eliminated. After removing the SAFM module, the improved YOLOv5 model was equivalent to the traditional YOLOv5 model. It can be seen from Table 2 that the performance of the improved model in identifying and detecting building cracks and and the performance of crack positioning was reduced after the SAFM module was removed, indicating that the SAFM module can effectively enhance the target detection performance of the YOLOv5 model.

After verifying the effectiveness of the introduced attention mechanism, three target detection algorithms, Faster-R-CNN, YOLOv4, and YOLOv6, were used to compare with the improved YOLOv5 algorithm. The training and validation sets were used to check whether the four algorithms were overfitting, and the test set was employed to verify the performance comparison among the four algorithms. The results are displayed in Table 3, and the partial detection results for building crack detection are presented in Fig. 3. It can be seen from the table that the recognition effects of the four algorithms on the cracks in the training set were all excellent, and the differences among them were not significant. However, for the validation set, except for the improved YOLOv5 algorithm, the recognition effects of the other three algorithms declined. The algorithms were trained with the samples in the training set, and they were adjusted according to the recognition errors during the training process, so they performed well for the training set. However, the validation set contained new samples for the algorithms, and the algorithms used the parameters obtained after training to make predictions. Except for the improved YOLOv5 algorithm, the other three algorithms suffered from over-fitting, which made them perform excellently when dealing with training samples but led to a significant decrease in performance when facing unknown new samples. When the test set was used, the improved YOLOv5 algorithm had the best detection and localization performance for building cracks, followed by the YOLOv4 and YOLOv6 algorithms, and the Faster-R-CNN algorithm had the worst performance. As can be seen from Fig. 2, the improved algorithm could effectively identify crack types and fit crack locations effectively. The YOLOv6 algorithm also identified crack types but was less precise in locating crack locations than the improved YOLOv5 model. The YOLOv4 and Faster-R-CNN algorithms misidentified crack types and were rough in crack location.

Table 2. Results of ablation experiments

Evaluation indicator

Traditional YOLOv5

Improved YOLOv5

Precision

0.852

0.989

Recall rate

0.843

0.988

F1

0.847

0.988

mAP

0.789

0.987

Intersection over union (IOU)

0.841

0.986

Table 3. Performance comparison of different building crack target detection algorithms

Sample set

Evaluation indicator

Faster-R-CNN

YOLOv4

YOLOv6

Improved YOLOv5

Training set

Precision

0.989

0.991

0.992

0.990

Recall rate

0.987

0.990

0.991

0.989

F1

0.988

0.990

0.990

0.989

mAP

0.989

0.989

0.987

0.988

IOU

0.988

0.988

0.986

0.987

Validation set

Precision

0.782

0.806

0.854

0.990

Recall rate

0.781

0.815

0.848

0.989

F1

0.781

0.805

0.845

0.989

mAP

0.733

0.799

0.828

0.989

IOU

0.742

0.813

0.847

0.988

Test set

Precision

0.784

0.801

0.853

0.989

Recall rate

0.783

0.811

0.849

0.988

F1

0.783

0.806

0.847

0.988

mAP

0.734

0.798

0.829

0.987

IOU

0.741

0.812

0.848

0.986

Y0L0v6 (adjacent edge-through fracture cracks) Improved Y0L0v5 (adjacent edge-through fracture cracks)

Fig. 3. Partial detection results of different building crack target detection algorithms

Conclusions

This paper introduced the SAFM module with an attention mechanism to the YOLOv5 model to improve its detection performance. Simulation experiments were conducted. Ablation experiments were first conducted on the improved YOLOv5 model to verify the effectiveness of the SAFM module, The improved algorithm was also compared with the other three algorithms. It was found that the attention mechanism of the SAFM module could effectively improve the target detection performance of the YOLOv5 model. The improved YOLOv5 algorithm had the best detection and location performance for building cracks, followed by the YOLOv4 and YOLOv6 algorithms, and the Faster-R-CNN algorithm performed worst. In terms of the detection results of building crack images, the improved YOLOv5 model could identify crack types and effectively fit crack locations.