Automation of monitoring construction works based on laser scanning from unmanned aerial vehicles

Original article

The current stage of development of the construction industry is characterized by the gradual introduction of digital technologies, which are responsible for solving a number of production tasks that were previously performed “manually” (that is, directly by a person) [1–3]. The main advantages of digital technologies in construc- tion include high accuracy, speed and completeness of obtaining the information necessary for decision-making, elimination of dependence on insufficient qualifications and (or) dishonesty of a human performer, cost reduction through the use of unmanned vehicles, specialized software, and also risk reduction [4–6].

The most common digital technology in modern construction can be considered the construction of an Build-

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS ing information Modeling (BIM) [7–8]. At its core, it acts as a digital twin of a construction object as a subject of management, which makes it possible to carry out the most important management function of monitoring the performance of work. The idea of a BIM model is that at the design stage, a three-dimensional digital model of an object is created (including). Then, in the course of construction, using unmanned vehicles, surface scanning technologies, sensors, sensors, specialized software, new 3D models are created with a certain frequency, demonstrating progress in the construction work [9–10]. Moreover, these operations are performed in automatic or semi-automatic mode.

Thus, decision makers quickly receive the necessary high-quality information for control and corrective actions, in addition, automation makes it possible to reduce the costs of monitoring and control processes. Random errors that a human employee can make are excluded, and targeted falsification of data becomes almost impossible. It can also be added to the above that from 2021 in Russia it becomes mandatory to create digital models of objects that are built at the expense of budgetary funds.

However, the analysis of the literature shows that the development of technologies for creating BIM models and their implementation in practice faces serious problems and limitations. This is due to insufficient knowledge and development of a number of theoretical and applied aspects of digital modeling in construction.

Thus, in modern conditions, the expediency of using unmanned aerial vehicles (UAV) for conducting aerial photography of construction sites and obtaining the necessary information has already been established [11]. For example, in [12], the problem of monitoring the state of ground heating networks was solved on the basis of thermal imaging from the UAV, which makes it possible to identify the places of coolant leaks that require repair. There are developments for remote noncontact construction control using aerial photography, which demonstrate sufficient quality of the data obtained [13–14]. It should also be noted that the use of UAV eliminates the need for people to stay in potentially hazardous industrial areas.

As is known, UAVs can be equipped with various instruments and devices that collect the information necessary for the user – photo and video cameras, thermal imaging cameras, three-dimensional laser scanners (lidars), etc. [15–17]. For construction management, laser scanning is of the greatest importance as the most suitable technology for obtaining data for the subsequent construction of three-dimensional models. During the operation of the lidar, a laser beam is directed to the object under study, and then the time and direction of its reflection are recorded. As a result, a dense cloud of points is formed, which, after processing, makes it possible to obtain three-dimensional models [18]. Dense point clouds provide a fairly high accuracy of the results, because the shooting speed reaches several million points per minute [19].

The construction of building information models using UA and laser scanning, as well as other data collection technologies, has already been discussed in a number of papers. In particular, D. Moon et al. demonstrate in their study that laser-scanning point clouds do not provide the required accuracy of information for designing earthworks for the construction of large objects. To solve this problem, it was proposed to combine the results of laser scanning with aerial photography data [20]. A similar research methodology was used by Y.H. Jo, S. Hong when building a digital twin of historical objects in order to plan their restoration and repair [21].

However, this approach increases the payload on the UAV, reduces the time of one flight (due to the growth of the payload mass), and increases the overall duration and cost of work. This problem can be partially solved by using UAVs with a higher maximum takeoff weight and flight duration, but such equipment in itself is much more expensive and may be inaccessible under the conditions of sanctions and restrictions on imports to Russia. In addition, with the simultaneous use of information in different formats, the need for computing power increases, the data processing time increases, and there is a need for additional specialized software. Therefore, the joint use of aerial photography and laser scanning can be promising mainly for large unique objects, and not for the mass practice of construction control and monitoring.

There are also developments on the joint use of laser scanning both from the Earth’s surface by a human operator and with UAVs [22–24]. This improves the quality of the resulting model, but at the same time leads to a large amount of redundant information (“noise”). In addition, significant costs are required for the work of operators on the surface, it is necessary to have hand-held laser scanners in sufficient quantities, etc. It is no coincidence that this approach is also most often used to build models of unique objects of historical heritage and is recommended by UNESCO for the three-dimensional reconstruction of historical and cultural monuments [25]. According to [26], the use of two methods for obtaining data for building models leads to an increase in time and financial costs from 20–30% to 50% or more, which is not always compensated by data quality [27].

An analysis of the literature also showed that most of the works deal with the scientific and technical problem of obtaining a model of a building that has already been built based on the results of a single survey. In addition to the studies already cited, one can note the experience of studying deformations and displacements of the building of the Institute for Advanced Horticultural Research of Transylvania (Cluj-Napoca, Romania), which was built

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS on soft soils [28]. In this work, deformations and displacements of the facade were identified, which basically correspond to the allowable deviations. In [29], the building of the Boschi Observatory (Indonesia) was surveyed, which is located on a geological fault line and requires annual monitoring.

In contrast to the existing studies, this paper considers the use of UAV laser scanning for monitoring and control of construction activities. This task differs in that scanning should be carried out not once or annually (as in the case of already built buildings), but once a day or even more often, and on a large number of objects. Consequently, the issue of cost reduction comes to the fore and becomes the most relevant. Conducting combined ground and airborne scanning on a daily basis is not possible in most cases due to both financial and time constraints.

Therefore, further research is needed on airborne laser scanning technology as the cheapest and fastest technical solution. The authors believe that when using efficient data processing algorithms, it can provide the required measurement accuracy. We add that, in accordance with the data of a systematic review [30], the most promising areas of research in the studied subject area are precisely the control of costs for hardware and software, the improvement of data processing capabilities, and the automation of these processes.

As partially noted earlier, in modern realities, it is also necessary to take into account the complication of access to the most modern technical solutions, including restrictions on the export of industrial UAVs from the China, the termination of the official business “Leica Company” (a leading manufacturer of ground-based laser scanning systems) in Russia. Consequently, there is a need for the most efficient use of existing equipment, as well as for a wider use of equipment that does not fall under the sanctions imposed on Russia in 2022–2023.

However, published Russian studies on laser scanning in construction have not yet reflected the possibility of using this technology in construction control. Separate works of domestic authors (as well as foreign studies cited above) consider the construction of models of objects of cultural and historical heritage based on ground and air laser scanning data [31–32]. A popular area of research is the use of laser scanning (usually mobile, with a lidar placed on a car) to control the evenness of roads [33–35]. This is favored by the simple geometric characteristics of the roadway (disturbance of evenness of a solid planar object is immediately identified even with low lidar performance). These developments are not focused on controlling the construction of objects of a more complex configuration (buildings, structures).

Thus, in the processes of control and monitoring of construction works, it is promising to use the technology of airborne laser scanning with a UAV, which has

sufficient accuracy and an acceptable cost. This technology should solve the problem of correctly comparing the initial design model of the construction object with the actual state. In addition, you need to track the compliance of planned and actual models in accordance with the construction schedule and scan results for a specific date. Therefore, algorithms for transforming and comparing the corresponding point clouds are very important. Considering all of the above, the purpose of this study is to develop and test a technology for automating the monitoring and control of construction work based on laser scanning data from the UAV.

METHODS AND MATERIALS

The study used UAV “DJI Matrice 600 Pro” (manufactured in China) of the “hexacopter” type, with a flight speed of up to 65 km/h, a maximum take-off weight of 15.5 kg, a maximum flight altitude of 2500 m, a radio communication range of 5000 m, a flight duration of up to 18 minutes. BVS can be used at temperatures from –10оC to +50оC and wind speeds up to 8 m/s. It is also equipped with an onboard satellite navigation system receiver.

For scanning, the UAV was equipped with an AGM-MS3.200 airborne laser scanner (manufactured in the Russian Federation) weighing 1.3 kg, with a scanning frequency of 600 kHz (600 thousand times per second), a maximum scanning distance of 150 m, a view of 360о, distance determination accuracy 3 cm, coordinate determination accuracy 5 cm. The device can be used at temperatures from –20оC to +40оC. A general view of the equipment involved in the study is shown in Figure 1.

The design height of airborne laser scanning was 100 m. The flight task was planned in the UgCS program. The digital orthomosaic was developed using the “Agisoft Metashape Professional”. To process the point clouds, the Credo Scan 3D program was used. When determining the exact trajectory of the movement of the UAV, the online service “AGM Posworks web”, for the correction of data on the movement OF THE UAV and the elimination of noise – the software “AGM ScanWorks”.

When developing an algorithm for comparing dense point clouds, the methodology of non-binary data trees and the method of constructing octant trees were used. The processing of dense point clouds, the software implementation of the algorithm was carried out in the application “Control system for monitoring construction work at facilities that have passed state examination” (the authors’ own development, performed at the Institute of Digitalization of Kemerovo State University, Kemerovo, Russia). The site of the tank farm for the storage and transshipment of oil and oil products in the city of Kemerovo (Kemerovo region – Kuzbass, Russia) was used as the object of work.

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS

Fig. 1. General view of the equipment used in the study (on the left – UAV, on the right – an airborne laser scanner on a suspension)

RESULTS AND DISCUSSION

At the initial stage of the research, a digital orthophotomap of the area was created, which is necessary for the subsequent placement of laser scanning data on construction sites on it. To create an orthophotomap, the original (unprocessed) digital materials of aerial photography, elements of internal orientation of photographs, elements of external orientation of photographs (centers of projections of aerial photographs) were used. A digital orthophotomap is derived from aerial photographs with overlapping boundaries to provide a continuous image of a construction site without seams or cuts. To build a digital orthophotomap, the following steps were performed:

• 1)    orthorectification of aerial photographs using the centers of their projections;

• 2)    automatic identification of images, construction of a sparse point digital terrain model;

• 3)    control of the position and construction of a digital sparse point model of the surface using control signs;

• 4)    construction of a dense point digital surface model;

• 5)    building an orthophotomap;

• 6)    cropping of the orthophotomap and digital surface model along the boundaries of the work object.

Further work was carried out directly on the creation of a dense cloud of points based on laser scanning data. At the same time, information about the UAV flight trajectory, data from the UAV inertial system, and the results of measurements using lidar were used . Creating a dense point cloud included the following steps:

• 1)    obtaining the exact trajectory of the movement of the UAV relative to the point of the base station (KEME);

• 2)    automatic introduction of corrective corrections of the inertial system to the range measurement data;

• 3)    automatic filtering of the preliminary version of the dense cloud of points from noise.

Thus, the authors obtained a digital orthophotomap of the area where construction work is being carried out, as well as two dense point clouds – from the results of aerial photography and from the results of laser scanning. Before proceeding to the use of the obtained results for the development of automation technology for monitoring and control of construction work, it was necessary to evaluate the quality of the data obtained, compare them with each other (to assess the correctness of calculations, eliminate gross errors, confirm compliance with regulatory requirements). For this purpose, dense point clouds were loaded into the Credo Scan 3D program, and a vertical cutting plane was built according to characteristic objects in the territory where construction work is being carried out. The resulting point cloud profile along the cutting plane is shown in Figure 2.

The blue line corresponds to the point cloud from the laser scan on August 23, 2022; the red line corresponds to the point cloud from the laser scan on December 17, 2022. The green line corresponds to the dense point cloud obtained from aerial photography.

The error in determining the coordinates of points δ (according to the instructions for the laser scanner used in the study) is calculated by the following formula:

δ = 15 + 1 × 10^–3× D , (1)

where D is the distance to the measured object, km.

The results of the surveys must comply with the regulatory requirements for the results of topographic aerial photography in terms of the error in calculating the coordinates and density of points per 1 square meter (accord-

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS

Fig. 2. The resulting profile of point clouds along the cutting plane

ing to GOST R 59328-2021 “Topographic aerial photography. Technical requirements”). Since the distance to the measured object is equal to the design flight height, that is, 100 m, the error in calculating the coordinates of points in space in accordance with (1) will be: 15 + 1 × 10–3 × 0.1 = 0.015 m. This corresponds to the regulatory requirements for accuracy determining the coordinates of points in airborne laser scanning. Table 1 shows the characteristics of dense point clouds from the results of laser scanning. As can be seen from the data in Table 1, the dot density per square meter also complies with current regulatory requirements.

After receiving and processing the initial data of laser scanning and aerial photography at the beginning of construction, it was further necessary to develop an algorithm for comparing the initial data and the results that reflect the progress in the construction work on certain dates. Such an algorithm should process two point clouds obtained for different dates and select fragments of the construction object that were created during the corresponding period. During the study, the authors developed the following linear algorithm (Figure 3). For convenience of perception and presentation, the blocks of the algorithm are numbered on the left.

Let us characterize the main blocks of the algorithm. During the implementation of the first block, one of the dense point clouds is loaded, while the order (first of all, the results of an earlier or later survey are entered) does not matter, since the algorithm provides for the selection of the desired comparison sequence. The second block is the block of operations. Here the coordinate system of the first cloud is converted to the local coordinate system. Aerial photography data is also used for conversion, since each point in the image is attached to certain geographic coordinates that have high accuracy.

It should be noted that the more accurate the coordinates (the more decimal places the corresponding records have), the higher the need for computing power, the greater the load on them, in addition, the computation time is lengthened. In order to reduce the processing time and the load on computing power, the authors propose to transfer the coordinate system from the original to the local one.

This is done in the fourth, arithmetic block of the algorithm. In the fifth block of the algorithm, the point clouds are aligned with each other in the local coordinate system. Further, in the sixth data input-output block, the roles of the clouds are selected. The cloud obtained earlier is the source or reference cloud. The second cloud is the comparison cloud, which is the result of a later survey. It will calculate and visualize distances between points. Since more and more new fragments of the object (for example, floors) usually appear as construction work progresses, the later point cloud will contain more information than the original one. Accordingly, the reference cloud is determined according to the earlier survey date. As a result of the execution of the sixth block of the algorithm, the calculated distances (generated scalar fields) are placed on the compared cloud.

Then, during the execution of the seventh block of the algorithm, the distances between the points of the reference and compared clouds are calculated. For the

Table 1

Characteristics of dense point clouds from laser scanning results

Scan date	Number of points, pcs.	Area, square meters	Dot density, units per square meter
08/23/2022	28447422	118229	240.6
12/17/2022	14633928	118229	123.8

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS

Fig. 3. Algorithm for comparing dense point clouds for different dates, used to monitor and control construction work

calculation, the “near neighbor” method is used, when for each point of the compared cloud there is the nearest point in the source cloud, the Euclidean distance between them is calculated. Then this distance is attributed to the point of the compared cloud, as a result of which the corresponding scalar field is formed on the basis of this cloud.

A feature of the proposed algorithm is that the search for points in order to rationally use computing power and reduce operating time is carried out not throughout the entire source cloud, but within the boundaries of a predetermined space. The space of combined clouds is represented as an octree, where each node in the octant tree divides the space into eight new octants (Figure 4).

Each of the internal nodes of the octree contains information about 8 “descendants”, underlying nodes, each of which is 2 times smaller than the “ancestor” in size. This data structure allows 3D coordinates to be stored using 3 bytes instead of 12 bytes. Also, the search time for the smallest subtree that contains the desired coordinate point is significantly reduced. The search for the nearest coordinate point is not carried out over the entire source cloud, but only within the boundaries of the octant.

The ninth block of the algorithm displays the resulting point cloud, that is, the result of the comparison of the reference and compared clouds. This cloud should be rendered using a color scheme to display the scale of the received distances. When using such a cloud, it is possible to directly identify the fragments of an object that appeared during a certain time, as well as to accurately measure their position in a three-dimensional coordinate system.

The algorithm developed by the authors is implemented in the program (web application) “Management system for monitoring construction work at facilities that have passed the state examination”. Consider its use for monitoring construction work. Upon entering the application, the user selects two point clouds for comparison in accordance with the dates of interest to him. After performing the comparison, the program displays the resulting point cloud (Figure 5).

As can be seen from the data in Figure 5, the software allows you to get a visual representation of the changes in a three-dimensional object over a certain period, which is set by the user. The greater the distance between the points in the cloud, the closer the color scale to the red part of the spectrum. The minimum distances correspond to the blue part of the spectrum. Judging by Figure 5, during the analyzed period, the height of the object located in the upper left part of the resulting point cloud increased significantly. Significant progress has also been made in the construction of a cylindrical reservoir (upper right of the resulting point cloud). There were no significant changes in the rest of the construction site (the image is dominated by blue).

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS

Fig. 4. An example of structuring space points using octrees [36]

Fig. 5. An example of visualization of the resulting point cloud in the program “Management system for monitoring construction work at facilities that have passed the state examination”

The program also provides a number of tools for accurately assessing the geometric parameters of construction objects and their changes. In particular, the function «Construction of a profile of heights» is provided. It allows you to estimate the height difference between the selected points (Figure 6).

Each point has coordinate values in a three-dimensional system, which allows you to accurately set its posi-

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS

Fig. 6. An example of constructing a height profile between selected points in the program “Management system for monitoring construction work at facilities that have passed the state examination”

tion on a real object. Profiles (graphs) of the height difference, as can be seen from Figure 6, can be downloaded in .csv (2D) and .las (3 D).

Also, according to the data of dense point clouds used in the program “Management system for monitoring construction work at facilities that have passed the state examination”, you can perform the following operations necessary for the user (decision maker):

• 1)    measuring the distance between points in order to obtain accurate distances between objects and calculate their geometric dimensions;

• 2)    measuring the coordinates of points to accurately determine the position of a point in three-dimensional space and detect an object of interest to the user on the ground;

• 3)    measurement of area and volume, which makes it possible to calculate the volume and mass of materials used over a certain period of time.

CONCLUSION

Existing publications and technologies for building BIM-models of buildings have certain limitations and disadvantages for constant use in the processes of control and monitoring of construction objects (rather high cost, duration of surveying and data processing, the need to use expensive machinery and equipment). It is possible to speed up and reduce the cost of monitoring construction sites based on laser scanning technology exclusively from UAVs (without ground imaging) with subsequent automated data processing according to appropriate algorithms. In this case, aerial photography and the construction of an orthophotomap are carried out once at the beginning of construction, and then only airborne laser scanning is performed at the required frequency.

For data processing in the study, a linear algorithm was developed that allows comparing point clouds for differ- ent dates and identifying changes in point coordinates. A distinctive feature of the algorithm is that the search for points during comparison is not carried out over the entire cloud, but within the boundaries of the area specified by the octant. This reduces the load on computing power and reduces data processing time. The result of the algorithm execution is the resulting point cloud as a result of comparing two clouds (with different survey dates).

The developed algorithm received a software implementation in the web application “Management system for monitoring construction work at facilities that have passed the state examination” created by the authors. When controlling and monitoring construction works, the user (decision maker) has the opportunity to select the dates on which the point clouds were received and compare them. The resulting point cloud will be generated automatically. Its visualization is provided using a color scheme (red color corresponds to the largest changes, blue color corresponds to the minimum changes). The web application also provides the functions of constructing a height profile between selected points, measuring the coordinates of points and the distances between them, measuring the areas and volumes of construction objects, and calculating the materials used.

Thus, the developed technology makes it possible to monitor and control construction sites at a high speed, at low costs (since it does not require time-consuming and expensive laser scanning from the Earth’s surface), and makes it possible to make the most of the existing hardware and instrument base without involving hard-to-reach high-cost equipment. The developed algorithm also helps to reduce time costs and reduce the load on computing power. The software implementation of the construction work control and monitoring technology enables decision makers to quickly obtain the necessary information, track changes at the construction site, and compare them with planned data.

RESEARCH RESULTS OF SCIENTISTS AND SPECIALISTS