Roof-DeGAN: гибридная GAN с межмасштабным вниманием для восстановления областей крыш на аэрофотоснимках

The restoration of images and building contours from aerial photographs for creating up-to-date terrain maps and cadastral assessment of capital construction objects is one of the main tasks addressed by the PLC «Roscadastr». Effective solution of this problem is of significant interest for cartography, urban infrastructure monitoring, and urban planning [1 , 2] . However, in practice, aerial photographs often contain defects caused by atmospheric factors (cloudiness, fog), temporary objects (construction equipment, vehicles, tree canopies), or technical limitations of imaging (sensor noise, low resolution). The quality of restoration directly affects the accuracy of subsequent automated analysis, including semantic and instance segmentation, 3D model construction, and condition assessment of capital construction ob jects.

Traditional image restoration methods, such as Navier-Stokes inpainting [3] and patch-matching based on similarity search [4] , demonstrate limited effectiveness when dealing with complex roof structures. These approaches rely predominantly on low-level features (pixel intensity, color and structure gradients) and do not account for object semantics, leading to blurred boundaries, distortion of geometric shapes (roof slopes, ridges, valleys), and the appearance of unnatural texture artifacts. When reconstructing large damaged areas, such methods cannot generate fundamentally new content, which is critical for aerial photographs with large defects.

Modern deep learning methods offer three main directions for solving this problem: generative adversarial networks (GANs), diffusion probabilistic models, and their hybrid combinations.

Generative adversarial networks remain in demand due to their high inference speed and robustness to limited training data. The foundations of adversarial training were presented in [5] , and conditional GANs (cGANs) for image-to-image translation have become a basic approach to restoration. However, classical architectures such as Pix2Pix [6] and ESRGAN [7] show degraded quality when processing large masks due to the limited capacity of the discriminator and insufficient penalty for high-frequency distortions, which is particularly noticeable on roofing material textures.

Diffusion probabilistic models demonstrate high generation quality. The base DDPM architecture [8] and its development for restoration tasks [9] achieve high realism but require thousands of inference iterations and large amounts of data. In 2025, specialized adaptations for remote sensing appeared: SatDiff [10] based on Stable Diffusion provides high-quality satellite image restoration; the KAO method [11] introduces kernel-adaptive optimization, outperforming previous approaches on structural tasks; the Image Characteristic-Guided method [12] accounts for low-rank image properties. Despite superior metrics, these models retain high computational complexity and low inference speed.

Hybrid methods combine the advantages of GANs and diffusion. Modern hybrid GAN implementations, such as DeGAN [13] , BD-VITGAN [14] , and TAMGAN [15] , integrate transformers, dense connections, and multi-level attention, significantly improving structural consistency and perceptual quality in remote sensing tasks. Other approaches, such as DSEPGAN [16] , achieve a balance between speed, determinism, and restoration quality, which is particularly important for specialized aerial roof imaging tasks.

The current state of the field is characterized by fragmentation: most methods are developed for general datasets; diffusion models require excessive resources; and hybrid approaches are insufficiently adapted to the specific characteristics of aerial roof images. There is no specialized method that combines robustness to small sample sizes, deterministic inference, high perceptual quality, and moderate computational complexity. The present work aims to address these shortcomings and continues the research described by the author in [17 , 18] . A hybrid approach is proposed for restoring damaged roof areas in aerial photographs, based on a conditional generative adversarial network DeGAN with transformer blocks in the generator and a multi-level discriminator with cross-level attention. The composite loss function combines adversarial components, pixel-wise comparison, perceptual features, and diffusion enhancement, achieving a balance between geometric reconstruction accuracy and naturalness of roofing material textures.

Section 1 presents a formal problem statement for the restoration of missing roof areas. Section 2 describes the architecture of Roof-DeGAN, including the structure of the generator with transformer blocks (equipped with a dynamic attention sparsification mechanism) and a multi-scale discriminator, as well as the composite loss function. Section 3 presents the datasets used (ZRG for pre-training and the target dataset with synthetic damage), data preprocessing methods, as well as details of the two-stage model training (pre-training for segmentation and fine-tuning for restoration) and a list of quantitative evaluation metrics (PSNR, SSIM, LPIPS, FID). Section 4 analyzes the training dynamics and model convergence. Section 5 provides a quantitative comparison of the proposed approach with state-of-the-art methods. Section 6 presents an ablation study to evaluate the contribution of each component of the loss function, as well as the impact of pre-training on ZRG. Section 7 analyzes the restoration quality as a function of the damage area size. Section 8 discusses the limitations and promising directions for future research. The paper concludes with Section 9 and a list of references.

1.    Problem Statement

The problem of restoring damaged or occluded areas of building roofs in aerial photographs can be formulated as a conditional image generation task. Let I gt G R ^HxWx3 be the original undamaged RGB image of a building roof of size H x W pixels. A binary mask of the damaged region M G { 0,1 } ^HxW defines the pixels to be restored: M ij = 1 corresponds to a damaged pixel, M ij = 0 corresponds to intact regions. The damaged image is formed as I in = I gt 0 (1 — M ), where 0 denotes element-wise multiplication.

The objective is to adjust the parameters θ of a parametric function G _g ( • ) such that, for the pair (I in , M ), it restores the complete image I rec = G g (I in ,M ), minimizing the generator loss function L g . A key feature of the problem is the need to generate new content while strictly preserving the geometric integrity of roof architectural elements (slopes, ridges, valleys) and the semantic consistency of roofing material textures (tiles, metal tiles, slate) with the surrounding context. This distinguishes it from general image inpainting tasks and requires specialized hybrid approaches (e.g., GANs with diffusion that account for geometric and semantic features).

( a ) damaged image I in

( b ) damage mask (I in ,M )

( c ) restored image I rec

• Figure 1. Example of restoring a damaged roof area

Figure 1 presents a sequence of three images illustrating the restoration process of an occluded roof area — the original damaged image, the damage mask (tree canopy), and the restoration result.

The necessity of solving this problem is driven by the practical needs of the PLC «Roscadastr». In the mass processing of aerial photographs for cadastral registration and updating of cartographic materials, a key step is the extraction of building contours and the formation of digital maps. However, the presence of damaged or occluded areas in the images (e.g., covered by tree canopies, cloud shadows, or temporary objects) causes standard segmentation and contour extraction algorithms to fail to correctly determine roof boundaries. An unreconstructed, noisy image does not allow obtaining a reliable binary object mask, which directly affects:

• ^•    the accuracy of contour maps — at the location of an occluded roof fragment, the contour is either broken or erroneously drawn along the boundary of the overlapping ob ject (e.g., a tree canopy);

• ^•    the formation of digital elevation and terrain models — the height reference of building corners is distorted;

• ^•    cadastral registration — the area of the object is incorrectly calculated, leading to legal and economic consequences.

2.    Roof-DeGAN Architecture

Manual processing of thousands of images with such defects is extremely labor-intensive and time-consuming. Developing an effective image restoration model allows automatic filling of occluded areas with semantically consistent content while preserving roof geometry. As a result, mapping algorithms receive a complete image as input, ensuring correct contour extraction, area calculation, and the formation of reliable cartographic products. At the same time, processing time is reduced from several minutes per image to fractions of a second, which is critically important when working with regional and federal aerial photography archives.

The proposed model is a hybrid generative adversarial network in which the generator restores occluded image regions while the discriminator evaluates the plausibility of the resulting fragments. To improve restoration quality, modern components are employed: transformer blocks for capturing global context, dense connections for better feature propagation, and a diffusion enhancement component for increased stability and naturalness of textures.

The generator is built upon an architecture combining transformer blocks and dense convolutional connections, specifically adapted for satellite and aerial image restoration tasks. It consists of three main parts: a contracting path (encoder), a bottleneck layer, and an expanding path (decoder). Skip connectors between corresponding levels of the contracting and expanding paths transmit low-level features.

The contracting path consists of a sequence of blocks, each including a self-attention mechanism, a fully connected layer for capturing long-range dependencies in the image, and a dense convolutional block that performs the transformation:

y = LeakyReLU (BN (Conv _3x3 (x))) + x, where

Conv _3x30 is a 2D convolution with a 3 x 3 kernel,

BN is batch normalization,

LeakyReLU( - ) is an activation function with a negative slope.

The number of feature channels increases from 64 to 1024 (in the bottleneck), while the spatial resolution is halved at each level by using a convolution stride of 2.

The expanding path is symmetric to the contracting path and uses transposed convolutions to increase resolution. At each level, the following transformation is performed:

y = TransposeConv _{4x4 s} =2 (Concat (x dec , x enc )) + ResidualBlock(x), where

TransposeConv _{4x4 s=2}0 is a transposed convolution with a 4 x 4 kernel and stride 2,

Concat ( x d_ec , x _enc ) is the concatenation (channel-wise joining) of features from the current level of the expanding path and the corresponding level of the contracting path via a skip connector,

ResidualBlock( x ) is an additional residual block for stabilizing training.

Skip connectors transmit detailed low-level features (boundaries, local textures), which prevents blurring of restored regions.

The generator’s output layer consists of a 1 x 1 convolution with a tanh( - ) activation function, which maps pixel values to the range [ - 1,1]:

I = tanh (Conv1x1(z)), where I︁ is the restored image and z is the output of the last block of the expanding path.

The discriminator is designed as a multi-level network with a cross-scale attention mechanism. It evaluates plausibility not on the entire image at once, but simultaneously at several levels of detail (from small fragments to large regions), linking information across different feature scales for more accurate assessment of restored fragment consistency. The discriminator’s input is the concatenation of the damaged image and the damage mask. Its output is a set of probability maps at different resolutions. Each level includes a convolutional backbone with a LeakyReLU( - ) activation function and spectral normalization, as well as a cross-scale attention module that links information between levels of detail. This approach provides accurate assessment of high-frequency details (roofing material textures) and roof geometric elements, while also improving training stability through gradient diversity.

Standard transformer blocks in the generator have quadratic computational complexity O(n 2 ) and do not account for the specific characteristics of aerial roof images, which contain large homogeneous areas (flat slopes) alongside fine details (tiles, joints). To address this problem, Roof-DeGAN introduces a Dynamic Sparse Attention (DSA) mechanism. At each encoder level, for each patch, a local variability measure is computed:

Vi = Var(pi) + Var(Vpi), where pi are the pixel values in the i-th patch and ∇pi is the gradient. For patches with low variability, attention is computed on a combination of:

• •    predictable neighboring patches (e.g., within a 3 x 3 window);

• •    30% of randomly selected patches across the entire image.

Patches with high variability (v i ≥ τ) are processed fully. The threshold τ is dynamically adjusted as a percentile of the distribution { v _i } at the current level (with т = P 50 , the 50th percentile, i.e., the median of the { v i } distribution). DSA reduces computational complexity from O(n ² ) to O(n • k), where k ^ n is the effective attention size, and enables processing of higher-resolution images without quality loss on textured regions.

The organization of Roof-DeGAN is shown in Figure 2.

The overall loss function of the model L G consists of several components:

(1) L G    ^ pix

pix

+ ^ adv L adv

+ ^ perc L perc ⁺ ^ diff L diff

+ ^ color L color .

The pixel-wise component ( L pix ) is responsible for accurate brightness matching of each pixel in intact regions and ensures the geometric integrity of the image. This component (the L 1 loss function) guarantees that the model does not arbitrarily alter the brightness and shape of ob jects in undamaged parts of the image [6] .

The adversarial component ( L adv ) forces the generator to create fragments so realistic that the discriminator cannot distinguish them from real ones. It is responsible for texture realism and edge sharpness [5] .

The perceptual component ( L perc ) compares images not pixel-wise but in the space of high-level features extracted by a pretrained convolutional

Generator

Discriminator

Figure 2. Main components of the Roof-DeGAN model network, e.g., VGG-16 [19]. This component allows the model to capture object structure and material texture (tiles, slate, metal), ignoring minor pixel shifts that do not affect human visual perception.

The diffusion component ( L dif f ) uses a pretrained DDPM denoising model [8] as a realism expert. It forces the restored image to behave like the ground truth during the diffusion noising and restoration process. This approach is based on ideas presented in [9 , 12] and effectively suppresses high-frequency artifacts typical of GANs, while also stabilizing training.

The color consistency component (Lcolor) penalizes the model for producing unnatural hues by comparing color distributions in the restored and ground truth images. The implementation is based on a differentiable histogram loss [20]. This component preserves the overall color palette characteristic of a specific roofing material type (e.g., terracotta tile color or gray slate shade), preventing the appearance of «acidic» or faded tones [12].

The optimal values of the coefficients λ were determined experimentally during the loss function study (see Section 6) .

3.    Training Details and Evaluation Metrics 3.1.    Pretraining on ZRG (Segmentation Stage)

To form initial representations of roof geometry and textures, the private Zeitview Rooftop Geometry (ZRG) dataset was used [21] .1 The dataset contains 22,334 annotated RGB images of residential buildings captured by unmanned aerial vehicles at an altitude of 3–5 meters above the roofs. The image resolution is <1 cm/pixel, allowing fine roof elements (individual tiles, joints, ventilation outlets) to be distinguished. Each image is accompanied by:

• ^•    a roof segmentation mask (binary roof/background mask);

• ^•    a 3D wireframe (polygonal model of roof faces);

• ^•    information about overhanging vegetation and shadows (natural occlusions).

The choice of ZRG is justified by the following advantages over classical aerial image datasets (Inria [22] , ISPRS Potsdam [23] ). Unlike Inria and ISPRS Potsdam, which contain only binary building/non-building annotations, ZRG provides detailed roof segmentation masks and 3D wireframes. The resolution of ZRG (<1 cm/pixel) significantly exceeds that of Inria (30 cm/pixel) and ISPRS Potsdam (5 cm/pixel), enabling the distinction of fine roof elements. Furthermore, ZRG includes natural occlusions (shadows, overhanging trees) absent in classical datasets, focuses on residential and suburban development (in contrast to the urban focus of Inria), and comprises over 22,000 annotated ob jects compared to 38 patches in ISPRS Potsdam.

The Roof-DeGAN generator was pretrained on the ZRG dataset in binary roof segmentation mode. The architecture at this stage included an encoder-decoder with transformer blocks and dense skip connections (as in the target model), but without the adversarial component. The loss function at the pretraining stage:

Lpretrain   ApixLpix + ApercLperc, where

L pix is the L 1 loss (pixel-wise comparison with the roof mask),

L perc is the perceptual loss on VGG-16 features [19] .

The coefficients were A _pix = 1.0, A _perc = 0.05.

Training continued for 65 epochs using the Adam optimizer ( Ir = 10 ^-4 , в 1 = 0.5, в 2 = 0.999), with a mini-batch size of 16 images. The discriminator weights were not initialized at this stage.

• 3.2.    Weight Transfer and Fine-Tuning (Restoration Stage)

• 3.3.    Target Dataset and Synthetic Damage Mask Generation

  The target dataset was formed from the aerial photography archives of PLC«Roscadastr» 2 and includes 2000 RGB images of building roofs (1600 for training, 200 for validation, and 200 for testing). All images were captured by unmanned aerial vehicles (UAVs) at an altitude of 30–50 m above roof level, which for typical garden plots ( « 8x8 m) provides a spatial resolution of « 3 cm/pixel. The size of each image after preprocessing is 256 x 256 pixels, corresponding to one entire building with a small surrounding background. The dataset covers the following roofing material types: metal tiles (about 35%), ceramic tiles (25%), slate (20%), bituminous shingles (10%), and other coverings (roll roofing, flat roofs, copper) — 10%. By development type, the images are distributed as follows: dense urban (10%), suburban (15%), and garden/dacha (75%). All images undergo manual verification for the absence of global defects (gaps exceeding 50% of the area, strong atmospheric distortions such as haze or glare), ensuring the purity of the ground truth data.

After pretraining, the encoder and decoder weights were copied into the generator of the target Roof-DeGAN model. The generator’s output layer was replaced: instead of a single channel (binary mask), three channels (RGB) were set, and its weights were randomly initialized. The discriminator was randomly initialized.

Fine-tuning was performed on the target dataset (1600 images with synthetic damage masks) using the full loss function (1) . The optimal coefficients were A pix = 1.0, A adv = 0.1, A _perc = 0.05, A diff = 0.01, A coior = 0.05. Training continued for 30 epochs with early stopping (training was halted if the loss on the validation set (200 images) did not decrease for 5 consecutive epochs). According to the training dynamics (Figures 4 and 6) , the best metrics were achieved at epoch 25, after which a sharp degradation was recorded at epoch 26 (adversarial training collapse). Therefore, the weights corresponding to epoch 25 were saved as the final model.

The mini-batch size was 32 images. Thanks to the use of mixed precision (FP16) and the Flash Attention mechanism, training ran stably on an NVIDIA A100 80 GB GPU. Choosing a batch size of 32 provides a balance between computational efficiency and gradient smoothness necessary for adversarial training.

Since obtaining real aerial photographs with precise «occluded area» masks (e.g., under a tree canopy or construction equipment) is laborintensive and expensive, synthetic damage mask generation is used. In image restoration tasks, this approach is standard and allows control over the shape, size, and position of defects. The damage mask M G { 0,1 } ^HxW is generated for each training example according to the following algorithm. First, with specified probabilities (40%, 30%, and 30% respectively), one of three damage types is selected: rectangular/polygonal (simulating occlusion by construction equipment or temporary structures), realistic tree crown, or irregular composite (modeling cloud shadows or tangled objects).

Particular attention is paid to generating tree crown masks, which, unlike primitive geometric shapes (ellipses or rectangles), reproduce characteristic features of real tree vegetation. The developed method includes the following crown types (examples are shown in Figure 3) :

(a) Branchy crown

(b) Pyramidal crown

(c) Spreading crown

Figure 3. Examples of synthetic tree crown masks

• ^•    Branchy (old oak, elm) — crowns with prominent large branches extending beyond the main mass;

• ^•    Pyramidal (spruce) — classic cone-shaped crowns with a characteristic form and «fluffy» texture;

• ^•    Spreading (pine) — irregular, asymmetric crowns with uneven branch growth.

• 3.4.    Quality Evaluation Metrics

For each crown type, the following parameters are specified: the area A _m is chosen from a uniform distribution within one of three ranges — 10–15%, 20–25%, or 30–35% of the total image area, allowing evaluation of model robustness to varying degrees of occlusion (see Section 7) . For branchy forms, the number and length of branches are additionally adjusted; for pyramidal forms — the trunk inclination angle and degree of «fluffiness»; for spreading forms — the degree of asymmetry and the number of crown growth centers. The mask location is chosen with equal probability at the center, edge, or corner of the roof, so that the model learns to restore both isolated defects and edge discontinuities.

After shape generation, the mask undergoes post-processing: first, morphological opening with a 3 x 3 kernel is applied to remove isolated spurious pixels at mask boundaries; then Gaussian blurring with &  = 1 is applied, followed by binarization at a threshold of 0.5. This creates smooth but sharp transition boundaries between damaged and intact regions, preventing the appearance of sharp brightness steps that the model could use as a «shortcut» to bypass learning. Additionally, for tree crown masks, internal gaps (simulating spaces between branches) are generated with 50% probability, and protruding branches with 40% probability, which increases realism and complicates the restoration task.

The damaged image is formed as I in = I gt 0 (1 — M ), i.e., pixels corresponding to the mask are zeroed out, while intact regions remain unchanged. Mask generation is performed on-the-fly during data loading (with a batch size of 32 images). For the validation and test sets, masks and damaged images are fixed and saved once, ensuring reproducibility of comparison across different models and configurations.

The proposed mask generation approach reflects the most common occlusion cases encountered in practice: tree vegetation (realistic tree crowns of various types covering 15–35% of the area), equipment and temporary ob jects (rectangular masks covering 10–20%), and clouds and shadows (large irregular masks covering 20–35%). Limitations of the method include the absence of halftone and gradient occlusion simulations (e.g., dappled shadows from foliage), as well as masks with occlusion exceeding 50% of the area (such cases require generation «from scratch» rather than restoration and are beyond the scope of this task). Nevertheless, as shown in Section 7 (Figure 9) , even at 35% occlusion the restoration quality can become unacceptable, which justifies the choice of the specified damage area ranges.

For quantitative assessment of restoration quality, the following metrics were used: Peak Signal-to-Noise Ratio (PSNR) [24] , Structural Similarity Index (SSIM) [25] , and Learned Perceptual Image Patch Similarity (LPIPS) [26] based on features of a pretrained VGG-16 network. All metrics were calculated exclusively on the pixels defined by the damage mask M , ensuring correct comparison of methods in restoration tasks. LPIPS was chosen due to the high correlation of deep VGG-16 features with human perception of roofing material textures and shapes [27] .

Additionally, the Frechet Inception Distance (FID) [28] between distributions of restored and ground truth images on the test set was used to evaluate texture generation quality. A low FID value indicates that the generated fragments are not only pixel-wise similar to the originals but also statistically indistinguishable from real roof images.

To evaluate the accuracy of restoring roof geometric contours (slopes, ridges, valleys), the Boundary F1 metric was used [1] . Boundaries are extracted using the Canny edge detector (thresholds 50, 150), after which the F1-score is computed with a tolerance of 3 pixels. The metric is calculated only within the damage region M and characterizes the suitability of the restored image for automatic contouring in cadastral systems.

To assess the reliability of the quality metrics, the bootstrap method was used (1000 resampling iterations with replacement on the test set of 200 images). Results are presented as mean ± standard deviation; for the FID metric, 95% confidence intervals are additionally provided.

For visual validation of the results, an expert evaluation method was also used: three specialists in the field of cadastral registration performed a comparative analysis of restored images on a scale from 1 to 5, evaluating geometric accuracy and texture realism. The averaged Mean Opinion Score (MOS) confirmed the correlation of objective metrics with subjective quality perception. All metrics were calculated both on the entire test set (200 images) and separately for three damage groups (10–15%, 20–25%, 30–35%), allowing evaluation of model performance for various degrees of roof occlusion.

The experiments were conducted on a computing cluster with the following configuration: NVIDIA Tesla A100 SXM GPU (80 GB memory), Intel Xeon Gold 6248R CPU (24 cores), 128 GB DDR4 RAM. The software used was Python 3.12 and PyTorch 2.5 with CUDA 12.x support. For reproducing basic functionality and experiments with small datasets (pretraining on Inria), a simplified version of the model is also provided as an interactive Jupyter notebook in the Google Colab cloud environment.

4.    Training Dynamics and Model Convergence

Figure 4 presents the curves of the main loss components on the validation set during model training.

Figure 4. Dynamics of generator and discriminator loss functions on the validation set during Roof-DeGAN training

In the first 10 epochs, a rapid decrease in generator loss is observed (from 7.82 to 3.94), corresponding to the initial training phase when the model learns the coarse structure of images and general roof geometric features. The discriminator loss on real images decreases monotonically (from 0.89 to 0.58), indicating an improved ability to distinguish ground truth samples. Simultaneously, the discriminator loss on generated images increases (from 1.12 to 1.45), reflecting the intensification of gradients against the generator typical of adversarial training.

In the 10–20 epoch interval, the rate of generator loss decrease slows down, and the model begins to refine roofing material textures and architectural element boundaries. The discriminator loss on real images continues to decrease smoothly (to 0.41 by epoch 20), while on generated images it increases (to 1.62), indicating that the balance between generator and discriminator is maintained.

In the 20–25 epoch interval, further quality improvement is observed: generator loss decreases to a minimum value of 1.32 by epoch 25, discriminator loss on real images stabilizes at 0.37, and on generated images at 1.60. Quality metrics reach peak values: PSNR — 34.5 dB, SSIM — 0.972, LPIPS — 0.052, FID— 17.6.

At epoch 26, a sharp degradation occurs: generator loss increases to 2.38, and quality metrics show a decline (PSNR drops to 33.42 dB, LPIPS increases to 0.058, FID— to 18.7). Further training up to 30 epochs does not lead to quality recovery: generator loss fluctuates in the range of 2.35–2.41, metrics remain at worse values. This served as the basis for applying early stopping at epoch 26, saving the best model obtained at epoch 25. Common GAN stabilization techniques (R1 regularization and spectral normalization) were not used in this work. Stability was achieved primarily through architectural solutions (DSA, multi-scale discriminator) and the composite loss function. The application of these methods is a promising direction for future work.

Figure 6 shows the dynamics of the quality metrics PSNR, SSIM, LPIPS, and FID on the validation set during model training. Analysis of this figure confirms the conclusions drawn from the loss curves. The PSNR and SSIM metrics actively increase until epoch 20, after which the growth rate slows. Maximum values are achieved at epoch 25: PSNR = 34.5 dB, SSIM = 0.972. The LPIPS metric decreases to 0.052, and FID to 17.6, indicating a significant improvement in perceptual quality and naturalness of roofing material textures. At epoch 26, a deterioration of all metrics is recorded: PSNR drops to 33.42 dB, SSIM to 0.970, LPIPS increases to 0.058, FID to 18.7.

In the 27–30 epoch interval, a plateau at degraded values is observed, indicating the onset of overfitting — the model begins to over-adapt to the textures of the training set at the expense of generalization ability on new images.

The obtained results confirm the effectiveness of the chosen stabilization techniques (early stopping, label smoothing) and demonstrate that the proposed hybrid Roof-DeGAN approach achieves a balance between convergence speed, perceptual quality, and robustness to limited training data.

Figure 5. Dynamics of geometric (PSNR, SSIM) and perceptual (LPIPS, FID) metrics on the validation set during Roof-DeGAN training

5.    Quantitative Comparison with State-of-the-Art Methods

To objectively evaluate the effectiveness of the proposed approach, a comparison was conducted with a number of state-of-the-art image restoration methods, including traditional algorithms, pure diffusion models, and hybrid GAN architectures. Testing was performed on an identical test set of 200 images with the same synthetic and natural damage masks covering 15–25% of the area. All methods were investigated in their standard configurations on the same hardware (NVIDIA A100 SXM GPU and Intel Xeon Gold 6248R CPU). The results are presented in Table 1 . The symbol f in the table header indicates that higher values are better, while ^ indicates that lower values are better. Inference time is reported for images of size 256 x 256 pixels.

Analysis of this table allows the following conclusions to be drawn.

Superiority over traditional methods . Roof-DeGAN significantly outperforms the classical Navier-Stokes and PatchMatch algorithms across all metrics: the gain in PSNR is 8.0–9.7 dB, and FID is reduced by a factor of 8–10. Visually, this corresponds to the transition from artifact-ridden and blurry reconstructions to detailed images with preserved roof geometry and textures.

Table 1. Comparison of Roof-DeGAN with state-of-the-art methods on the test set ( N = 200)

Methods	PSNR ↑	SSIM ↑	LPIPS ↓	FID ↓	P1	t2
Traditional
Navier–Stokes [3]	24.18	0.742	0.312	187.3	—	0.8
PatchMatch [4]	25.92	0.801	0.245	142.8	—	2.3
Diffusion-based
DDPM [8]	28.45	0.873	0.118	78.6	112	28.5
SatDiff [10]	30.45	0.941	0.065	38.2	189	41.3
KAO [11]	31.12	0.952	0.058	32.7	205	38.9
GAN-based
Pix2Pix [6]	28.92	0.885	0.142	92.5	54	0.12
ESRGAN [7]	29.34	0.902	0.098	67.3	67	0.18
DeGAN [13]	31.85	0.958	0.082	29.4	62	0.14
New method
Roof-DeGAN	33.7 ± 1.2	0.971 ± 0.008	0.048 ± 0.011	17.8 ± 2.4	48	0.15

¹ P — the number of model parameters in millions;

² t — the inference time per image in seconds

Roof-DeGAN: a hybrid GAN for roof inpainting          209

Comparison with diffusion models . Pure diffusion models (DDPM, SatDiff, KAO) demonstrate high quality but require tens of seconds for inference and significant amounts of training data. The proposed hybrid method outperforms them in PSNR by 2.8–5.5 dB and achieves substantially better FID (17.8 vs. 32.7–78.6) with an inference time 250–270 times smaller (0.15 s vs. 38–41 s). This makes the approach practically applicable for real-time processing of large aerial image archives.

Comparison with GAN architectures . Compared to baseline GAN models (Pix2Pix, ESRGAN) and other hybrid approaches such as DeGAN, the proposed method shows a PSNR gain of up to 5.0 dB and a significant improvement in FID (by a factor of 1.7–5.2 times). The LPIPS value of 0.048 confirms the high perceptual quality of restored textures, surpassing those of the compared GAN architectures (in our case, Pix2Pix = 0.142, ESRGAN = 0.098, DeGAN = 0.082) — an improvement of 41–66% relative to the closest competitor. Moreover, the model has fewer parameters (48M vs. 54–205M), providing computational resource savings. The superiority of Roof-DeGAN over the closest competitors is statistically significant: the 95% confidence intervals of the metrics do not overlap. For example, the difference in PSNR between Roof-DeGAN (33.7 ± 1.2 dB) and KAO (31.1 ± 1.8 dB) is 2.6 dB with non-overlapping intervals, confirming the robustness of the proposed method’s advantage.

The obtained results demonstrate that the integration of transformer blocks, multi-level attention, and diffusion enhancement allows the proposed method to achieve the best balance between restoration quality and computational efficiency among the considered approaches.

6.    Impact of Loss Function Components

To quantitatively assess the contribution of each component of the composite loss function, a comparative analysis of five model configurations differing in the set of optimized criteria was conducted. The five model configurations were trained with different component combinations until the optimal epoch according to the early stopping criterion. The results are presented in Table 2, with the best values highlighted in red.

Table 2. Ablation study of the main loss function components on the validation set

Configuration (components from (1) )	PSNR ↑    SSIM ↑    LPIPS ↓    FID ↓      Visual characteristics
No pretraining (all components from (1) )	28.34    0.912     0.112     54.2 Noticeable artifacts, smoothed textures
B ( λ pix L pix )	Blurred textures, ..    .    . smoothed boundaries
A ( λ pix L pix + λ adv L adv )	31.12    0.952     0.082     48.2 Sharp boundaries, local texture artifacts
P ( λ pix L pix + λ perc L perc )	31.45    0.958     0.068     39.7 Natural textures, excessive smoothing
D ( λ pi x L pi x + λ a d v L a d v + λ perc L perc + λ diff L diff )	Good geometry and texture, 32.18    0.965     0.058     28.9 minor color shifts
C ( λ pix L pix + λ adv L adv + λ perc L perc + λ diff L diff + λ color L color )	Most realistic textures, 34.5     0.972     0.052     17.6 geometry, and color consistency

Roof-DeGAN: a hybrid GAN for roof inpainting          211

Analysis of Table 2 reveals the following. The baseline pixel-wise configuration B ensures geometric integrity (position of slopes, ridges, valleys) but leads to characteristic blurring of roofing material textures (PSNR = 29.84 dB, LPIPS = 0.098).

Adding the adversarial component A significantly improves boundary sharpness and adds high-frequency details, but without perceptual and diffusion components, local artifacts arise (LPIPS = 0.082, FID remains high).

Including the perceptual component P without the adversarial part achieves LPIPS = 0.068, but visually the images appear overly smoothed due to the absence of high-frequency details that the adversarial component provides.

The best LPIPS value (0.058) among incomplete configurations is achieved with the combined use of adversarial, perceptual, and diffusion components D. Adding diffusion enhancement significantly improves structural consistency and reduces FID (to 28.9), though minor color shifts are observed.

The full configuration C, including all five components, combines their advantages and achieves the best geometric and perceptual metrics on the validation set: PSNR = 34.5 dB, SSIM = 0.972, LPIPS = 0.052, FID = 17.6.

Comparison of the «No pretraining» configuration and the full configuration C shows that the proposed two-stage approach (pretraining on ZRG followed by fine-tuning) provides a PSNR gain of 6.16 dB, an LPIPS reduction of 0.060, and an FID reduction of 36.6 compared to training from scratch. This confirms the effectiveness of transfer learning from the roof segmentation task to the image restoration task.

The ablation study (Table 2) confirmed the necessity of using all five loss function components. To achieve the best balance between geometric accuracy and perceptual quality, the weight coefficients λ were tuned. The optimal values obtained during hyperparameter optimization on the validation set were:

• ^•    λ _pix = 1.0 (baseline pixel correspondence),

• •    λ adv = 0.1 (realism),

• ^•    λ _perc = 0.05 (texture correction),

• ^•    λ diff = 0.01 (geometry preservation),

• ^•    λ color = 0.05 (color balance).

  £ 28

  

  0.90

  ■Л 0.80

  0.70

  Epoch

  Epoch

  B (Baseline) - 0.098

  A (Adversarial) - 0.082

  P (Perceptual) - 0.068

  D (Diffusion) - 0.058

  C (Color) - 0.052

  Saved best weights (epoch 25)

  
  

  0.85

  
  

  0.75

  
  

  0.225

  0.200

  0.175

  £ 0.150

  0.125

  0.100

  0.075

  0.050

  

  0        5       10       15       20       25       30

  Epoch

  
  
  

  Epoch

  Figure 6. Dynamics of geometric (PSNR, SSIM) and perceptual (LPIPS, FID) metrics on the validation set

  
  

Figure 6 show the dynamics of metric changes on the validation set.

Pixel-wise component ( λ _pix = 1.0 ). This component serves as a basic regularizer, ensuring the preservation of unchanged image regions and approximate brightness correspondence of restored fragments to the ground truth. The value λ _pi_x = 1.0 was chosen as the reference because L _pi_x ( L ₁ loss) has a natural scale comparable to the total contribution of the remaining components. Reducing λ pix to 0.5 leads to noticeable blurring of architectural element boundaries (PSNR decreases by 1.2 dB), while increasing it to 2.0 suppresses the adversarial and perceptual components, making textures overly smoothed.

Adversarial component (λadv = 0.1). GAN training tends to be unstable at high adversarial component values. Experimentally, it was found that at Xadv > 0.2 the discriminator converges too quickly, generating vanishing gradients for the generator and leading to mode collapse after just 10-12 epochs. At Xadv < 0.05 the influence of the adversarial component becomes negligible: the model generates geometrically correct but overly smooth textures (LPIPS > 0.07). The value Xadv = 0.1 ensures a stable adversarial equilibrium where the discriminator remains sufficiently «strong» to provide informative gradients but does not suppress the generator. It should be noted that increasing Xadv from 0.05 to 0.1 improves Boundary F1 from 0.89 to 0.93, confirming the importance of the adversarial component for sharp geometric boundaries.

Perceptual component (X _perc = 0.05 ). The perceptual loss, computed on VGG-16 features, has a substantially larger scale compared to the L 1 loss. Direct use of X _perc = 1.0 leads to dominance of this component and the appearance of characteristic artifacts — excessive texturing and «hallucination» of fine details absent in the ground truth. Reducing the coefficient to 0.05 preserves the positive effect of perceptual learning (naturalness of tile and slate textures) without biasing the overall loss function. The ablation study (Table 2) confirms that removing L _perc increases LPIPS by 0.030, while doubling X _perc to 0.1 does not yield significant improvement but slows convergence.

Diffusion component (X diff = 0.01 ). Diffusion enhancement is based on a pretrained DDPM model acting as a «realism expert». The scale of the diffusion loss varies substantially depending on the noise level and current state of the generator. The value X diff = 0.01 was chosen so that the component exerts a stabilizing influence (suppression of high-frequency GAN artifacts, FID reduction) without dominating the pixel-wise and adversarial components. At X diff >  0.05 the model begins to copy texture features of the pretrained diffusion model, leading to excessive smoothing of fine roof details. At X diff <  0.005 the positive effect of diffusion enhancement becomes statistically insignificant.

Color consistency component (Xcolor = 0.05). The color loss, based on a differentiable histogram, has a scale sensitive to image size and the number of histogram bins. For 256 x 256 pixel images, a coefficient of 0.05 is optimal: it effectively suppresses unnatural color shifts (e.g., the appearance of «acidic» hues when restoring terracotta tiles) but does not lead to averaging of color clusters. At λcolor = 0.1, slight «fading» of saturated roofing material colors is observed (reduction in color diversity according to the Colorfulness Index metric [29] by 12%).

Resulting balance. Thus, the chosen coefficients provide a balanced contribution of each component to the overall loss function, as confirmed by the ablation study results: the full configuration achieves PSNR = 34.5 dB, SSIM = 0.972, LPIPS = 0.052, and FID = 17.6 on the validation set, outperforming all incomplete configurations (Table 2) across the combination of geometric and perceptual metrics.

7.    Analysis of Restoration Quality as a Function of Damage Area

To assess the model’s robustness to varying damage scales, the test set of 200 images was divided into three groups: small damage (10–15% of area), medium damage (20–25%), and large damage (30–35%). Small damage accounts for approximately 85% of all images in the test set, while medium and large damage account for 10% and 5%, respectively.

The comparison results with state-of-the-art methods are presented in Table 3. The best values are highlighted in red. As can be seen from the table, Roof-DeGAN consistently outperforms all compared approaches across all damage ranges. For small damage (10–15%), the gain in PSNR is 1.5–9.7 dB compared to traditional methods and 1.5–2.1 dB relative to modern diffusion models (SatDiff, KAO). Meanwhile, SSIM improves to 0.986 versus 0.742–0.958 for competitors, and FID decreases to 18.2 versus 32.7–187.3.

For medium damage (20–25%), the advantage increases: PSNR improvement reaches 2.0–8.3 dB, SSIM rises to 0.971 versus 0.718–0.952, LPIPS decreases to 0.059 versus 0.058–0.328 for competitors, and FID decreases to 22.5 versus 35.8–192.6.

	Table 3. Restoration quality as a function of damage area
Methods	10–15%                 20–25%               30–35% PSNR↑ SSIM↑ LPIPS↓  FID↓    PSNR↑ SSIM↑ LPIPS↓  FID↓    PSNR↑ SSIM↑ LPIPS↓  FID↓
Traditional methods Navier-Stokes [3] PatchMatch [4]	24,18  0,742  0,312  187,3     23,45  0,718  0,328  192,6     22,10  0,682  0,356  205,4 25,92  0,801  0,245  142,8     25,12  0,778  0,262  148,5     23,85  0,741  0,289  162,1
Diffusion models DDPM [8] SatDiff [10] KAO [11]	28,45  0,873  0,118   78,6     27,80  0,852  0,132   82,4     26,15  0,814  0,158   89,7 31,78  0,949  0,068   38,2     30,45  0,941  0,065   41,5     28,92  0,912  0,092   52,3 32,41  0,958  0,061   32,7     31,12  0,952  0,058   35,8     29,67  0,923  0,085   44,1

Hybrid and GAN-based methods

Pix2Pix [6] ESRGAN [7] DeGAN baseline [13]

28,92  0,885  0,142   92,5     28,15  0,865  0,156   98,2     26,78  0,828  0,182  110,6 29,34  0,902  0,098   67,3     28,65  0,882  0,112   72,1     27,40  0,845  0,138   84,9 31,85  0,958  0,082   29,4     31,02  0,948  0,088   33,7     29,78  0,922  0,102   41,2

New method

Roof-DeGAN

33,87 0,986 0,045   18,2     33,42 0,971 0,059   22,5     31,25 0,948 0,074   29,8

216      Igor V. Vinokurov, Georgy M. Lapankov, Georgy D. Umarov     EN RU

Method

M

Figure 7. Comparison of geometric (PSNR, SSIM) and perceptual (LPIPS, FID) metrics of Roof-DeGAN with the best competitors

10-15%

20-25%

30-35%

Roof-DeGAN

DcGAN Method

Particularly revealing is the behavior under large damage (30–35%): the proposed method maintains high quality (PSNR = 31.25 dB, SSIM = 0.948, LPIPS = 0.074, FID = 29.8), whereas traditional methods drop to 22.10–23.85 dB (FID up to 205.4), and pure diffusion models drop to 28.92–29.67 dB (FID up to 44.1–89.7). The PSNR gain over the best competitors (KAO, DeGAN) is 1.6–2.3 dB, SSIM improves by 0.025–0.026, LPIPS is comparable (0.074 vs. 0.058–0.085 for competitors), and FID decreases by 11.3–14.4 units. The comparison results of geometric and perceptual metrics of the Roof-DeGAN model with the best competitors are shown in Figure 7.

Table 4 presents a comparison of methods using the Boundary F1 metric. The best result is highlighted in red.

Table 4. Comparison of methods using the Boundary F1 metric (averaged over all damage types)

Methods		Boundary F1 ↑
Traditional methods	Navier-Stokes [3] PatchMatch [4]	0.38 0.46
	DDPM [8]	0.62
Diffusion models	SatDiff [10]	0.79
	KAO [11]	0.82
	Pix2Pix [6]	0.54
Hybrid and GAN-based methods ESRGAN [7]		0.58
	DeGAN baseline [13]	0.81
New method	Roof-DeGAN	0.91

( a ) Original images

( b ) Successful restoration results

Figure 8. Garden house roofs before and after successful restoration using the proposed method

Examples of successful roof restoration on the PLC «Roscadastr» test dataset are shown in Figure 8.

( a ) Original image

( b ) Unsuccessful restoration results

Figure 9. Example of unsuccessful roof image restoration using the proposed method

For each of the three examples, the original image (partially occluded by tree canopies) and the result produced by the proposed model are shown. The degree of restoration complexity increases from left to right: 10% occlusion (multi-slope roof) — minimal restoration; 20% occlusion (gable roof) — texture and slope geometry restoration; 35% occlusion (multi-slope roof) — significant reconstruction of corners and texture.

For small and medium damage (10–20%), the restoration quality is more than acceptable: slope geometry is preserved, and roofing material texture is reproduced faithfully. For large damage (35% and above), significant distortions of roofing texture and roof shape may be observed in the restored image. The proportion of completely unsuccessful results does not exceed 3–5% of the total number of processed images; in these cases, the quality remains unacceptable for subsequent automated cadastral processing (Figure 9) .

8.    Limitations and future work

Despite the high quantitative and qualitative results, the Roof-DeGAN model has a number of limitations that open up directions for further development.

Dependence on mask quality. The model requires a binary mask M of the damaged region. In real-world conditions, automatic segmentation of occlusions (tree canopies, shadows, equipment) rarely achieves ideal quality. Qualitative analysis shows that with minor segmentation errors (loU € [0.85; 0.95]), the model maintains high robustness— artifacts are primarily localized at mask boundaries and do not affect the internal geometry of slopes due to skip connections and the multi-level discriminator. With substantial mask distortions (loU < 0.8), degradation of restoration quality is observed: «double contour» effects appear, along with local texture distortions and disruption of ridge geometry, as the generator either attempts to restore already visible areas or leaves part of the occlusion untouched.

Image resolution . For processing high-resolution aerial photographs significantly exceeding 256 x 256 pixels, a promising direction is the integration of the proposed Roof-DeGAN architecture with patched inference frameworks such as SAHI (Slicing Aided Hyper Inference). This approach involves splitting the original image and corresponding damage mask M into overlapping patches of fixed resolution, independently processing each patch with the trained model, and then aggregating the results using weighted averaging in overlap regions to minimize boundary artifacts. This strategy preserves inference computational efficiency when working with images of size 1024 x 1024 and larger, ensuring continuity of roofing material textures and geometric consistency of roof architectural elements. As an alternative scaling path, transitioning to more efficient transformer blocks with linear attention complexity (Swin Transformer v2 [30] , EfficientViT [31] , etc.) may be considered.

Representation of rare classes and scenarios . The dataset covers the main types of roofing materials (tiles, metal tiles, slate, bituminous shingles), but inadequately represents rare materials (thatch, copper, slate, green roofs, membrane coverings) and complex weather conditions (snow, rain, shadows from neighboring buildings). Expanding the dataset with synthetic images and applying domain adaptation methods will significantly improve the model’s generalization capability.

Training computational complexity. At the inference stage, the model demonstrates high efficiency (0.15 s per 256 x 256 image on an NVIDIA Tesla A100 SXM GPU). However, training requires approximately 6 hours of compute time on the same hardware. The significant resource intensity of training limits scaling to larger datasets and high-resolution images. Promising optimization directions include knowledge distillation, weight quantization, and replacing the base architecture with lighter convolutional networks [28].

Limitations of diffusion enhancement . The diffusion component increases generation stability and texture realism, but also increases computational load and in some cases leads to slight smoothing of fine details with very dense masks. Further optimization (working in latent space or reducing the number of diffusion steps) will eliminate this drawback.

Sensitivity to damage localization . Although the model demonstrates high average boundary restoration accuracy according to the Boundary F1 metric (Table 4) , this result is achieved under conditions where damage masks are predominantly located in central areas of slopes (which corresponds to 85% of the test set). With large damage (30–35%) affecting roof boundaries, accuracy drops to 0.68–0.74 (Figure 9) . This limitation is related to the fact that skip connections and cross-scale attention cannot convey geometry if the entire boundary is damaged. A promising solution is the integration of 3D roof wireframes from the ZRG dataset into the training process.

Limited applicability . Experimental validation of the model was conducted exclusively on PLC «Roscadastr» data. The obtained results may not generalize to aerial photographs taken under different conditions. To expand the model’s applicability domain, additional validation on other datasets is required, along with model fine-tuning if necessary.

In the current version of the study, the target dataset is limited to 2000 images. To further improve the model’s generalization capability, expansion of the sample to 5000+ images is planned, including rare types of roofing materials and natural occlusions (cloudiness, seasonal vegetation changes). Future development directions also include the use of aerial photograph time series (restoration across multiple dates) and the transition to three-dimensional roof geometry restoration.

9.    Conclusion

In the course of this work, a hybrid generative model Roof-DeGAN was developed for restoring occluded areas of building roofs in aerial photographs. The proposed architecture combines transformer blocks for capturing global context, dense convolutional connections for improved feature propagation, and a cross-scale attention mechanism in a multi-level discriminator to enhance training stability.

The main results of the work are as follows:

• ^•    An encoder-decoder generator architecture was developed with transformer blocks featuring dynamic sparse attention, which reduces complexity from O(n 2 ) to Ofak) by adaptively skipping homogeneous image regions.

• ^•    A multi-level discriminator was created that evaluates the plausibility of restored fragments at different scales, improving training stability and texture quality.

• ^•    A two-stage training method was proposed and experimentally validated: pretraining on the ZRG dataset in roof segmentation mode followed by weight transfer to the restoration task. It was shown that this approach provides a PSNR gain of 6.16 dB compared to training from scratch (Table 2) .

• ^•    The high effectiveness of the proposed approach was experimentally confirmed: on the test set, PSNR = 33.7 dB, SSIM = 0.971, LPIPS = 0.048, and FID = 17.8 were achieved, surpassing state-of-the-art methods on the PLC «Roscadastr» dataset (Table 1) . On the test set, the average Boundary F1 metric was 0.91. For damage areas of 10–15%, the value reaches 0.96; for 20–25%, it reaches 0.88; and for 30–35%, it decreases to 0.74 (Table 4) . This confirms that the model reliably restores roof geometry under moderate damage, although boundary accuracy predictably decreases with extensive occlusions.

The obtained results can be used in automated Earth remote sensing data processing systems, in updating cartographic materials, in urban development monitoring and building roof condition assessment tasks, as well as in related fields requiring the restoration of occluded image fragments.

Future research will focus on adapting the developed model to account for temporal vegetation dynamics, integrating data from other spectral ranges, and applying the proposed approach to related tasks: shadow removal, restoration of damaged archival images, and improving the quality of images captured under adverse weather conditions.