一种基于注意力嵌入对抗网络的全色锐化方法

AESGGAN: an attention embedded adversarial network for pansharpening

摘要:

全色锐化旨在将低空间分辨率的多光谱图像和高空间分辨率的全色图像进行融合，生成一幅高空间分辨率的多光谱图像.伴随卷积神经网络的发展，涌现出很多基于CNN的全色锐化方法.这些用于全色锐化的CNN模型大都未考虑不同通道特征和不同空间位置特征对最终锐化结果的影响.并且仅使用基于像素的1-范数或2-范数作为损失函数对锐化结果与参考图像进行评估，易导致锐化结果过于平滑，空间细节缺失.为了解决上述问题，本文提出一种嵌入注意力机制，并辅以空间结构信息对抗损失的生成对抗网络模型.该网络模型由2个部分组成：一个生成器网络模型和一个判别器网络模型.嵌入通道注意力机制和空间注意力机制的生成器将低分辨多光谱图像和全色图像融合为高质量的高分辨多光谱图像.判别器以patch-wise判别的方式对锐化结果与参考图像的梯度进行一致性检验，以确保锐化结果的空间细节信息.最后，在3种典型数据集上的对比实验验证了所提出方法的有效性.

Abstract:

Pansharpening aims to fuse low-resolution multispectral image with high-resolution panchromatic image to generate a high-resolution multispectral image. With the development of Convolutional Neural Network (CNN), many CNN-based pansharpening methods have appeared and achieved promising performance. However, most of CNN-based pansharpening methods did not consider that the features in different channel dimensions and spatial dimensions have the different importance to generate a good result. In addition, only L1-norm or L2-norm is used as the loss function in the pixel domain to examine the distortion between the pansharpening results and the reference images, which usually cause the pansharpening results appear overly smooth and lack spatial detail information. In order to address the two problems, the authors proposed an attention embedded adversarial network with spatial structure information adversarial loss. This network consists of two parts: the generator and the discriminator. The channel attention and spatial attention embedded generator fuses low-resolution multispectral image and panchromatic image into a high quality high-resolution multispectral image. In order to ensure the spatial information of pansharpening results, the discriminator verifies the consistency of the gradient of pansharpening results and reference image by a patch-wise way. Finally, comparative experiments on three typical datasets verify the effectiveness of the proposed method.