圖像紋理合成

In

在

背景 (Background)

In Single Image Super-Resolution, objective metric-based (like Mean Squared Error) CNN models might give good PSNR values but typically produce over-smoothed images, and thereby lack the ability to capture high-frequency features in an image. EnhanceNet is a Generative Adversarial Network which focusses on generating realistic textures along with higher perceptual quality rather than just improving on the PSNR values.

在單圖像超分辨率中，基于客觀指標(如均方誤差)的CNN模型可能會提供良好的PSNR值，但通常會產生過于平滑的圖像，從而缺乏捕獲圖像中高頻特征的能力。 EnhanceNet是一個生成對抗網絡 ，它專注于生成逼真的紋理以及更高的感知質量，而不僅僅是改善PSNR值。

網絡架構 (Network Architecture)

EnhanceNet to generate a 4x Super-resolved ImageEnhanceNet生成4倍超分辨率圖像

• EnhanceNet has a generic CNN architecture with a feed-forward fully convolutional network comprising of 10 residual blocks as they help in faster convergence of a model.

  EnhanceNet具有通用的CNN架構，該架構具有由10個殘差塊組成的前饋全卷積網絡，因為它們有助于更快地收斂模型。

• Instead of the convolution transpose layers, nearest neighbor upsampling followed by a convolution layer is used in the upsampling part of the network to avoid unnecessary artifacts.
  代替卷積轉置層，在網絡的上采樣部分中使用最近的鄰居上采樣，然后是卷積層，以避免不必要的偽像。
• Finally, a bicubic interpolation of the low-resolution image is added to the reconstructed output to avoid any color shifts and to ensure training stability.
  最后，將低分辨率圖像的雙三次插值添加到重建的輸出中，以避免任何顏色偏移并確保訓練穩定性。

損失函數 (Loss Functions)

Versions of EnhanceNet based on combinations of loss functions基于損失函數組合的EnhanceNet版本

These are the pillars of EnhanceNet, the performance of various combinations of these losses were studied.

這些是EnhanceNet的Struts，研究了這些損失的各種組合的性能。

(i) Pixel wise MSE-loss in the image domain (Le)

(i)圖像域的像素級MSE損失( Le )

est: Network Estimated Image; HR: High-Resolution Imageest：網絡估算的圖片； HR：高分辨率圖像

This is a baseline approach loss which is the mean squared error between estimated image and ground-truth.

這是基線進近損耗，它是估計圖像與地面真實性之間的均方誤差。

(ii) Perceptual Loss in feature space (Lp)

(ii)特征空間的感知損失( Lp )

est: Network Estimated Image; HR: High-Resolution Imageest：網絡估算的圖片； HR：高分辨率圖像

Perceptual losses are generally based on generating outputs for the super-resolved image from different layers of the famous VGG network (which is the feature map — φ). In a pre-trained VGG-19 network, mapping an image to the initial convolutional layers’ feature space focusses on high-frequency features (detailed pixel-content) while mapping to the later layers emphasize the primary structure of an image. So, a combination of second and fifth max-pooling layers of the VGG19 network are used to calculate the perceptual loss.

感知損失通常基于從著名VGG網絡(即特征圖φ)的不同層生成的超分辨圖像的輸出。 在預訓練的VGG-19網絡中，將圖像映射到初始卷積層的特征空間集中在高頻特征(詳細的像素內容)上，而映射到后面的層則強調圖像的主要結構。 因此，VGG19網絡的第二和第五最大池化層的組合用于計算感知損失。

(iii) Texture Matching Loss (Lt)

(iii)紋理匹配損耗( Lt )

est: Network Estimated Image; HR: High-Resolution Imageest：網絡估算的圖片； HR：高分辨率圖像

This is based on the style-transfer paper which transfers a texture-style from one painting to another. Patches of the image(16x16 pixels) are used to compute this loss to concentrate on the local texture matching with the high-resolution image. φ represents a feature map generated from the VGG19 network. G(f) is the Gram function which calculates the product of the matrix with its transpose.

這基于樣式轉移紙，該樣式轉移紙將紋理樣式從一幅畫轉移到另一幅畫。 圖像的補丁(16x16像素)用于計算此損失，以集中于與高分辨率圖像匹配的局部紋理。 φ表示從VGG19網絡生成的特征圖。 G(f)是Gram函數，可計算矩陣與其轉置的乘積。

(iv) Adversarial Loss (La)

(iv)對抗損失( La )

This is the typical GAN minimax loss which tries to optimize the performance of both generator and discriminator.

這是典型的 GAN minimax損耗，它試圖優化生成器和鑒別器的性能。

評價 (Evaluation)

4x super-resolved images using different variants of EnhanceNet使用EnhanceNet的不同變體形式的4倍超分辨率圖像 PSNR values for the super-resolved images on different datasets不同數據集上超分辨圖像的PSNR值

• From the above two figures, it can be observed that ENet-PAT perceptually looks the closest to HR image, while the ENet-E baseline-approach image looks very blurry though it seems to produce the highest PSNR values.
  從以上兩個圖可以看出，ENet-PAT在感覺上看起來最接近HR圖像，而ENet-E基線方法圖像看起來非常模糊，盡管它似乎產生最高的PSNR值。
• ENet-P tends to create sharper edges than ENet-E but creates checkered artifacts in place of new texture generation.
  與ENet-E相比，ENet-P傾向于創建更銳利的邊緣，但是會生成方格狀的瑕疵來代替新的紋理生成。
• ENet-PA produces better details but also has high-frequency noise which is not desired.
  ENet-PA可以產生更好的細節，但也具有不希望出現的高頻噪聲。
• Texture loss in ENet-PAT helps it create meaningful local textures, and reduces the noise and artifacts to a large extent.
  ENet-PAT中的紋理損失有助于其創建有意義的局部紋理，并在很大程度上減少噪聲和偽影。

翻譯自: https://medium.com/swlh/enhancenet-single-image-super-resolution-through-automated-texture-synthesis-c0fe252d077d

圖像紋理合成

總結

以上是生活随笔為你收集整理的图像纹理合成_EnhanceNet：通过自动纹理合成实现单图像超分辨率的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。