CycleGAN的原理可以概述為：
將一類圖片轉換成另一類圖片 。也就是說，現在有兩個樣
本空間，X和Y，我們希望把X空間中的樣本轉換成Y空間中
的樣本。（獲取一個數據集的特征，并轉化成另一個數據
集的特征）
這樣來看：實際的目標就是學習從X到Y的映射。我們設這
個映射為F。它就對應著GAN中的 生成器 ，F可以將X中的
圖片x轉換為Y中的圖片F(x)。對于生成的圖片，我們還需要
GAN中的 判別器 來判別它是否為真實圖片，由此構成對抗
生成網絡

在足夠大的樣本容量下，網絡可以將相同的輸入圖像集合
映射到目標域中圖像的任何隨機排列，其中任何學習的映
射可以歸納出與目標分布匹配的輸出分布（即：映射F完全
可以將所有x都映射為Y空間中的同一張圖片，使損失無效
化）。
因此，單獨的對抗損失Loss不能保證學習函數可以
將單個輸入Xi映射到期望的輸出Yi。
對此，作者又提出了所謂的“循環一致性損失”
（cycle consistency loss）。

我們希望能夠把 domain A 的圖片（命名為 a）轉
化為 domain B 的圖片（命名為圖片 b）。
為了實現這個過程，我們需要兩個生成器 G_AB 和
G_BA，分別把 domain A 和 domain B 的圖片進行
互相轉換。
將X的圖片轉換到Y空間后，應該還可以轉換回來。
這樣就杜絕模型把所有X的圖片都轉換為Y空間中的
同一張圖片了
最后為了訓練這個單向 GAN 需要兩個 loss，分別是
生成器的重建 loss 和判別器的判別 loss。
判別 loss：判別器 D_B 是用來判斷輸入的圖片是否
是真實的 domain B 圖片


CycleGAN 其實就是一個 A→B 單向 GAN 加上一個
B→A 單向 GAN。兩個 GAN 共享兩個生成器，然
后各自帶一個判別器，所以加起來總共有兩個判別器
和兩個生成器。
一個單向 GAN 有兩個 loss，而 CycleGAN 加起來
總共有四個 loss。


對顏色、紋理等的轉換效果比較好，對多樣性高的、
多變的轉換效果不好（如幾何轉換）

代碼

import tensorflow as tf import glob from matplotlib import pyplot as plt %matplotlib inline AUTOTUNE = tf.data.experimental.AUTOTUNE import os os.listdir('../input/apple2orange/apple2orange')

imgs_A = glob.glob('../input/apple2orange/apple2orange/trainA/*.jpg')

imgs_B = glob.glob('../input/apple2orange/apple2orange/trainB/*.jpg')

test_A = glob.glob('../input/apple2orange/apple2orange/testA/*.jpg') test_B = glob.glob('../input/apple2orange/apple2orange/testB/*.jpg')

def read_jpg(path):img = tf.io.read_file(path)img = tf.image.decode_jpeg(img, channels=3)return img def normalize(input_image):input_image = tf.cast(input_image, tf.float32)/127.5 - 1return input_image def load_image(image_path):image = read_jpg(image_path)image = tf.image.resize(image, (256, 256))image = normalize(image)return image train_a = tf.data.Dataset.from_tensor_slices(imgs_A) train_b = tf.data.Dataset.from_tensor_slices(imgs_B) test_a = tf.data.Dataset.from_tensor_slices(test_A) test_b = tf.data.Dataset.from_tensor_slices(test_B) BUFFER_SIZE = 200 train_a = train_a.map(load_image, num_parallel_calls=AUTOTUNE).cache().shuffle(BUFFER_SIZE).batch(1) train_b = train_b.map(load_image, num_parallel_calls=AUTOTUNE).cache().shuffle(BUFFER_SIZE).batch(1) test_a = test_a.map(load_image, num_parallel_calls=AUTOTUNE).cache().shuffle(BUFFER_SIZE).batch(1) test_b = test_b.map(load_image, num_parallel_calls=AUTOTUNE).cache().shuffle(BUFFER_SIZE).batch(1) data_train = tf.data.Dataset.zip((train_a, train_b)) data_test = tf.data.Dataset.zip((test_a, test_b)) plt.figure(figsize=(6, 3)) for img, musk in zip(train_a.take(1), train_b.take(1)):plt.subplot(1,2,1)plt.imshow(tf.keras.preprocessing.image.array_to_img(img[0]))plt.subplot(1,2,2)plt.imshow(tf.keras.preprocessing.image.array_to_img(musk[0]))

實例歸一化

!pip install tensorflow_addons import tensorflow_addons as tfa OUTPUT_CHANNELS = 3 def downsample(filters, size, apply_batchnorm=True): # initializer = tf.random_normal_initializer(0., 0.02)result = tf.keras.Sequential()result.add(tf.keras.layers.Conv2D(filters, size, strides=2, padding='same',use_bias=False))if apply_batchnorm:result.add(tfa.layers.InstanceNormalization())result.add(tf.keras.layers.LeakyReLU())return result def upsample(filters, size, apply_dropout=False): # initializer = tf.random_normal_initializer(0., 0.02)result = tf.keras.Sequential()result.add(tf.keras.layers.Conv2DTranspose(filters, size, strides=2,padding='same',use_bias=False))result.add(tfa.layers.InstanceNormalization())if apply_dropout:result.add(tf.keras.layers.Dropout(0.5))result.add(tf.keras.layers.ReLU())return result def Generator():inputs = tf.keras.layers.Input(shape=[256,256,3])down_stack = [downsample(64, 4, apply_batchnorm=False), # (bs, 128, 128, 64)downsample(128, 4), # (bs, 64, 64, 128)downsample(256, 4), # (bs, 32, 32, 256)downsample(512, 4), # (bs, 16, 16, 512)downsample(512, 4), # (bs, 8, 8, 512)downsample(512, 4), # (bs, 4, 4, 512)downsample(512, 4), # (bs, 2, 2, 512)downsample(512, 4), # (bs, 1, 1, 512)]up_stack = [upsample(512, 4, apply_dropout=True), # (bs, 2, 2, 1024)upsample(512, 4, apply_dropout=True), # (bs, 4, 4, 1024)upsample(512, 4, apply_dropout=True), # (bs, 8, 8, 1024)upsample(512, 4), # (bs, 16, 16, 1024)upsample(256, 4), # (bs, 32, 32, 512)upsample(128, 4), # (bs, 64, 64, 256)upsample(64, 4), # (bs, 128, 128, 128)]# initializer = tf.random_normal_initializer(0., 0.02)last = tf.keras.layers.Conv2DTranspose(OUTPUT_CHANNELS, 4,strides=2,padding='same',activation='tanh') # (bs, 256, 256, 3)x = inputs# Downsampling through the modelskips = []for down in down_stack:x = down(x)skips.append(x)skips = reversed(skips[:-1])# Upsampling and establishing the skip connectionsfor up, skip in zip(up_stack, skips):x = up(x)x = tf.keras.layers.Concatenate()([x, skip])x = last(x)return tf.keras.Model(inputs=inputs, outputs=x) generator_x = Generator() # a——>o generator_y = Generator() # o——>a #tf.keras.utils.plot_model(generator, show_shapes=True, dpi=64) def Discriminator(): # initializer = tf.random_normal_initializer(0., 0.02)inp = tf.keras.layers.Input(shape=[256, 256, 3], name='input_image')down1 = downsample(64, 4, False)(inp) # (bs, 128, 128, 64)down2 = downsample(128, 4)(down1) # (bs, 64, 64, 128)down3 = downsample(256, 4)(down2) # (bs, 32, 32, 256)zero_pad1 = tf.keras.layers.ZeroPadding2D()(down3) # (bs, 34, 34, 256)conv = tf.keras.layers.Conv2D(512, 4, strides=1,use_bias=False)(zero_pad1) # (bs, 31, 31, 512)norm1 = tfa.layers.InstanceNormalization()(conv)leaky_relu = tf.keras.layers.LeakyReLU()(norm1)zero_pad2 = tf.keras.layers.ZeroPadding2D()(leaky_relu) # (bs, 33, 33, 512)last = tf.keras.layers.Conv2D(1, 4, strides=1)(zero_pad2) # (bs, 30, 30, 1)return tf.keras.Model(inputs=inp, outputs=last) discriminator_x = Discriminator() # discriminator a discriminator_y = Discriminator() # discriminator o #tf.keras.utils.plot_model(discriminator, show_shapes=True, dpi=64) loss_object = tf.keras.losses.BinaryCrossentropy(from_logits=True) def discriminator_loss(disc_real_output, disc_generated_output):real_loss = loss_object(tf.ones_like(disc_real_output), disc_real_output)generated_loss = loss_object(tf.zeros_like(disc_generated_output), disc_generated_output)total_disc_loss = real_loss + generated_lossreturn total_disc_loss def generator_loss(disc_generated_output):gan_loss = loss_object(tf.ones_like(disc_generated_output), disc_generated_output)return gan_loss LAMBDA = 7 def calc_cycle_loss(real_image, cycled_image):loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))return LAMBDA * loss1 generator_x_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5) generator_y_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5) discriminator_x_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5) discriminator_y_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5) def generate_images(model, test_input):prediction = model(test_input, training=True)plt.figure(figsize=(15,15))display_list = [test_input[0], prediction[0]]title = ['Input Image', 'Predicted Image']for i in range(2):plt.subplot(1, 2, i+1)plt.title(title[i])# getting the pixel values between [0, 1] to plot it.plt.imshow(display_list[i] * 0.5 + 0.5)plt.axis('off')plt.show() @tf.function def train_step(image_a, image_b):with tf.GradientTape(persistent=True) as tape:fake_b = generator_x(image_a, training=True)cycled_a = generator_y(fake_b, training=True)fake_a = generator_y(image_b, training=True)cycled_b = generator_x(fake_a, training=True)disc_real_a = discriminator_x(image_a, training=True)disc_real_b = discriminator_y(image_b, training=True)disc_fake_a = discriminator_x(fake_a, training=True)disc_fake_b = discriminator_y(fake_b, training=True)gen_x_loss = generator_loss(disc_fake_b)gen_y_loss = generator_loss(disc_fake_a)total_cycle_loss = (calc_cycle_loss(image_a, cycled_a) + calc_cycle_loss(image_b, cycled_b))# 總生成器損失 = 對抗性損失 + 循環損失。total_gen_x_loss = gen_x_loss + total_cycle_losstotal_gen_y_loss = gen_y_loss + total_cycle_lossdisc_x_loss = discriminator_loss(disc_real_a, disc_fake_a)disc_y_loss = discriminator_loss(disc_real_b, disc_fake_b)# 計算生成器和判別器損失。generator_x_gradients = tape.gradient(total_gen_x_loss, generator_x.trainable_variables)generator_y_gradients = tape.gradient(total_gen_y_loss, generator_y.trainable_variables)discriminator_x_gradients = tape.gradient(disc_x_loss, discriminator_x.trainable_variables)discriminator_y_gradients = tape.gradient(disc_y_loss, discriminator_y.trainable_variables)# 將梯度應用于優化器。generator_x_optimizer.apply_gradients(zip(generator_x_gradients, generator_x.trainable_variables))generator_y_optimizer.apply_gradients(zip(generator_y_gradients, generator_y.trainable_variables))discriminator_x_optimizer.apply_gradients(zip(discriminator_x_gradients,discriminator_x.trainable_variables))discriminator_y_optimizer.apply_gradients(zip(discriminator_y_gradients,discriminator_y.trainable_variables)) def fit(train_ds, test_ds, epochs):for epoch in range(epochs+1):for img_a, img_b in train_ds:train_step(img_a, img_b)print ('.', end='')if epoch % 5 == 0:print()for test_a, test_b in test_ds.take(1):print("Epoch: ", epoch)generate_images(generator_x, test_a)generate_images(generator_x, test_a) EPOCHS = 100 fit(data_train, data_test, EPOCHS)

總結

以上是生活随笔為你收集整理的GAN生成对抗网络-CycleGAN原理与基本实现-图像转换-10的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。