當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

数据集准备及数据预处理_1.准备数据集

發布時間：2023/12/15 编程问答 34 豆豆

生活随笔收集整理的這篇文章主要介紹了数据集准备及数据预处理_1.准备数据集小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

數據集準備及數據預處理

In this post I will introduce an example of how to upgrade legacy tensorflow codes to new style. tensorflow 2.0 recommends using Keras which is a well known high level API. Their sequential style approach makes your code more simple. And the script would be worked line by line, so we can easily understand the working process.

在這篇文章中，我將介紹一個如何將舊版tensorflow代碼升級為新樣式的示例。 tensorflow 2.0建議使用Keras，這是眾所周知的高級API。他們的順序樣式方法使您的代碼更簡單。而且腳本將逐行工作，因此我們可以輕松地了解工作過程。

Let me introduce a guide by my BEGAN repository

讓我介紹一下BEGAN存儲庫中的指南

1.準備數據集 (1. Prepare a dataset)

For preparing the dataset, at the previous version, I made the ImageIterator class which provides preprocessing and iterator. Obviously, it is not quite hard, cos we can find a kind guide on the homepage for that. But it looks cumbersome. In tensorflow 2.0 you don’t have to make that kind of class, we just use ImageDataGenerator which is a predefined API for data generation. We can set parameters for preprocessing and data augmentation when we create ImageDataGenerator. I gave some parameters for normalization and horizontal flipping as data augmentation as shown in below.

為了準備數據集，在以前的版本中，我制作了ImageIterator 類，其提供預處理和迭代器。顯然，這并不難，因為我們可以在主頁上找到一種類似的指南。但這看起來很麻煩。在tensorflow 2.0中，您不必創建此類，我們只使用ImageDataGenerator ，這是用于數據生成的預定義API。創建ImageDataGenerator時，可以設置用于預處理和數據擴充的參數。我提供了一些參數進行標準化和水平翻轉，作為數據增強，如下所示。

2.設計一個網絡。 (2. Design a network.)

The began network consists of a decoder and an encoder. In the previous repository, to implement a Convolution Layer, I used 3 low level functions as shown in below.

初始網絡由解碼器和編碼器組成。在先前的存儲庫中，為了實現卷積層，我使用了3個低級函數，如下所示。

And I also predefined some parameters about layer weights.

我還預定義了一些有關圖層權重的參數。

So, someone used to define a custom function for reducing code lines and better readability.

因此，曾經有人定義了一個自定義函數以減少代碼行并提高可讀性。

But, with Keras, I only needed just a line.

但是，有了Keras，我只需要一條線。

The high level API includes parameters on how to define activation and bias. So we don’t have to make any custom function. We just use predefined Keras APIs. I could design the decoder network using about 20 lines. Awesome!!

高級API包含有關如何定義激活和偏差的參數。因此，我們不必進行任何自定義功能。我們只使用預定義的Keras API。我可以使用大約20條線設計解碼器網絡。太棒了!!

Before training the network we have to make a model which takes inputs and returns outputs. Keras provides an API for that. Look at the codes to feel how simple it is.

在訓練網絡之前，我們必須先建立一個模型，該模型接受輸入并返回輸出。 Keras為此提供了一個API。查看代碼以了解它有多簡單。

It is interesting I didn’t write any lines for defining variable scope like previous codes. It works automatically on the Keras API.

有趣的是，我沒有寫任何行來定義變量范圍，如先前的代碼。它可以在Keras API上自動運行。

3.訓練和測試模型 (3. Train and Test a Model)

3.1 Train

3.1火車

What I really love about Keras is I don’t have to use sess.run anymore. I think that the old style process makes codes quite complex. Look at the flow, we can easily follow these lines how this is gonna work.

我真正喜歡Keras的地方是我不再需要使用sess.run 。我認為舊樣式的流程使代碼變得相當復雜。看一下流程，我們可以很容易地遵循以下原則。

The model only had been trained for 5 hours on a Nvidia 1080TI. But the generator could show quite impressive outputs.

該模型僅在Nvidia 1080TI上訓練了5個小時。但是生成器可能會顯示出令人印象深刻的輸出。

3.2 Test

3.2測試

Look at the super tiny test codes. There’s no excuse for not updating your legacy repository.

看一下超小的測試代碼。沒有任何理由不更新舊版存儲庫。

I also updated my DCGAN repository. Visit the repository, switch branches and check updated codes. I derived help from a tutorial that introduced the tensorflow homepage. I would like to recommend you read the tutorial too.

我還更新了DCGAN存儲庫。訪問存儲庫，切換分支并檢查更新的代碼。我從介紹tensorflow主頁的教程中獲得了幫助。我也建議您閱讀本教程。

翻譯自: https://medium.com/@fabulousjeong/updating-old-tensorflow-codes-to-new-tensorflow-2-0-style-ed4dca5f42f