無監督學習 k-means

有關深層學習的FAU講義 (FAU LECTURE NOTES ON DEEP LEARNING)

These are the lecture notes for FAU’s YouTube Lecture “Deep Learning”. This is a full transcript of the lecture video & matching slides. We hope, you enjoy this as much as the videos. Of course, this transcript was created with deep learning techniques largely automatically and only minor manual modifications were performed. Try it yourself! If you spot mistakes, please let us know!

這些是FAU YouTube講座“ 深度學習 ”的 講義 。 這是演講視頻和匹配幻燈片的完整記錄。 我們希望您喜歡這些視頻。 當然，此成績單是使用深度學習技術自動創建的，并且僅進行了較小的手動修改。 自己嘗試！ 如果發現錯誤，請告訴我們！

導航 (Navigation)

Previous Lecture / Watch this Video / Top Level / Next Lecture

上一個講座 / 觀看此視頻 / 頂級 / 下一個講座

Welcome back to deep learning! Today, we want to continue talking about unsupervised methods and look into a very popular technique that is the so-called autoencoder.

歡迎回到深度學習！ 今天，我們要繼續談論無監督方法，并研究一種非常流行的技術，即所謂的自動編碼器。

An example for “auto”-encoding shapes (pun intended). Image created using gifify. Source: YouTube一個“自動”編碼形狀的示例(用于雙關語)。 使用gifify創建的圖像 。 資料來源： YouTube

So, here are our slides. Part two of our lecture and the topic autoencoders. Well, the concept of the autoencoder is that we want to use the ideas of feed-forward neural networks. You could say that a feed-forward neural network is a function of xthat produces some encoding y.

所以，這是我們的幻燈片。 我們的講座的第二部分和主題自動編碼器。 好吧，自動編碼器的概念是我們要使用前饋神經網絡的思想。 您可以說前饋神經網絡是x的函數，它產生一些編碼y 。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Now, the problem is: “How can we generate a loss in such a constellation?” The idea is rather simple. We add an additional layer here on top and the purpose of the layer is to compute decoding. So, we have another layer that is g(y). g(y) produces some x hat. The loss that we can then define is that x hat and x need to be the same. So the autoencoder tries to learn an approximation of the identity. Well, that sounds rather simple. To be honest, if we have exactly the same number of nodes in the input and in the hidden layer for y here, then the easiest solution would probably be the identity. So why is this useful at all?

現在的問題是：“我們如何在這樣的星座中產生損失？” 這個想法很簡單。 我們在此處添加一個附加層，該層的目的是計算解碼。 因此，我們還有另一個層g( y )。 g( y )產生一些x帽。 我們可以定義的損失是x hat和x必須相同。 因此，自動編碼器嘗試了解身份的近似值。 好吧，聽起來很簡單。 老實說，如果我們在輸入中和在y的隱藏層中具有完全相同數量的節點，那么最簡單的解決方案可能就是身份。 那么，這到底為什么有用呢？

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Well, let’s look at some loss functions. What you can typically use is a loss function that then operates here on x and some x’. It can be proportional to a negative log-likelihood function where you have p(x|x’) and resulting functions. Then, in a similar way, as we’ve seen earlier in this class, you can use the squared L2 norm where you assume your probability density of a function to be a normal distribution with uniform variance. Then, you end up with the L2 loss. It is simply xminus x’ and this encapsulated in an L2 norm. Of course, you can also do things like cross-entropy. So, if you assume the Bernoulli distribution, you see that we end up exactly with our cross-entropy. This is simply the sum over the weighted xtimes the logarithm of x’ subscript i plus 1-x subscript i times the logarithm of 1 — x’ subscript i. Remember that if you want to use it this way, then your x’s need to be in the range of probabilities. So, if you want to apply this kind of loss function, then you may want to use it in combination with a softmax function.

好吧，讓我們看一些損失函數。 通常可以使用的是損失函數，然后在此處對x和某些x '進行運算。 它可以與負對數似然函數成比例，其中您有p( x | x ')和所得函數。 然后，以類似的方式，就像我們在本課程前面所看到的那樣 ，您可以使用平方L2范數，在此假設函數的概率密度為具有均方差的正態分布。 然后，您最終會遭受L2損失。 它就是x減去x '，并封裝在L2范數中。 當然，您也可以做交叉熵之類的事情。 因此，如果您假設伯努利分布，您會發現我們恰好以交叉熵結束。 這僅是加權x乘以x '下標i的對數加上1- x下標i乘以1 — x '下標i的對數的總和。 請記住，如果您想以這種方式使用它，則x必須在概率范圍內。 因此，如果您想應用這種損失函數，則可能希望將其與softmax函數結合使用。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Ok, so here are some typical strategies to construct such autoencoders. I think one of the most popular ones is the undercomplete autoencoder. So here, you enforce information compression by using fewer neurons in the hidden layer. You try to find a transform that does essentially a dimensionality reduction to the hidden layer. Then, you try to expand from this hidden layer onto the original data domain and try to find a solution that produces a minimum loss. So, you try to learn compression here. By the way, if you do this with linear layers and squared L2 norm, you essentially learn a principal component analysis (PCA). If you use it with nonlinear layers, you end up with something like a nonlinear PCA generalization.

好的，這是構造此類自動編碼器的一些典型策略。 我認為最流行的一種是不完整的自動編碼器。 因此，在這里，您可以通過在隱藏層中使用較少的神經元來實施信息壓縮。 您嘗試找到一種實質上減少了隱藏層維數的變換。 然后，您嘗試從該隱藏層擴展到原始數據域，并嘗試找到產生最小損失的解決方案。 因此，您嘗試在此處學習壓縮。 順便說一句，如果您使用線性層和平方L2范數執行此操作，則實際上您將學習主成分分析(PCA)。 如果將其與非線性層一起使用，則最終會出現類似非線性PCA泛化的情況。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

There are also things like these sparse autoencoders. Here, we have a different idea. We even increase the number of neurons, to resemble a one-hot encoded vector. So, you may say: “Why would you increase the number of your neurons? Then, you could even find a much simpler solution like the identity and neglect a couple of those neurons!” So, this idea will not work straightforwardly. You have to enforce sparsity which is also coining the name sparse autoencoder. Here, you have to enforce sparsity in the activations using some additional regularization. For example, you can do this with an L1 norm on the activations in y. Remember, the sparsity in the sparse autoencoder stems from sparsity in the activations, not from sparsity in the weights. If you look at your identity, then you see this is simply a diagonal matrix with ones on the diagonal. So, this would also be a very sparse solution. So, again enforce the sparsity on the activations, not on the weights.

也有類似這些稀疏自動編碼器的東西。 在這里，我們有一個不同的想法。 我們甚至增加了神經元的數量，類似于一個熱編碼的載體。 因此，您可能會說：“為什么要增加神經元的數量？ 然后，您甚至可以找到一種更簡單的解決方案，例如身份驗證，而忽略其中的兩個神經元！” 因此，這個想法不會直接起作用。 您必須實施稀疏性，這也造就了稀疏自動編碼器的名稱。 在這里，您必須使用一些其他正則化方法來強制激活稀疏性。 例如，您可以對y中的激活使用L1范數來執行此操作。 請記住，稀疏自動編碼器中的稀疏性源自激活中的稀疏性，而不是權重中的稀疏性。 如果您看一下自己的身份，那么您會看到這只是一個對角矩陣，對角線上有一個。 因此，這也是一個非常稀疏的解決方案。 因此，再次對激活而不是權重實施稀疏性。

Sparse autoencoding in action. Image created using gifify. Source: YouTube有效的稀疏自動編碼。 使用gifify創建的圖像 。 資料來源： YouTube

What else can be done? Well, you can use autoencoder variations. You can combine it essentially with all the recipes we’ve learned so far in this class. You can build convolutional autoencoders. There, you replace the fully connected layers with convolutional layers and you can optionally also add pooling layers.

還有什么可以做的？ 好吧，您可以使用自動編碼器變體。 您可以將它與我們在課堂上到目前為止學到的所有食譜結合起來。 您可以構建卷積自動編碼器。 在那里，您可以用卷積層替換完全連接的層，還可以選擇添加池化層。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

There is the denoising autoencoder which is also a very interesting concept. There you corrupt the input with noise and the target is then the noise-free original sample. So, this then results in a trained system that does not just do dimensionality reduction or finding a sparse representation. At the same time, it also performs denoising and you could argue that this is an additional regularization that is similar to dropout but essentially applied to the input layers. There’s also a very interesting paper called noise2noise where they show that you can even build such denoising auto-encoders even if you have a noisy target. The key here then is that you have a different noise pattern in the input and in the target and this way you can also train a denoising autoencoder at least if you build on top of convolutional autoencoders.

有降噪自動編碼器，這也是一個非常有趣的概念。 在那里，您會用噪聲破壞輸入，然后目標就是無噪聲的原始樣本。 因此，這導致了一個訓練有素的系統，該系統不僅減少了維數或發現了稀疏表示。 同時，它還會執行去噪，您可能會說這是一個附加的正則化，類似于丟包，但實際上應用于輸入層。 還有一篇非常有趣的論文，稱為noise2noise ，它們表明即使您有嘈雜的目標，您也可以構建這種去噪自動編碼器。 然后，這里的關鍵是您在輸入和目標中具有不同的噪聲模式，并且至少在基于卷積自動編碼器的情況下，這種方式也可以訓練去噪自動編碼器。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Now, you can even go as far as using the denoising autoencoder as a generative model. The idea here is that if you sample your corruption model, your noise model frequently. Then, the denoising autoencoder learns the probability of the respective input. So, if you have x as a typical sample, then by iteratively applying the noise and the denoising will reproduce this sample very often. So, you can then use a Markov chain and you alternate the denoising model and the corruption process. This then leads to an estimator of p(x). To be honest, this is often expensive and it’s very hard to assess the convergence which is the reason why so-called variational autoencoders are much more common. We will talk about those variational autoencoders in a couple of slides.

現在，您甚至可以將去噪自動編碼器用作生成模型。 這里的想法是，如果您對損壞模型進行采樣，那么您的噪聲模型就會頻繁出現。 然后，去噪自動編碼器學習各個輸入的概率。 因此，如果您將x作為典型樣本，則通過迭代應用噪聲和去噪將非常頻繁地重現該樣本。 因此，您可以使用馬爾可夫鏈，然后交替使用降噪模型和破壞過程。 然后，這導致了p( x )的估計。 老實說，這通常很昂貴，并且很難評估收斂性，這就是所謂的變分自動編碼器更為普遍的原因。 我們將在幾張幻燈片中討論那些可變自動編碼器。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

There are some variations that we need to introduce first. There’s the so-called stacked autoencoder. In the stacked autoencoder, you essentially put an autoencoder inside of an autoencoder. This allows us to build essentially deep autoencoders. We see this here in the concept. Autoencoder 1 is using in the hidden layer the Autoencoder 2 which is indicated by the blue nodes. You can stack those into a single autoencoder.

首先需要引入一些變體。 有所謂的堆疊式自動編碼器。 在堆疊式自動編碼器中，您實際上是將自動編碼器放在自動編碼器內部。 這使我們能夠構建本質上很深的自動編碼器。 我們在概念中看到了這一點。 自動編碼器1在隱藏層中使用由藍色節點表示的自動編碼器2。 您可以將它們堆疊到單個自動編碼器中。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

This gives rise to the following model and then the idea is to have a gradual dimensionality reduction. You already have guessed this: this is also something that is very commonly used together with convolutions and pooling.

這產生了以下模型，然后的想法是逐步減小維數。 您已經猜到了：這也是卷積和池化一起非常常用的東西。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Now let’s go into the concept of the variational autoencoders. This is a bit of a different concept. In the traditional autoencoders, you try to compute a deterministic feature vector that describes the attributes of the input in some kind of latent space. So let’s say, you had a latent space that characterizes different features, for example, of a face. Then, you could argue that this encoder then takes the image and projects it onto some probably unknown latent attributes. Here, we give them some additional interpretation. So for every attribute, you have a certain score and from the score then you generate the original image again. The key difference in variational auto-encoders is that you use a variational approach to learn the latent representation. It allows you to describe the latent space in a probabilistic manner.

現在讓我們進入變分自動編碼器的概念。 這有點不同。 在傳統的自動編碼器中，您嘗試計算確定性特征向量，該特征向量描述某種潛在空間中輸入的屬性。 假設您有一個潛在的空間，可以表征不同的特征，例如面部。 然后，您可能會爭辯說，此編碼器隨后會拍攝圖像并將其投影到一些可能未知的潛在屬性上。 在這里，我們給他們一些附加的解釋。 因此，對于每個屬性，您都有一定的分數，然后從該分數中再次生成原始圖像。 可變自動編碼器的主要區別在于，您使用可變方法來學習潛在表示。 它允許您以概率方式描述潛在空間。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So the idea is, it’s not just a simple category per dimension but instead you want to describe a probability distribution for each dimension. So, if we only had the regular autoencoders, you would see the scaling on the left-hand side here per image. The variable smile would be a discrete value, where you select one point where more or less smiling is available. In contrast, the variational autoencoder allows you to describe a probability distribution over the property “smile”. You can see here that in the first row there is not so much smiling. The second row — the Mona Lisa — you’re not sure whether she is smiling or not. People have been arguing over this for centuries. You can see that the variational autoencoder is able to describe this uncertainty by adding a variance over this latent variable. So, we can see we’re not sure whether this is a smile or not. Then, we have very clear instances of smiling, and then, of course, the distribution has a much lower standard deviation.

因此，想法是，這不僅是每個維度的簡單類別，而且您想描述每個維度的概率分布。 因此，如果我們只有常規的自動編碼器，則在每個圖像的左側都會看到縮放比例。 可變笑容將是一個離散值，您可以在其中選擇一個或多或少可以使用笑容的點。 相反，變分自動編碼器允許您描述屬性“微笑”的概率分布。 您可以在這里看到在第一行中沒有那么多微笑。 第二行-蒙娜麗莎-您不確定她是否在微笑。 數百年來，人們一直在為此爭論。 您會看到，通過在此潛在變量上添加方差，變分自動編碼器能夠描述這種不確定性。 因此，我們可以看到我們不確定這是否是微笑。 然后，我們有非常明顯的微笑實例，然后，當然，分布具有低得多的標準偏差。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So how would this work then? Well, you have some encoder that maps your input onto the latent attribute and the decoder is then sampling this distribution in order to produce the final output. So, this means that we have the representation as a probability distribution that enforces a continuous and smooth latent space representation. Similar latent space vectors should correspond to similar reconstructions.

那么，這將如何工作？ 好吧，您有一些編碼器將輸入映射到潛在屬性，然后解碼器對該分布進行采樣以產生最終輸出。 因此，這意味著我們將表示形式作為概率分布來實施連續且平滑的潛在空間表示。 相似的潛在空間向量應對應于相似的重構。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Well, the assumption here is that we have some hidden latent variable z that generates some observation x. Then you can train the variational autoencoder by determining the distribution of z. Then, the problem is that the computation of this distribution p(z|x) is usually intractable. So, we somehow have to approximate our p(z|x) by a tractable distribution. This means that the tractable distribution q then leads to the problem that we have to determine the parameters of q. For example, you can use a Gaussian distribution for this. Then, you can use this to define Kullback-Leibler Divergence between P(z|x) and q(z|x). This is equivalent to maximizing the reconstruction likelihood of the expected value of the logarithm of p(x|z) minus the KL divergence between q(z|x) and p(z). This forces q(z|x) to be similar to the true prior distribution p(z).

好吧，這里的假設是，我們有一些隱藏的潛在變量z生成了一些觀測值x。 然后，您可以通過確定z的分布來訓練變分自動編碼器。 然后，問題在于該分布p( z | x )的計算通常是棘手的。 因此，我們必須以某種易處理的分布來近似逼近我們的p( z | x )。 這意味著可處理的分布q然后導致我們必須確定q的參數的問題。 例如，您可以為此使用高斯分布。 然后，您可以使用它來定義P( z | x )和q( z | x )之間的Kullback-Leibler散度。 這等效于最大化p( x | z )對數的期望值減去q( z | x )和p( z )之間的KL散度的重構可能性。 這迫使q( z | x )類似于真實的先驗分布p( z )。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So now, p(z) is often assumed to be the isotropic Gaussian distribution. Determining now q(z|x) boils down to estimating the parameter vectors μ and σ. So, we do this using a neural network. Who might have guessed that? We estimate our q(z|x) and p(x|z). So here, you see the general outline. Again, you see this autoencoder structure and we have this encoder q(z|x) that produces the latent space representation and then our p(x|z) that produces again our output x’. x’ is supposed to be similar to x.

因此，現在通常假定p( z )是各向同性的高斯分布。 現在確定q( z | x )歸結為估計參數向量μ和σ 。 因此，我們使用神經網絡來做到這一點。 誰會猜到呢？ 我們估計q( z | x )和p( x | z )。 因此，在這里，您將看到總體輪廓。 再次，您會看到這種自動編碼器結構，并且我們有一個編碼器q( z | x )生成潛在空間表示，然后是我們的p( x | z )再次生成我們的輸出x '。 x '應該類似于x 。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So, let’s look at this in some more detail. You can see that in the encoder branch, we essentially have a classic home feed-forward neural network that reduces the dimensionality. What we do in the following is we introduce one key change. This is indicated here in light red and light green. The key idea here now is that in the specific layer that is indicated by the colors, we change the interpretation of the activations. In those neurons, they’re essentially just feed-forward layers, but we interpret the top two neurons here as means of our latent distribution and the bottom two neurons as variances of the latent distributions. Now, the key problem is that in order to go to the next layer, we have to sample from those distributions in order to get the actual observations and perform the decoding on them. So how can this potentially be done? Well, we have a problem here because we cannot backpropagate through the random sampling process.

因此，讓我們更詳細地看一下。 您可以看到，在編碼器分支中，我們本質上有一個經典的前饋神經網絡，它可以降低維數。 下面我們要做的是介紹一個關鍵的更改。 在此以淺紅色和淺綠色指示。 現在的關鍵思想是，在由顏色指示的特定層中，我們更改激活的解釋。 在這些神經元中，它們本質上只是前饋層，但我們在這里將頂部的兩個神經元解釋為潛在分布的方式，將底部的兩個神經元解釋為潛在分布的方差。 現在，關鍵問題是，為了進入下一層，我們必須從這些分布中進行采樣，以便獲得實際的觀測值并對其進行解碼。 那么如何才能做到這一點呢？ 好吧，我們這里有一個問題，因為我們無法在隨機采樣過程中反向傳播。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So the idea here is that the encoder produces the means and variances. Then, we produce some z that is a sampling of our function q(z|x). So, this is a random node and the randomness has the problem that we don’t know how to backpropagate for this node. So, the key element that we introduced in the forward-pass here is that we reparameterize z. z is determined deterministically as the μ plus σ times some ε. ε is the random information. It now stems from a random generator that is simply connected to z. For example, you can choose it to be a Gaussian distribution with zero mean and unit variance. So, this then allows us to backpropagate into μ and σ because the random node is here on the right-hand side and we don’t need to backpropagate into the random node. So, by this reparametrization trick, we can even introduce sampling into a network as a layer. So, this is pretty exciting and as you have already guessed. this is also very nice because we can then use this noise in order to generate random samples from this specific distribution. So, we can use the right-hand side of our autoencoder in order to produce new samples. All of this is deterministic in the backpropagation and you can compute your gradients in the same way as we used to. You can apply your backpropagation algorithm as before.

因此，這里的想法是編碼器產生均值和方差。 然后，我們生成一些z ，它是函數q( z | x )的采樣。 因此，這是一個隨機節點，并且隨機性具有我們不知道如何向該節點反向傳播的問題。 因此，我們在前向傳遞中引入的關鍵元素是重新參數化z 。 當μ加σ乘以ε時，確定地確定z 。 ε是隨機信息。 現在，它源自簡單地連接到z的隨機生成器。 例如，您可以將其選擇為均值和單位方差為零的高斯分布。 因此，這使我們可以反向傳播為μ和σ，因為隨機節點在右側，并且我們不需要反向傳播為隨機節點。 因此，通過這種重新參數化技巧，我們甚至可以將采樣作為一個層引入網絡。 因此，這非常令人興奮，正如您已經猜到的那樣。 這也非常好，因為我們可以使用該噪聲以便從此特定分布生成隨機樣本。 因此，我們可以使用自動編碼器的右側來生成新樣本。 所有這些在反向傳播中都是確定性的，您可以像以前一樣計算梯度。 您可以像以前一樣應用反向傳播算法。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So what effect does this have? Well here, we have some latent space visualizations. If you use only the reconstruction loss, you can see that samples are well separated but there is no smooth transition. Obviously, if you only had the KL divergence then you would not be able to describe the original data. So, you need the combination where you’re simultaneously optimizing with respect to the input distributions and the respective KL divergence. So, this is pretty cool because then we can generate new data by sampling from the distributions in the latent space and then you reconstruct with the decoder the diagonal prior enforces independent latent variables

那么這有什么作用呢？ 在這里，我們有一些潛在的空間可視化。 如果僅使用重建損失，則可以看到樣本分離得很好，但是沒有平滑過渡。 顯然，如果僅存在KL散度，則將無法描述原始數據。 因此，您需要組合，同時針對輸入分布和各自的KL散度進行優化。 因此，這很酷，因為這樣我們就可以通過從潛在空間中的分布中采樣來生成新數據，然后使用解碼器重構對角先驗強制獨立的潛在變量

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So, we can encode different factors of variation. Here, we have an example where we smoothly vary the degree of smiling and the head pose. You can see that this kind of disentangling actually works.

因此，我們可以編碼不同的變異因素。 在這里，我們有一個示例，其中我們平穩地改變了微笑的程度和頭部姿勢。 您可以看到這種解開確實有效。

Latent space exploration in variational autoencoders. Image created using gifify. Source: YouTube變分自動編碼器中的潛在空間探索。 使用gifify創建的圖像 。 資料來源： YouTube

So, let’s summarize. The variational autoencoder is a probabilistic model that allows you to generate data from an intractable density. We are able to optimize a variational lower bound instead and this is trained by backpropagation using reparametrization.

因此，讓我們總結一下。 可變自動編碼器是一種概率模型，可讓您根據難以處理的密度生成數據。 我們能夠優化變分下界，這是通過使用重新參數化的反向傳播進行訓練的。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

The pros are that we have a principled approach to generative modeling. The latent space representation can be very useful for other tasks but the cons are that this only maximizes a lower bound of likelihood. So, the samples in standard models are often of lower quality compared to generative adversarial networks. This is still an active area of research. So, in this video, I gave you a very nice wrap up of autoencoders and the important techniques are, of course, the simple autoencoder, the undercomplete autoencoder, the sparse autoencoder, and the stacked autoencoder. Finally, we’ve been looking into variational autoencoders in order to be able to also generate data and in order to be able to describe probability distributions in our latent variable space.

優點是我們有一種生成建模的原則方法。 潛在空間表示對于其他任務可能非常有用，但是缺點是這只會使可能性的下限最大化。 因此，與生成對抗網絡相比，標準模型中的樣本通常質量較低。 這仍然是一個活躍的研究領域。 因此，在此視頻中，我為您提供了很好的自動編碼器包裝，而重要的技術當然是簡單的自動編碼器，欠完善的自動編碼器，稀疏的自動編碼器和堆疊式自動編碼器。 最后，我們一直在研究變分自動編碼器，以便能夠生成數據并能夠描述潛在變量空間中的概率分布。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

In the next video, we will see that data generation is a very interesting task and we can essentially do it from unsupervised data. This is why we will look into the generative adversarial networks. You will see that this is a very interesting technique that has led to widespread use in many different applications in order to use the power of deep neural networks to perform data generation. So, I hope you liked this video. We discussed very important points of unsupervised learning and I think these are state-of-the-art techniques that are widely used. So, if you like this video, please stay tuned and looking forward to meeting you in the next one. Thank you very much. Goodbye!

在下一個視頻中，我們將看到數據生成是一個非常有趣的任務，并且我們可以從無監督的數據中完成它。 這就是為什么我們將研究生成對抗網絡的原因。 您將看到，這是一種非常有趣的技術，已導致在許多不同的應用程序中得到廣泛使用，以便利用深度神經網絡的強大功能執行數據生成。 所以，我希望你喜歡這個視頻。 我們討論了無監督學習的重要方面，我認為這些是被廣泛使用的最新技術。 因此，如果您喜歡此視頻，請繼續關注，期待與您在下一個視頻中見面。 非常感謝你。 再見！

If you liked this post, you can find more essays here, more educational material on Machine Learning here, or have a look at our Deep LearningLecture. I would also appreciate a follow on YouTube, Twitter, Facebook, or LinkedIn in case you want to be informed about more essays, videos, and research in the future. This article is released under the Creative Commons 4.0 Attribution License and can be reprinted and modified if referenced. If you are interested in generating transcripts from video lectures try AutoBlog.

如果你喜歡這篇文章，你可以找到這里更多的文章 ，更多的教育材料，機器學習在這里 ，或看看我們的深入 學習 講座 。 如果您希望將來了解更多文章，視頻和研究信息，也歡迎關注YouTube ， Twitter ， Facebook或LinkedIn 。 本文是根據知識共享4.0署名許可發布的 ，如果引用，可以重新打印和修改。 如果您對從視頻講座中生成成績單感興趣，請嘗試使用AutoBlog 。

鏈接 (Links)

Link — Variational Autoencoders: Link — NIPS 2016 GAN Tutorial of GoodfellowLink — How to train a GAN? Tips and tricks to make GANs work (careful, noteverything is true anymore!) Link - Ever wondered about how to name your GAN?

鏈接 —可變自動編碼器： 鏈接 — Goodfellow的NIPS 2016 GAN教程鏈接 —如何訓練GAN？ 使GAN正常工作的提示和技巧(小心，什么都沒了！) 鏈接 -是否想知道如何命名GAN？

翻譯自: https://towardsdatascience.com/unsupervised-learning-part-2-b1c130b8815d

無監督學習 k-means

總結

以上是生活随笔為你收集整理的无监督学习 k-means_无监督学习-第2部分的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。