信號處理深度學習機器學習

機器學習性能與兩種關鍵信號處理算法(快速傅里葉變換和最小均方預測)的有趣對比。 (A fun comparison of machine learning performance with two key signal processing algorithms — the Fast Fourier Transform and the Least Mean Squares prediction.)

Signal processing has given us a bag of tools that have been refined and put to very good use in the last fifty years. There is autocorrelation, convolution, Fourier and wavelet transforms, adaptive filtering via Least Mean Squares (LMS) or Recursive Least Squares (RLS), linear estimators, compressed sensing and gradient descent, to mention a few. Different tools are used to solve different problems, and sometimes, we use a combination of these tools to build a system to process signals.

信號處理為我們提供了一整套工具，這些工具在過去的五十年中已經完善并得到了很好的使用。 有自相關，卷積，傅立葉和小波變換，通過最小均方(LMS)或遞歸最小二乘(RLS)進行自適應濾波，線性估計器，壓縮感測和梯度下降等。 使用不同的工具來解決不同的問題，有時，我們結合使用這些工具來構建用于處理信號的系統。

Machine Learning, or the deep neural networks, is much simpler to get used to because the underlying mathematics is fairly straightforward regardless of what network architecture we use. The complexity and the mystery of neural networks lie in the amount of data they process to get the fascinating results we currently have.

機器學習(即深度神經網絡)要習慣得多，因為無論我們使用哪種網絡架構，基礎數學都非常簡單明了。 神經網絡的復雜性和奧秘在于它們處理的數據量，以獲得我們目前擁有的迷人結果。

時間序列預測 (Time Series Prediction)

This article is an effort to compare the performance of a neural network for a few key signal processing algorithms. Let us look at time series prediction as the first example. We will implement a three layer sequential deep neural network to predict the next sample of a signal. We will also do it the traditional way by using a tap delay filter and adapting the weights based on the mean square error — this is the LMS filtering, an iterative approach to the optimal Weiner filter for estimating signal from noisy measurement. We will then compare the prediction error between the two methods. So, let us get started with writing the code!

本文旨在比較幾種關鍵信號處理算法的神經網絡性能。 讓我們將時間序列預測作為第一個示例。 我們將實現一個三層順序的深度神經網絡，以預測信號的下一個樣本。 我們還將通過使用抽頭延遲濾波器并根據均方誤差調整權重來實現傳統方法-這是LMS濾波 ，這是一種針對最優Weiner濾波器的迭代方法，用于從噪聲測量中估計信號。 然后，我們將比較兩種方法之間的預測誤差。 因此，讓我們開始編寫代碼！

Let us first import all the usual python libraries we need. Since we are going to be using the TensorFlow and Keras framework, we will import them too.

讓我們首先導入我們需要的所有常用python庫。 由于我們將使用TensorFlow和Keras框架，因此我們也將其導入。

神經網絡預測 (Prediction with Neural Networks)

Let us start building our 3 layer Neural network now. The input layer takes 64 samples and produces 32 samples. The hidden layer maps these 32 outputs from the first layer to 8 samples. The final layer maps these 8 samples in to 1 predicted output. Remember that the input size is provided with the input_shape parameter in the first layer.

現在讓我們開始構建3層神經網絡。 輸入層獲取64個樣本，并產生32個樣本。 隱藏層將這32個輸出從第一層映射到8個樣本。 最后一層將這8個樣本映射為1個預測輸出。 請記住，輸入大小隨第一層的input_shape參數提供。

We will use the Adam optimizer without bothering about what it is. That is the benefit of TensorFlow, we don’t need to know every detail about all the processing required for neural network to build one using this amazing framework. If we find out that the Adam optimizer doesn’t work as well, we will simply try another optimizer — RMSprop for example.

我們將使用Adam優化器而不用擔心它是什么。 這就是TensorFlow的好處，我們不需要了解神經網絡使用此驚人框架構建一個神經網絡所需的所有處理的每個細節。 如果發現Adam優化器不能正常工作，我們將簡單地嘗試另一個優化器 -例如RMSprop。

Let us now create a time series, a simple superposition of sine waves. We will then add noise to it to mimic a real world signal.

現在讓我們創建一個時間序列，一個正弦波的簡單疊加。 然后，我們將向其添加噪聲以模擬現實世界的信號。

Now that we have the data, let us think about how to feed this data to the neural network for training. We know the network takes 64 samples at the input and produces one output sample. Since we want to train the network to predict the next sample, we want to use the 65th sample as the output label.

現在我們有了數據，讓我們考慮如何將這些數據饋送到神經網絡進行訓練。 我們知道網絡在輸入處獲取64個樣本，并生成一個輸出樣本。 由于我們要訓練網絡以預測下一個樣本，因此我們希望將第65個樣本用作輸出標簽。

The first input set is therefore from sample 0 to sample 63 (the first 64 samples) and the first label is sample 64 (the 65th sample). The second input set can either be a separate set of 64 samples (non-overlapping window), or we can choose to have a sliding window and take 64 samples from sample 1 to sample 64. Let us follow the sliding window approach, just to generate a lot of training data from the time series we have.

因此，第一個輸入集是從樣本0到樣本63(前64個樣本)，而第一個標簽是樣本64(第65個樣本)。 第二個輸入集可以是64個樣本的單獨集合(不重疊的窗口)，也可以選擇有一個滑動窗口并從樣本1到64取64個樣本。讓我們遵循滑動窗口的方法，從我們擁有的時間序列中生成很多訓練數據。

Also note that we are using the noisy samples as the input while using the noiseless data as the label. We want the neural network to predict the actual signal even in presence of noise.

還要注意，我們將噪聲樣本用作輸入，而將無噪聲數據用作標簽。 我們希望神經網絡即使在存在噪聲的情況下也能預測實際信號。

Let us look at the sizes of the time series data and the training data. See that we generated 5000 samples for our time series data, but we created 3935 x 64 = 251840 samples of input data to our neural network.

讓我們看一下時間序列數據和訓練數據的大小。 看到我們為時間序列數據生成了5000個樣本，但是我們為神經網絡創建了3935 x 64 = 251840個輸入數據樣本。

The shape of train_data is the number of input sets x input length. Here, we have 3935 batches of input, each input being 64 samples long.

train_data的形狀是輸入集的數量x輸入長度。 在這里，我們有3935批輸入，每個輸入長64個樣本。

print(y.shape, train_data.shape, train_labels.shape)(5000,) (3935, 64) (3935,)

We are now ready to train the neural network. Let us instantiate the model first. The model summary provides information on how many layers, what is the output shape, and the number of parameters we need to train for this neural network.

現在我們準備訓練神經網絡。 讓我們首先實例化模型。 該模型摘要提供了有關此神經網絡需要訓練的層數，輸出形狀以及參數數量的信息。

For the first layer, we have 64 inputs and 32 outputs. A dense layer implemets the equation y = f(Wx + b), where f is the activation function, W is the weight matrix and b is the bias. We can immediately see that W is a 64 x 32 matrix, and b is a 32 x 1 vector. This gives us 32 x 64 + 32 = 2080 parameters to train for the first layer. The reader can do similar computations to verify the parameters for second and third layer, as an exercise in understanding. After all, you wouldn’t be reading this article unless you are a beginner to Machine Learning and eager to get started :)

對于第一層，我們有64個輸入和32個輸出。 致密層實現等式y = f(Wx + b) ，其中f是激活函數， W是權重矩陣， b是偏差。 我們可以立即看到W是64 x 32的矩陣， b是32 x 1的向量。 這給我們提供了32 x 64 + 32 = 2080個參數來訓練第一層。 讀者可以進行類似的計算以驗證第二層和第三層的參數，作為理解的練習。 畢竟，除非您是機器學習的初學者并且渴望入門，否則您將不會閱讀本文：)

model = dnn_keras_tspred_model()Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 32) 2080
_________________________________________________________________
dense_1 (Dense) (None, 8) 264
_________________________________________________________________
dense_2 (Dense) (None, 1) 9
=================================================================
Total params: 2,353
Trainable params: 2,353
Non-trainable params: 0
_________________________________________________________________

Alright, onward to training then. Since we are using the Keras framework, training is as simple as calling the fit() method. With TensorFlow, we need to do a little more work, but that is for another article.

好吧，那就繼續訓練吧。 由于我們使用的是Keras框架，因此培訓就像調用fit()方法一樣簡單。 使用TensorFlow，我們需要做更多的工作，但這是另一篇文章。

Let us use 100 epochs, which just means that we will use the same training data again and again to train the neural network and will do so 100 times. In each epoch, the network uses the number of batches of input and label sets to train the parameters.

讓我們使用100個紀元 ，這意味著我們將一次又一次地使用相同的訓練數據來訓練神經網絡，并且將進行100次。 在每個紀元中，網絡使用輸入和標簽集的批次數量來訓練參數。

Let us use the datetime to profile how long this training takes, and the history value that is returned as a Python Dictionary to get the validation loss after each epoch.

讓我們使用日期時間來描述該訓練需要多長時間，并使用歷史記錄值作為Python字典返回，以獲取每個時期后的驗證損失。

DNN training done. Time elapsed: 10.177171 s

Now that the network is trained, and we see that the validation loss has decreased over epochs to the point that it has flattened out (indicating further training doesn’t yield any significant improvements), let us use this network to see how well it performs against test data.

既然已經對網絡進行了培訓，并且我們看到驗證損失已經減少了幾步，直到達到平坦為止(表明進一步的培訓不會帶來任何明顯的改善)，讓我們使用該網絡來查看其效果如何根據測試數據。

Let us create the test data set exactly the same way as we created the training data sets, but use only that part of the time series that we have not used for training before. We want to surprise the neural network with data it has not seen before to know how well it can perform.

讓我們創建測試數據集的方式與創建訓練數據集的方式完全相同，但是只使用時間序列中我們從未用于訓練的那部分。 我們想用之前從未見過的數據讓神經網絡感到驚訝，以了解其性能如何。

We will now call the predict() method in Keras framework to get the outputs of the neural network for the test data set. This step is different if we use the TensorFlow framework, but we will cover that in another article.

現在，我們將在Keras框架中調用predict()方法，以獲取測試數據集的神經網絡輸出。 如果我們使用TensorFlow框架，則此步驟有所不同，但我們將在另一篇文章中介紹。

As we see, the prediction from neural network is very close to the actual noise free data!

如我們所見，來自神經網絡的預測非常接近實際的無噪聲數據！

LMS算法的預測 (Prediction with LMS algorithm)

We will use a L=64 tap filter to predict the next sample. We don’t need that large a filter, but let us keep the number of inputs per output sample same as what we used for neural network.

我們將使用L = 64抽頭濾波器來預測下一個樣本。 我們不需要那么大的濾波器，但讓我們將每個輸出樣本的輸入數量保持與用于神經網絡的數量相同。

The filter coefficients (or weights) are obtained by computing the error between predicted and measured sample, and adjusting the weights based on the correlation between mean square error and the input measurements.

濾波器系數(或權重)是通過計算預測樣本和測量樣本之間的誤差，并基于均方誤差和輸入測量值之間的相關性來調整權重而獲得的。

As you see in the code, yrlms[k] is the filter output when the inputs are ypn[k-L:k], the error is computed as the difference between the noisy measured value ypn[k] and the filter output yrlms[k]. The correlation between measurement and error is given by the product of ypn[k-L:k] and e, and mu is the LMS step size (or learning rate).

如您在代碼中看到的， yrlms [k]是輸入為ypn [kL：k]時的濾波器輸出，誤差計算為噪聲測量值ypn [k]與濾波器輸出yrlms [k]之差 。 測量和誤差之間的相關性由ypn [kL：k]與e的乘積給出，而mu是LMS步長(或學習率)。

As we see, the LMS prediction is equally good, despite having much lower complexity.

如我們所見，盡管LMS預測的復雜度要低得多，但它同樣好。

(64,) (1064,)

LMS和神經網絡之間的預測結果比較 (Comparing Prediction Results between LMS and Neural Network)

Before we close this section, let us compare the error between LMS prediction and the neural network prediction. To be fair, I ignored the initial portion of the LMS to give it time to converge when measuring the mean square error and SNR. Despute that, we see that the neural network performance is 5 dB better than the LMS performance!

在結束本節之前，讓我們比較LMS預測和神經網絡預測之間的誤差。 公平地說，我忽略了LMS的初始部分，以便在測量均方誤差和SNR時有時間收斂。 對此，我們看到神經網絡性能比LMS性能好5 dB！

Neural network SNR: 19.986311477279084
LMS Prediction SNR: 14.93359076022336

快速傅立葉變換 (Fast Fourier Transform)

Alright, a neural network beat LMS by 5 dB in signal prediction, but let us see if a neural network can be trained to do the Fourier Transform. We will compare it to the FFT (Fast Fourier Transform) from SciPy FFTPack. The FFT algorithm is at the heart of signal processing, can the neural network be trained to mimic that too? Let us find out…

好了，神經網絡在信號預測方面比LMS勝5 dB，但讓我們看看是否可以訓練神經網絡進行傅立葉變換。 我們將其與SciPy FFTPack的FFT(快速傅立葉變換)進行比較。 FFT算法是信號處理的核心，是否可以訓練神經網絡來模仿它？ 讓我們找出……

We will use the same signal we created before, the superposition of sine waves, to evaluate FFT as well. Let us look at the FFT ouput first.

我們將使用之前創建的相同信號(正弦波的疊加)來評估FFT。 首先讓我們看一下FFT輸出。

Let us create a neural network model to mimic the FFT now. In contrast to the model we created before where we have 64 inputs but only one output, this model needs to generate 64 outputs for every 64 sample input set.

讓我們創建一個神經網絡模型來模擬FFT。 與之前創建的具有64個輸入但只有一個輸出的模型相比，該模型需要為每64個樣本輸入集生成64個輸出。

And since FFT inputs and outputs are complex, we need twice the number of samples at the input, arranged as real followed by imaginary. Since the outputs are also complex, we again 2 x NFFT samples.

而且由于FFT的輸入和輸出很復雜，因此我們需要在輸入處添加兩倍數量的采樣，按實數排列，然后按虛數排列。 由于輸出也很復雜，因此我們再次進行2 x NFFT采樣。

To train this neural network model, let us use random data generated using numpy.random.normal and set the labels based on the FFT routine from the SciPy FFTPack that we are comparing with.

為了訓練該神經網絡模型，讓我們使用使用numpy.random.normal生成的隨機數據，并根據與之比較的SciPy FFTPack中的FFT例程設置標簽。

The rest of the code is fairly similar to the previous neural network training. Here, I am running 10,000 batches at a time, and I have an outer for loop to do multiple sets of 10,000 batches if the network needs more training. Note that this needs the model to be created outside the for loop, so that the weights are not reinitialized.

其余代碼與先前的神經網絡訓練非常相似。 在這里，我一次運行10,000個批次，并且如果網絡需要更多培訓，我有一個外部for循環可以執行多組10,000個批次。 請注意，這需要在for循環之外創建模型，以便不會重新初始化權重。

See from model summary that there are almost 50,000 parameters for just a 64 point FFT. We can reduce this a bit since we are only evaluating real inputs while keeping the imaginary parts as zero, but the goal here is to quickly compare if the neural network can be trained to do the Fourier Transform.

從模型摘要中可以看出，僅64點FFT就有近50,000個參數。 我們可以減少一點，因為我們只評估實數輸入，而將虛部保持為零，但是這里的目標是快速比較神經網絡是否可以訓練以進行傅立葉變換。

Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_3 (Dense) (None, 128) 16512
_________________________________________________________________
dense_4 (Dense) (None, 128) 16512
_________________________________________________________________
dense_5 (Dense) (None, 128) 16512
=================================================================
Total params: 49,536
Trainable params: 49,536
Non-trainable params: 0
_________________________________________________________________
DNN training done. Time elapsed: 30.64511 s

Training is done. Let us now test the network using the same input samples we created for LMS. We compare the neural network output to the FFT ouput and they are identical! How amazing is that!

培訓完成。 現在讓我們使用為LMS創建的相同輸入樣本來測試網絡。 我們將神經網絡輸出與FFT輸出進行比較，它們是相同的！ 那太神奇了！

Let us do one last evaluation before we conclude this article. We will compare the neural network output with the FFT output for some random input data, and see how the mean square error and SNR looks like.

在結束本文之前，讓我們做最后一個評估。 對于某些隨機輸入數據，我們將神經網絡輸出與FFT輸出進行比較，并查看均方誤差和SNR的樣子。

Running the code below, we get a decent 23.64 dB SNR. While we do see some samples every now and then where the error is high, for most part, the error is very small. Given that we trained the neural network for only 10,000 batches, this is a pretty good result!

運行下面的代碼，我們得到不錯的23.64 dB SNR。 雖然我們時不時會看到一些樣本，但誤差很大，但在大多數情況下，誤差很小。 鑒于我們只訓練了10,000個批次的神經網絡，這是一個相當不錯的結果！

Neural Network SNR compared to SciPy FFT: 23.64254974707859

摘要 (Summary)

Being stuck inside during Covid-19, it was a fun weekend project to compare machine learning performance to some key signal processing algorithms. We see that machine learning can do what signal processing can, but has inherently higher complexity, with the benefit of being generalizable to different problems. The signal processing algorithms are optimal for the job in terms of complexity, but are specific to the particular problems they solve. We can’t use FFT in place of LMS or vice versa, while we can use the same neural network processor, and just load a different set of weights to solve a different problem. That is the versatility of neural networks.

被困在Covid-19大會期間，這是一個有趣的周末項目，旨在將機器學習性能與一些關鍵信號處理算法進行比較。 我們看到機器學習可以完成信號處理所能完成的工作，但是固有地具有更高的復雜度，其優點是可以推廣到不同的問題。 信號處理算法在復雜性方面最適合該工作，但是特定于它們解決的特定問題。 我們不能使用FFT代替LMS，反之亦然，而我們可以使用相同的神經網絡處理器，而只是加載不同的權重集來解決不同的問題。 那就是神經網絡的多功能性。

And with that note, I’ll conclude this article. I hope you had as much fun reading this as I had putting this together. Please leave your feedback too if you found it helpful and learnt a thing or two!

有了這個注釋，我將結束本文。 希望您閱讀這些內容和我整理這些內容一樣開心。 如果您覺得有幫助并且學到了一兩件事，也請留下您的反饋！

翻譯自: https://towardsdatascience.com/machine-learning-and-signal-processing-103281d27c4b

信號處理深度學習機器學習

總結

以上是生活随笔為你收集整理的信号处理深度学习机器学习_机器学习与信号处理的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。