參照代碼理解,比較直觀：

# These are the weights between the input layer and the hidden layer.self.weights_0_1 = np.zeros((self.input_nodes,self.hidden_nodes))# These are the weights between the hidden layer and the output layer.self.weights_1_2 = np.random.normal(0.0, self.output_nodes**-0.5, (self.hidden_nodes, self.output_nodes))# The input layer, a two-dimensional matrix with shape 1 x input_nodesself.layer_0 = np.zeros((1,input_nodes))

一共三層：weights_0_1[input_nodes,hidden_nodes],weights_1_2[hidden_nodes,output_nodes]

#### Implement the forward pass here ####### Forward pass #### Input Layerself.update_input_layer(review)# Hidden layerlayer_1 = self.layer_0.dot(self.weights_0_1)# Output layerlayer_2 = self.sigmoid(layer_1.dot(self.weights_1_2))

輸入和權重點乘：weights中每一列對應的是一個節點和前一層所有節點的權重
[1,input_nodes] · [input_nodes,hidden_nodes] = [1,hidden_nodes]

#### Implement the backward pass here ####### Backward pass #### Output errorlayer_2_error = layer_2 - self.get_target_for_label(label) # Output layer error is the difference between desired target and actual output.layer_2_delta = layer_2_error * self.sigmoid_output_2_derivative(layer_2)# Backpropagated errorlayer_1_error = layer_2_delta.dot(self.weights_1_2.T) # errors propagated to the hidden layerlayer_1_delta = layer_1_error # hidden layer gradients - no nonlinearity so it's the same as the error

反向計算誤差：
先有輸出層誤差，關注如何從i層誤差到i-1層誤差：
經過激活函數，無法相當于經過一個放大器：layer_2_error * self.sigmoid_output_2_derivative(layer_2)
i-1誤差，考慮一個節點，誤差來自于i層所有節點誤差加權求和，
比如weights_1_2[hidden_nodes,output_nodes]，求第1個節點誤差，對應的權重weights_1_2[1,output_nodes]，即第一行，所有計算方式為：layer_1_error = layer_2_delta.dot(self.weights_1_2.T)

# Update the weightsself.weights_1_2 -= layer_1.T.dot(layer_2_delta) * self.learning_rate # update hidden-to-output weights with gradient descent stepself.weights_0_1 -= self.layer_0.T.dot(layer_1_delta) * self.learning_rate # update input-to-hidden weights with gradient descent step

得到了每一層的誤差delta之后，下面就考慮如何更新權重？
從現有數據怎么得到$?E$/$?W$?

$V^k$為第k層的值，未添加激活函數的k層節點值

前半部分通過反向傳播一個一個計算,已經計算出來了。
[hidden_nodes,1] · [1,output_nodes] = [hidden_nodes,output_nodes]

layer_2_delta = layer_2_error * self.sigmoid_output_2_derivative(layer_2) self.weights_1_2 -= layer_1.T.dot(layer_2_delta) * self.learning_rate

采取向量的方式理解也比較直觀

$?ER/?W^{(l)}={\delta }^{(l+1)}(a^{(l)}{)}^T$
${\delta }^{(l)}=((W^{(l)}{)}^T{\delta }^{(l+1)})°f\prime (a^{(l)})$這個公式網上好多地方寫錯了.

總結

以上是生活随笔為你收集整理的Backpropagation 总结的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。