文章目錄

• 基類Layer的實現：
• 激活層的實現:
• CostLayer的實現：

基類Layer的實現：

前面已經提到過一個layer的包含：shape，激活函數，梯度的處理以及輸出層的處理。

import numpy as npclass Layer:def __init__(self, shape):self.shape = shapedef _activate(self, x, predict):passdef __str__(self):return self.__class__.__name__def __repr__(self):return str(self)@propertydef name(self):return str(self)# v^i = f^(u^i) , y is v^kdef derivative(self, y):pass# forward pass : XW + bdef activate(self, x, w, bias):return x.dot(w) + bias# y = v^i, w = w^i, prev_delta = delta^i+1# f' = f *(1-f), so if y is v^i, it is easy to derivate# 反向傳播計算誤差，非輸出層的處理def bp(self, y, w, pre_delta):return pre_delta.dot(w.T)*self.derivative(y)

激活層的實現:

class Sigmoid(Layer):def _activate(self, x):return 1 / (1 + np.exp(-x))def derivative(self, y):return y*(1-y)

CostLayer的實現：

涉及到２個部分處理，一是損失的計算方式，而是特殊的變換，還有最后一層的誤差需要單獨計算。

# ============================================================================= # 沒有激活函數、但可能會有特殊的變換函數（比如說 Softmax），同時還需要定義某個損失函數 # 定義導函數時，需要考慮到自身特殊的變換函數并計算相應的、整合后的梯度 # ============================================================================= # 輸出層有兩個功能：一是特殊的變換，二是計算最后一層的梯度，最后一層的梯度需要特殊處理 class CostLayer(Layer):"""初始化結構self._available_cost_functions：記錄所有損失函數的字典self._available_transform_functions：記錄所有特殊變換函數的字典self._cost_function、self._cost_function_name：記錄損失函數及其名字的兩個屬性self._transform_function 、self._transform：記錄特殊變換函數及其名字的兩個屬性"""def __init__(self, shape, cost_function="MSE", transform=None):super(CostLayer, self).__init__(shape)self._available_cost_functions = {"MSE": CostLayer._mse,"SVM": CostLayer._svm,"CrossEntropy": CostLayer._cross_entropy}self._available_transform_functions = {"Softmax": CostLayer._softmax,"Sigmoid": CostLayer._sigmoid}self._cost_function_name = cost_functionself._cost_function = self._available_cost_functions[cost_function]if transform is None and cost_function == "CrossEntropy":self._transform = "Softmax"self._transform_function = CostLayer._softmaxelse:self._transform = transformself._transform_function = self._available_transform_functions.get(transform,None)def __str__(self):return self._cost_function_namedef _activate(self, x, predict):if self._transform_function is None:return xreturn self._transform_function(x)def _derivative(self, y, delta=None):pass@staticmethoddef safe_exp(x):return np.exp(x - np.max(x,axis=1,keepdims=True))# 特殊變換函數@staticmethoddef _softmax(y, diff=False):if diff:return y*(1-y)exp_y = CostLayer.safe_exp(y)return exp_y / np.sum(exp_y, axis=1, keepdims=True)@staticmethoddef _sigmoid(y, diff=False):if diff:return y * (1 - y)return 1 / (1 + np.exp(-y))# 單獨計算輸出層的誤差def bp_first(self, y, y_pred):if self._cost_function_name == "CrossEntropy" and (self._transform == "Softmax" or self._transform =="Sigmoid"):return y - y_pred # 否則、就只能用普適性公式進行計算：# （沒有特殊變換函數）# （有特殊變換函數）dy = -self._cost_function(y, y_pred)if self._transform_function is None:return dyreturn dy * self._transform_function(y_pred, diff=True)@propertydef calculate(self):return lambda y, y_pred: self._cost_function(y,y_pred,False)# 損失函數@staticmethoddef _mse(y, y_pred, diff=True):if diff:return y_pred - yreturn 0.5*np.average((y-y_pred)**2)@staticmethoddef _cross_entropy(y, y_pred, diff=True, eps=1e-8):if diff:return -y / (y_pred + eps) + (1-y)/ (1- y_pred +eps)return np.average(-y * np.log(y_pred + eps) - (1 - y) * np.log(1 - y_pred + eps))

總結

以上是生活随笔為你收集整理的深度学习：tensorflow Layers的实现，numpy实现深度学习(二)的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。