加粗樣式邊學(xué)邊記錄，感謝莫煩大神的教學(xué)視頻，獲益良多，之前已經(jīng)看完了吳恩達(dá)Andrew Ng的視頻，但對Tensorflow的使用還是有很多不懂的地方，還是要花些時間好好學(xué)學(xué)tensorflow和keras。

---

莫煩大神的視頻學(xué)習(xí)地址

---

Tensorflow 基礎(chǔ)框架

基于tensorflow構(gòu)建的三層（單隱層）神經(jīng)網(wǎng)絡(luò)如下圖所示：

上圖中，圓形和正方形的節(jié)點被稱為node，在node間流動的數(shù)據(jù)流（多維數(shù)組）被稱為張量（tensor）。

張量 Tensor ：

階數(shù)說明

0階張量 = 標(biāo)量（Scalar）	也就是一個數(shù)值，如$[1]$
1階張量 = 向量（Vector）	比如一維的$[1,2,3]$
2階張量 = 矩陣（Matrix）	比如$[[1,2,3],[4,5,6],[7,8,9]]$
$\dots$	$\dots$
n階張量 = n維數(shù)組

tensor和node之間關(guān)系：

如果輸入tensor的維度是$5000\times 64$，表示有5000個訓(xùn)練樣本，每個樣本有64個特征，則輸入層必須有64個node來接受這些特征。

---

例子

創(chuàng)建數(shù)據(jù)訓(xùn)練函數(shù)，目標(biāo)是要獲得公式y(tǒng) = 0.1*x+0.3中的0.1和0.3這兩個值，即0.1為權(quán)重weight和0.3為偏執(zhí)bias

import tensorflow as tf import numpy as np # create data x_data=np.random.rand(100).astype(np.float32) y_data=x_data*0.1+0.3### create tensorflow structure start## #參數(shù)定義（一維,第二個是取值范圍的下限，第三個參數(shù)是取值范圍的上限） Weights = tf.Variable(tf.random_uniform([1],-1.0,1.0)) biases = tf.Variable(tf.zeros([1]))y = Weights*x_data+biases#計算預(yù)測值和真實值的差值 loss=tf.reduce_mean(tf.square(y-y_data)) optimizer = tf.train.GradientDescentOptimizer(0.5) #learning_rate train= optimizer.minimize(loss)#建立完以上這些變量，還需要在NN中初始化這些變量variables init=tf.initialize_all_variables() ###create tensorflow structure end ###sess = tf.Session() sess.run(init) # Very importantfor step in range(201):sess.run(train)if step % 20 == 0: #每隔20步打印一次print(step,sess.run(Weights),sess.run(biases)) WARNING:tensorflow:From /home/will/anaconda3/envs/py35/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py:118: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Use `tf.global_variables_initializer` instead. 0 [0.36647058] [0.20860067] 20 [0.16722272] [0.25959927] 40 [0.11913844] [0.28849784] 60 [0.10544876] [0.2967253] 80 [0.10155129] [0.2990677] 100 [0.10044166] [0.2997346] 120 [0.10012573] [0.29992446] 140 [0.10003577] [0.29997852] 160 [0.10001019] [0.2999939] 180 [0.10000289] [0.29999828] 200 [0.10000083] [0.2999995]

這時輸出的Weight就很接近1了，bias也很接近0.3

---

Session 會話控制

import tensorflow as tfmatrix1 = tf.constant([[3,3]]) # 一行兩列的matrix matrix2 = tf.constant([[2],[2]]) #兩行一列的matrixproduct = tf.matmul(matrix1,matrix2) #matrix multipy矩陣乘法。而在np中，是np.dot(m1,m2)類似的功能#method 1 #sess = tf.Session() #result = sess.run(product) # tensorflow思考模式：每run一次就會執(zhí)行一次這個結(jié)構(gòu) #print(result) #sess.close() #有跟沒有差不多，有就系統(tǒng)一點#method 2 with tf.Session() as sess: #意思是打開tf.Session并以sess命名它result2 = sess.run(product)print(result2)

---

Variable變量

import tensorflow as tfstate = tf.Variable(0,name='counter') # 只有定義成一個變量才是一個變量，不像Python里的 #print(state.name) one = tf.constant(1) #變量 + 常亮 等于 變量 new_value = tf.add(state,one) update = tf.assign(state,new_value) #將new_value變量加載到state上，所以當(dāng)前state的狀態(tài)等于new_value #這里的state就作為儲存新數(shù)值的載體#如果在tensorflow中設(shè)置了變量，那么接下來這一步很重要 init=tf.initialize_all_variables()#must have if define variable#初始化素有的變量，還需要session.run才能激活 with tf.Session() as sess:sess.run(init)for _ in range(3):sess.run(update)print(sess.run(state)) 1 2 3

只要在tensorflow中設(shè)置了變量，記得要在初始化.

---

Placeholder傳入值

placeholder 是 Tensorflow 中的占位符，暫時儲存變量.

Tensorflow 如果想要從外部傳入data, 那就需要用到 tf.placeholder(), 然后以這種形式傳輸數(shù)據(jù) sess.run(***, feed_dict={input: **})
也就是喂數(shù)據(jù)。

placeholder和variable的區(qū)別：
通俗講：placeholder 是你輸入自己數(shù)據(jù)的接口, variable 是網(wǎng)絡(luò)自身的變量, 通常不是你來進(jìn)行修改, 而是網(wǎng)絡(luò)自身會改動更新.

import tensorflow as tfinput1=tf.placeholder(tf.float32) #tensorflow里大多數(shù)情況只能處理float32的情況 #也可以規(guī)定數(shù)組大小，如兩行兩列的數(shù)組：input1=tf.placeholder(tf.float32,[2,2]) input2=tf.placeholder(tf.float32)output=tf.multiply(input1,input2)with tf.Session() as sess:print(sess.run(output,feed_dict={input1:[7.],input2:[2.]})) #用placeholder跟feed_dict是一個綁定的關(guān)系，用了就需要給它輸入數(shù)據(jù) [14.]

---

添加層 def add_layer()

定義添加神經(jīng)層的函數(shù)def add_layer(),它有四個參數(shù)：輸入值、輸入的大小、輸出的大小和激勵函數(shù)，我們設(shè)定默認(rèn)的激勵函數(shù)是None。

import tensorflow as tfdef add_layer(inputs,in_size,out_size,activation_function=None): #None的情況就是線性激活函數(shù)Weights = tf.Variable(tf.random_normal([in_size,out_size]))biases = tf.Variable(tf.zeros([1,out_size])+0.1) #在機(jī)器學(xué)習(xí)中bias推薦不為0Wx_plus_b = tf.matmul(inputs,Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b)return outputs

---

建造神經(jīng)網(wǎng)絡(luò)

import tensorflow as tf import numpy as npdef add_layer(inputs,in_size,out_size,activation_function=None): #None的情況就是線性激活函數(shù)Weights = tf.Variable(tf.random_normal([in_size,out_size]))biases = tf.Variable(tf.zeros([1,out_size])+0.1) #在機(jī)器學(xué)習(xí)中bias推薦不為0Wx_plus_b = tf.matmul(inputs,Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b)return outputsx_data = np.linspace(-1,1,300)[:,np.newaxis] #-1到1的范圍內(nèi)300個單位,即有300行，300個例子 noise = np.random.normal(0,0.05,x_data.shape) y_data = np.square(x_data) - 0.5 + noisexs = tf.placeholder(tf.float32,[None,1]) #輸入只有1個特征，所以這里是1，輸出也一樣 ys = tf.placeholder(tf.float32,[None,1]) # None的意思是無論輸入多少個例子都o(jì)kl1 = add_layer(xs,1,10,activation_function=tf.nn.relu) prediction = add_layer(l1,10,1,activation_function=None) #構(gòu)建的是——輸入層1個、隱藏層10個、輸出層1個的神經(jīng)網(wǎng)絡(luò)。# 對求和的所有數(shù)求一個平均值，也就是平均的誤差是多少 loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),reduction_indices=[1])) train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss) #learning_rateinit = tf.initialize_all_variables() sess = tf.Session() sess.run(init)for i in range(1000):sess.run(train_step,feed_dict={xs:x_data,ys:y_data})if i % 50 == 0:print(sess.run(loss,feed_dict={xs:x_data,ys:y_data})) #loss也是要用到placeholder，只要用到placeholder都要加上feed_dict 0.53408515 0.0043351008 0.003585369 0.0032615305 0.0030836272 0.0029586344 0.0028774897 0.0028177483 0.0027668795 0.0027297884 0.0026985474 0.0026706108 0.0026462309 0.0026261543 0.0026080583 0.002592326 0.0025791153 0.0025680475 0.0025586013 0.0025521303

訓(xùn)練結(jié)果

---

輸出結(jié)果可視化

優(yōu)化過程中，結(jié)果可視化可以給我們指引優(yōu)化方向

import tensorflow as tf import numpy as np import matplotlib.pyplot as plt # 作為python中輸出結(jié)果可視化的一個模塊 %matplotlib #在jupyter notebook加上這個才能看到動態(tài)圖def add_layer(inputs,in_size,out_size,activation_function=None): #None的情況就是線性激活函數(shù)Weights = tf.Variable(tf.random_normal([in_size,out_size]))biases = tf.Variable(tf.zeros([1,out_size])+0.1) #在機(jī)器學(xué)習(xí)中bias推薦不為0Wx_plus_b = tf.matmul(inputs,Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b)return outputsx_data = np.linspace(-1,1,300)[:,np.newaxis] #-1到1的范圍內(nèi)300個單位,即有300行，300個例子 noise = np.random.normal(0,0.05,x_data.shape) y_data = np.square(x_data) - 0.5 + noisexs = tf.placeholder(tf.float32,[None,1]) #輸入只有1個特征，所以這里是1，輸出也一樣 ys = tf.placeholder(tf.float32,[None,1]) # None的意思是無論輸入多少個例子都o(jì)kl1 = add_layer(xs,1,10,activation_function=tf.nn.relu) prediction = add_layer(l1,10,1,activation_function=None) #構(gòu)建的是——輸入層1個、隱藏層10個、輸出層1個的神經(jīng)網(wǎng)絡(luò)。# 對求和的所有數(shù)求一個平均值，也就是平均的誤差是多少 loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),reduction_indices=[1])) train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss) #learning_rateinit = tf.initialize_all_variables() sess = tf.Session() sess.run(init)fig = plt.figure() ax = fig.add_subplot(1,1,1) ax.scatter(x_data,y_data) plt.ion() #主程序如果不加這個，就會在運行完plt.show()之后暫停 plt.show() for i in range(1000):sess.run(train_step,feed_dict={xs:x_data,ys:y_data})if i % 50 == 0:#print(sess.run(loss,feed_dict={xs:x_data,ys:y_data})) #loss也是要用到placeholder，只要用到placeholder都要加上feed_dicttry:ax.lines.remove(lines[0]) #在圖片里去除lines的第一個線段，lines也只有一個線段，不然會有很多條線，最后密密麻麻的線except Exception:passprediction_value = sess.run(prediction,feed_dict={xs:x_data})lines = ax.plot(x_data,prediction_value,'r-',lw=5) #將prediction的結(jié)果以曲線的形式plot上去，不是點的形式, lw是線的寬度plt.pause(0.1) #plot過程中暫停0.1s

如果是在jupyter notebook上跑的注意了，如果不加%matplotlib只會輸出散點圖，連紅色曲線都沒有，加上這個指令可以另外打開一個窗口顯示。

各種優(yōu)化器

Tensorflow 中的優(yōu)化器會有很多不同的種類。最基本, 也是最常用的一種就是GradientDescentOptimizer。

其實都是優(yōu)化learning_rate

Tensorboard 可視化好幫手

先放圖片，下圖這里一個塊都可以打開，而且塊需要在原來的代碼中加入一些代碼把它們框起來，以及可以命名它們

import tensorflow as tfdef add_layer(inputs,in_size,out_size,activation_function=None): #None的情況就是線性激活函數(shù)with tf.name_scope('layer'): with tf.name_scope('weights'):Weights = tf.Variable(tf.random_normal([in_size,out_size]),name='W')with tf.name_scope('biases'):biases = tf.Variable(tf.zeros([1,out_size])+0.1,name='b') #在機(jī)器學(xué)習(xí)中bias推薦不為0with tf.name_scope('Wx_plus_b'):Wx_plus_b = tf.matmul(inputs,Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b)return outputs#define placeholder for inputs to network #下面這句意思其實就是在tensorboard里看到的inputs層大框架下包含x_input和y_input with tf.name_scope('inputs'): #要縮進(jìn)，別忘了xs = tf.placeholder(tf.float32,[None,1],name='x_input')ys = tf.placeholder(tf.float32,[None,1],name='y_input')#add hidden layer l1 = add_layer(xs,1,10,activation_function=tf.nn.relu) #add output layer prediction = add_layer(l1,10,1,activation_function=None)#the error between prediction and real data with tf.name_scope('loss'): loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),reduction_indices=[1]),name='loss') with tf.name_scope('train'): train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss) #learning_rateinit = tf.initialize_all_variables() sess = tf.Session() writer = tf.summary.FileWriter("logs/",sess.graph) #graph就是把之前編輯的所有信息收集起來放到文件夾里，這里是logs/ #important step sess.run(init)

然后可以在/home/username下看到logs文件夾，然后打開terminal，輸入tensorboard --logdir='logs/'
然后會看到

will@will-450R5G-450R5U:~$ tensorboard --logdir='logs/' TensorBoard 1.10.0 at http://will-450R5G-450R5U:6006 (Press CTRL+C to quit)

按住Ctrl點擊里面的網(wǎng)址就可以打開tensorboard可視化助手查看當(dāng)前的神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)了。

---

如果在jupyter notebook里面重復(fù)運行同一段代碼生成可視化結(jié)果，可能會報錯
這時把jupyter notebook的kernel重啟把里面的緩存數(shù)據(jù)清空（最簡單就是把整個jupyter notebook重啟），再把/logs目錄下的結(jié)果清空，再打開jupyter notebook運行代碼就可以了。

---

可視化Weights、bias及l(fā)oss等數(shù)據(jù)

這個的代碼運行完了，只能看到graph的內(nèi)容，看不到histogram的內(nèi)容和event的內(nèi)容，奇怪，不知道什么情況，有哪位大神知道的，望指教

from __future__ import print_function import tensorflow as tf import numpy as np#這里加多了一個參數(shù)，n_layer是添加層的名字 def add_layer(inputs, in_size, out_size, n_layer, activation_function=None):# add one more layer and return the output of this layerlayer_name = 'layer%s' % n_layer #從這里可以看出with tf.name_scope(layer_name):with tf.name_scope('weights'):Weights = tf.Variable(tf.random_normal([in_size, out_size]), name='W')#可視化Weights的變化量，第一個參數(shù)是名稱，第二參數(shù)是輸入tf.summary.histogram(layer_name + '/weights', Weights)with tf.name_scope('biases'):biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, name='b')#可視化biases的變化量tf.summary.histogram(layer_name + '/biases', biases)with tf.name_scope('Wx_plus_b'):Wx_plus_b = tf.add(tf.matmul(inputs, Weights), biases)if activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b, )#可視化outputs的變化量tf.summary.histogram(layer_name + '/outputs', outputs)return outputs# Make up some real data x_data = np.linspace(-1, 1, 300)[:, np.newaxis] noise = np.random.normal(0, 0.05, x_data.shape) y_data = np.square(x_data) - 0.5 + noise# define placeholder for inputs to network with tf.name_scope('inputs'):xs = tf.placeholder(tf.float32, [None, 1], name='x_input')ys = tf.placeholder(tf.float32, [None, 1], name='y_input')# add hidden layer l1 = add_layer(xs, 1, 10, n_layer=1, activation_function=tf.nn.relu) # add output layer prediction = add_layer(l1, 10, 1, n_layer=2, activation_function=None)# the error between prediciton and real data with tf.name_scope('loss'):loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),reduction_indices=[1]))#可視化loss的變化量，這個很重要，用scalar的話，不會出現(xiàn)在histogram里面，而是出現(xiàn)在event里面#也可以改成在histogram里面顯示，跟上面變量顯示一樣的代碼格式tf.summary.scalar('loss', loss)with tf.name_scope('train'):train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)sess = tf.Session() #給所有訓(xùn)練圖合并，tf.merge_all_summaries()方法會對我們所有的 summaries 合并到一起. merged = tf.summary.merge_all()writer = tf.summary.FileWriter("logs/", sess.graph)init = tf.global_variables_initializer() sess.run(init)for i in range(1000):sess.run(train_step, feed_dict={xs: x_data, ys: y_data})if i % 50 == 0:result = sess.run(merged,feed_dict={xs: x_data, ys: y_data}) #merged也是要運行才能記錄結(jié)果的 writer.add_summary(result, i)

---

高階內(nèi)容

Classification 分類任務(wù)

這里需要MNIST的數(shù)據(jù)集，這個需要翻墻才能下載，一開始不知道老是報錯，數(shù)據(jù)集不大，建議找找資源跑一下，挺有收獲的。
數(shù)據(jù)集下載地址：http://yann.lecun.com/exdb/mnist/
其實運行代碼就會自動下載，不需要手動下載，手動下載需要把壓縮包放在/home/user/MNIST_data/目錄下。

import tensorflow as tf from tensorflow.examples.tutorials.mnist import input_data #number 1 to 10 data #下面這句命令是幫助你下載這個數(shù)據(jù)集 mnist = input_data.read_data_sets('MNIST_data', one_hot=True)def add_layer(inputs, in_size, out_size, activation_function=None,):# add one more layer and return the output of this layerWeights = tf.Variable(tf.random_normal([in_size, out_size]))biases = tf.Variable(tf.zeros([1, out_size]) + 0.1,)Wx_plus_b = tf.matmul(inputs, Weights) + biasesif activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b,)return outputsdef compute_accuracy(v_xs,v_ys):global predictiony_pre = sess.run(prediction,feed_dict={xs:v_xs})#下面開始計算真實值與預(yù)測值之間的誤差correct_prediction = tf.equal(tf.argmax(y_pre,1),tf.argmax(v_ys,1))accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))result = sess.run(accuracy,feed_dict={xs:v_xs,ys:v_ys})return result#define placeholder for inputs to network xs = tf.placeholder(tf.float32,[None,784]) #784表示像素點，28×28 ys = tf.placeholder(tf.float32,[None,10])#add output layer prediction = add_layer(xs,784,10,activation_function=tf.nn.softmax)#the error between prediction and real data cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys*tf.log(prediction),reduction_indices=[1])) #loss train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)sess = tf.Session() #important step sess.run(tf.initialize_all_variables())for i in range(1000):batch_xs,batch_ys = mnist.train.next_batch(100) #從下載好的database提取100個樣本sess.run(train_step,feed_dict={xs:batch_xs,ys:batch_ys})if i%50 ==0:print(compute_accuracy(mnist.test.images,mnist.test.labels))

這一節(jié)沒有跑，之前做Andrew Ng的作業(yè)也跑過一個類似的，識別數(shù)字手勢的。

Dropout解決overfitting

""" Please note, this code is only for python 3+. If you are using python 2+, please modify the code accordingly. """ import tensorflow as tf from sklearn.datasets import load_digits from sklearn.model_selection import train_test_split from sklearn.preprocessing import LabelBinarizer# load data digits = load_digits() X = digits.data y = digits.target y = LabelBinarizer().fit_transform(y) #使y的標(biāo)簽成為one-hot vector，在對應(yīng)位置上顯示1表示對應(yīng)的數(shù)字 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.3) #拆分?jǐn)?shù)據(jù)集def add_layer(inputs, in_size, out_size, layer_name, activation_function=None, ):# add one more layer and return the output of this layerWeights = tf.Variable(tf.random_normal([in_size, out_size]))biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, )Wx_plus_b = tf.matmul(inputs, Weights) + biases# here to dropout，dropout操作Wx_plus_b = tf.nn.dropout(Wx_plus_b, keep_prob)if activation_function is None:outputs = Wx_plus_belse:outputs = activation_function(Wx_plus_b, )tf.summary.histogram(layer_name + '/outputs', outputs)return outputs# define placeholder for inputs to network keep_prob = tf.placeholder(tf.float32) #dropout操作 xs = tf.placeholder(tf.float32, [None, 64]) # 8x8 ys = tf.placeholder(tf.float32, [None, 10])# add output layer l1 = add_layer(xs, 64, 50, 'l1', activation_function=tf.nn.tanh) prediction = add_layer(l1, 50, 10, 'l2', activation_function=tf.nn.softmax)# the loss between prediction and real data cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction),reduction_indices=[1])) # loss tf.summary.scalar('loss', cross_entropy) train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)sess = tf.Session() merged = tf.summary.merge_all() # summary writer goes in here train_writer = tf.summary.FileWriter("logs/train", sess.graph) test_writer = tf.summary.FileWriter("logs/test", sess.graph)# tf.initialize_all_variables() no long valid from # 2017-03-02 if using tensorflow >= 0.12 if int((tf.__version__).split('.')[1]) < 12 and int((tf.__version__).split('.')[0]) < 1:init = tf.initialize_all_variables() else:init = tf.global_variables_initializer() sess.run(init) for i in range(500):# here to determine the keeping probability，0.5表示保留50%的節(jié)點，0.6就保留60%sess.run(train_step, feed_dict={xs: X_train, ys: y_train, keep_prob: 0.5})if i % 50 == 0:# record loss #這里的keep_prob要設(shè)置成1，因為這里是要記錄數(shù)據(jù)，所以要記錄全部的數(shù)據(jù)train_result = sess.run(merged, feed_dict={xs: X_train, ys: y_train, keep_prob: 1}) test_result = sess.run(merged, feed_dict={xs: X_test, ys: y_test, keep_prob: 1})train_writer.add_summary(train_result, i) test_writer.add_summary(test_result, i)

生成的logs/文件夾下有test和train兩個文件夾，terminal輸入tensorboard --logdir='logs/'，然后打開網(wǎng)址，就可以查看dropout的效果，但是很奇怪，test數(shù)據(jù)集好像沒跑成功，只顯示train的效果。

---

CNN部分

關(guān)鍵內(nèi)容：建立卷積層、池化層

import tensorflow as tf from tensorflow.examples.tutorials.mnist import input_data # number 1 to 10 data mnist = input_data.read_data_sets('MNIST_data', one_hot=True)def compute_accuracy(v_xs, v_ys):global predictiony_pre = sess.run(prediction, feed_dict={xs: v_xs, keep_prob: 1})correct_prediction = tf.equal(tf.argmax(y_pre,1), tf.argmax(v_ys,1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))result = sess.run(accuracy, feed_dict={xs: v_xs, ys: v_ys, keep_prob: 1})return resultdef weight_variable(shape):initial = tf.truncated_normal(shape,stddev=0.1) #shape表示生成張量的維度，mean是均值，stddev是標(biāo)準(zhǔn)差return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.1, shape=shape) #創(chuàng)建常量return tf.Variable(initial)def conv2d(x,W):#stride[1,x_movement,y_movement,1] #Must have strides[0]=strides[3]=1,就是上面的第一和第四個數(shù)return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding='SAME') def max_pool_2x2(x):#stride[1,x_movement,y_movement,1] return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')# define placeholder for inputs to network xs = tf.placeholder(tf.float32, [None, 784]) # 28x28 ys = tf.placeholder(tf.float32, [None, 10]) keep_prob = tf.placeholder(tf.float32) #-1代表先不考慮輸入的圖片例子多少這個維度 #后面的1是channel的數(shù)量，因為我們輸入的圖片是黑白的，因此channel是1，例如如果是RGB圖像，那么channel就是3 x_image = tf.reshape(xs,[-1,28,28,1]) # print(x_image.shape) #[n_samples,28,28,1]## conv1 layer ## #卷積核patch/kernel 5×5 #in size 1是image的厚度，因為是黑白圖片channel是1 #out size 32輸出是32個featuremap即32個filters W_conv1 = weight_variable([5,5,1,32]) # b_conv1 = bias_variable([32]) h_conv1 = tf.nn.relu(conv2d(x_image,W_conv1)+b_conv1) #output size 28*28*32 h_pool1 = max_pool_2x2(h_conv1) #output size 14*14*32## conv2 layer ## W_conv2 = weight_variable([5,5,32,64]) # b_conv2 = bias_variable([64]) h_conv2 = tf.nn.relu(conv2d(h_pool1,W_conv2)+b_conv2) #output size 14*14*64 h_pool2 = max_pool_2x2(h_conv2) #output size 7*7*64## func1 layer ## W_fc1 = weight_variable([7*7*64,1024]) b_fc1 = bias_variable([1024])h_pool2_flat = tf.reshape(h_pool2,[-1,7*7*64]) #[n_smaples,7,7,64] ->> [n_samples,7*7*64] h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1) h_fc1_drop = tf.nn.dropout(h_fc1,keep_prob)## func2 layer ## W_fc2 = weight_variable([1024,10]) b_fc2 = bias_variable([10]) prediction = tf.nn.softmax(tf.matmul(h_fc1_drop,W_fc2) + b_fc2)# the error between prediction and real data cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction),reduction_indices=[1])) # loss train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) #這里用Adam優(yōu)化器，不用Gradient Descentsess = tf.Session() # important step # tf.initialize_all_variables() no long valid from # 2017-03-02 if using tensorflow >= 0.12 if int((tf.__version__).split('.')[1]) < 12 and int((tf.__version__).split('.')[0]) < 1:init = tf.initialize_all_variables() else:init = tf.global_variables_initializer() sess.run(init)for i in range(1000):batch_xs, batch_ys = mnist.train.next_batch(100)sess.run(train_step, feed_dict={xs: batch_xs, ys: batch_ys, keep_prob: 0.5})if i % 50 == 0:print(compute_accuracy( mnist.test.images[:1000], mnist.test.labels[:1000]))

運行結(jié)果：

Extracting MNIST_data/train-images-idx3-ubyte.gz Extracting MNIST_data/train-labels-idx1-ubyte.gz Extracting MNIST_data/t10k-images-idx3-ubyte.gz Extracting MNIST_data/t10k-labels-idx1-ubyte.gz 0.124 0.736 0.824 0.88 0.899 0.908 0.92 0.932 0.931 0.94 0.949 0.949 0.944 0.958 0.955 0.956 0.96 0.963 0.959 0.965

測試集的準(zhǔn)確率已經(jīng)很高了。

Saver保存和讀取

參數(shù)保存方法（目前TF只能保存參數(shù)，不能保存整個網(wǎng)絡(luò)結(jié)構(gòu)）：

import tensorflow as tf## Save to file # remember to define the same dtype and shape when restore W = tf.Variable([[1,2,3],[3,4,5]],dtype=tf.float32,name="weights") b = tf.Variable([[1,2,3]],dtype=tf.float32,name="biases") #定義dtype很重要init = tf.global_variables_initializer()saver = tf.train.Saver() #存儲變量with tf.Session() as sess:sess.run(init)save_path = saver.save(sess,"my_net/save_net.ckpt") #第二個是保存路徑print("Save to path:",save_path)

提取方法：

import tensorflow as tf import numpy as np # restore variables # redefine the same shape and same type for your variables #即重新加載參數(shù)需要重定義相同形狀和類型的變量在裝載參數(shù) W = tf.Variable(np.arange(6).reshape((2,3)),dtype=tf.float32,name="weights") b = tf.Variable(np.arange(3).reshape((1,3)),dtype=tf.float32,name="biases")# not need init step saver = tf.train.Saver() with tf.Session() as sess:#提取變量saver.restore(sess,"my_net/save_net.ckpt")print("weights:",sess.run(W))print("biases:",sess.run(b))

如果使用jupyter notebook在同一個窗口使用保存和提取，會報錯，分開使用就可以了。
輸出結(jié)果：

INFO:tensorflow:Restoring parameters from my_net/save_net.ckpt weights: [[1. 2. 3.][3. 4. 5.]] biases: [[1. 2. 3.]]

遷移學(xué)習(xí)transfer learning

""" This is a simple example of transfer learning using VGG. Fine tune a CNN from a classifier to regressor. Generate some fake data for describing cat and tiger length. Fake length setting: Cat - Normal distribution (40, 8) Tiger - Normal distribution (100, 30) The VGG model and parameters are adopted from: https://github.com/machrisaa/tensorflow-vgg Learn more, visit my tutorial site: [莫煩Python](https://morvanzhou.github.io) """from urllib.request import urlretrieve import os import numpy as np import tensorflow as tf import skimage.io import skimage.transform import matplotlib.pyplot as pltdef download(): # download tiger and kittycat imagecategories = ['tiger', 'kittycat']for category in categories:os.makedirs('./for_transfer_learning/data/%s' % category, exist_ok=True)with open('./for_transfer_learning/imagenet_%s.txt' % category, 'r') as file:urls = file.readlines()n_urls = len(urls)for i, url in enumerate(urls):try:urlretrieve(url.strip(), './for_transfer_learning/data/%s/%s' % (category, url.strip().split('/')[-1]))print('%s %i/%i' % (category, i, n_urls))except:print('%s %i/%i' % (category, i, n_urls), 'no image')def load_img(path):img = skimage.io.imread(path)img = img / 255.0# print "Original Image Shape: ", img.shape# we crop image from centershort_edge = min(img.shape[:2])yy = int((img.shape[0] - short_edge) / 2)xx = int((img.shape[1] - short_edge) / 2)crop_img = img[yy: yy + short_edge, xx: xx + short_edge]# resize to 224, 224resized_img = skimage.transform.resize(crop_img, (224, 224))[None, :, :, :] # shape [1, 224, 224, 3]return resized_imgdef load_data():imgs = {'tiger': [], 'kittycat': []}for k in imgs.keys():dir = './for_transfer_learning/data/' + kfor file in os.listdir(dir):if not file.lower().endswith('.jpg'):continuetry:resized_img = load_img(os.path.join(dir, file))except OSError:continueimgs[k].append(resized_img) # [1, height, width, depth] * nif len(imgs[k]) == 400: # only use 400 imgs to reduce my memory loadbreak# fake length data for tiger and cattigers_y = np.maximum(20, np.random.randn(len(imgs['tiger']), 1) * 30 + 100)cat_y = np.maximum(10, np.random.randn(len(imgs['kittycat']), 1) * 8 + 40)return imgs['tiger'], imgs['kittycat'], tigers_y, cat_yclass Vgg16:vgg_mean = [103.939, 116.779, 123.68]def __init__(self, vgg16_npy_path=None, restore_from=None):# pre-trained parameterstry:self.data_dict = np.load(vgg16_npy_path, encoding='latin1').item()except FileNotFoundError:print('Please download VGG16 parameters from here https://mega.nz/#!YU1FWJrA!O1ywiCS2IiOlUCtCpI6HTJOMrneN-Qdv3ywQP5poecM\nOr from my Baidu Cloud: https://pan.baidu.com/s/1Spps1Wy0bvrQHH2IMkRfpg')self.tfx = tf.placeholder(tf.float32, [None, 224, 224, 3])self.tfy = tf.placeholder(tf.float32, [None, 1])# Convert RGB to BGRred, green, blue = tf.split(axis=3, num_or_size_splits=3, value=self.tfx * 255.0)bgr = tf.concat(axis=3, values=[blue - self.vgg_mean[0],green - self.vgg_mean[1],red - self.vgg_mean[2],])# pre-trained VGG layers are fixed in fine-tuneconv1_1 = self.conv_layer(bgr, "conv1_1")conv1_2 = self.conv_layer(conv1_1, "conv1_2")pool1 = self.max_pool(conv1_2, 'pool1')conv2_1 = self.conv_layer(pool1, "conv2_1")conv2_2 = self.conv_layer(conv2_1, "conv2_2")pool2 = self.max_pool(conv2_2, 'pool2')conv3_1 = self.conv_layer(pool2, "conv3_1")conv3_2 = self.conv_layer(conv3_1, "conv3_2")conv3_3 = self.conv_layer(conv3_2, "conv3_3")pool3 = self.max_pool(conv3_3, 'pool3')conv4_1 = self.conv_layer(pool3, "conv4_1")conv4_2 = self.conv_layer(conv4_1, "conv4_2")conv4_3 = self.conv_layer(conv4_2, "conv4_3")pool4 = self.max_pool(conv4_3, 'pool4')conv5_1 = self.conv_layer(pool4, "conv5_1")conv5_2 = self.conv_layer(conv5_1, "conv5_2")conv5_3 = self.conv_layer(conv5_2, "conv5_3")pool5 = self.max_pool(conv5_3, 'pool5')# detach original VGG fc layers and# reconstruct your own fc layers serve for your own purposeself.flatten = tf.reshape(pool5, [-1, 7*7*512])self.fc6 = tf.layers.dense(self.flatten, 256, tf.nn.relu, name='fc6')self.out = tf.layers.dense(self.fc6, 1, name='out')self.sess = tf.Session()if restore_from: saver = tf.train.Saver()#初始restore_from為None，訓(xùn)練完之后就可以使用相應(yīng)的參數(shù)保存文件，這是路徑saver.restore(self.sess, restore_from) else: # training graphself.loss = tf.losses.mean_squared_error(labels=self.tfy, predictions=self.out)self.train_op = tf.train.RMSPropOptimizer(0.001).minimize(self.loss)self.sess.run(tf.global_variables_initializer())def max_pool(self, bottom, name):return tf.nn.max_pool(bottom, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', name=name)def conv_layer(self, bottom, name):with tf.variable_scope(name): # CNN's filter is constant, NOT Variable that can be trainedconv = tf.nn.conv2d(bottom, self.data_dict[name][0], [1, 1, 1, 1], padding='SAME')lout = tf.nn.relu(tf.nn.bias_add(conv, self.data_dict[name][1]))return loutdef train(self, x, y):loss, _ = self.sess.run([self.loss, self.train_op], {self.tfx: x, self.tfy: y})return lossdef predict(self, paths):fig, axs = plt.subplots(1, 2)for i, path in enumerate(paths):x = load_img(path)length = self.sess.run(self.out, {self.tfx: x})axs[i].imshow(x[0])axs[i].set_title('Len: %.1f cm' % length)axs[i].set_xticks(()); axs[i].set_yticks(())plt.show()def save(self, path='./for_transfer_learning/model/transfer_learn'):saver = tf.train.Saver()saver.save(self.sess, path, write_meta_graph=False)def train():tigers_x, cats_x, tigers_y, cats_y = load_data()# plot fake length distributionplt.hist(tigers_y, bins=20, label='Tigers')plt.hist(cats_y, bins=10, label='Cats')plt.legend()plt.xlabel('length')plt.show()xs = np.concatenate(tigers_x + cats_x, axis=0)ys = np.concatenate((tigers_y, cats_y), axis=0)vgg = Vgg16(vgg16_npy_path='./for_transfer_learning/vgg16.npy')print('Net built')for i in range(100):b_idx = np.random.randint(0, len(xs), 6)train_loss = vgg.train(xs[b_idx], ys[b_idx])print(i, 'train loss: ', train_loss)vgg.save('./for_transfer_learning/model/transfer_learn') # save learned fc layersdef eval():vgg = Vgg16(vgg16_npy_path='./for_transfer_learning/vgg16.npy',restore_from='./for_transfer_learning/model/transfer_learn')vgg.predict(['./for_transfer_learning/data/kittycat/000129037.jpg', './for_transfer_learning/data/tiger/391412.jpg'])if __name__ == '__main__':# download()# train()eval()

先運行download()把圖片都下載下來，只需要訓(xùn)練最后的兩層，全連接層和輸出層，運行train()，最后運行eval()用樣本里的圖片測試模型。

總結(jié)

以上是生活随笔為你收集整理的莫烦---Tensorflow学习的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。