文章目錄

• • 1. 谷歌Colab設(shè)置
  • 2. 編寫代碼
  • 3. flask 微服務(wù)
  • 4. 打包到容器
  • 5. 容器托管

參考 基于深度學(xué)習(xí)的自然語(yǔ)言處理

使用這篇文章的數(shù)據(jù)(情感分類)進(jìn)行學(xué)習(xí)。

1. 谷歌Colab設(shè)置

Colab 地址

• 新建筆記本
  

• 設(shè)置
  

• 選擇 GPU/TPU 加速計(jì)算
  

• 測(cè)試 GPU 是否分配

import tensorflow as tf tf.test.gpu_device_name()

輸出：

/device:GPU:0

• 上傳數(shù)據(jù)至谷歌云硬盤，并在Colab中加載
  
  
  
• 解壓數(shù)據(jù)
  

2. 編寫代碼

import numpy as np import pandas as pddata = pd.read_csv("yelp_labelled.txt", sep='\t', names=['sentence', 'label'])data.head() # 1000條數(shù)據(jù)# 數(shù)據(jù) X 和 標(biāo)簽 y sentence = data['sentence'].values label = data['label'].values# 訓(xùn)練集 測(cè)試集拆分 from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(sentence, label, test_size=0.2, random_state=1)#%%max_features = 2000# 文本向量化 from keras.preprocessing.text import Tokenizer tokenizer = Tokenizer(num_words=max_features) tokenizer.fit_on_texts(X_train) # 訓(xùn)練tokenizer X_train = tokenizer.texts_to_sequences(X_train) # 轉(zhuǎn)成 [[ids...],[ids...],...] X_test = tokenizer.texts_to_sequences(X_test) vocab_size = len(tokenizer.word_index)+1 # +1 是因?yàn)閕ndex 0, 0 不對(duì)應(yīng)任何詞，用來(lái)padmaxlen = 50 # pad 保證每個(gè)句子的長(zhǎng)度相等 from keras.preprocessing.sequence import pad_sequences X_train = pad_sequences(X_train, maxlen=maxlen, padding='post') # post 尾部補(bǔ)0，pre 前部補(bǔ)0 X_test = pad_sequences(X_test, maxlen=maxlen, padding='post')#%%embed_dim = 256 hidden_units = 64from keras.models import Model, Sequential from keras.layers import Dense, LSTM, Embedding, Bidirectional, Dropout model = Sequential() model.add(Embedding(input_dim=max_features,output_dim=embed_dim,input_length=maxlen)) model.add(Bidirectional(LSTM(hidden_units))) model.add(Dropout(0.3)) model.add(Dense(1, activation='sigmoid')) # 二分類sigmoid, 多分類 softmaxmodel.compile(optimizer='adam', loss='binary_crossentropy',metrics=['accuracy']) model.summary() from keras.utils import plot_model plot_model(model, show_shapes=True, to_file='model.jpg') # 繪制模型結(jié)構(gòu)到文件#%%history = model.fit(X_train,y_train,batch_size=64,epochs=100,verbose=2,validation_split=0.1) # verbose 是否顯示日志信息，0不顯示，1顯示進(jìn)度條，2不顯示進(jìn)度條 loss, accuracy = model.evaluate(X_train, y_train, verbose=1) print("訓(xùn)練集：loss {0:.3f}, 準(zhǔn)確率：{1:.3f}".format(loss, accuracy)) loss, accuracy = model.evaluate(X_test, y_test, verbose=1) print("測(cè)試集：loss {0:.3f}, 準(zhǔn)確率：{1:.3f}".format(loss, accuracy))# 繪制訓(xùn)練曲線 from matplotlib import pyplot as plt import pandas as pd his = pd.DataFrame(history.history) loss = history.history['loss'] val_loss = history.history['val_loss'] acc = history.history['accuracy'] val_acc = history.history['val_accuracy']plt.plot(loss, label='train Loss') plt.plot(val_loss, label='valid Loss') plt.title('Training and Validation Loss') plt.legend() plt.grid() plt.show()plt.plot(acc, label='train Acc') plt.plot(val_acc, label='valid Acc') plt.title('Training and Validation Acc') plt.legend() plt.grid() plt.show()#%%model.save('trained_model.h5')import pickle with open('trained_tokenizer.pkl','wb') as f:pickle.dump(tokenizer, f)# 下載到本地 from google.colab import files files.download('trained_model.h5') files.download('trained_tokenizer.pkl')

3. flask 微服務(wù)

• 以下內(nèi)容不懂，抄一遍

編寫 app.py

# Flask import pickle import numpy as np from keras.preprocessing.sequence import pad_sequences from keras.models import load_model def load_var():global model, tokenizermodel = load_model('trained_model.h5')model.make_predict_function()with open('trained_tokenizer.pkl','rb') as f:tokenizer = pickle.load(f)maxlen = 50 def process_txt(text):x = tokenizer.texts_to_sequences(text)x = pad_sequences(x, maxlen=maxlen, padding='post')return x#%%from flask import Flask, request, jsonify app = Flask(__name__)@app.route('/') def home_routine():return "hello NLP!"#%%@app.route("/prediction",methods=['POST']) def get_prediction():if request.method == 'POST':data = request.get_json()x = process_txt(data)prob = model.predict(x)pred = np.argmax(prob, axis=-1)return str(pred)#%%if __name__ == "__main__":load_var()app.run(debug=True)# 上線階段應(yīng)該為 app.run(host=0.0.0.0, port=80)

• 運(yùn)行 python app.py
• windows cmd 輸入：
  Invoke-WebRequest -Uri 127.0.0.1:5000/prediction -ContentType 'application/json' -Body '["The book was very poor", "Very nice", "bad, oh no", "i love you"]' -Method 'POST'

返回預(yù)測(cè)結(jié)果：

4. 打包到容器

• 后序需要用 Docker 將 應(yīng)用程序包裝到容器中

5. 容器托管

• 容器托管到網(wǎng)絡(luò)服務(wù)，如 AWS EC2 實(shí)例

總結(jié)

以上是生活随笔為你收集整理的NLP项目工作流程的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。