根據人臉做年齡預測

• • 說明
  • 環境配置
  • mtcnn庫檢測人臉
  • MTCNN模型實現
  • • PNet
    • RNet
    • ONet
  • 數據準備
  • • 運行PNet
  • 運行RNet
  • • 運行ONet
    • 年齡預測

說明

做一個針對人臉做不同的年齡預測，需要用到MTCNN模型和SSR-Net模型，MTCNN模型用于做人臉檢測和人臉對齊，在MTCNN中，使用深度學習方法結合NMS和邊界框回歸，將人臉區域坐標和關鍵點坐標進行識別，相比較機器學習方法，MTCNN能更好地識別不同情況下的人臉。

我實在華為云服務器上跑的，結果ok

MTCNN模型的詳解可以參考：link

環境配置

python3 TensorFlow 1.13.1 mtcnn==0.0.8

mtcnn庫檢測人臉

將圖片存儲在image_path中

import numpy as np import cv2 import tensorflow as tf import random from PIL import Imageimage_path = "path" img = Image.open(image_path) img = np.array(img)

調用mtcnn庫，進行人臉區域檢測，并顯示檢測結果

from mtcnn.mtcnn import MTCNN as mtcnndetector = mtcnn() detected = detector.detect_faces(img)# 打印檢測結果 detected

將檢測結果繪制在圖片上

# 繪圖部分 box = detected[0]["box"] res_img = cv2.rectangle(img, (box[0],box[1]),(box[0]+box[2],box[1]+box[3]), 0, 1)keypoints = detected[0]["keypoints"] res_img = cv2.circle(res_img, keypoints['left_eye'], 1, 255, 4) res_img = cv2.circle(res_img, keypoints['right_eye'], 1, 255, 4) res_img = cv2.circle(res_img, keypoints['nose'], 1, 255, 4) res_img = cv2.circle(res_img, keypoints['mouth_left'], 1, 255, 4) res_img = cv2.circle(res_img, keypoints['mouth_right'], 1, 255, 4)res_img = Image.fromarray(res_img) res_img

MTCNN模型實現

MTCNN網絡分為三部分：PNet RNet ONet 對應于
人臉/非人臉分類分類結果，人臉邊界框以及人臉關鍵點位置。

NMS（non maximum suppression）非極大值抑制 當我們進行人臉檢測時，可能會對同一張人臉區域有多個邊界框檢測結果，雖然這些檢測結果都有很高的置信度，但是我們只需要置信度最高的檢測結果，所以進行局部最大值檢測，將不是最大值的預測結果去掉，完成邊界框篩選的任務。NMS被應用在很多目標檢測模型當中，例如R-CNN，Faster R-CNN，Mask R-CNN等。

from src.align.detect_face import Network from src.align.detect_face import rerec, pad from src.align.detect_face import nms from src.align.detect_face import imresample from src.align.detect_face import generateBoundingBox

PNet

我們使用全卷積網絡：Proposal 網絡（PNet），來生成人臉區域備選框，然后備選框通過邊界框回歸進行校正。校正后，應用NMS來將高度重復的備選框進行篩選。

class PNet(Network):def setup(self):(self.feed('data') .conv(3, 3, 10, 1, 1, padding='VALID', relu=False, name='conv1').prelu(name='PReLU1').max_pool(2, 2, 2, 2, name='pool1').conv(3, 3, 16, 1, 1, padding='VALID', relu=False, name='conv2').prelu(name='PReLU2').conv(3, 3, 32, 1, 1, padding='VALID', relu=False, name='conv3').prelu(name='PReLU3').conv(1, 1, 2, 1, 1, relu=False, name='conv4-1').softmax(3,name='prob1'))(self.feed('PReLU3') .conv(1, 1, 4, 1, 1, relu=False, name='conv4-2'))

RNet

PNet生成的所有人臉備選框都被輸入另一個卷積網絡，叫做Refine網絡（RNet）。RNet將大量錯誤的人臉信息去掉，同樣通過邊界框回歸進行校正，以及通過NMS進行篩選。

class RNet(Network):def setup(self):(self.feed('data') #pylint: disable=no-value-for-parameter, no-member.conv(3, 3, 28, 1, 1, padding='VALID', relu=False, name='conv1').prelu(name='prelu1').max_pool(3, 3, 2, 2, name='pool1').conv(3, 3, 48, 1, 1, padding='VALID', relu=False, name='conv2').prelu(name='prelu2').max_pool(3, 3, 2, 2, padding='VALID', name='pool2').conv(2, 2, 64, 1, 1, padding='VALID', relu=False, name='conv3').prelu(name='prelu3').fc(128, relu=False, name='conv4').prelu(name='prelu4').fc(2, relu=False, name='conv5-1').softmax(1,name='prob1'))(self.feed('prelu4') #pylint: disable=no-value-for-parameter.fc(4, relu=False, name='conv5-2'))

ONet

ONet與RNet相似，但是在ONet將輸出5個人臉關鍵點位置，全稱為Output Network，作為最后一層網絡，將輸出人臉區域坐標以及人臉關鍵點坐標。

class ONet(Network):def setup(self):(self.feed('data') #pylint: disable=no-value-for-parameter, no-member.conv(3, 3, 32, 1, 1, padding='VALID', relu=False, name='conv1').prelu(name='prelu1').max_pool(3, 3, 2, 2, name='pool1').conv(3, 3, 64, 1, 1, padding='VALID', relu=False, name='conv2').prelu(name='prelu2').max_pool(3, 3, 2, 2, padding='VALID', name='pool2').conv(3, 3, 64, 1, 1, padding='VALID', relu=False, name='conv3').prelu(name='prelu3').max_pool(2, 2, 2, 2, name='pool3').conv(2, 2, 128, 1, 1, padding='VALID', relu=False, name='conv4').prelu(name='prelu4').fc(256, relu=False, name='conv5').prelu(name='prelu5').fc(2, relu=False, name='conv6-1').softmax(1, name='prob1'))(self.feed('prelu5') #pylint: disable=no-value-for-parameter.fc(4, relu=False, name='conv6-2'))(self.feed('prelu5') #pylint: disable=no-value-for-parameter.fc(10, relu=False, name='conv6-3'))

數據準備

# 打開原圖 test_img = Image.open(image_path) test_img# 進行圖片預處理 test_img = np.array(test_img) img_size = np.asarray(test_img.shape)[0:2] factor_count=0 minsize = 20 total_boxes=np.empty((0,9)) points=np.empty(0) h=test_img.shape[0] # h=410 w=test_img.shape[1] # w=599minl=np.amin([h, w]) # minl = [410,599] 中最小值 410 m=12.0/minsize # m=12/20 minl=minl*m # minl = 410*12/20 = 410* 0.6 factor = 0.709 scales=[]while minl>=12:scales += [m*np.power(factor, factor_count)]minl = minl*factor factor_count += 1# first stage for scale in scales:hs=int(np.ceil(h*scale)) #大于等于該值的最小整數ws=int(np.ceil(w*scale))im_data = cv2.resize(test_img, (ws, hs), interpolation=cv2.INTER_AREA)im_data = (im_data-127.5)*0.0078125img_x = np.expand_dims(im_data, 0)img_y = np.transpose(img_x, (0,2,1,3))

運行PNet

運行PNet，并加載預訓練權重

with tf.Graph().as_default():with tf.Session() as sess:with tf.variable_scope('pnet'):data = tf.placeholder(tf.float32, shape=(None, None, None, 3), name="input")pnet = PNet({'data':data})pnet.load("./src/align/PNet.npy", sess)out = sess.run(('pnet/conv4-2/BiasAdd:0', 'pnet/prob1:0'), feed_dict={'pnet/input:0':img_y})# boundingbox regression 結果 out0 = np.transpose(out[0], (0,2,1,3)) # face classification 結果 out1 = np.transpose(out[1], (0,2,1,3))threshold = 0.5 boxes, reg = generateBoundingBox(out1[0,:,:,1].copy(), out0[0,:,:,:].copy(), scale, threshold) print("PNet產生結果為："+str(boxes.shape))total_boxes = boxes.copy()# 邊界框繪制函數 def draw_bboxes(img, total_boxes):for i in range(total_boxes.shape[0]):r = random.randint(0, 255)g = random.randint(0, 255)b = random.randint(0, 255)x1 = int(total_boxes[:,0][i])y1 = int(total_boxes[:,1][i])x2= int(total_boxes[:,2][i])y2 = int(total_boxes[:,3][i])img = cv2.rectangle(img,(x1,y1),(x2,y2), (r,g,b), 2)return img

將PNet預測結果進行篩選和回歸，結果繪制在圖片上

img = Image.open(image_path) img = np.array(img) Image.fromarray(draw_bboxes(img,total_boxes)) total_boxes=np.empty((0,9)) pick = nms(boxes.copy(), 0.7, 'Union')if boxes.size>0 and pick.size>0:boxes = boxes[pick,:]total_boxes = np.append(total_boxes, boxes, axis=0) print("篩選之后結果為："+str(total_boxes.shape)) # 繪制篩選后的邊界框 img = Image.open(image_path) img = np.array(img)# 進行nms計算 參數為0.7 pick = nms(total_boxes.copy(), 0.6, 'Union') total_boxes = total_boxes[pick,:] print(total_boxes.shape)# 邊界框回歸 regw = total_boxes[:,2]-total_boxes[:,0] regh = total_boxes[:,3]-total_boxes[:,1] qq1 = total_boxes[:,0]+total_boxes[:,5]*regw qq2 = total_boxes[:,1]+total_boxes[:,6]*regh qq3 = total_boxes[:,2]+total_boxes[:,7]*regw qq4 = total_boxes[:,3]+total_boxes[:,8]*regh total_boxes = np.transpose(np.vstack([qq1, qq2, qq3, qq4, total_boxes[:,4]])) print(total_boxes.shape) img = Image.open(image_path) img = np.array(img)# 將邊界框形狀轉為正方形 total_boxes = rerec(total_boxes.copy()) print(total_boxes)# 將邊界框坐標整理成整數 total_boxes[:,0:4] = np.fix(total_boxes[:,0:4]).astype(np.int32) print(total_boxes) dy, edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(total_boxes.copy(), w, h)img = Image.open(image_path) img = np.array(img) Image.fromarray(draw_bboxes(img,total_boxes))

運行RNet

MTCNN的PNet計算結束后，可以看到已經有若干個邊界框已經被預測出來。接下來我們將進行RNet預測，通過RNet預測之后，邊界框將更加準確。

numbox = total_boxes.shape[0] tempimg = np.zeros((24,24,3,numbox)) for k in range(0,numbox):tmp = np.zeros((int(tmph[k]),int(tmpw[k]),3))tmp[dy[k]-1:edy[k],dx[k]-1:edx[k],:] = img[y[k]-1:ey[k],x[k]-1:ex[k],:]if tmp.shape[0]>0 and tmp.shape[1]>0 or tmp.shape[0]==0 and tmp.shape[1]==0:tempimg[:,:,:,k] = imresample(tmp, (24, 24))else:print(0) tempimg = (tempimg-127.5)*0.0078125 tempimg1 = np.transpose(tempimg, (3,1,0,2)) with tf.Graph().as_default():with tf.Session() as sess:with tf.variable_scope('rnet'):data = tf.placeholder(tf.float32, shape=(None, 24, 24, 3), name="input")rnet = RNet({'data':data})rnet.load("./src/align/RNet.npy", sess)out = sess.run(('rnet/conv5-2/conv5-2:0', 'rnet/prob1:0'), feed_dict={'rnet/input:0':tempimg1})# 檢測到的人臉坐標 out0 = np.transpose(out[0]) out1 = np.transpose(out[1])score = out1[1,:] threshold = 0.7 ipass = np.where(score>0.2) total_boxes = np.hstack([total_boxes[ipass[0],0:4].copy(), np.expand_dims(score[ipass].copy(),1)]) mv = out0[:,ipass[0]] if total_boxes.shape[0]>0:pick = nms(total_boxes, threshold, 'Union')total_boxes = total_boxes[pick,:]print(total_boxes)img = Image.open(image_path) img = np.array(img)from src.align.detect_face import bbreg# 邊界框回歸 total_boxes = bbreg(total_boxes.copy(), np.transpose(mv[:,pick])) print(total_boxes) # 邊界框整理成正方形 total_boxes = rerec(total_boxes.copy()) print(total_boxes)img = Image.open(image_path) img = np.array(img) Image.fromarray(draw_bboxes(img,total_boxes))

運行ONet

最后，我們進行ONet預測，不僅使人臉的邊界框檢測更加準確，這一步還將關鍵點檢測出來。

numbox = total_boxes.shape[0] total_boxes = np.fix(total_boxes).astype(np.int32) dy, edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(total_boxes.copy(), w, h)tempimg = np.zeros((48,48,3,numbox)) for k in range(0,numbox):tmp = np.zeros((int(tmph[k]),int(tmpw[k]),3))tmp[dy[k]-1:edy[k],dx[k]-1:edx[k],:] = img[y[k]-1:ey[k],x[k]-1:ex[k],:]if tmp.shape[0]>0 and tmp.shape[1]>0 or tmp.shape[0]==0 and tmp.shape[1]==0:tempimg[:,:,:,k] = imresample(tmp, (48, 48))else:print(0) tempimg = (tempimg-127.5)*0.0078125 tempimg1 = np.transpose(tempimg, (3,1,0,2)) with tf.Graph().as_default():with tf.Session() as sess:with tf.variable_scope('onet'):data = tf.placeholder(tf.float32, shape=(None, 48, 48, 3), name="input")onet = ONet({'data':data})rnet.load("./src/align/ONet.npy", sess)out = sess.run(('onet/conv6-2/conv6-2:0', 'onet/conv6-3/conv6-3:0', 'onet/prob1:0'), feed_dict={'onet/input:0':tempimg1})# 人臉區域邊界框預測結果 out0 = np.transpose(out[0]) # 人臉關鍵點預測結果 out1 = np.transpose(out[1]) # 人臉區域置信度 out2 = np.transpose(out[2])score = out2[1,:] points = out1 # threshold = 0.7 ipass = np.where(score>0.7) points = points[:,ipass[0]] total_boxes = np.hstack([total_boxes[ipass[0],0:4].copy(), np.expand_dims(score[ipass].copy(),1)]) mv = out0[:,ipass[0]]w = total_boxes[:,2]-total_boxes[:,0]+1 h = total_boxes[:,3]-total_boxes[:,1]+1 points[0:5,:] = np.tile(w,(5, 1))*points[0:5,:] + np.tile(total_boxes[:,0],(5, 1))-1 points[5:10,:] = np.tile(h,(5, 1))*points[5:10,:] + np.tile(total_boxes[:,1],(5, 1))-1 if total_boxes.shape[0]>0:total_boxes = bbreg(total_boxes.copy(), np.transpose(mv))pick = nms(total_boxes.copy(), 0.7, 'Min')total_boxes = total_boxes[pick,:]points = points[:,pick] img = Image.open(image_path) img = np.array(img)r = random.randint(0, 255) g = random.randint(0, 255) b = random.randint(0, 255)point_color = (r, g, b) for i in range(5):cv2.circle(img,(int(points[i]),int(points[i+5])),1, point_color, 4)Image.fromarray(draw_bboxes(img,total_boxes))

年齡預測

我們使用SSR-Net模型預測年齡

加載模型

from SSRNET_model import SSR_netweight_file = "./ssrnet_3_3_3_64_1.0_1.0.h5"img_size = 64 stage_num = [3,3,3] lambda_local = 1 lambda_d = 1 model = SSR_net(img_size,stage_num, lambda_local, lambda_d)() model.load_weights(weight_file)

模型層級結構

model.summary()

準備輸入數據

faces = np.empty((len(detected), img_size, img_size, 3)) faces.shape

獲取人臉區域圖片，并縮放
將人臉檢測結果進行裁剪和縮放

ad = 0.4img_h, img_w, _ = np.shape(img)for i,d in enumerate(detected):if d['confidence'] >=0.95 :x1,y1,w,h = d['box']x2 = x1 + wy2 = y1 + hxw1 = max(int(x1 - ad * w), 0)yw1 = max(int(y1 - ad * h), 0)xw2 = min(int(x2 + ad * w), img_w - 1)yw2 = min(int(y2 + ad * h), img_h - 1)img = cv2.resize(img[yw1:yw2+1, xw1:xw2+1, :], (img_size, img_size))faces[i,:,:,:] = imgres_img = Image.fromarray(img) res_img

預測年齡
將人臉區域圖片輸入模型，獲得預測結果

res = model.predict(faces)print("預測年齡為："+str(int(res[0])))

總結

以上是生活随笔為你收集整理的根据人脸做年龄预测的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。