文章目錄

• 前言
• 場景
• 開發(fā)前準備
• • 在項目級gradle里添加華為maven倉
  • 在應用級的build.gradle里面加上SDK依賴
  • 在AndroidManifest.xml文件里面申請相機、訪問網(wǎng)絡和存儲權(quán)限
• 代碼開發(fā)關(guān)鍵步驟
• • 設置人臉檢測器
  • 這里我們通過相機回調(diào)拿到相機幀數(shù)據(jù),并通過調(diào)用人臉檢測器拿到人臉輪廓點后寫入FacePointEngine供貼紙濾鏡使用
  • 介紹如何設計貼紙,首先看一下貼紙數(shù)JSON數(shù)據(jù)定義
  • 我們制作貓耳貼紙JSON文件,通過人臉索引找到眉心84號點和鼻尖85號點分別貼上耳朵和鼻子,然后把它和圖片都放在assets目錄下
  • 這里渲染貼紙紋理我們使用GLSurfaceView,使用起來比TextureView簡單, 首先在onSurfaceChanged實例化貼紙濾鏡,傳入貼紙路徑并開啟相機
  • 然后在onSurfaceChanged初始化貼紙濾鏡
  • 最后通過onDrawFrame把貼紙繪制到屏幕
• Demo效果
• • • 源碼

前言

---

在這個美即真理、全民娛樂的時代，可愛有趣的人臉貼紙在各大美顏軟件中得到了廣泛的應用，現(xiàn)在已經(jīng)不僅局限于相機美顏類軟件中，在社交、娛樂類的app中對人臉貼紙、AR貼紙的需求也非常廣泛。本文詳細介紹了集成華為HMS ML kit人臉識別實現(xiàn)2d貼紙的集成過程，在后面的文章中我們還會介紹3D貼紙的開發(fā)過程，歡迎大家關(guān)注哦~

場景

---

在美顏相機、美圖app以及社交類app(如抖音、微博、微信)等需要對拍照，或者對照片進行處理的app都會構(gòu)建自己特有的貼紙的需求。

開發(fā)前準備

---

在項目級gradle里添加華為maven倉

打開AndroidStudio項目級build.gradle文件

增量添加如下maven地址：

buildscript {{ maven {url 'http://developer.huawei.com/repo/'}} } allprojects {repositories { maven { url 'http://developer.huawei.com/repo/'}} }

在應用級的build.gradle里面加上SDK依賴

// Face detection SDK. implementation 'com.huawei.hms:ml-computer-vision-face:2.0.1.300' // Face detection model. implementation 'com.huawei.hms:ml-computer-vision-face-shape-point-model:2.0.1.300'

在AndroidManifest.xml文件里面申請相機、訪問網(wǎng)絡和存儲權(quán)限

<!--相機權(quán)限--> <uses-feature android:name="android.hardware.camera" /> <uses-permission android:name="android.permission.CAMERA" /> <!--寫權(quán)限--> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> <!--讀權(quán)限--> <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

代碼開發(fā)關(guān)鍵步驟

---

設置人臉檢測器

MLFaceAnalyzerSetting detectorOptions; detectorOptions = new MLFaceAnalyzerSetting.Factory().setFeatureType(MLFaceAnalyzerSetting.TYPE_UNSUPPORT_FEATURES).setShapeType(MLFaceAnalyzerSetting.TYPE_SHAPES).allowTracing(MLFaceAnalyzerSetting.MODE_TRACING_FAST).create(); detector = MLAnalyzerFactory.getInstance().getFaceAnalyzer(detectorOptions);

這里我們通過相機回調(diào)拿到相機幀數(shù)據(jù),并通過調(diào)用人臉檢測器拿到人臉輪廓點后寫入FacePointEngine供貼紙濾鏡使用

@Override public void onPreviewFrame(final byte[] imgData, final Camera camera) {int width = mPreviewWidth;int height = mPreviewHeight;long startTime = System.currentTimeMillis();//設置前后攝方向一致if (isFrontCamera()){mOrientation = 0;}else {mOrientation = 2;}MLFrame.Property property =new MLFrame.Property.Creator().setFormatType(ImageFormat.NV21).setWidth(width).setHeight(height).setQuadrant(mOrientation).create();ByteBuffer data = ByteBuffer.wrap(imgData);// 調(diào)用人臉檢測接口SparseArray<MLFace> faces = detector.analyseFrame(MLFrame.fromByteBuffer(data,property));//判斷是否獲取到人臉信息if(faces.size()>0){MLFace mLFace = faces.get(0);EGLFace EGLFace = FacePointEngine.getInstance().getOneFace(0);EGLFace.pitch = mLFace.getRotationAngleX();EGLFace.yaw = mLFace.getRotationAngleY();EGLFace.roll = mLFace.getRotationAngleZ() - 90;if (isFrontCamera())EGLFace.roll = -EGLFace.roll;if (EGLFace.vertexPoints == null) {EGLFace.vertexPoints = new PointF[131];}int index = 0;// 獲取一個人的輪廓點坐標并轉(zhuǎn)化到openGL歸一化坐標系下的浮點值for (MLFaceShape contour : mLFace.getFaceShapeList()) {if (contour == null) {continue;}List<MLPosition> points = contour.getPoints();for (int i = 0; i < points.size(); i++) {MLPosition point = points.get(i);float x = ( point.getY() / height) * 2 - 1;float y = ( point.getX() / width ) * 2 - 1;if (isFrontCamera())x = -x;PointF Point = new PointF(x,y);EGLFace.vertexPoints[index] = Point;index++;}}// 插入人臉對象FacePointEngine.getInstance().putOneFace(0, EGLFace);// 設置人臉個數(shù)FacePointEngine.getInstance().setFaceSize(faces!= null ? faces.size() : 0);}else{FacePointEngine.getInstance().clearAll();}long endTime = System.currentTimeMillis();Log.d("TAG","Face detect time: " + String.valueOf(endTime - startTime)); }

ML kit接口返回的人臉輪廓點情況如圖所示：

介紹如何設計貼紙,首先看一下貼紙數(shù)JSON數(shù)據(jù)定義

public class FaceStickerJson {public int[] centerIndexList; // 中心坐標索引列表，有可能是多個關(guān)鍵點計算中心點public float offsetX; // 相對于貼紙中心坐標的x軸偏移像素public float offsetY; // 相對于貼紙中心坐標的y軸偏移像素public float baseScale; // 貼紙基準縮放倍數(shù)public int startIndex; // 人臉起始索引，用于計算人臉的寬度public int endIndex; // 人臉結(jié)束索引，用于計算人臉的寬度public int width; // 貼紙寬度public int height; // 貼紙高度public int frames; // 貼紙幀數(shù)public int action; // 動作，0表示默認顯示，這里用來處理貼紙動作等public String stickerName; // 貼紙名稱，用于標記貼紙所在文件夾以及png文件的public int duration; // 貼紙幀顯示間隔public boolean stickerLooping; // 貼紙是否循環(huán)渲染public int maxCount; // 最大貼紙渲染次數(shù) ... }

我們制作貓耳貼紙JSON文件,通過人臉索引找到眉心84號點和鼻尖85號點分別貼上耳朵和鼻子,然后把它和圖片都放在assets目錄下

{"stickerList": [{"type": "sticker","centerIndexList": [84],"offsetX": 0.0,"offsetY": 0.0,"baseScale": 1.3024,"startIndex": 11,"endIndex": 28,"width": 495,"height": 120,"frames": 2,"action": 0,"stickerName": "nose","duration": 100,"stickerLooping": 1,"maxcount": 5}, {"type": "sticker","centerIndexList": [83],"offsetX": 0.0,"offsetY": -1.1834,"baseScale": 1.3453,"startIndex": 11,"endIndex": 28,"width": 454,"height": 150,"frames": 2,"action": 0,"stickerName": "ear","duration": 100,"stickerLooping": 1,"maxcount": 5}] }

這里渲染貼紙紋理我們使用GLSurfaceView,使用起來比TextureView簡單, 首先在onSurfaceChanged實例化貼紙濾鏡,傳入貼紙路徑并開啟相機

@Override public void onSurfaceCreated(GL10 gl, EGLConfig config) {GLES30.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);mTextures = new int[1];mTextures[0] = OpenGLUtils.createOESTexture();mSurfaceTexture = new SurfaceTexture(mTextures[0]);mSurfaceTexture.setOnFrameAvailableListener(this);//將samplerExternalOES 輸入到紋理中cameraFilter = new CameraFilter(this.context);//設置assets目錄下人臉貼紙路徑String folderPath ="cat";stickerFilter = new FaceStickerFilter(this.context,folderPath);//創(chuàng)建屏幕濾鏡對象screenFilter = new BaseFilter(this.context);facePointsFilter = new FacePointsFilter(this.context);mEGLCamera.openCamera(); }

然后在onSurfaceChanged初始化貼紙濾鏡

@Override public void onSurfaceChanged(GL10 gl, int width, int height) {Log.d(TAG, "onSurfaceChanged. width: " + width + ", height: " + height);int previewWidth = mEGLCamera.getPreviewWidth();int previewHeight = mEGLCamera.getPreviewHeight();if (width > height) {setAspectRatio(previewWidth, previewHeight);} else {setAspectRatio(previewHeight, previewWidth);}// 設置畫面的大小,創(chuàng)建FrameBuffer,設置顯示尺寸cameraFilter.onInputSizeChanged(previewWidth, previewHeight);cameraFilter.initFrameBuffer(previewWidth, previewHeight);cameraFilter.onDisplaySizeChanged(width, height);stickerFilter.onInputSizeChanged(previewHeight, previewWidth);stickerFilter.initFrameBuffer(previewHeight, previewWidth);stickerFilter.onDisplaySizeChanged(width, height);screenFilter.onInputSizeChanged(previewWidth, previewHeight);screenFilter.initFrameBuffer(previewWidth, previewHeight);screenFilter.onDisplaySizeChanged(width, height);facePointsFilter.onInputSizeChanged(previewHeight, previewWidth);facePointsFilter.onDisplaySizeChanged(width, height);mEGLCamera.startPreview(mSurfaceTexture); }

最后通過onDrawFrame把貼紙繪制到屏幕

@Override public void onDrawFrame(GL10 gl) {int textureId;// 清除屏幕和深度緩存GLES30.glClear(GLES30.GL_COLOR_BUFFER_BIT | GLES30.GL_DEPTH_BUFFER_BIT);//更新獲取一張圖mSurfaceTexture.updateTexImage();//獲取SurfaceTexture轉(zhuǎn)化矩陣mSurfaceTexture.getTransformMatrix(mMatrix);//設置相機顯示轉(zhuǎn)化矩陣cameraFilter.setTextureTransformMatrix(mMatrix);//繪制相機紋理textureId = cameraFilter.drawFrameBuffer(mTextures[0],mVertexBuffer,mTextureBuffer);//繪制貼紙紋理textureId = stickerFilter.drawFrameBuffer(textureId,mVertexBuffer,mTextureBuffer);//繪制到屏幕screenFilter.drawFrame(textureId , mDisplayVertexBuffer, mDisplayTextureBuffer);if(drawFacePoints){facePointsFilter.drawFrame(textureId, mDisplayVertexBuffer, mDisplayTextureBuffer);} }

這樣我們的貼紙就畫到人臉上了.

Demo效果

---

源碼

---

Demo源碼已上傳Github，地址請戳: https://github.com/HMS-Core/hms-ml-demo/tree/master/Face2D-Sticker，大家可以做參考做基于場景的優(yōu)化

欲了解更多詳情，請參閱：

• 華為開發(fā)者聯(lián)盟官網(wǎng)：https://developer.huawei.com/consumer/en/hms
• 獲取開發(fā)指導文檔：https://developer.huawei.com/consumer/en/doc/development
• 參與開發(fā)者討論請到Reddit社區(qū)：https://www.reddit.com/r/HMSCore/
• 下載demo和示例代碼請到Github：https://github.com/HMS-Core
• 解決集成問題請到Stack Overflow：https://stackoverflow.com/questions/tagged/huawei-mobile-services?tab=Newest

---

原文鏈接：https://developer.huawei.com/consumer/cn/forum/topicview?tid=0203324526929930082&fid=18

原作者：旭小夜 于 2021-04-07 15:19:24 編輯

總結(jié)

以上是生活随笔為你收集整理的【转载】超简单集成HMS ML Kit 人脸检测实现可爱2D贴纸的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。