背景

本文主要講的是基于Android平臺實現RTMP的技術方案設計，基礎架構圖如下：

組網注意事項

1. 組網：無線組網，需要好的AP模塊才能撐得住大的并發流量，推送端到AP，最好是有線網鏈接；

2. 服務器部署：SRS或NGINX，服務器可以和Windows平臺的教師機部署在一臺機器；

3. 教師端：如教師有移動的PAD，可以直接推到RTMP服務器，然后共享出去；

4. 學生端：直接拉取服務端的RTMP流播放即可；

5. 教師和學生互動：學生端如需作為示范案例，屏幕數據共享給其他同學，只需請求同屏，數據反推到RTMP服務器，其他學生查看即可。

6. 擴展監控：如果需要更進一步的技術方案，如教師端想監控學生端的屏幕情況，可以有兩種方案，如學生端直接推RTMP過來，或者，學生端啟動內置RTSP服務，教師端想看的時候，隨時看即可（亦可輪詢播放）。

Android端對接

推送分辨率如何設定或縮放？

Android設備，特別是高分屏，拿到的視頻原始寬高非常大，如果推原始分辨率，編碼和上行壓力大，所以，一般建議，適當縮放，比如寬高縮放至2/3，縮放一般建議等比例縮放，縮放寬高建議16字節對齊。

private void createScreenEnvironment() {sreenWindowWidth = mWindowManager.getDefaultDisplay().getWidth();screenWindowHeight = mWindowManager.getDefaultDisplay().getHeight();Log.i(TAG, "screenWindowWidth: " + sreenWindowWidth + ",screenWindowHeight: "+ screenWindowHeight);if (sreenWindowWidth > 800){if (screenResolution == SCREEN_RESOLUTION_STANDARD){scale_rate = SCALE_RATE_HALF;sreenWindowWidth = align(sreenWindowWidth / 2, 16);screenWindowHeight = align(screenWindowHeight / 2, 16);}else if(screenResolution == SCREEN_RESOLUTION_LOW){scale_rate = SCALE_RATE_TWO_FIFTHS;sreenWindowWidth = align(sreenWindowWidth * 2 / 5, 16);}}Log.i(TAG, "After adjust mWindowWidth: " + sreenWindowWidth + ", mWindowHeight: " + screenWindowHeight);int pf = mWindowManager.getDefaultDisplay().getPixelFormat();Log.i(TAG, "display format:" + pf);DisplayMetrics displayMetrics = new DisplayMetrics();mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);mScreenDensity = displayMetrics.densityDpi;mImageReader = ImageReader.newInstance(sreenWindowWidth,screenWindowHeight, 0x1, 6);mMediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);}

橫豎屏自動適配

橫豎屏狀態下，采集的屏幕寬高不一樣，如果橫豎屏切換，這個時候，需要考慮到橫豎屏適配問題，確保比如豎屏狀態下，切換到橫屏時，推拉流兩端可以自動適配，橫豎屏自動適配，編碼器需要重啟，拉流端，需要能自動適配寬高變化，自動播放。

public void onConfigurationChanged(Configuration newConfig) {try {super.onConfigurationChanged(newConfig);if (this.getResources().getConfiguration().orientation == Configuration.ORIENTATION_LANDSCAPE) {Log.i(TAG, "onConfigurationChanged cur: LANDSCAPE");} else if (this.getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT) {Log.i(TAG, "onConfigurationChanged cur: PORTRAIT");}if(isPushingRtmp || isRecording || isRTSPPublisherRunning){stopScreenCapture();clearAllImages();createScreenEnvironment();setupVirtualDisplay();}} catch (Exception ex) {}}

補幀策略

好多人不理解為什么要補幀，實際上，屏幕采集的時候，屏幕不動的話，不會一直有數據下去，這個時候，比較好的做法是，保存最后一幀數據，設定一定的補幀間隔，確保不會因為幀間距太大，導致播放端幾秒都收不到數據，當然，如果服務器可以緩存GOP，這個問題迎刃而解。

異常網絡處理、事件回調機制

回答：如果是走RTMP，網絡抖動或者其他網絡異常，需要有好重連機制和狀態回饋機制。

class EventHandeV2 implements NTSmartEventCallbackV2 {@Overridepublic void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);String publisher_event = "";switch (id) {case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:publisher_event = "開始..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:publisher_event = "連接中..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:publisher_event = "連接失敗..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:publisher_event = "連接成功..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:publisher_event = "連接斷開..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:publisher_event = "關閉..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:publisher_event = "開始一個新的錄像文件 : " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:publisher_event = "已生成一個錄像文件 : " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:publisher_event = "發送時延: " + param1 + " 幀數:" + param2;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:publisher_event = "快照: " + param1 + " 路徑：" + param3;if (param1 == 0) {publisher_event = publisher_event + "截取快照成功..";} else {publisher_event = publisher_event + "截取快照失敗..";}break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:publisher_event = "RTSP服務URL: " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:publisher_event ="服務器不支持RTSP推送, 推送的RTSP URL: " + param3;break;}String str = "當前回調狀態：" + publisher_event;Log.i(TAG, str);Message message = new Message();message.what = PUBLISHER_EVENT_MSG;message.obj = publisher_event;handler.sendMessage(message);}}

部分屏幕數據采集

回答：我們遇到的好多場景下，教室端，會拿出來3/4的區域用來投遞給學生看，1/4的區域，用來做一些指令等操作，這個時候，就需要考慮屏幕區域裁剪：

/*** 投遞裁剪過的RGBA數據** @param data: RGBA data** @param rowStride: stride information** @param width: width** @param height: height** @param clipedLeft: 左; clipedTop: 上; clipedwidth: 裁剪后的寬; clipedHeight: 裁剪后的高; 確保傳下去裁剪后的寬、高均為偶數** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);

文字、圖片水印

好多場景下，同屏者會把公司logo，和一定的文字信息展示在推送端，這個時候，需要考慮到文字和圖片水印問題：

/*** Set Text water-mark(設置文字水印)* * @param fontSize: it should be "MEDIUM", "SMALL", "BIG"* * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".* * @param xPading, yPading: the distance of the original picture.* * <pre> The interface is only used for setting font water-mark when publishing stream. </pre> * * @return {0} if successful*/public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);/*** Set Text water-mark font file name(設置文字水印字體路徑)** @param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf** @return {0} if successful*/public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);/*** Set picture water-mark(設置png圖片水印)* * @param picPath: the picture working path, e.g: /sdcard/logo.png* * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".* * @param picWidth, picHeight: picture width & height* * @param xPading, yPading: the distance of the original picture.* * <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre> * * @return {0} if successful*/public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);

屏幕權限獲取|數據采集

采集推送之前，需要獲取屏幕權限，拿到屏幕數據后，調用SDK接口，完成推送或錄像操作即可：

@TargetApi(Build.VERSION_CODES.LOLLIPOP)private boolean startScreenCapture() {Log.i(TAG, "startScreenCapture..");setupMediaProjection();setupVirtualDisplay();return true;}private int align(int d, int a) {return (((d) + (a - 1)) & ~(a - 1));}@SuppressWarnings("deprecation")@SuppressLint("NewApi")private void createScreenEnvironment() {sreenWindowWidth = mWindowManager.getDefaultDisplay().getWidth();screenWindowHeight = mWindowManager.getDefaultDisplay().getHeight();Log.i(TAG, "screenWindowWidth: " + sreenWindowWidth + ",screenWindowHeight: "+ screenWindowHeight);if (sreenWindowWidth > 800){if (screenResolution == SCREEN_RESOLUTION_STANDARD){scale_rate = SCALE_RATE_HALF;sreenWindowWidth = align(sreenWindowWidth / 2, 16);screenWindowHeight = align(screenWindowHeight / 2, 16);}else if(screenResolution == SCREEN_RESOLUTION_LOW){scale_rate = SCALE_RATE_TWO_FIFTHS;sreenWindowWidth = align(sreenWindowWidth * 2 / 5, 16);screenWindowHeight = align(screenWindowHeight * 2 / 5, 16);}}Log.i(TAG, "After adjust mWindowWidth: " + sreenWindowWidth + ", mWindowHeight: " + screenWindowHeight);int pf = mWindowManager.getDefaultDisplay().getPixelFormat();Log.i(TAG, "display format:" + pf);DisplayMetrics displayMetrics = new DisplayMetrics();mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);mScreenDensity = displayMetrics.densityDpi;mImageReader = ImageReader.newInstance(sreenWindowWidth,screenWindowHeight, 0x1, 6);mMediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);}@SuppressLint("NewApi")private void setupMediaProjection() {mMediaProjection = mMediaProjectionManager.getMediaProjection(MainActivity.mResultCode, MainActivity.mResultData);}@SuppressLint("NewApi")private void setupVirtualDisplay() {mVirtualDisplay = mMediaProjection.createVirtualDisplay("ScreenCapture", sreenWindowWidth, screenWindowHeight,mScreenDensity,DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,mImageReader.getSurface(), null, null);mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {@Overridepublic void onImageAvailable(ImageReader reader) {Image image = mImageReader.acquireLatestImage();if (image != null) {processScreenImage(image);//image.close();}}}, null);}private void startRecorderScreen() {Log.i(TAG, "start recorder screen..");if (startScreenCapture()) {new Thread() {@Overridepublic void run() {Log.i(TAG, "start record..");}}.start();}}private ByteBuffer deepCopy(ByteBuffer source) {int sourceP = source.position();int sourceL = source.limit();ByteBuffer target = ByteBuffer.allocateDirect(source.remaining());target.put(source);target.flip();source.position(sourceP);source.limit(sourceL);return target;}/*** Process image data as desired.*/@SuppressLint("NewApi")private void processScreenImage(Image image) {if(!isPushingRtmp && !isRecording &&!isRTSPPublisherRunning){image.close();return;}/*final Image.Plane[] planes = image.getPlanes();width_ = image.getWidth();height_ = image.getHeight();row_stride_ = planes[0].getRowStride();ByteBuffer buf = deepCopy(planes[0].getBuffer());*/// Log.i("OnScreenImage", "new image");pushImage(image);}@SuppressLint("NewApi")private void stopScreenCapture() {if (mVirtualDisplay != null) {mVirtualDisplay.release();mVirtualDisplay = null;}}

基礎初始化

private void InitAndSetConfig() {//開始要不要采集音頻或視頻，請自行設置publisherHandle = libPublisher.SmartPublisherOpen(this.getApplicationContext(),audio_opt, video_opt, sreenWindowWidth,screenWindowHeight);if ( publisherHandle == 0 ){return;}Log.i(TAG, "publisherHandle=" + publisherHandle);libPublisher.SetSmartPublisherEventCallbackV2(publisherHandle, new EventHandeV2());if(videoEncodeType == 1){int h264HWKbps = setHardwareEncoderKbps(true, sreenWindowWidth,screenWindowHeight);Log.i(TAG, "h264HWKbps: " + h264HWKbps);int isSupportH264HWEncoder = libPublisher.SetSmartPublisherVideoHWEncoder(publisherHandle, h264HWKbps);if (isSupportH264HWEncoder == 0) {Log.i(TAG, "Great, it supports h.264 hardware encoder!");}}else if (videoEncodeType == 2){int hevcHWKbps = setHardwareEncoderKbps(false, sreenWindowWidth,screenWindowHeight);Log.i(TAG, "hevcHWKbps: " + hevcHWKbps);int isSupportHevcHWEncoder = libPublisher.SetSmartPublisherVideoHevcHWEncoder(publisherHandle, hevcHWKbps);if (isSupportHevcHWEncoder == 0) {Log.i(TAG, "Great, it supports hevc hardware encoder!");}}if(is_sw_vbr_mode){int is_enable_vbr = 1;int video_quality = CalVideoQuality(sreenWindowWidth,screenWindowHeight, true);int vbr_max_bitrate = CalVbrMaxKBitRate(sreenWindowWidth,screenWindowHeight);libPublisher.SmartPublisherSetSwVBRMode(publisherHandle, is_enable_vbr, video_quality, vbr_max_bitrate);}//音頻相關可以參考SmartPublisher工程/*if (!is_speex){// set AAC encoderlibPublisher.SmartPublisherSetAudioCodecType(publisherHandle, 1);}else{// set Speex encoderlibPublisher.SmartPublisherSetAudioCodecType(publisherHandle, 2);libPublisher.SmartPublisherSetSpeexEncoderQuality(publisherHandle, 8);}libPublisher.SmartPublisherSetNoiseSuppression(publisherHandle, is_noise_suppression ? 1: 0);libPublisher.SmartPublisherSetAGC(publisherHandle, is_agc ? 1 : 0);*/// libPublisher.SmartPublisherSetClippingMode(publisherHandle, 0);//libPublisher.SmartPublisherSetSWVideoEncoderProfile(publisherHandle, sw_video_encoder_profile);//libPublisher.SmartPublisherSetSWVideoEncoderSpeed(publisherHandle, sw_video_encoder_speed);// libPublisher.SetRtmpPublishingType(publisherHandle, 0);libPublisher.SmartPublisherSetFPS(publisherHandle, 18); //幀率可調libPublisher.SmartPublisherSetGopInterval(publisherHandle, 18*3);//libPublisher.SmartPublisherSetSWVideoBitRate(publisherHandle, 1200, 2400); //針對軟編碼有效，一般最大碼率是平均碼率的二倍libPublisher.SmartPublisherSetSWVideoEncoderSpeed(publisherHandle, 3);//libPublisher.SmartPublisherSaveImageFlag(publisherHandle, 1);}

準備推送|錄像|啟動RTSP服務

@SuppressWarnings("deprecation")@Overridepublic void onStart(Intent intent, int startId) {// TODO Auto-generated method stubsuper.onStart(intent, startId);Log.i(TAG, "onStart++");if (libPublisher == null)return;clearAllImages();screenResolution = intent.getExtras().getInt("SCREENRESOLUTION");videoEncodeType = intent.getExtras().getInt("VIDEOENCODETYPE");push_type = intent.getExtras().getInt("PUSHTYPE");Log.i(TAG, "push_type: " + push_type);mWindowManager = (WindowManager) getSystemService(Service.WINDOW_SERVICE);// 窗口管理者createScreenEnvironment();startRecorderScreen();//如果同時推送和錄像，設置一次就可以InitAndSetConfig();if ( publisherHandle == 0 ){stopScreenCapture();return;}if(push_type == PUSH_TYPE_RTMP){String publishURL = intent.getStringExtra("PUBLISHURL");Log.i(TAG, "publishURL: " + publishURL);if (libPublisher.SmartPublisherSetURL(publisherHandle, publishURL) != 0) {stopScreenCapture();Log.e(TAG, "Failed to set publish stream URL..");if (publisherHandle != 0) {if (libPublisher != null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle = 0;}}return;}}//啟動傳遞數據線程post_data_thread = new Thread(new DataRunnable());Log.i(TAG, "new post_data_thread..");is_post_data_thread_alive = true;post_data_thread.start();//錄像相關++is_need_local_recorder = intent.getExtras().getBoolean("RECORDER");if(is_need_local_recorder){ConfigRecorderParam();int startRet = libPublisher.SmartPublisherStartRecorder(publisherHandle);if( startRet != 0 ){isRecording = false;Log.e(TAG, "Failed to start recorder..");}else{isRecording = true;}}//錄像相關——if(push_type == PUSH_TYPE_RTMP){Log.i(TAG, "RTMP Pusher mode..");//推流相關++int startRet = libPublisher.SmartPublisherStartPublisher(publisherHandle);if (startRet != 0) {isPushingRtmp = false;Log.e(TAG, "Failed to start push rtmp stream..");return;}else{isPushingRtmp = true;}//推流相關--}else if(push_type == PUSH_TYPE_RTSP){Log.i(TAG, "RTSP Internal Server mode..");rtsp_handle_ = libPublisher.OpenRtspServer(0);if (rtsp_handle_ == 0) {Log.e(TAG, "創建rtsp server實例失敗! 請檢查SDK有效性");} else {int port = 8554;if (libPublisher.SetRtspServerPort(rtsp_handle_, port) != 0) {libPublisher.CloseRtspServer(rtsp_handle_);rtsp_handle_ = 0;Log.e(TAG, "創建rtsp server端口失敗! 請檢查端口是否重復或者端口不在范圍內!");}//String user_name = "admin";//String password = "12345";//libPublisher.SetRtspServerUserNamePassword(rtsp_handle_, user_name, password);if (libPublisher.StartRtspServer(rtsp_handle_, 0) == 0) {Log.i(TAG, "啟動rtsp server 成功!");} else {libPublisher.CloseRtspServer(rtsp_handle_);rtsp_handle_ = 0;Log.e(TAG, "啟動rtsp server失敗! 請檢查設置的端口是否被占用!");return;}isRTSPServiceRunning = true;}if(isRTSPServiceRunning){Log.i(TAG, "onClick start rtsp publisher..");String rtsp_stream_name = "stream1";libPublisher.SetRtspStreamName(publisherHandle, rtsp_stream_name);libPublisher.ClearRtspStreamServer(publisherHandle);libPublisher.AddRtspStreamServer(publisherHandle, rtsp_handle_, 0);if (libPublisher.StartRtspStream(publisherHandle, 0) != 0) {Log.e(TAG, "調用發布rtsp流接口失敗!");return;}isRTSPPublisherRunning = true;}}//如果同時推送和錄像，Audio啟動一次就可以了CheckInitAudioRecorder();Log.i(TAG, "onStart--");}private void stopPush() {if(!isPushingRtmp){return;}if (!isRecording && !isRTSPPublisherRunning) {if (audioRecord_ != null) {Log.i(TAG, "stopPush, call audioRecord_.StopRecording..");audioRecord_.Stop();if (audioRecordCallback_ != null) {audioRecord_.RemoveCallback(audioRecordCallback_);audioRecordCallback_ = null;}audioRecord_ = null;}}if (libPublisher != null) {libPublisher.SmartPublisherStopPublisher(publisherHandle);}if (!isRecording && !isRTSPPublisherRunning) {if (publisherHandle != 0) {if (libPublisher != null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle = 0;}}}}

停止推送|錄像|RTSP服務

private void stopRecorder() {if(!isRecording){return;}if (!isPushingRtmp && !isRTSPPublisherRunning) {if (audioRecord_ != null) {Log.i(TAG, "stopRecorder, call audioRecord_.StopRecording..");audioRecord_.Stop();if (audioRecordCallback_ != null) {audioRecord_.RemoveCallback(audioRecordCallback_);audioRecordCallback_ = null;}audioRecord_ = null;}}if (libPublisher != null) {libPublisher.SmartPublisherStopRecorder(publisherHandle);}if (!isPushingRtmp && !isRTSPPublisherRunning) {if (publisherHandle != 0) {if (libPublisher != null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle = 0;}}}}//停止發布RTSP流private void stopRtspPublisher() {if(!isRTSPPublisherRunning){return;}if (!isPushingRtmp && !isRecording) {if (audioRecord_ != null) {Log.i(TAG, "stopRtspPublisher, call audioRecord_.StopRecording..");audioRecord_.Stop();if (audioRecordCallback_ != null) {audioRecord_.RemoveCallback(audioRecordCallback_);audioRecordCallback_ = null;}audioRecord_ = null;}}if (libPublisher != null) {libPublisher.StopRtspStream(publisherHandle);}if (!isPushingRtmp && !isRecording) {if (publisherHandle != 0) {if (libPublisher != null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle = 0;}}}}//停止RTSP服務private void stopRtspService() {if(!isRTSPServiceRunning){return;}if (libPublisher != null && rtsp_handle_ != 0) {libPublisher.StopRtspServer(rtsp_handle_);libPublisher.CloseRtspServer(rtsp_handle_);rtsp_handle_ = 0;}}

感興趣的開發者可酌情參考。

總結

以上是生活随笔為你收集整理的Android对接实现内网无纸化会议|智慧教室|实时同屏功能的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。