opencv圖像深度-1

OpenCV空間AI競賽 (OpenCV Spatial AI Competition)

Recently, the people at OpenCV launched the OpenCV Spatial AI competition sponsored by Intel as part of OpenCV’s 20th anniversary celebration. The main objective of the competition is to develop applications that benefit from the features of the new OpenCV AI Kit with Depth (OAK-D). The competition consists of two phases, the winners of the first phase were able to obtain an OAK-D for free to develop their application and the second phase winners will receive a cash prize of up to $3,000.

最近，OpenCV人們發起了由英特爾贊助的OpenCV Spatial AI競賽 ，這是OpenCV成立20周年慶典的一部分。 競賽的主要目的是開發利用新的帶有深度的OpenCV AI套件(OAK-D)的功能的應用程序。 比賽分為兩個階段，第一階段的獲勝者可以免費獲得OAK-D來開發其應用程序，第二階段的獲勝者將獲得最高3,000美元的現金獎勵。

The OAK-D contains a 12 MP RGB camera for deep neural inference and a stereo camera for depth estimation in real time using Intel’s Myriad X Vision Processing Unit (VPU).

OAK-D包含一個12 MP RGB攝像頭，用于深度神經推理，以及一個立體聲攝像頭，用于使用英特爾的Myriad X視覺處理單元(VPU)進行實時深度估計。

If you want to know more about the OAK-D, make sure to check the interview by Ritesh Kanjee to Brandon Gilles, who is the Chief Architect of the OpenCV AI Kit. The kit has raised over $800,000 as part of their Kickstarter capaign with mote than 4,000 supporters. If you are interested, you can also find more about the kit in Luxoni’s community slack channel (https://luxonis-community.slack.com/)

如果您想進一步了解OAK-D，請確保檢查 由僅限Ritesh Kanjee布蘭登吉爾采訪 ，誰是OpenCVAI包的首席架構師。 該套件已籌集了超過800,000美元，作為其Kickstarter活動的一部分，其中有4,000多個支持者。 如果您有興趣，還可以在Luxoni的社區松弛頻道( https://luxonis-community.slack.com/ )中找到有關該工具包的更多信息。

Due to the interesting features that the OAK-D, I decided to apply for the OpenCV Spatial AI competition and was lucky enough to be selected as one of the winners of Phase 1. You can also check the projects for the rest of the Phase 1 winners here.

由于OAK-D具有有趣的功能，我決定參加OpenCV Spatial AI競賽，很幸運地被選為第1階段的獲獎者之一。您還可以檢查第1階段其余部分的項目這里的贏家。

This publication is part of a series of post where I will be describing my journey developing with the new OAK-D as part of my competition project.

該出版物是一系列帖子的一部分，我將在競賽項目中描述使用新型OAK-D進行開發的過程。

擬議制度 (Proposed System)

Illustration of how the output of the proposed system could detect people wearing a mask and their distance to the user.提出的系統的輸出如何檢測戴口罩的人及其與用戶的距離的圖示。

The title of my proposal is “Social distancing feedback for visually impaired people using a wearable camera”. Due to the current worldwide outbreak of COVID-19, social distancing has become a new social norm as a measure to prevent the widespread of the pandemic.

我建議的標題是“ 使用可穿戴式攝像頭為視障人士提供的社會疏遠反饋 ”。 由于當前在全球范圍內爆發了COVID-19，因此，社會隔離已成為一種新的社會規范，可以作為一種預防大流行的措施。

However, visually impaired people are struggling to keep independence in the new socially distanced normal1,2. For blind people, it is not possible to easily confirm if they are keeping the social distance with the people around them. As an example, a video in the Royal National Institute of Blind People (RNIB) Twitter account showed the difficulties blind people are struggling with in their daily life due to social distancing.

但是，視障人士正在努力保持與新的社會隔離的正常人的獨立性1,2,3。 對于盲人，無法輕易確認他們是否與周圍的人保持社交距離。 例如， 皇家國家盲人研究所(RNIB)Twitter帳戶中的一段視頻顯示，盲人由于社會疏遠而在日常生活中遇到的困難。

Moreover, common solutions for the blind such as white cane or dog cannot assist the blind to keep the social distance. Even worse, blind people cannot know if the people close to them is wearing a mask or not, thus they suffer a higher risk of infection.

此外，盲人的常見解決方案(如白手杖或狗)無法幫助盲人保持社交距離。 更糟糕的是，盲人無法知道附近的人是否戴著口罩，因此感染的風險更高。

For those reasons, the objective of my project is to develop a feedback system for the blind that informs about the distance to other people around and whether someone is not wearing a mask.

出于這些原因，我的項目的目的是為盲人開發一種反饋系統，該系統可以告知與周圍其他人的距離以及有人是否沒有戴口罩。

For that type of project, where the depth and Artificial Intelligence needs to be combined in real time, the OAK-D is the ideal system. As shown in one example of the DepthAI experiments, the OAK-D is able to detect in real time the position of the faces in an image and whether they are wearing a mask or not. By combining this information with the depth information obtained from the stereo cameras, it is possible to estimate the position of the people around the user and if someone is not wearing a mask.

對于那種需要實時結合深度和人工智能的項目，OAK-D是理想的系統。 如DepthAI實驗的一個示例所示，OAK-D能夠實時檢測圖像中人臉的位置以及他們是否戴著口罩。 通過將該信息與從立體攝像機獲得的深度信息結合起來，可以估計用戶周圍的人的位置以及是否有人沒有戴口罩。

Then, the system will inform the user about the distance to the people around using five haptic motors attached to the OAK-D board. The haptic motors will be related to 5 direction angles: -40, -20, 0, 20 and 40 degrees. For example, if the system detects that there is a person near at an angle of -20 degrees (as in the image above), then the second motor from the left will vibrate. Moreover, in order to inform about how close the person is, the intensity of the motor will change as the detected person gets closers. Finally, if the system detect that there is a person not wearing a mask, the system will inform the user by changing the vibration pattern.

然后，系統將使用連接到OAK-D板上的五個觸覺電機，告知用戶與周圍人的距離。 觸覺電機將與5個方向角相關：-40，-20、0、20和40度。 例如，如果系統檢測到有人在-20度角附近(如上圖所示)，則左側的第二個電動機將振動。 此外，為了告知人有多近，電動機的強度會隨著檢測到的人越來越近而變化。 最后，如果系統檢測到有人沒有戴口罩，則系統將通過更改振動模式來通知用戶。

Windows安裝程序和初始測試 (Windows Setup and Initial Testing)

This week I received the OpenCV AI Kit. As shown in the image below, the kit contains a OAK-D, a USB-C cable, a 5V (3A) wall charger and a 3D printed GoPro mount.

這周我收到了OpenCV AI工具包。 如下圖所示，該套件包含OAK-D，USB-C電纜，5V(3A)壁式充電器和3D打印的GoPro支架。

Note: The Raspberry Pi Zero was not included with the kit, it was added only for comparing dimensions.注意：套件中不包含Raspberry Pi Zero，它只是為了比較尺寸而添加的。

The OAK-D has a small size (46 x 100 mm) with a T shape. Actually, the lower part of the board matches with the width of the Raspberry Pi Zero, so the system combining both boards can have a compact size as shown in the image below.

OAK-D具有T形的小尺寸(46 x 100毫米)。 實際上，開發板的下部與Raspberry Pi Zero的寬度匹配，因此結合了兩個開發板的系統可以具有緊湊的尺寸，如下圖所示。

In order to connect with the OAK-D, the people at Luxonis have developed the DepthAI Python API. The DepthAI API is open source and can run in different Operating Systems including Ubuntu, Raspbian and macOS. In the case of Windows 10, as of today (August 8, 2020) it is still experimental. However, following the instructions descibed in here, the process was quite easy. One important note, if you do not want to have to compile the API, make sure to use the Python 3.7 (32 bit). I tried to use the Python 3.8 (32 bit) but it did not work correctly, so make sure you are using the correct Python version.

為了與OAK-D連接，Luxonis的人們開發了DepthAI Python API 。 DepthAI API是開源的，可以在包括Ubuntu，Raspbian和macOS在內的不同操作系統中運行。 對于Windows 10，截至今天(2020年8月8日)仍處于試驗階段。 但是，按照此處描述的說明進行操作非常容易。 重要提示 ，如果您不想編譯API，請確保使用Python 3.7(32位) 。 我嘗試使用Python 3.8(32位)，但無法正常工作，因此請確保您使用的是正確的Python版本。

Once I installed the depthAI API and its dependencies, I was able to run the default demo by running the following command (make sure to be in the depthai folder):

安裝depthAI API及其依賴項后，我可以通過運行以下命令來運行默認演示(確保位于depthai文件夾中)：

python depthai.py

This demo by default runs the MobileNet SSD object detection model that can detect 20 different types of objects (bicycle, car, cat…) inside an image. Moreover, the demo combines the bounding box of the detected object with the depth information of the stereo cameras to provide the 3D position of each detected object. As an example, below, I show an output of the demo for the detection of a water bottle, which is one of the classes that can detect the default demo model.

默認情況下，此演示運行MobileNet SSD對象檢測模型，該模型可以檢測圖像中的20種不同類型的對象(自行車，汽車，貓……)。 此外，該演示將檢測到的對象的邊界框與立體相機的深度信息相結合，以提供每個檢測到的對象的3D位置。 在下面的示例中，我顯示了用于檢測水瓶的演示的輸出，這是可以檢測默認演示模型的類之一。

The demo was able to track the object and calculate the depth at 30 fps without any problem. By looking at the Python code of the depthai.py script, I saw that the demo can be configured to other modes by add arguments when running the demo. For example, running the following code it is possible to obtain the colorized depth (Note: only works for the Refactory version for Windows 10, in the original repository the configuration has changed):

該演示能夠跟蹤對象并以30 fps的速度計算深度，沒有任何問題。 通過查看depthai.py腳本的Python代碼，我看到可以在運行演示時通過添加參數將演示配置為其他模式。 例如，運行以下代碼，就有可能獲得彩色深度( 注意：僅適用于Windows 10的Refactory版本 ，在原始存儲庫中，配置已更改)：

python depthai.py --streams depth_color_hDepth output using the OAK-D.使用OAK-D的深度輸出。

Overall, the depth looks pretty good with some black regions on the left of the background. However, that region contains glass panels and probably the stereo camera system cannot extract many features from it, so that why no depth was provided for those regions.

總體而言，深度看起來不錯，背景左側有一些黑色區域。 但是，該區域包含玻璃面板，因此立體相機系統可能無法從中提取許多功能，因此為什么沒有為這些區域提供深度。

深度估算：OAK-D與Azure Kinect DK (Depth Estimation: OAK-D vs. Azure Kinect DK)

Even though the depth estimation of the OAK-D is not its main feature, I wanted to compare the depth estimation of the OAK-D with the latest Azure Kinect DK. For that purpose, I modified wrote a small Python script (hello_depth.py) that reads the raw deoth values and displays the depth as in the Azure Kinect.

盡管OAK-D的深度估計不是其主要功能，但我想將OAK-D的深度估計與最新的Azure Kinect DK進行比較。 為此，我修改了編寫一個小的Python腳本( hello_depth.py )，該腳本讀取原始的牙齒值并像Azure Kinect一樣顯示深度。

As for the Azure Kinect, I used the depth estimation example program in my Python repository for the Azure Kinect SDK. In the image below the estimated depth for both devices is compared.

對于Azure Kinect，我在Python存儲庫中使用了Azure Kinect SDK的深度估算示例程序。 在下面的圖像中，比較了兩個設備的估計深度。

Comparison of the depth estimation for the Azure Kinect and the OAK-D.Azure Kinect和OAK-D的深度估計的比較。

As it can be observed, even though the OAK-D uses a estereo camera the results are very good. Particularly, in the case of the table, the OAK-D was able to estimate the depth correctly whereas the TOF sensor in the Azure Kinect failed.

可以觀察到，即使OAK-D使用酯化相機，效果也非常好。 特別是在桌子的情況下，OAK-D能夠正確估計深度，而Azure Kinect中的TOF傳感器卻無法使用。

This is all for this first part, in the next part I will test the face mask detection example using the OAK-D. I will also be uploading all the scripts for this project in my repository https://github.com/ibaiGorordo/Social-Distance-Feedback.

這是第一部分的全部內容，在下一部分中，我將使用OAK-D測試面罩檢測示例 。 我還將在我的存儲庫https://github.com/ibaiGorordo/Social-Distance-Feedback中上傳該項目的所有腳本。

翻譯自: https://towardsdatascience.com/opencv-spatial-ai-competition-journey-part-1-e76593d456fe

opencv圖像深度-1

總結

以上是生活随笔為你收集整理的opencv图像深度-1_OpenCV空间AI竞赛之旅（第1部分-初始设置+深度）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。