當前位置：首頁 > 人文社科 > 生活经验 >内容正文

生活经验

PCL：点云特征描述子3D_object_recognition_(descriptors)

發布時間：2023/11/27 生活经验 26 豆豆

生活随笔收集整理的這篇文章主要介紹了 PCL：点云特征描述子3D_object_recognition_(descriptors) 小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

PCL官網：https://pointclouds.org/

翻譯自該網站：http://robotica.unileon.es/index.php/PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)

參考鏈接：https://mp.weixin.qq.com/s/m3tlvBYrDG8wZZr7lWTOpA

http://www.pclcn.org/study/shownews.php?id=101

1、概述

Local descriptors

1）PFH

2）PFH

3)RSD ? ?? （后邊的除了VFH自己會用到，詳細理解，其他的都只是粘貼英文）

4）3DSC

5）USC

6）SHOT

7）Spin image

9）NARF

Obtaining a range image

10）Planar projection

11）Extracting borders

12）Finding keypoints

14）RoPS

Global descriptors

1）VFH

2）CVFH

3）OUR-CVFH

4）ESF

5）GFPFH

6）GRSD

Saving and loading

Visualization

PCLHistogramVisualizer

PCLPlotter

PCL Viewer

現在是學習點云處理最有趣應用之一的基礎知識的時候了:3D對象識別。與2D識別類似，該技術依賴于在云中找到好的關鍵點(特征點)，并將它們與之前保存的一組關鍵點進行匹配。但是3D比2D有幾個優勢:也就是說，我們能夠相對于傳感器精確地估計物體的確切位置和方向。此外，3D對象識別往往對于雜亂的環境有更強大的魯棒性(比如在比較擁擠的場景，前面的物體會遮擋住在后邊背景中物體)。最后，掌握物體形狀的信息將有助于避免碰撞或抓取操作。

在第一個教程中，我們將了解什么是描述符，PCL中有多少類型可用，以及如何計算它們。

1、概述

3D對象識別的基礎是在兩個不同的云之間找到一組對應關系，其中一個包含我們要尋找的對象。為此，我們需要一種以明確的方式比較點的方法。到現在為止，我們已經使用了存儲XYZ坐標，RGB顏色的點，但是這些屬性都不夠獨特。在兩次連續掃描中，盡管屬于不同表面，但兩個點仍可以共享相同的坐標，and using the color information takes us back to 2D recognition, will all the lightning related problems。

feature（特征）討論見文章：https://blog.csdn.net/m0_37957160/article/details/108405087

在之前的教程中，在介紹法線之前我們討論了特征（feature）。法線是特征的一個例子，因為它們編碼關于點附近的信息。也就是說，計算時要考慮到相鄰的點，這樣我們就可以了解周圍的表面是怎樣的了。但這還不夠。一個特征要想是最佳的，它必須滿足以下條件：

? ?? 1.It must be robust to transformations: 像平移和旋轉這樣的剛體變換(并不改變點之間距離的變換)一定不會影響特征。Even if we play with the cloud a bit beforehand, there should be no difference。

? ?? 2.It must be robust to noise:產生的噪聲的測量誤差不應對特征估計造成太大影響。（對噪聲魯棒性）

? ? 3.It must be resolution invariant: ??如果進行不同密度的采樣(如執行下降采樣)，結果必須相同或相似。

這就是描述符的作用。點的特征越復雜(和精確)，他們就會編碼大量關于周圍幾何圖形的信息。其目的是在多個點云之間明確地識別一個點，不管噪聲、分辨率或變換。另外，描述符獲取一些關于它們所屬對象的額外數據，比如視點(可以讓我們檢索姿態)。

Finding correspondences between point features of two clouds (image from http://pointclouds.org/

PCL中實現了許多3D描述符。每一個描述符都有它自己的方法來計算一個點的唯一值。例如，某些方法會使用該點及其相鄰點的法線角度之間的差異。其他方法則使用兩點之間的距離。因此，對于一定的標準目標，一些方法可能是好的也可能是壞的。一個給定的描述符可能是尺度不變的，而另一個描述符可能對于遮擋和物體局部視圖是更好的。選擇哪種描述符取決于你要做什么。

關于直方圖的介紹見鏈接：

計算完必要的值（the necessary values）之后，執行附加步驟以減小描述符的大小：并將結果合并到直方圖中。為此，將構成描述符的每個變量的值范圍劃分為n個細分，并計算每個變量中出現的次數。試著想象一個計算單個變量的描述符，它的范圍從1到100，我們選擇為它創建10個bins(容器)，因此第一個bin將收集1到10之間的所有匹配項，第二個bin將收集11到20之間的所有匹配項，依此類推。我們看一下第一個點-鄰居對的變量值，它是27，所以我們將第三個bin的值（也就是20到30的bin劃分）增加1。我們一直這樣做，直到獲得該關鍵點的最終直方圖。必須根據該變量的描述性來仔細選擇bin的大小（變量不必共享相同數量的bins，并且bins的大小也不必相同；例如，如果前一個示例中的大多數值都在50-100范圍內，那么在該范圍內使用更多較小大小的容器將是明智的)。

描述符可分為兩大類:全局描述符和局部描述符。計算和使用每一個(比如識別管道)的過程是不同的，因此本文將在各自的部分中進行解釋。

Tutorials:

How 3D Features work in PCL：鏈接如下：博客翻譯鏈接如下
Overview and Comparison of Features：鏈接如下：https://github.com/PointCloudLibrary/pcl/wiki/Overview-and-Comparison-of-Features ? 博客翻譯鏈接如下：
How does a good feature look like?鏈接如下：博客翻譯鏈接如下

Publication:

Point Cloud Library: Three-Dimensional Object Recognition and 6 DoF Pose Estimation (Aitor Aldoma et al., 2012)

下表將告訴你PCL中有多少描述符，以及它們的一些特性:

? ? ??

? Values marked with an asterisk (*) indicate that the descriptor's size depends on some parameter(s), and the one given is for the default values.

?? Descriptors without their own custom PointType use the generic "pcl::Histogram<>" type. See Saving and loading(鏈接：http://robotica.unileon.es/index.php/PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#Saving_and_loading)

Optionally, you can download a document with a less simple version of the table. Page format is A4, landscape:

Local descriptors

局部描述符是為我們作為輸入的單個點計算的。他們對物體是什么樣子沒有概念（不知道物體是什么），他們只是描述了這個點周圍的局部幾何是怎么樣的。通常，您的任務是選擇需要為哪些點計算描述符:即關鍵點。在大多數情況下，您可以通過執行下采樣并選擇所有剩余點來擺脫困境，但是可以使用關鍵點檢測器，例如用于NARF（見下方）或ISS的檢測器（見鏈接：http://robotica.unileon.es/index.php/PCL/OpenNI_tutorial_5:_3D_object_recognition_(pipeline)#ISS）。

Local descriptors are used for object recognition and registration（見鏈接：http://robotica.unileon.es/index.php/PCL/OpenNI_tutorial_3:_Cloud_processing_(advanced)#Registration）. Now we will see which ones are implemented into PCL.

1）PFH

PFH代表點特征直方圖。它是PCL提供的最重要的描述符之一，也是其他(如FPFH)的基礎。PFH試圖通過分析該點附近法線方向的差異來捕獲該點周圍的幾何信息(正因為如此，不精確的法線估計可能產生低質量的描述符)。

首先，該算法將附近的所有點配對（不僅將選定的關鍵點與其鄰居配對，而且還將鄰居與其自身配對）(First, the algorithm pairs all points in the vicinity (not just the chosen keypoint with its neighbors, but also the neighbors with themselves).)。然后，對于每一對，根據其法線計算固定坐標系。在此框架下，法線之間的差異可以用3個角度變量進行編碼。保存這些變量以及點之間的歐幾里得距離，然后在計算所有點對時將它們合并到直方圖中。 The final descriptor is the concatenation of the histograms of each variable (4 in total). （總共4個）。

-----自己個人理解-----

點特征的描述子一般是基于點坐標、法向量、曲率來描述某個點周圍的幾何特征。用點特征描述子不能提供特征之間的關系，減少了全局特征信息。因此誕生了一直基于直方圖的特征描述子：PFH--point feature histogram（點特征直方圖）。

2.PFH的原理

PFH通過參數化查詢點和緊鄰點之間的空間差異，形成了一個多維直方圖對點的近鄰進行幾何描述，直方圖提供的信息對于點云具有平移旋轉不變性，對采樣密度和噪聲點具有穩健性。PFH是基于點與其鄰近之間的關系以及它們的估計法線，也即是它考慮估計法線之間的相互關系，來描述幾何特征。

為了計算兩個點（?ps is de?ned as the source point and pt as the target point）及其相關法線之間的偏差，在其中一個點上定義了一個固定坐標系。

使用上圖的uvw坐標系，法線ns，nt之間的偏差可以用一組角度表示

d是兩點之間的歐氏距離，?，利用α,φ,θ,d，四個元素可以構成PFH描述子。

問題來了，PFH翻譯成點特征直方圖，四個元素和直方圖有什么關系？

首先計算查詢點Pq近鄰內的對應的所有四個元素，如圖所示，表示的是一個查詢點(Pq) 的PFH計算的影響區域，Pq 用紅色標注并放在圓球的中間位置，半徑為r， (Pq)的所有k鄰元素（即與點Pq的距離小于半徑r的所有點）全部互相連接在一個網絡中。最終的PFH描述子通過計算鄰域內所有兩點之間關系而得到的直方圖，因此存在一個O(k) 的計算復雜性。?

為了創建最終的直方圖，將所有四元素組以統計的方式放入一個直方圖中，這個過程首先把每個特征值范圍劃分為b個子區間，并統計落在每個子區間的點數量，前三個元素均是角度，都和法向量有關系，可以將三個元素標準化并放到同一個區間內。

橫坐標表示角度，縱坐標表示區間內點云的數量。

3.FPFH的由來

具有n個點的點云p的點特征直方圖的理論計算復雜度為o（nk^2），其中k是點云p中每個點p的鄰近數。在密集點鄰域中計算點特征柱狀圖可以表示映射框架中的主要瓶頸之一。本節提出了PFH公式的簡單化，稱為快速點特征直方圖（FPFH:fast point feature histograms），它將算法的計算復雜度降低到O（NK），同時仍然保留了PFH的大部分判別能力。

4.FPFH的原理

step1，只計算每個查詢點Pq和它鄰域點之間的三個特征元素（參考PFH），在這里不同于PFH：PFH是計算鄰域點所有組合的特征元素，而這一步只計算查詢點和近鄰點之間的特征元素。如下圖，第一個圖是PFH計算特征過程，即鄰域點所有組合的特征值（圖中所有連線，包括但不限于Pq和Pk之間的連線），第二個圖是step1中計算內容，只需要計算Pq（查詢點）和緊鄰點（圖2中紅線部分）之間的特征元素。可以看出降低了復雜度我們稱之為SPFH（simple point feature histograms）。

(看清楚兩個圖的區別，一個是PFH所有點之間的關系，一個是FPFH只考慮當前關鍵點與其相鄰節點之間的直接連接，刪除相鄰節點之間的附加連接。)

step2，重新確定k近鄰域，為了確定查詢點Pq的近鄰點Pk的SPFH值、查詢點Pq和近鄰的距離以及k的數值（一般使用半徑kdtree搜索，只能確定某半徑范圍內的近鄰點，不能確定具體的查詢點與近鄰的距離、k數值----PS:應該是這樣，不過重新確定k近鄰主要還是計算查詢點Pq的近鄰點Pk的SPFH值），則

Wk權重，一般為距離。

這里的FPFH是由查詢點的簡化特征直方圖SPFH（）加上周邊K鄰域內的各個鄰域點的SPFH()的加權和兩部分構成，SPFH（)或SPFH(pk)的每個區間內的值都是整數，且區間內的值加起來等于鄰域內的鄰域點數。但是每個區間經過后半部的加權之后，使得最終的FPFH(pq)的每個區間變成了小數。

5.二者區別和聯系

（1）FPFH沒有對近鄰點所有組合進行計算可能漏掉一些重要點對

（2）PFH特征模型是對查詢點周圍精確的鄰域半徑內，而FPFH還包括半徑r范圍以外的額外點對（不過在2r內，這是由于計算SPFH(Pk)導致的）

（3）FPFH降低了復雜度，可以在實時場景中使用

（4）因為重新計算權重，所以FPFH結合SPFH值，重新獲取重要的緊鄰點對幾何信息

（5）在FPFH中，通過分解三元組（三個角特征）簡化了合成的直方圖，即簡單地創建b個相關的的特征直方圖，每個特征維數（dimension）對應一個直方圖（bin），并將它們連接在一起。pcl默認，in PFH assume the number of quantum bins (i.e. subdivision intervals in a feature’s value range),bins（b）=5即子區間數量（為什么默認5不知道），三個角特征元素，5^3=125,（至于為什么5的3次方不知道？？？）也就是一個查詢點就有125個子區間，PFHSignature125的由來。這樣有一個問題：對于點云特別是稀疏點云來說，很多區間存在0值，即直方圖上存在冗余空間。因此，在FPFH中，通過分解三元組（三個角特征）簡化了合成的直方圖，即簡單地創建b個不相關的的特征直方圖，每個特征維數（dimension）對應一個直方圖（bin），并將它們連接在一起。pcl默認FPFH的b=11，3*11=33，也就是FPFHSignature33的由來。

參考文獻：RUSU博士論文，以及RUSU發表的會議論文Fast Point Feature Histograms (FPFH) for 3D Registration

參考鏈接：http://www.pclcn.org/study/shownews.php?id=101

-------個人理解結束，接著翻譯理解------

計算描述符在PCL是非常容易的，PFH也不例外:

#include <pcl/io/pcd_io.h>
#include <pcl/features/normal_3d.h>
#include <pcl/features/pfh.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.//儲存點云對象pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the normals.//儲存法線對象pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing the PFH descriptors for each point.建立用于存儲每個點的PFH描述符的對象。pcl::PointCloud<pcl::PFHSignature125>::Ptr descriptors(new pcl::PointCloud<pcl::PFHSignature125>());// Read a PCD file from disk.//讀取if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.（注意:您通常現在執行向下采樣。這里省略了，為了簡單起見，但是要注意計算可能會花費很長時間。）// Estimate the normals.//計算發現pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloud);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZ>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZ>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// PFH estimation object.pcl::PFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::PFHSignature125> pfh;pfh.setInputCloud(cloud);pfh.setInputNormals(normals);pfh.setSearchMethod(kdtree);// Search radius, to look for neighbors. Note: the value given here has to be// larger than the radius used to estimate the normals.pfh.setRadiusSearch(0.05);pfh.compute(*descriptors);
}

As you can see, PCL uses the "PFHSignature125" type to save the descriptor to. This means that the descriptor's size is 125 (the dimensionality of the feature vector). Dividing a feature in D dimensional space in B divisions requires a total of? bins. The original proposal makes use of the distance between the points, but the implementation of PCL does not, as it was not considered discriminative enough (specially in 2.5D scans, where the distance between points increases the further away from the sensor). Hence, the remaining features (with one dimension each) can be divided in 5 divisions, resulting in a 125-dimensional () vector.

The final object that stores the computed descriptors can be handled like a normal cloud (even saved to, or read from, disk), and it has the same number of "points" than the original one. The "PFHSignature125" object at position 0, for example, stores the PFH descriptor for the "PointXYZ" point at the same position in the cloud.

Input: Points, Normals, Search method, Radius
Output: PFH descriptors
Tutorial: Point Feature Histograms (PFH) descriptors
Publications:
- Persistent Point Feature Histograms for 3D Point Clouds (Radu Bogdan Rusu et al., 2008)
- Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments (Radu Bogdan Rusu, 2009, section 4.4, page 50)
API: pcl::PFHEstimation(官網API函數)
Download（這個鏈接沒有）

2）PFH

? PFH給出了準確的結果，但它有一個缺點:它的計算成本太高，無法實時執行。對于考慮了n個關鍵點和k個鄰居的云，它的復雜度為。因此，一個派生描述符被創建，名為FPFH (Fast Point Feature Histogram)。

FPFH只考慮當前關鍵點與其相鄰節點之間的直接連接，刪除相鄰節點之間的附加連接。這將復雜性降低到O(nk)。因此，得到的直方圖稱為SPFH(簡化點特征直方圖)。參考系和角變量一如既往地計算。

FPFH只考慮當前關鍵點與其相鄰節點之間的直接連接，刪除相鄰節點之間的附加連接。這將復雜性降低到O(nk)。因此，得到的直方圖稱為SPFH(簡化點特征直方圖)。參考系和角變量一如既往地計算。（The reference frame and the angular variables are computed as always. ）

為了解決這些額外連接的丟失，在計算完所有直方圖之后，需要執行一個附加步驟：將點的鄰居的SPFH與自己的鄰居的SPFH“合并”，并根據距離加權。這樣的效果是可以給出遠至所用半徑2倍的點的點表面信息。最后，將3個直方圖（不使用距離）連接起來組成最終的描述符。

(To account for the loss of these extra connections, an additional step takes place after all histograms have been computed: the SPFHs of a point's neighbors are "merged" with its own, weighted according to the distance. This has the effect of giving a point surface information of points as far away as 2 times the radius used. Finally, the 3 histograms (distance is not used) are concatenated to compose the final descriptor. )

#include <pcl/io/pcd_io.h>
#include <pcl/features/normal_3d.h>
#include <pcl/features/fpfh.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the normals.pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing the FPFH descriptors for each point.pcl::PointCloud<pcl::FPFHSignature33>::Ptr descriptors(new pcl::PointCloud<pcl::FPFHSignature33>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.// Estimate the normals.pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloud);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZ>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZ>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// FPFH estimation object.pcl::FPFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::FPFHSignature33> fpfh;fpfh.setInputCloud(cloud);fpfh.setInputNormals(normals);fpfh.setSearchMethod(kdtree);// Search radius, to look for neighbors. Note: the value given here has to be// larger than the radius used to estimate the normals.fpfh.setRadiusSearch(0.05);fpfh.compute(*descriptors);
}

類“ FPFHEstimationOMP”中提供了利用多線程優化（帶有OpenMP API）的FPFH估計的其他實現。其接口與標準未優化的實現相同。使用它會大大提高多核系統的性能，這意味著更快的計算時間。請記住要改為包含標題“ fpfh_omp.h”。

Input: Points, Normals, Search method, Radius
Output: FPFH descriptors
Tutorial: Fast Point Feature Histograms (FPFH) descriptors
Publications:
- Fast Point Feature Histograms (FPFH) for 3D Registration (Radu Bogdan Rusu et al., 2009)
- Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments (Radu Bogdan Rusu, 2009, section 4.5, page 57)
API:
- pcl::FPFHEstimation
- pcl::FPFHEstimationOMP
Download

3)RSD ? ?? （后邊的除了VFH自己會用到，詳細理解，其他的都只是粘貼英文）

先翻譯未深入理解，用到再看

The Radius-Based Surface Descriptor encodes the radial relationship of the point and its neighborhood. For every pair of the keypoint with a neighbor, the algorithm computes the distance between them, and the difference between their normals. Then, by assuming that both points lie on the surface of a sphere, said sphere is found by fitting not only the points, but also the normals (otherwise, there would be infinite possible spheres). Finally, from all the point-neighbor spheres, only the ones with the maximum and minimum radii are kept and saved to the descriptor of that point.

As you may have deduced already, when two points lie on a flat surface, the sphere radius will be infinite. If, on the other hand, they lie on the curved face of a cylinder, the radius will be more or less the same as that of the cylinder. This allows us to tell objects apart with RSD. The algorithm takes a parameter that sets the maximum radius at which the points will be considered to be part of a plane.

? ? 基于半徑的表面描述符對點及其鄰域的徑向關系進行編碼。對于與鄰居的每對關鍵點，算法都會計算它們之間的距離以及它們的法線之間的差。然后，通過假設兩個點都位于球體的表面上，不僅可以擬合這些點，還可以擬合法線來找到該球體（否則，將存在無限可能的球體）。最后，在所有點相鄰球體中，只有半徑最大和最小的球體才會保留并保存到該點的描述符中。
? ? 正如您可能已經推論的那樣，當兩個點位于一個平面上時，球體半徑將是無限的。另一方面，如果它們位于圓柱體的曲面上，則半徑將與圓柱體的半徑大致相同。這使我們可以區分對象與RSD。該算法采用一個參數，該參數設置將點視為平面一部分的最大半徑。

這是編譯RSD描述符的代碼:

#include <pcl/io/pcd_io.h>
#include <pcl/features/normal_3d.h>
#include <pcl/features/rsd.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the normals.pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing the RSD descriptors for each point.pcl::PointCloud<pcl::PrincipalRadiiRSD>::Ptr descriptors(new pcl::PointCloud<pcl::PrincipalRadiiRSD>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.// Estimate the normals.pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloud);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZ>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZ>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// RSD estimation object.pcl::RSDEstimation<pcl::PointXYZ, pcl::Normal, pcl::PrincipalRadiiRSD> rsd;rsd.setInputCloud(cloud);rsd.setInputNormals(normals);rsd.setSearchMethod(kdtree);// Search radius, to look for neighbors. Note: the value given here has to be// larger than the radius used to estimate the normals.rsd.setRadiusSearch(0.05);// Plane radius. Any radius larger than this is considered infinite (a plane).rsd.setPlaneRadius(0.1);// Do we want to save the full distance-angle histograms?rsd.setSaveHistograms(false);rsd.compute(*descriptors);
}

NOTE: This code will only compile with PCL versions 1.8 and above (the current trunk).

Optionally, you can use the "setSaveHistograms()" function to enable the saving of the full distance-angle histograms, and then use "getHistograms()" to retrieve them.

Input: Points, Normals, Search method, Radius, Maximum Radius, [Subdivisions], [Save full histograms]
Output: RSD descriptors
Publications:
- General 3D Modelling of Novel Objects from a Single View (Zoltan-Csaba Marton et al., 2010)
- Combined 2D-3D Categorization and Classification for Multimodal Perception Systems (Zoltan-Csaba Marton et al., 2011)
API: pcl::RSDEstimation
Download

4）3DSC

The 3D Shape Context is a descriptor that extends its existing 2D counterpart to the third dimension. It works by creating a support structure (a sphere, to be precise) centered at the point we are computing the descriptor for, with the given search radius. The "north pole" of that sphere (the notion of "up") is pointed to match the normal at that point. Then, the sphere is divided in 3D regions or bins. In the first 2 coordinates (azimuth and elevation) the divisions are equally spaced, but in the third (the radial dimension), divisions are logarithmically spaced so they are smaller towards the center. A minimum radius can be specified to prevent very small bins, that would be too sensitive to small changes in the surface.

For each bin, a weighted count is accumulated for every neighboring point that lies within. The weight depends on the volume of the bin and the local point density (number of points around the current neighbor). This gives the descriptor some degree of resolution invariance.

We have mentioned that the sphere is given the direction of the normal. This still leaves one degree of freedom (only two axes have been locked, the azimuth remains free). Because of this, the descriptor so far does not cope with rotation. To overcome this (so the same point in two different clouds has the same value), the support sphere is rotated around the normal N times (a number of degrees that corresponds with the divisions in the azimuth) and the process is repeated for each, giving a total of N descriptors for that point.

You can compute the 3DSC descriptor the following way:

#include <pcl/io/pcd_io.h>
#include <pcl/features/normal_3d.h>
#include <pcl/features/3dsc.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the normals.pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing the 3DSC descriptors for each point.pcl::PointCloud<pcl::ShapeContext1980>::Ptr descriptors(new pcl::PointCloud<pcl::ShapeContext1980>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.// Estimate the normals.pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloud);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZ>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZ>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// 3DSC estimation object.pcl::ShapeContext3DEstimation<pcl::PointXYZ, pcl::Normal, pcl::ShapeContext1980> sc3d;sc3d.setInputCloud(cloud);sc3d.setInputNormals(normals);sc3d.setSearchMethod(kdtree);// Search radius, to look for neighbors. It will also be the radius of the support sphere.sc3d.setRadiusSearch(0.05);// The minimal radius value for the search sphere, to avoid being too sensitive// in bins close to the center of the sphere.sc3d.setMinimalRadius(0.05 / 10.0);// Radius used to compute the local point density for the neighbors// (the density is the number of points within that radius).sc3d.setPointDensityRadius(0.05 / 5.0);sc3d.compute(*descriptors);
}

Input: Points, Normals, Search method, Radius, Minimal radius, Point density radius
Output: 3DSC descriptors
Publication:
- Recognizing Objects in Range Data Using Regional Point Descriptors (Andrea Frome et al., 2004)
API: pcl::ShapeContext3DEstimation
Download

5）USC

The Unique Shape Context descriptor extends the 3DSC by defining a local reference frame, in order to provide an unique orientation for each point. This not only improves the accuracy of the descriptor, it also reduces its size, as computing multiple descriptors to account for orientation is no longer necessary.

You can check the second publication listed below to learn more about how the LRF is computed.

#include <pcl/io/pcd_io.h>
#include <pcl/features/usc.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the USC descriptors for each point.pcl::PointCloud<pcl::UniqueShapeContext1960>::Ptr descriptors(new pcl::PointCloud<pcl::UniqueShapeContext1960>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.// USC estimation object.pcl::UniqueShapeContext<pcl::PointXYZ, pcl::UniqueShapeContext1960, pcl::ReferenceFrame> usc;usc.setInputCloud(cloud);// Search radius, to look for neighbors. It will also be the radius of the support sphere.usc.setRadiusSearch(0.05);// The minimal radius value for the search sphere, to avoid being too sensitive// in bins close to the center of the sphere.usc.setMinimalRadius(0.05 / 10.0);// Radius used to compute the local point density for the neighbors// (the density is the number of points within that radius).usc.setPointDensityRadius(0.05 / 5.0);// Set the radius to compute the Local Reference Frame.usc.setLocalRadius(0.05);usc.compute(*descriptors);
}

NOTE: This code will only compile with PCL versions 1.8 and above (the current trunk). For 1.7 and below, change UniqueShapeContext1960 to ShapeContext1980, and edit CMakeLists.txt.

Input: Points, Radius, Minimal radius, Point density radius, Local radius
Output: USC descriptors
Publication:
- Unique Shape Context for 3D Data Description (Federico Tombari et al., 2010)
API: pcl::UniqueShapeContext
Download

6）SHOT

SHOT stands for Signature of Histograms of Orientations. Like 3DSC, it encodes information about the topology (surface) withing a spherical support structure. This sphere is divided in 32 bins or volumes, with 8 divisions along the azimuth, 2 along the elevation, and 2 along the radius. For every volume, a one-dimensional local histogram is computed. The variable chosen is the angle between the normal of the keypoint and the current point within that volume (to be precise, the cosine, which was found to be better suitable).

When all local histograms have been computed, they are stitched together in a final descriptor. Like the USC descriptor, SHOT makes use of a local reference frame, making it rotation invariant. It is also robust to noise and clutter.

#include <pcl/io/pcd_io.h>
#include <pcl/features/normal_3d.h>
#include <pcl/features/shot.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the normals.pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing the SHOT descriptors for each point.pcl::PointCloud<pcl::SHOT352>::Ptr descriptors(new pcl::PointCloud<pcl::SHOT352>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.// Estimate the normals.pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloud);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZ>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZ>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// SHOT estimation object.pcl::SHOTEstimation<pcl::PointXYZ, pcl::Normal, pcl::SHOT352> shot;shot.setInputCloud(cloud);shot.setInputNormals(normals);// The radius that defines which of the keypoint's neighbors are described.// If too large, there may be clutter, and if too small, not enough points may be found.shot.setRadiusSearch(0.02);shot.compute(*descriptors);
}

Like with FPFH, a multithreading-optimized variant is available with "SHOTEstimationOMP", that makes use of OpenMP. You need to include the header "shot_omp.h". Also, another variant that uses the texture for matching is available, "SHOTColorEstimation", with an optimized version too (see the second publication for more details). It outputs a "SHOT1344" descriptor.

Input: Points, Normals, Radius
Output: SHOT descriptors
Publications:
- Unique Signatures of Histograms for Local Surface Description (Federico Tombari et al., 2010)
- A Combined Texture-Shaped Descriptor for Enhanced 3D Feature Matching (Federico Tombari et al., 2011)
API:
- pcl::SHOTEstimation,
- pcl::SHOTColorEstimation
- pcl::SHOTEstimationOMP
- pcl::SHOTColorEstimationOMP
Download

7）Spin image

The Spin Image (SI) is the oldest descriptor we are going to see here. It has been around since 1997, but it still sees some use for certain applications. It was originally designed to describe surfaces made by vertices, edges and polygons, but it has been since adapted for point clouds. The descriptor is unlike all others in that the output resembles an image that can be compared with another with the usual means.

The support structure used is a cylinder, centered at the point, with a given radius and height, and aligned with the normal. This cylinder is divided radially and vertically into volumes. For each one, the number of neighbors lying inside is added up, eventually producing a descriptor. Weighting and interpolation are used to improve the result. The final descriptor can be seen as a grayscale image where dark areas correspond to volumes with higher point density.

#include <pcl/io/pcd_io.h>
#include <pcl/features/normal_3d.h>
#include <pcl/features/spin_image.h>// A handy typedef.
typedef pcl::Histogram<153> SpinImage;int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the normals.pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing the spin image for each point.pcl::PointCloud<SpinImage>::Ptr descriptors(new pcl::PointCloud<SpinImage>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.// Estimate the normals.pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloud);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZ>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZ>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// Spin image estimation object.pcl::SpinImageEstimation<pcl::PointXYZ, pcl::Normal, SpinImage> si;si.setInputCloud(cloud);si.setInputNormals(normals);// Radius of the support cylinder.si.setRadiusSearch(0.02);// Set the resolution of the spin image (the number of bins along one dimension).// Note: you must change the output histogram size to reflect this.si.setImageWidth(8);si.compute(*descriptors);
}

The Spin Image estimation object provides more methods for tuning the estimation, so checking the API is recommended.

Input: Points, Normals, Radius, Image resolution
Output: Spin images
Publications:
- Spin-Images: A Representation for 3-D Surface Matching (Andrew Edie Johnson, 1997)
- Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes (Andrew Edie Johnson and Martial Hebert, 1999)
API: pcl::SpinImageEstimation
Download

8）RIFT

The Rotation-Invariant Feature Transform, like the spin image, takes some concepts from 2D features, in this case from the Scale-Invariant Feature Transform (SIFT). It is the only descriptor seen here that requires intensity information in order to compute it (it can be obtained from the RGB color values). This means, of course, that you will not be able to use RIFT with standard XYZ clouds, you also need the texture.

In the first step, a circular patch (with the given radius) is fitted on the surface the point lies on. This patch is divided into concentric rings, according to the chosen distance bin size. Then, an histogram is populated with all the point's neighbors lying inside a sphere centered at that point and with the mentioned radius. The distance and the orientation of the intensity gradient at each point are considered. To make it rotation invariant, the angle between the gradient orientation and the vector pointing outward from the center of the patch is measured.

The authors' original implementation uses 4 rings and 8 histogram orientations, which produce a descriptor of size 32. RIFT is not robust to texture flipping, though this was never considered a big issue.

#include <pcl/io/pcd_io.h>
#include <pcl/point_types_conversion.h>
#include <pcl/features/normal_3d.h>
#include <pcl/features/intensity_gradient.h>
#include <pcl/features/rift.h>// A handy typedef.
typedef pcl::Histogram<32> RIFT32;int
main(int argc, char** argv)
{// Object for storing the point cloud with color information.pcl::PointCloud<pcl::PointXYZRGB>::Ptr cloudColor(new pcl::PointCloud<pcl::PointXYZRGB>);// Object for storing the point cloud with intensity value.pcl::PointCloud<pcl::PointXYZI>::Ptr cloudIntensity(new pcl::PointCloud<pcl::PointXYZI>);// Object for storing the intensity gradients.pcl::PointCloud<pcl::IntensityGradient>::Ptr gradients(new pcl::PointCloud<pcl::IntensityGradient>);// Object for storing the normals.pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing the RIFT descriptor for each point.pcl::PointCloud<RIFT32>::Ptr descriptors(new pcl::PointCloud<RIFT32>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZRGB>(argv[1], *cloudColor) != 0){return -1;}// Note: you would usually perform downsampling now. It has been omitted here// for simplicity, but be aware that computation can take a long time.// Convert the RGB to intensity.pcl::PointCloudXYZRGBtoXYZI(*cloudColor, *cloudIntensity);// Estimate the normals.pcl::NormalEstimation<pcl::PointXYZI, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloudIntensity);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZI>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZI>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// Compute the intensity gradients.pcl::IntensityGradientEstimation < pcl::PointXYZI, pcl::Normal, pcl::IntensityGradient,pcl::common::IntensityFieldAccessor<pcl::PointXYZI> > ge;ge.setInputCloud(cloudIntensity);ge.setInputNormals(normals);ge.setRadiusSearch(0.03);ge.compute(*gradients);// RIFT estimation object.pcl::RIFTEstimation<pcl::PointXYZI, pcl::IntensityGradient, RIFT32> rift;rift.setInputCloud(cloudIntensity);rift.setSearchMethod(kdtree);// Set the intensity gradients to use.rift.setInputGradient(gradients);// Radius, to get all neighbors within.rift.setRadiusSearch(0.02);// Set the number of bins to use in the distance dimension.rift.setNrDistanceBins(4);// Set the number of bins to use in the gradient orientation dimension.rift.setNrGradientBins(8);// Note: you must change the output histogram size to reflect the previous values.rift.compute(*descriptors);
}

Input: Points, Search method, Intensity gradients, Radius, Distance bins, Gradient bins
Output: RIFT descriptors
Publication:
- A Sparse Texture Representation Using Local Affine Regions (Svetlana Lazebnik et al., 2005)
API:
- pcl::RIFTEstimation
- pcl::IntensityGradientEstimation
- pcl::IntensityGradient
Download

9）NARF

The Normal Aligned Radial Feature is the only descriptor here that does not take a point cloud as input. Instead, it works with range images. A range image is a common RGB image in which the distance to the point that corresponds to a certain pixel is encoded as a color value in the visible light spectrum: the points that are closer to the camera would be violet, while the points near the maximum sensor range would be red.

NARF also requires us to find suitable keypoints to compute the descriptor for. NARF keypoints are located near an object's corners, and this also requires to find the borders (transitions from foreground to background), which are trivial to find with a range image. Because of this lengthy pipeline, I will describe the whole process in different sections.

Obtaining a range image

Because we always work with point clouds, I will now explain how you can convert one into a range image, in order to use it for the NARF descriptor. PCL provides a couple of handy classes to perform the conversion, given that you fill the camera data correctly.

A range image can be created in two ways. First, we can use spherical projection, which would give us an image similar to the ones produced by a LIDAR sensor. Second, we can use planar projection, which is better suitable for camera-like sensors as the Kinect or the Xtion, and will not have the characteristic distortion of the first one.

Spherical projection

The following code will take a point cloud and create a range image from it, using spherical projection:

#include <pcl/io/pcd_io.h>
#include <pcl/range_image/range_image.h>
#include <pcl/visualization/range_image_visualizer.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Parameters needed by the range image object:// Angular resolution is the angular distance between pixels.// Kinect: 57° horizontal FOV, 43° vertical FOV, 640x480 (chosen here).// Xtion: 58° horizontal FOV, 45° vertical FOV, 640x480.float angularResolutionX = (float)(57.0f / 640.0f * (M_PI / 180.0f));float angularResolutionY = (float)(43.0f / 480.0f * (M_PI / 180.0f));// Maximum horizontal and vertical angles. For example, for a full panoramic scan,// the first would be 360o. Choosing values that adjust to the real sensor will// decrease the time it takes, but don't worry. If the values are bigger than// the real ones, the image will be automatically cropped to discard empty zones.float maxAngleX = (float)(60.0f * (M_PI / 180.0f));float maxAngleY = (float)(50.0f * (M_PI / 180.0f));// Sensor pose. Thankfully, the cloud includes the data.Eigen::Affine3f sensorPose = Eigen::Affine3f(Eigen::Translation3f(cloud->sensor_origin_[0],cloud->sensor_origin_[1],cloud->sensor_origin_[2])) *Eigen::Affine3f(cloud->sensor_orientation_);// Noise level. If greater than 0, values of neighboring points will be averaged.// This would set the search radius (e.g., 0.03 == 3cm).float noiseLevel = 0.0f;// Minimum range. If set, any point closer to the sensor than this will be ignored.float minimumRange = 0.0f;// Border size. If greater than 0, a border of "unobserved" points will be left// in the image when it is cropped.int borderSize = 1;// Range image object.pcl::RangeImage rangeImage;rangeImage.createFromPointCloud(*cloud, angularResolutionX, angularResolutionY,maxAngleX, maxAngleY, sensorPose, pcl::RangeImage::CAMERA_FRAME,noiseLevel, minimumRange, borderSize);// Visualize the image.pcl::visualization::RangeImageVisualizer viewer("Range image");viewer.showRangeImage(rangeImage);while (!viewer.wasStopped()){viewer.spinOnce();// Sleep 100ms to go easy on the CPU.pcl_sleep(0.1);}
}

Here you can see an example of the output range image:

Input: Points, Angular resolution, Maximum angle, Sensor pose, Coordinate frame, Noise level, Maximum range, Border size
Output: Range image (spherical projection)
Tutorials:
- How to create a range image from a point cloud
- How to visualize a range image
API: pcl::RangeImage
Download

10）Planar projection

As mentioned, planar projection will give better results with clouds taken from a depth camera:

#include <pcl/io/pcd_io.h>
#include <pcl/range_image/range_image_planar.h>
#include <pcl/visualization/range_image_visualizer.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Parameters needed by the planar range image object:// Image size. Both Kinect and Xtion work at 640x480.int imageSizeX = 640;int imageSizeY = 480;// Center of projection. here, we choose the middle of the image.float centerX = 640.0f / 2.0f;float centerY = 480.0f / 2.0f;// Focal length. The value seen here has been taken from the original depth images.// It is safe to use the same value vertically and horizontally.float focalLengthX = 525.0f, focalLengthY = focalLengthX;// Sensor pose. Thankfully, the cloud includes the data.Eigen::Affine3f sensorPose = Eigen::Affine3f(Eigen::Translation3f(cloud->sensor_origin_[0],cloud->sensor_origin_[1],cloud->sensor_origin_[2])) *Eigen::Affine3f(cloud->sensor_orientation_);// Noise level. If greater than 0, values of neighboring points will be averaged.// This would set the search radius (e.g., 0.03 == 3cm).float noiseLevel = 0.0f;// Minimum range. If set, any point closer to the sensor than this will be ignored.float minimumRange = 0.0f;// Planar range image object.pcl::RangeImagePlanar rangeImagePlanar;rangeImagePlanar.createFromPointCloudWithFixedSize(*cloud, imageSizeX, imageSizeY,centerX, centerY, focalLengthX, focalLengthX,sensorPose, pcl::RangeImage::CAMERA_FRAME,noiseLevel, minimumRange);// Visualize the image.pcl::visualization::RangeImageVisualizer viewer("Planar range image");viewer.showRangeImage(rangeImagePlanar);while (!viewer.wasStopped()){viewer.spinOnce();// Sleep 100ms to go easy on the CPU.pcl_sleep(0.1);}
}

Input: Points, Image size, Projection center, Focal length, Sensor pose, Coordinate frame, Noise level, Maximum range
Output: Range image (planar projection)
Tutorials:
- How to create a range image from a point cloud
- How to visualize a range image
API: pcl::RangeImagePlanar
Download

If you prefer to do the conversion in real time while you inspect the cloud, PCL ships with an example that fetches an "openni_wrapper::DepthImage" from an OpenNI device and creates the range image from it. You can adapt the code of the example example from tutorial 1 to save it to disk with the function pcl::io::saveRangeImagePlanarFilePNG().

11）Extracting borders

NARF keypoints are located near the edges of objects in the range image, so in order to find them, we first have to extract the borders. A border is defined as an abrupt change from foreground to background. In a range image, this can be easily seen because there is a "jump" in the depth value of two adjacent pixels.

There are three types of borders. Object borders consist of the pixels (or points) located on the very edge of an object (the outermost points still belonging to the object). Shadow borders are points in the background on the edge of occlusions (empty areas in the background due to the objects in front covering them). Notice that, when the cloud is seen from the sensor's viewpoint, object and shadow points will seem adjacent. Finally, veil points are points interpolated between the previous two which appear in scans taken with LIDAR sensors, so we do not have to worry about them here.

The algorithm basically compares a point's depth with the values of its neighbors, and if a big difference is found, we know it is due to a border. Points closer to the sensor will be marked as object borders, and the other ones as shadow borders.

PCL provides a class for extracting borders of a range image:

#include <pcl/io/pcd_io.h>
#include <pcl/range_image/range_image_planar.h>
#include <pcl/features/range_image_border_extractor.h>
#include <pcl/visualization/range_image_visualizer.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the borders.pcl::PointCloud<pcl::BorderDescription>::Ptr borders(new pcl::PointCloud<pcl::BorderDescription>);// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Convert the cloud to range image.int imageSizeX = 640, imageSizeY = 480;float centerX = (640.0f / 2.0f), centerY = (480.0f / 2.0f);float focalLengthX = 525.0f, focalLengthY = focalLengthX;Eigen::Affine3f sensorPose = Eigen::Affine3f(Eigen::Translation3f(cloud->sensor_origin_[0],cloud->sensor_origin_[1],cloud->sensor_origin_[2])) *Eigen::Affine3f(cloud->sensor_orientation_);float noiseLevel = 0.0f, minimumRange = 0.0f;pcl::RangeImagePlanar rangeImage;rangeImage.createFromPointCloudWithFixedSize(*cloud, imageSizeX, imageSizeY,centerX, centerY, focalLengthX, focalLengthX,sensorPose, pcl::RangeImage::CAMERA_FRAME,noiseLevel, minimumRange);// Border extractor object.pcl::RangeImageBorderExtractor borderExtractor(&rangeImage);borderExtractor.compute(*borders);// Visualize the borders.pcl::visualization::RangeImageVisualizer* viewer = NULL;viewer = pcl::visualization::RangeImageVisualizer::getRangeImageBordersWidget(rangeImage,-std::numeric_limits<float>::infinity(),std::numeric_limits<float>::infinity(),false, *borders, "Borders");while (!viewer->wasStopped()){viewer->spinOnce();// Sleep 100ms to go easy on the CPU.pcl_sleep(0.1);}
}

You can use the extractor's "getParameters()" function to get a pcl::RangeImageBorderExtractor::Parameters struct with the settings that will be used.

Input: Range image
Output: Borders
Tutorial: How to extract borders from range images
Publication:
- Point Feature Extraction on 3D Range Scans Taking into Account Object Boundaries (Bastian Steder et al., 2011)
API: pcl::RangeImageBorderExtractor
Download

12）Finding keypoints

Citing the original publication: 文章鏈接見：

"We have the following requirements for our interest point extraction procedure:

It must take information about borders and the surface structure into account.
It must select positions that can be reliably detected even if the object is observed from another perspective.
The points must be on positions that provide stable areas for normal estimation or the descriptor calculation in general."

The procedure is the following: for every point in the range image, a score is computed that conveys how much the surface changes in its neighborhood (this is tuned with the support size σ, which is the diameter of the sphere used to find neighboring points). Also, the dominant direction of this change is computed. Then, this direction is compared with those of the neighbors, trying to find how stable the point is (if the directions are very different, that means the point is not stable, and that the surface around changes a lot). Points that are near the object's corners (but not exactly on the very edge) will be good keypoints, yet stable enough.

In PCL, NARF keypoints can be found this way:

#include <pcl/io/pcd_io.h>
#include <pcl/range_image/range_image_planar.h>
#include <pcl/features/range_image_border_extractor.h>
#include <pcl/keypoints/narf_keypoint.h>
#include <pcl/visualization/range_image_visualizer.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the keypoints' indices.pcl::PointCloud<int>::Ptr keypoints(new pcl::PointCloud<int>);// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Convert the cloud to range image.int imageSizeX = 640, imageSizeY = 480;float centerX = (640.0f / 2.0f), centerY = (480.0f / 2.0f);float focalLengthX = 525.0f, focalLengthY = focalLengthX;Eigen::Affine3f sensorPose = Eigen::Affine3f(Eigen::Translation3f(cloud->sensor_origin_[0],cloud->sensor_origin_[1],cloud->sensor_origin_[2])) *Eigen::Affine3f(cloud->sensor_orientation_);float noiseLevel = 0.0f, minimumRange = 0.0f;pcl::RangeImagePlanar rangeImage;rangeImage.createFromPointCloudWithFixedSize(*cloud, imageSizeX, imageSizeY,centerX, centerY, focalLengthX, focalLengthX,sensorPose, pcl::RangeImage::CAMERA_FRAME,noiseLevel, minimumRange);pcl::RangeImageBorderExtractor borderExtractor;// Keypoint detection object.pcl::NarfKeypoint detector(&borderExtractor);detector.setRangeImage(&rangeImage);// The support size influences how big the surface of interest will be,// when finding keypoints from the border information.detector.getParameters().support_size = 0.2f;detector.compute(*keypoints);// Visualize the keypoints.pcl::visualization::RangeImageVisualizer viewer("NARF keypoints");viewer.showRangeImage(rangeImage);for (size_t i = 0; i < keypoints->points.size(); ++i){viewer.markPoint(keypoints->points[i] % rangeImage.width,keypoints->points[i] / rangeImage.width,// Set the color of the pixel to red (the background// circle is already that color). All other parameters// are left untouched, check the API for more options.pcl::visualization::Vector3ub(1.0f, 0.0f, 0.0f));}while (!viewer.wasStopped()){viewer.spinOnce();// Sleep 100ms to go easy on the CPU.pcl_sleep(0.1);}
}

Input: Range image, Border extractor, Support Size
Output: NARF keypoints
Tutorial: How to extract NARF keypoint from a range image
Publication:
- Point Feature Extraction on 3D Range Scans Taking into Account Object Boundaries (Bastian Steder et al., 2011)
API: pcl::NarfKeypoint
Download

13）Computing the descriptor

We have created the range image from a point cloud, and we have extracted the borders in order to find good keypoints. Now it is time to compute the NARF descriptor for each keypoint.

The NARF descriptor encodes information about surface changes around a point. First, a local range patch is created around the point. It is like a small range image centered at that point, aligned with the normal (it would seem as if we were looking at the point along the normal). Then, a star pattern with n beams is overlaid onto the patch, also centered at the point. For every beam, a value is computed, that reflects how much the surface under it changes. The stronger the change is, and the closer to the center it is, the higher the final value will be. The n resulting values compose the final descriptor.

The descriptor right now is not invariant to rotations around the normal. To achieve this, the whole possible 360 degrees are binned into an histogram. The value of each bin is computed from the descriptor values according to the angle. Then, the bin with the highest value is considered the dominant orientation, and the descriptor is shifted according to it.

#include <pcl/io/pcd_io.h>
#include <pcl/range_image/range_image_planar.h>
#include <pcl/features/range_image_border_extractor.h>
#include <pcl/keypoints/narf_keypoint.h>
#include <pcl/features/narf_descriptor.h>int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the keypoints' indices.pcl::PointCloud<int>::Ptr keypoints(new pcl::PointCloud<int>);// Object for storing the NARF descriptors.pcl::PointCloud<pcl::Narf36>::Ptr descriptors(new pcl::PointCloud<pcl::Narf36>);// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Convert the cloud to range image.int imageSizeX = 640, imageSizeY = 480;float centerX = (640.0f / 2.0f), centerY = (480.0f / 2.0f);float focalLengthX = 525.0f, focalLengthY = focalLengthX;Eigen::Affine3f sensorPose = Eigen::Affine3f(Eigen::Translation3f(cloud->sensor_origin_[0],cloud->sensor_origin_[1],cloud->sensor_origin_[2])) *Eigen::Affine3f(cloud->sensor_orientation_);float noiseLevel = 0.0f, minimumRange = 0.0f;pcl::RangeImagePlanar rangeImage;rangeImage.createFromPointCloudWithFixedSize(*cloud, imageSizeX, imageSizeY,centerX, centerY, focalLengthX, focalLengthX,sensorPose, pcl::RangeImage::CAMERA_FRAME,noiseLevel, minimumRange);// Extract the keypoints.pcl::RangeImageBorderExtractor borderExtractor;pcl::NarfKeypoint detector(&borderExtractor);detector.setRangeImage(&rangeImage);detector.getParameters().support_size = 0.2f;detector.compute(*keypoints);// The NARF estimator needs the indices in a vector, not a cloud.std::vector<int> keypoints2;keypoints2.resize(keypoints->points.size());for (unsigned int i = 0; i < keypoints->size(); ++i)keypoints2[i] = keypoints->points[i];// NARF estimation object.pcl::NarfDescriptor narf(&rangeImage, &keypoints2);// Support size: choose the same value you used for keypoint extraction.narf.getParameters().support_size = 0.2f;// If true, the rotation invariant version of NARF will be used. The histogram// will be shifted according to the dominant orientation to provide robustness to// rotations around the normal.narf.getParameters().rotation_invariant = true;narf.compute(*descriptors);
}

Input: Range image, Key points, Support Size
Output: NARF descriptors
Tutorial: How to extract NARF Features from a range image
Publication:
- Point Feature Extraction on 3D Range Scans Taking into Account Object Boundaries (Bastian Steder et al., 2011)
API: pcl::NarfDescriptor
Download

14）RoPS

The Rotational Projection Statistics (RoPS) feature is a bit different from the other descriptors because it works with a triangle mesh, so a previous triangulation step is needed for generating this mesh from the cloud. Apart from that, most concepts are similar.

In order to compute RoPS for a keypoint, the local surface is cropped according to a support radius, so only points and triangles lying inside are taken into account. Then, a local reference frame (LRF) is computed, giving the descriptor its rotational invariance. A coordinate system is created with the point as the origin, and the axes aligned with the LRF. Then, for every axis, several steps are performed.

First, the local surface is rotated around the current axis. The angle is determined by one of the parameters, which sets the number of rotations. Then, all points in the local surface are projected onto the XY, XZ and YZ planes. For each one, statistical information about the distribution of the projected points is computed, and concatenated to form the final descriptor.

#include <pcl/io/pcd_io.h>
#include <pcl/point_types_conversion.h>
#include <pcl/features/normal_3d.h>
#include <pcl/surface/gp3.h>
#include <pcl/features/rops_estimation.h>// A handy typedef.
typedef pcl::Histogram<135> ROPS135;int
main(int argc, char** argv)
{// Object for storing the point cloud.pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);// Object for storing the normals.pcl::PointCloud<pcl::Normal>::Ptr normals(new pcl::PointCloud<pcl::Normal>);// Object for storing both the points and the normals.pcl::PointCloud<pcl::PointNormal>::Ptr cloudNormals(new pcl::PointCloud<pcl::PointNormal>);// Object for storing the ROPS descriptor for each point.pcl::PointCloud<ROPS135>::Ptr descriptors(new pcl::PointCloud<ROPS135>());// Read a PCD file from disk.if (pcl::io::loadPCDFile<pcl::PointXYZ>(argv[1], *cloud) != 0){return -1;}// Estimate the normals.pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normalEstimation;normalEstimation.setInputCloud(cloud);normalEstimation.setRadiusSearch(0.03);pcl::search::KdTree<pcl::PointXYZ>::Ptr kdtree(new pcl::search::KdTree<pcl::PointXYZ>);normalEstimation.setSearchMethod(kdtree);normalEstimation.compute(*normals);// Perform triangulation.pcl::concatenateFields(*cloud, *normals, *cloudNormals);pcl::search::KdTree<pcl::PointNormal>::Ptr kdtree2(new pcl::search::KdTree<pcl::PointNormal>);kdtree2->setInputCloud(cloudNormals);pcl::GreedyProjectionTriangulation<pcl::PointNormal> triangulation;pcl::PolygonMesh triangles;triangulation.setSearchRadius(0.025);triangulation.setMu(2.5);triangulation.setMaximumNearestNeighbors(100);triangulation.setMaximumSurfaceAngle(M_PI / 4); // 45 degrees.triangulation.setNormalConsistency(false);triangulation.setMinimumAngle(M_PI / 18); // 10 degrees.triangulation.setMaximumAngle(2 * M_PI / 3); // 120 degrees.triangulation.setInputCloud(cloudNormals);triangulation.setSearchMethod(kdtree2);triangulation.reconstruct(triangles);// Note: you should only compute descriptors for chosen keypoints. It has// been omitted here for simplicity.// RoPs estimation object.pcl::ROPSEstimation<pcl::PointXYZ, ROPS135> rops;rops.setInputCloud(cloud);rops.setSearchMethod(kdtree);rops.setRadiusSearch(0.03);rops.setTriangles(triangles.polygons);// Number of partition bins that is used for distribution matrix calculation.rops.setNumberOfPartitionBins(5);// The greater the number of rotations is, the bigger the resulting descriptor.// Make sure to change the histogram size accordingly.rops.setNumberOfRotations(3);// Support radius that is used to crop the local surface of the point.rops.setSupportRadius(0.025);rops.compute(*descriptors);
}

Input: Points, Triangles, Search method, Support radius, Number of rotations, Number of partition bins
Output: RoPS descriptors
Publication:
- Rotational Projection Statistics for 3D Local Surface Description and Object Recognition (Yulan Guo et al., 2013)
API: pcl::ROPSEstimation
Download

Global descriptors

Global descriptors encode object geometry. They are not computed for individual points, but for a whole cluster that represents an object. Because of this, a preprocessing step (segmentation) is required, in order to retrieve possible candidates.

Global descriptors are used for object recognition and classification, geometric analysis (object type, shape...), and pose estimation.

You should also know that many local descriptors can also be used as global ones. This can be done with descriptors that use a radius to search for neighbors (as PFH does). The trick is to compute it for one single point in the object cluster, and set the radius to the maximum possible distance between any two points (so all points in the cluster are considered as neighbors).

1）VFH

2）CVFH

3）OUR-CVFH

4）ESF

5）GFPFH

6）GRSD

Saving and loading

Visualization

PCLHistogramVisualizer

PCLPlotter

PCL Viewer

總結

以上是生活随笔為你收集整理的PCL：点云特征描述子3D_object_recognition_(descriptors)的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

上一篇： PCLPCL/OpenNI tutori
下一篇： PCL：英文参考链接