opencv 創建圖像

This project stemmed from my predilection of the visual arts — as a computing student, I’ve always envied art students in their studios with splodges of colors and pristine white canvases. However, when I’m faced with a blank canvas and a (few) bottle(s) of paint, I don’t know where to start. Without the hours of training Art students have, I simply can’t go very far unless I’m following some Bob Ross tutorial. That’s why I’ve decided to explore and see if I can code something that will enable me to create art “automatically”.

這個項目源于我對視覺藝術的偏愛-作為一名計算機專業的學生，??我總是在他們的工作室里用色彩斑p的原始畫布和白色的原始畫布羨慕藝術學生。 但是，當我面對一塊空白的畫布和幾瓶油漆時，我不知道從哪里開始。 如果沒有美術專業學生的訓練，除非我正在聽一些Bob Ross的教程，否則我走不了多遠。 這就是為什么我決定探索并查看是否可以編寫使我能夠“自動”創作藝術作品的原因。

I did this project a looong time ago for my final year project when I wanted to create some software program that can allow its users to sketch something stickman-like on a pre-selected background, select the category of what was drawn, and the programme will intelligently figure out which image to poisson-blend in. After that, the user can also select a neural transfer style to be applied on the image.

我在不久前的最后一個項目中做了這個項目，當時我想創建一些軟件程序，該程序可以讓用戶在預選的背景上繪制類似于stickman的東西，選擇繪制的類別，然后選擇該程序會智能地找出要進行泊松混合的圖像。此后，用戶還可以選擇要應用于圖像的神經傳遞樣式。

So in this article I will be giving a brief walkthrough of this project. Because there are many parts to it, I won’t be going in-depth with the technical implementations. However, if you’d like me to do so, leave a comment and let me know! :)

因此，在本文中，我將簡要介紹該項目。 因為涉及到很多部分，所以我不會深入探討技術實現。 但是，如果您希望我這樣做，請發表評論并告訴我！ :)

第1部分-背景圖片的選擇 (Part 1 — Selection of the background image)

Because I wanted the user to have a range of images to choose from, I’ve coded a python script that acted as an image search engine. If an image is selected as the user browses through a range of available background images, similar looking background images would be retrieved. The similarity is calculated based on colour histograms in the HSV colour space. A more detailed explanation can be found here.

因為我希望用戶可以選擇一系列圖像，所以我編寫了一個Python腳本，用作圖像搜索引擎。 如果在用戶瀏覽一系列可用的背景圖像時選擇了圖像，則將檢索相似外觀的背景圖像。 基于HSV顏色空間中的顏色直方圖計算相似度。 在這里可以找到更詳細的解釋。

Here’s the result. If a user selects this background image:

這是結果。 如果用戶選擇此背景圖片：

Similar looking images would be retrieved:

看起來相似的圖像將被檢索：

第2部分-開始繪制！ (Part 2 — Start drawing!)

This is the fun part! The program seeks to match whatever you’re drawing with the set of images in the database. So if you draw an image with the wings spread out versus if you draw an oval (for example) the retrieved results would be different. Additionally, because the programme is not as robust, the user would have to specify the category (for example bird or boat, or car). For this particular project, I have six categories. The drawing function is also coded in Python with openCV. Once the outline is drawn, the list of coordinates would be saved. 100 sample points (the number I used for my testing, this can vary — the more points the more accurate but the programme would take longer) was taken from this outline of the drawing.

這是有趣的部分！ 該程序試圖將您要繪制的內容與數據庫中的圖像集進行匹配。 因此，如果您繪制的圖像張開了翅膀，而繪制了橢圓形(例如)，則檢索到的結果將有所不同。 另外，由于程序不那么健壯，因此用戶必須指定類別(例如，鳥，船或汽車)。 對于這個特定的項目，我有六個類別。 繪圖功能也使用openCV在Python中編碼。 繪制輪廓后，將保存坐標列表。 從該圖形的輪廓中獲取了100個采樣點(我用于測試的數字可能會有所不同-采樣點越多越準確，但是程序將花費更長的時間)。

The drawing did not have to be this specific, it could be just a stickman bird figure too (or stickbird lmao)繪圖不必如此具體，它也可以只是火柴人的小鳥形象(或火柴人lmao)

第3部分-將樣本點與預處理的圖像描述符進行比較 (Part 3 — Comparing the sample points with pre-processed image descriptors)

Initially, I spent a very long time trying to figure out how to use image segmentation methods to get a clean cut of the target image objects so I could crop them out of their original images, figure out which one(s) matched my sketch the most, and them poisson-blend into my result image. However, this was really tricky to do and OpenCV’s image saliency techniques only worked well with certain types of images. For example,

最初，我花了很長時間試圖弄清楚如何使用圖像分割方法對目標圖像對象進行清晰的裁剪，以便可以將其從原始圖像中裁剪出來，從而找出與我的素描相匹配的圖像。大多數情況下，它們都將泊松融合到我的結果圖像中。 但是，這樣做確實很棘手，而且OpenCV圖像顯著性技術僅適用于某些類型的圖像。 例如，

For these images, clean masks can be obtained對于這些圖像，可以獲得干凈的口罩

However, if the background gets a little complicated or if the subject is not as clear, this happens:

但是，如果背景變得有點復雜或主題不太清楚，則會發生這種情況：

Hence, I decided to use the COCODataset instead.

因此，我決定改用COCODataset。

These are some of the masks of the “birds” category這些是“鳥類”類別的一些面具

With these masks, I can use the outline of my sketch to compare against the shape context histogram descriptors of these masks. How it works can be broken down into four steps:

有了這些蒙版，我可以使用草圖的輪廓與這些蒙版的形狀上下文直方圖描述符進行比較。 它的工作方式可以分為四個步驟：

Step 1: Finding the collection of points on shape contour

步驟1：在形狀輪廓上找到點的集合

Step 2: Computing the shape context

步驟2：計算形狀上下文

Step 3: Computing the cost matrix

步驟3：計算成本矩陣

Step 4: Finding the matching that minimizes total cost

步驟4：尋找可將總費用降至最低的匹配項

BY THE WAY, shape context is another fun topic to discuss that’ll take a full article by itself, so I shall not be discussing it here — but do let me know if that’ll be of interest!

順便說一句，形狀上下文是另一個有趣的話題，需要單獨閱讀全文，因此在這里我將不再討論它-但請告訴我是否有興趣！

Here’s some results:

結果如下：

So it works OKAY. The problem with the Hungarian matching algorithm is that it is extremely time costly — its O(n3)! This is also the part where I’m thinking of implementing some ML/DL techniques to speed up the process.

這樣就可以了。 匈牙利匹配算法的問題在于，它非常耗時-O(n3)！ 這也是我正在考慮實現某些ML / DL技術以加速該過程的部分。

第4部分-將最接近的匹配項混合到所選背景圖像中 (Part 4 — Blending the closest match into the selected background image)

The last part of this project (after using the target mask to “crop” the image out) is to use poisson blending to blend the image into our initially selected background image.

該項目的最后一部分(使用目標蒙版“裁剪”出圖像之后)是使用泊松混合將圖像混合到我們最初選擇的背景圖像中。

Poisson blending is one of the gradient domain image processing methods. Pixels in the resultant image are computed by solving a Poisson equation.

泊松混合是梯度域圖像處理方法之一。 通過求解泊松方程來計算結果圖像中的像素。

For each of the final pixels:

對于每個最終像素：

if mask(x,y) is 0:
final(x,y) = target(x,y)
else:
final(x,y) = 0
for each neighbor (nx,ny) in (x-1,y), (x+1,y), (x,y-1), (x,y+1):
final(x,y) += source(x,y) - source(nx,ny)
if mask(nx,ny) is 0:
final(x,y) += target(nx,ny)
else:
final(x,y) += final(nx,ny)

All masked pixels with a non-zero value affects each other’s final value. The matrix equation Ax=b is solved to compute everything simultaneously. The size of vectors x and b are both number of pixels in the target image — vector x is what we are solving for and contains all pixels in the final image while vector b is the guiding gradient plus the sum of all non-masked neighbour pixels in the target image. These non-masked neighbour pixels are the values of pixels at the mask boundary and the guiding gradient defines the second derivative of the final mask area. Matrix A is a sparse matrix and computes the gradient of the final image’s masked pixels.The equation for x is solved to get the final image. If the guiding gradient is zero, we are just solving a Laplace equation and the values at the mask boundary are blended smoothly across. With the gradient of the source image as the guiding gradient, the mask area takes on the appearance of the source image but is smoothly blended at the boundary.

具有非零值的所有蒙版像素都會影響彼此的最終值。 求解矩陣方程Ax = b可以同時計算所有內容。 向量x和b的大小都是目標圖像中的像素數-向量x是我們要求解的圖像，并且包含最終圖像中的所有像素，而向量b是引導梯度加所有未遮罩的相鄰像素之和在目標圖像中。 這些未遮罩的相鄰像素是遮罩邊界處的像素值，并且引導漸變定義了最終遮罩區域的二階導數。 矩陣A是一個稀疏矩陣，用于計算最終圖像的蒙版像素的梯度。求解x的方程以獲得最終圖像。 如果引導坡度為零，則我們正在求解拉普拉斯方程，并且將遮罩邊界處的值平滑地混合在一起。 以源圖像的梯度作為引導梯度，遮罩區域具有源圖像的外觀，但在邊界處平滑融合。

OpenCV’s seamless clone function is an implementation of the algorithm mentioned above.

OpenCV無縫克隆功能是上述算法的實現。

Some results:

一些結果：

The birds were not in the original background image — they were “blended” in這些鳥不在原始背景圖像中，而是在“

第5部分-使用預先訓練的模型在圖像上應用神經樣式傳遞 (Part 5 — Applying neural style transfer on the image with pre-trained models)

I did a separate article on this here! Do check it out if interested. But in a nutshell, what it does is it takes the image above and apply some cool art style to it. There is of course a range to choose from and the range largely depends on which pre-trained neural models you use.

我在這里做了另一篇文章！ 如果有興趣，請檢查一下。 簡而言之，它的作用是獲取上面的圖像并為其應用一些酷炫的藝術風格。 當然，有一個范圍可供選擇，范圍很大程度上取決于您使用的預訓練神經模型。

Here’s some examples:

這里有一些例子：

This project is actually far from done. Mainly because the matching algorithm takes too long and the pre-processing phase requires some manual intervention still. However, most of this (save for the last part) is just mathematical image manipulation techniques! If ML/DL techniques are used, the processing phase could potentially be improved. Nonetheless, I think it is still a fun project to work on and it made me learn a lot more about image processing techniques (and math lol).

這個項目實際上還遠遠沒有完成。 主要是因為匹配算法花費的時間太長，并且預處理階段仍然需要一些人工干預。 但是，大多數(最后一部分除外)只是數學圖像處理技術！ 如果使用ML / DL技術，則可能會改善處理階段。 盡管如此，我認為這仍然是一個有趣的項目，它使我學到了更多有關圖像處理技術(和數學笑)的知識。

Anyhow, let me know if I should do a more in-depth walkthrough of each part!

無論如何，請告訴我是否應該對每個部分進行更深入的演練！

Happy OpenCV-ing! ?(﹒??﹒?)?

OpenCV-ing祝您愉快！ ?(﹒??﹒?)?

翻譯自: https://medium.com/swlh/image-creation-for-non-artists-opencv-project-walkthrough-d56ee21db5b6

opencv 創建圖像

總結

以上是生活随笔為你收集整理的opencv 创建图像_非艺术家的图像创建（OpenCV项目演练）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。