數據庫課程設計結論

Living in the Pacific Northwest, the intertwined issues of salmon survival and river flow are frequently in the news, and the data we have on our salmon populations is a key piece in the conversation.

生活在西北太平洋地區，鮭魚生存和河流流量這兩個相互交織的問題經常成為新聞，而我們關于鮭魚種群的數據是談話中的關鍵部分。

I wanted to see if deep learning — object detection in particular — was up to the challenge of assisting with the real world problem of counting fish seen passing through a fish ladder.

我想看看深度學習(尤其是對象檢測)是否可以應對現實世界中對通過魚梯的魚進行計數的難題。

The answer is yes, and there are enough free or open source tools available to accomplish all aspects of the task.

答案是肯定的，并且有足夠的免費或開源工具可完成任務的所有方面。

Object Detection for Fish Counting魚類計數的目標檢測

This approach can be used to count other objects as well. If you have an idea and a place to grab a few images, you can use this walkthrough to help you get your own model running.

該方法也可以用于計算其他對象。 如果您有想法并可以獲取一些圖像，則可以使用本演練來幫助您運行自己的模型。

步驟1：一個有趣的問題 (Step 1: An intriguing issue)

The Pacific Northwest is home to many species of salmon whose lives follow a predictable pattern: hatch in fresh water, migrate to the ocean for the majority of their lives, and then migrate back upstream to their original fresh water hatch sites before they spawn and then die.

西北太平洋地區生活著許多種類的鮭魚，它們的生活遵循可預測的模式：在淡水中孵化，在大部分生命中移至海洋，然后在產卵前向上游遷移回到其原始淡水孵化場，死。

Salmon aren’t the only ones using the rivers though. Hydroelectric dams dot our rivers for hundreds of miles, potentially preventing the upstream migration. To solve this, our dams are all built with something called a ‘fish ladder.’ These constructions allow fish to jump upstream in short segments much as they would in an openly flowing river.

鮭魚并不是唯一使用河流的魚類。 水力發電大壩在我們的河流上遍布數百英里，有可能阻止上游遷移。 為了解決這個問題，我們的水壩全部建有所謂的“魚梯”。 這些結構使魚類可以像在一條開放的河流中那樣，在短段內向上游跳躍。

Hydroelectric dams provide fish ladders (foreground)水電大壩提供魚梯(前景)

Salmon ladders with viewing windows also provide us with the means to witness this migration and to collect data. Data on the number of salmon returning upstream can then be used to determine the length of the fishing season, set limits on fish per day, adjust the flow at the dam, and help us understand how well we’re doing in this balancing act.

帶有觀察窗的鮭魚梯也為我們提供了見證遷移和收集數據的方法。 然后，可以將有關返回上游的鮭魚數量的數據用于確定捕魚季節的長度，設置每天的魚類限制，調整水壩的流量，并幫助我們了解我們在這種平衡行動中的狀況。

Counting fish is currently performed by trained experts either as real time counts or by reviewing video recordings. Given how labor intensive this is, an assist from an expert tool could provide a nice boost to increase the amount of data taken, the speed at which it is available, or the number of locations where the data is recorded.

目前，計數魚類是由訓練有素的專家進行的，既可以實時計數，也可以通過查看錄像來進行。 考慮到這是多么勞動密集的工作，專家工具的幫助可以極大地提高所獲取的數據量，可用數據的速度或記錄數據的位置數量。

步驟2：圖像和預處理 (Step 2: Images and pre-processing)

Whichever project you choose, you’ll need to start with data. Don’t be dissuaded if you can’t start with ideal data; there’s still plenty to learn. I started with web scraped images of vacation photos taken at fish ladders. The images contained a variety of locations, lighting conditions, silhouettes of children, and possible post-production modifications. The images were so difficult to work with that all I could really label was what constituted a “fish”! I ran the rest of this flow with just that and worked out quite a few bugs, so remember that you can start simple. Once I had a sense of what I needed, I found my way to some better images recorded on a video.

無論選擇哪個項目，都需要從數據開始。 如果您無法從理想的數據入手，請不要被勸阻； 還有很多東西要學。 我從網上抓取了在魚梯上拍攝的度假照片的圖像開始。 圖像包含各種位置，光照條件，兒童剪影以及可能的后期制作修改。 這些圖像很難處理，以至于我只能標記出構成“魚”的東西！ 我僅以此來運行其余的流程，并解決了許多錯誤，因此請記住，您可以從簡單開始。 一旦感覺到需要的東西，我便找到了錄制視頻中更好圖像的方法。

For the images in video format, I used the free VLC Media Player tool to run the video and extract 300+ frames. I selected 317 images including 15 images with no fish at all.

對于視頻格式的圖像，我使用了免費的VLC Media Player工具來運行視頻并提取300多個幀。 我選擇了317張圖像，其中包括15張完全沒有魚的圖像。

To prepare these images for deep learning, I also needed to associate a correctly labeled box with each object to be detected. Trained experts label fish by species, mature vs. juvenile, and hatchery vs. wild.

為了準備這些圖像進行深度學習，我還需要將正確標記的框與要檢測的每個對象相關聯。 受過訓練的專家按魚種，成熟對幼魚以及孵化場對野生魚進行標記。

Salmon Identification鮭魚識別

Without fish identification expertise, it was still possible to create 3 classes:

沒有魚類鑒定專業知識，仍然有可能創建3個類別：

• ‘adipose’ for fish having an intact and visible adipose fin
  “脂肪”用于具有完整可見脂肪鰭的魚
• ‘no_adipose’ for fish having no adipose fin
  沒有脂肪的魚的“ no_adipose”
• ‘unknown’ for fish only partially in the viewing window, or whose adipose fin region is obscured by another fish or artifact
  魚只在部分觀察窗中或魚類的脂肪鰭區域被另一條魚或人工制品遮蓋，因此“未知”

Adipose Fin Identification脂肪鰭識別

The free tool labelImg worked well and I had my 317 fish images boxed and labeled in an afternoon.

免費工具labelImg運作良好，一個下午裝箱并標記了我的317條魚圖像。

One of the reasons I knew thatI could get a model working with only 300 or so images was the idea of augmentation. Augmentation takes the originals and constructs variations so that the model is exposed to a variety of changes such as lighting and orientation. With classification, this is easily performed with a few python libraries, but for object detection, the label boxes also need to be transformed if the image is flipped or rotated.

我知道可以得到一個僅處理300張左右圖像的模型的原因之一就是增強的想法。 增強采用原始圖像并構造變體，以便模型可以暴露于各種變化(例如光照和方向)。 使用分類，可以使用幾個python庫輕松地執行此操作，但是對于對象檢測，如果翻轉或旋轉圖像，則還需要轉換標簽框。

Rather than do these transforms by hand, I leveraged the free tools at Roboflow. I uploaded my images and my label file, selected options to create additional images with random amounts of blur, changes in brightness, and horizontal flip. After this step, I had 951 training images.

我沒有手動進行這些轉換，而是利用了Roboflow上的免費工具。 我上傳了圖像和標簽文件，并選擇了一些選項來創建具有隨機模糊量，亮度變化和水平翻轉的其他圖像。 經過這一步，我得到了951張訓練圖像。

步驟3：選擇并訓練模型 (Step 3: Select and train a model)

While it’s possible to build a home-grown object detection model, I opted to start with a known model for my project as a baseline before doing any tailoring. All of this can be done from the model library at Roboflow, and it’s possible to try out more than one.

盡管可以構建自家的對象檢測模型，但在進行任何剪裁之前，我選擇以項目的已知模型作為基準。 所有這些都可以從Roboflow的模型庫中完成 ，并且可以嘗試多個。

“You Only Look Once”. YOLO is a popular object detection machine learning model introduced in 2015 by a group of researchers at the University of Washington. Rather than pass an image classifier multiple times over an image to see if there was, say, a dog at the upper left, or maybe at the upper right, this new approach replaced the final layers of an image classifier with additional convolutional layers that allowed it to find all instances in one pass. The immediate improvement in speed was a major leap forward for computer vision and object detection. Since the original paper, the model has been improved several times and a new model built on this earlier foundation was released in June 2020 as YOLO v5. See the repository at https://github.com/ultralytics/yolov5 for more details on the model.

“你只看一次”。 YOLO是由華盛頓大學的一組研究人員于2015年引入的一種流行的對象檢測機器學習模型。 這種新方法不是在圖像上多次傳遞圖像分類器以查看是否在左上方或右上方有一條狗，而是用附加的卷積層替換了圖像分類器的最后一層，從而允許一次找到所有實例。 速度的立即提高是計算機視覺和物體檢測的重大飛躍。 自原始論文以來，該模型已進行了多次改進，并且在此較早基礎上構建的新模型于2020年6月作為YOLO v5發布。 有關該模型的更多詳細信息，請參見https://github.com/ultralytics/yolov5上的存儲庫。

Given the popularity, speed, and accuracy of YOLO, and the ease of leveraging the tools at Roboflow, trying out the YOLO v5 model was an obvious choice. Starting with a Google Colaboratory template that configured the environment and built the model, I customized this by uploading a new training set experimenting with various epochs and thresholds.

考慮到YOLO的受歡迎程度，速度和準確性，以及在Roboflow上利用工具的便利性 ，嘗試YOLO v5模型是一個顯而易見的選擇。 從配置環境并構建模型的Google Colaboratory模板開始，我通過上傳新的訓練集來進行自定義，該訓練集嘗試了各種時期和閾值。

步驟4：探索模型結果 (Step 4: Explore model results)

The results were impressive and informative.

結果令人印象深刻，內容豐富。

Impressive — the YOLO v5 model trained for 500 epochs in about an hour on the 900+ images, ran inference (prediction) on a new image in about 12 msec, and achieved mAP@0.5 of about 70%.

令人印象深刻 – YOLO v5模型在900幅以上的圖像上訓練了大約一個小時的500個歷元，在大約12毫秒內對新圖像進行了推斷(預測)，并達到了70％的mAP@0.5。

Model correctly labels fish in simple images模型可以在簡單圖像中正確標記魚

Individually viewing and rating the model’s success on test images would be a bit of a chore, though, so this is where the mAP@0.5 metric is valuable.

但是，單獨查看模型并在測試圖像上對模型的成功進行評分會比較麻煩，因此這是mAP@0.5指標很有價值的地方。

mAP@0.5

mAP@0.5

This metric contains two pieces. First, ‘mAP’ indicates the mean Average Precision or correctness of each of the 3 labels taking into account all labels. Second, ‘@0.5’ sets a threshold for how much of the predicted fish bounding box overlaps the original annotation. This second part is a key metric in object detection; it prevents the model from getting credit for guessing the correct fish but drawing a box around some other artifact (like a shadow) instead of an actual fish.

該指標包含兩部分。 首先，“ mAP”表示考慮了所有標簽的3個標簽中的每個標簽的平均平均精度或正確性。 第二，“ @ 0.5”為預測的魚邊界框與原始注釋重疊的閾值進行設置。 第二部分是對象檢測中的關鍵指標。 它會阻止模型因猜測正確的魚而獲得信譽，但會在其他一些人工物體(例如陰影)周圍繪制方框而不是實際的魚。

mAP@0.5 is a common metric for evaluating object detectionmAP@0.5是用于評估對象檢測的常用指標

This model achieved a mAP@0.5 of 70% —but is that good or bad? For some applications, it’s more than enough. In this particular application, there’s more to it than the label assigned to a single image, and ideally the results of a full solution need to be compared to an estimated error in our current fish counting methods.

該模型的mAP@0.5為70％，但這是好是壞？ 對于某些應用程序，這已經綽綽有余。 在這個特定的應用程序中，除了分配給單個圖像的標簽之外，還有更多其他功能，并且理想情況下，需要將完整解決方案的結果與我們當前的魚類計數方法中的估計誤差進行比較。

Informative — altered lighting conditions, shadows, and crowded conditions can result in both under and over counts. In the image below, the confidence threshold was intentionally set low at 0.4 to expose corner cases or images that provide a challenge for the model. Keep in mind that the goal is to see if we’re ready to use this for a real-life challenge, and that means looking for problems!

信息化 -改變照明條件，陰影和擁擠的條件下可能會導致這兩個下和超過計數。 在下圖中，將置信度閾值故意設置為0.4低，以暴露對模型提出挑戰的極端情況或圖像。 請記住，目標是查看我們是否準備好將其用于現實生活中的挑戰，這意味著尋找問題！

Model adds an extra fish with a low confidence score of 0.42模型添加了一個額外的魚，且魚的可信度低至0.42

In the actual application, when the objects are tracked from one frame of video to the next, shadows often move and the label disappears. This next level of challenge will be to address the object tracking from frame to frame so that there is exactly 1 count for each fish, no matter how much time it remains in the viewing window.

在實際應用中，當將對象從視頻的一幀跟蹤到下一幀時，陰影經常移動并且標簽消失。 接下來的挑戰將是解決從一個幀到另一個幀的對象跟蹤問題，以便每條魚的精確計數為1，無論它在觀察窗口中保留了多少時間。

步驟5：考慮后續步驟 (Step 5: Consider next steps)

Now the most important step — what did I learn?

現在最重要的一步-我學到了什么？

• Images with excellent lighting are required
  需要具有出色照明的圖像
• Viewing window height and width are not critical, but the depth needs to be carefully selected to reduce the number of fish that can obscure other fish
  查看窗口的高度和寬度不是關鍵，但是需要仔細選擇深度以減少可能使其他魚遮擋的魚的數量
• Correct species labels are required for training a model to separate sockeye, chinook, and coho in addition to other species. Not being a fish identification expert, there is the possibility that I mistook a scar for a small adipose fin. Correcting mislabeled images is another way to improve the model.
  訓練模型以分離其他種類的紅大馬，chinook和coho時，需要正確的物種標簽。 如果不是魚類鑒定專家，我可能會誤以為是小脂肪鰭的傷痕。 校正貼錯標簽的圖像是改善模型的另一種方法。
• Salmon swimming upstream in a fish ladder pause to rest for varying amounts of time. In some cases, they will swim slowly and maintain position, and at other times they will slow to the point that they drift backward with the current. This adds an additional level of complexity that will require an advanced system to track objects (fish) from one video frame to the next.
  在魚梯上游游的鮭魚會停下來休息不同的時間。 在某些情況下，它們會緩慢游動并保持姿勢，而在另一些情況下，它們會減速到隨著電流向后漂移的程度。 這增加了額外的復雜度，這將需要高級系統來跟蹤從一個視頻幀到下一個視頻幀的對象(魚)。

Most importantly, while the metrics above are from the better images, most of this learning happened on the first set of images, underscoring the point that excellent images are not required to make progress in trying out a project with deep learning and object detection. In other words, not finding ideal images shouldn’t hold you up from getting started!

最重要的是，盡管上述指標來自更好的圖像，但大多數學習是在第一組圖像上進行的，這突出說明了在嘗試進行深度學習和對象檢測的項目時，不需要出色的圖像即可取得進展。 換句話說，沒有找到理想的圖像不應阻止您入門！

結論 (Conclusion)

Based on the project so far, I think it’s fair to say that machine learning / AI / deep learning are ready to be taken out of the lab and applied to real world projects.

基于到目前為止的項目，我認為可以說機器學習/人工智能/深度學習已經可以從實驗室中帶出并應用于實際項目中了。

This is good news — as an indicator species, salmon help us assess the health of our environment. Accurate and timely fish counts are critical to ensuring their survival, and ours.

這是一個好消息-鮭魚作為指示物種，可以幫助我們評估環境的健康。 準確及時的魚類計數對于確保它們以及我們的生存至關重要。

Contact me! I’m open to new opportunities and actively looking for my next exciting data science gig.

聯絡我！ 我樂于接受新的機遇，并積極尋找下一個令人興奮的數據科學演出。

Link to Repository

鏈接到存儲庫

The notebooks used for this project are posted in github. Keep in mind that these are designed to be used on Google Colab and will not run locally. The 3rd notebook contains a pared down version that runs inference only using saved model weights.

用于該項目的筆記本發布在github中。 請記住，這些文件只能在Google Colab上使用，不能在本地運行。 第三個筆記本包含精簡版，僅使用保存的模型權重運行推斷。

https://github.com/jshaffer94247/Counting-Fish

https://github.com/jshaffer94247/Counting-Fish

翻譯自: https://medium.com/@jshaffer94247_86066/applying-deep-learning-to-environmental-issues-73a861f3a4ec

數據庫課程設計結論

總結

以上是生活随笔為你收集整理的数据库课程设计结论_结论的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。