机器学习模型 非线性模型_调试机器学习模型的终极指南
機(jī)器學(xué)習(xí)模型 非線性模型
You’ve divided your data into a training, development and test set, with the correct percentage of samples in each block, and you’ve also made sure that all of these blocks (specially development and test set) come from the same distribution.
您已將數(shù)據(jù)分為訓(xùn)練，開發(fā)和測(cè)試集，每個(gè)模塊中都有正確百分比的樣本，并且還確保所有這些模塊(特別是開發(fā)和測(cè)試集)都來自同一分布。
You’ve done some exploratory data analysis, gathered insights from this data, and chosen the best features for the task at hand. You’ve also chosen an evaluation metric that is well suited for your problem. Using this metric you will be able to iterate and change the hyper-parameters and configuration of your models in the quest to obtain the best possible performance.
您已經(jīng)進(jìn)行了一些探索性數(shù)據(jù)分析 ，從這些數(shù)據(jù)中收集了見解，并為當(dāng)前任務(wù)選擇了最佳功能 。 您還選擇了一個(gè)非常適合您的問題的評(píng)估指標(biāo)。 使用此度量，您將能夠迭代和更改模型的超參數(shù)和配置，以獲取最佳性能。
After all this, you pre-process the data, prepare it, and finally train a model (lets say a Support Vector Machine). You wait patiently, and once it has finished training you dispose yourself to evaluate the results, which are the following:
完成所有這些之后，您需要對(duì)數(shù)據(jù)進(jìn)行預(yù)處理，準(zhǔn)備并最終訓(xùn)練模型(例如說支持向量機(jī) )。 您需要耐心等待，一旦培訓(xùn)結(jié)束，您就可以評(píng)估以下結(jié)果：
Training set error: 8%
訓(xùn)練集錯(cuò)誤：8％
Development set error: 10%
開發(fā)設(shè)定錯(cuò)誤：10％
How should we look at these results? What can we compare them against? How can we improve them? Is it possible to do it?
我們應(yīng)該如何看待這些結(jié)果？ 我們可以將它們與什么進(jìn)行比較？ 我們?nèi)绾胃纳扑鼈?#xff1f; 有可能做到嗎？
In this post we will answer all of these questions in an easy, accessible manner. This guide is not a debugging guide about setting breakpoints in your code or seeing how training is evolving. It is about knowing what to do when your model is trained and built, how to correctly asses its performance, and seeing how you could improve it.
在這篇文章中，我們將以一種易于訪問的方式回答所有這些問題。 本指南不是關(guān)于在代碼中設(shè)置斷點(diǎn)或了解培訓(xùn)如何發(fā)展的調(diào)試指南。 它是關(guān)于知道在訓(xùn)練和構(gòu)建模型時(shí)該怎么做，如何正確評(píng)估其性能以及如何改進(jìn)它。
Lets get to it!
讓我們開始吧！
可以與我們的模型進(jìn)行比較嗎？ (What to compare our model against?)
When we build our first model and get the initial round of results, it is always desirable to compare this model against some already existing metric, to quickly asses how well it is doing. For this, we have two main strategies: Baseline models and Human-level performance.
當(dāng)我們建立第一個(gè)模型并獲得初始結(jié)果時(shí)，總是希望將這個(gè)模型與一些已經(jīng)存在的指標(biāo)進(jìn)行比較，以快速評(píng)估它的表現(xiàn)。 為此，我們有兩個(gè)主要策略：基線模型和人員水平的績(jī)效。
基準(zhǔn)模型 (Baseline models)
A baseline model is a very simple model that generally yields acceptable results in some kind of task. These results, given by the baseline, are the ones you should try to improve with your new shiny machine learning model.
基線模型是一個(gè)非常簡(jiǎn)單的模型，通常可以在某種任務(wù)中產(chǎn)生可接受的結(jié)果。 由基線給出的這些結(jié)果是您應(yīng)該嘗試使用新的閃亮的機(jī)器學(xué)習(xí)模型進(jìn)行改進(jìn)的結(jié)果。
In a few words, a baseline is a simple approach towards solving a problem that gives a good enough result, but that should be taken as a starting point for performance. If you build a model that does not surpass baseline model performance on some data, then you should probably rethink what you are doing.
簡(jiǎn)而言之，基線是解決問題的簡(jiǎn)單方法，可以給出足夠好的結(jié)果，但是應(yīng)該將其作為性能的起點(diǎn)。 如果您建立的模型在某些數(shù)據(jù)上沒有超過基準(zhǔn)模型性能，那么您可能應(yīng)該重新考慮您在做什么。
Lets see an example to get a better idea of how this works: In Natural Language Processing (NLP) one of the most common problems is that of Sentiment Analysis: detecting the mood, feeling or sentiment of a certain sentence, which could be positive, neutral or negative. A very simple model that can do this, is Naive Bayes: it is very transparent, fast on the training, and generally gives acceptable results, however, these are far from being optimal.
讓我們看一個(gè)例子，以更好地了解其工作原理：在自然語(yǔ)言處理(NLP)中，最常見的問題之一就是情感分析：檢測(cè)特定句子的情緒，感覺或情感，這可能是肯定的，中性或負(fù)面。 樸素的貝葉斯 ( Naive Bayes)是一個(gè)可以做到這一點(diǎn)的非常簡(jiǎn)單的模型：它非常透明，訓(xùn)練Swift，并且通常可以提供可接受的結(jié)果，但是，這些結(jié)果遠(yuǎn)非最佳。
Imagine, you gather some labelled data for sentiment analysis, pre-process the data, train a Naive Bayes model and get 65% accuracy. Because we are taking Naive Bayes as a Baseline model for this task, with every further model we build, we should aim to beat this 65% accuracy. If we train a Logistic Regression and get 55% performance, then we should probably re-think what we are doing.
想象一下，您收集了一些標(biāo)記數(shù)據(jù)以進(jìn)行情感分析 ，預(yù)處理數(shù)據(jù)，訓(xùn)練Naive Bayes模型并獲得65％的準(zhǔn)確性。 由于我們將樸素貝葉斯(Naive Bayes)作為該任務(wù)的基準(zhǔn)模型，因此我們構(gòu)建的每個(gè)其他模型都應(yīng)力爭(zhēng)達(dá)到65％的精度。 如果我們訓(xùn)練Logistic回歸并獲得55％的性能，那么我們可能應(yīng)該重新考慮我們?cè)谧鍪裁础?
We might come to the conclusion that non-neural models are not fit for this task, train an initial Recurrent Neural Network, and get 70%. Now, as we have beaten the baseline, we can try to keep improving this RNN to get better and better performance.
我們可能得出這樣的結(jié)論：非神經(jīng)模型不適合此任務(wù)，訓(xùn)練初始的遞歸神經(jīng)網(wǎng)絡(luò) ，并獲得70％的收益 。 現(xiàn)在，我們已經(jīng)突破了基準(zhǔn)線，可以嘗試不斷改進(jìn)此RNN，以獲得越來越好的性能。
員工績(jī)效 (Human Level Performance)
In the recent years, it has become usual for Machine Learning algorithms to not only produce excellent results in many fields, but to achieve even better results than human experts in those specific fields. Because of this, an useful metric to compare the performance of an algorithm on a certain task is Human Level Performance on that same task.
近年來，機(jī)器學(xué)習(xí)算法不僅在許多領(lǐng)域產(chǎn)生出色的結(jié)果，而且比那些特定領(lǐng)域的人類專家取得更好的結(jié)果已成為常態(tài)。 因此，用于比較算法在特定任務(wù)上的性能的有用度量是該任務(wù)上的人員水平性能 。
Lets see an example so that you can quickly grasp how this works. Imagine that a cardiovascular doctor can look at the health parameters of patients and diagnose with only three errors out of every one-hundred patients if the patient has a certain disease or not.
讓我們看一個(gè)示例，以便您可以快速了解其工作原理。 想象一下，如果有某種疾病，心血管醫(yī)生可以查看患者的健康參數(shù)，并且在每一百名患者中僅診斷出三個(gè)錯(cuò)誤。
Now, we build a Machine learning model to look at these same parameters and diagnose the absence or presence of this previous disease. If our model makes 10 errors out of every 100 diagnoses, then there is a lot of room for improvement, (the expert makes 7 fewer errors for every 100 patients; he has a 7% lower error rate), however, if our model makes 1 failed prediction out 100, it is surpassing human level performance, and therefore doing quite well.
現(xiàn)在，我們建立一個(gè)機(jī)器學(xué)習(xí)模型來查看這些相同的參數(shù)，并診斷出先前疾病的存在與否。 如果我們的模型每100次診斷中就有10個(gè)錯(cuò)誤，那么還有很大的改進(jìn)余地(專家每100名患者減少7個(gè)錯(cuò)誤；錯(cuò)誤率降低7％)，但是，如果我們的模型能夠1項(xiàng)失敗的預(yù)測(cè)中有100項(xiàng)超出了人類水平的表現(xiàn)，因此做得還不錯(cuò)。
Human level performance: 3% error
人員水平績(jī)效：3％的錯(cuò)誤
Model test data performance: 10% error
模型測(cè)試數(shù)據(jù)性能：10％的誤差
Alright, now that we have understood these two metrics, lets progress in the analysis of the results of our Machine Learning models taking Human-level performance as the metric to compare against.
好了，既然我們已經(jīng)理解了這兩個(gè)指標(biāo)，就可以在以人為水平的績(jī)效為指標(biāo)進(jìn)行比較的機(jī)器學(xué)習(xí)模型的結(jié)果分析中取得進(jìn)展。
與人類水平的表現(xiàn)比較 (Comparing to Human level performance)
Understanding how humans perform in a task can guide us towards how to reduce bias and variance. If you don’t know what Bias or Variance are, you can learn about it on the following post:
了解人類在一項(xiàng)任務(wù)中的表現(xiàn)可以指導(dǎo)我們?nèi)绾螠p少偏見和差異。 如果您不知道什么是“偏差”或“方差”，則可以在以下文章中了解它：
Despite humans being awesome at certain tasks, as we have said, Machines can become even better than them, and surpass human level performance. However, there is a certain threshold that neither humans or Machine learning models can surpass: Bayes Optimal error.
正如我們已經(jīng)說過的，盡管人類在某些任務(wù)上表現(xiàn)出色，但是機(jī)器可以變得比它們更好，并且可以超越人類水平的性能。 但是，人類或機(jī)器學(xué)習(xí)模型都無法超越某個(gè)特定的閾值： 貝葉斯最優(yōu)誤差。
Bayes optimal error is the best theoretical result that can be obtained for a certain task, and can not be improved by any kind of function, natural or artificial.
貝葉斯最佳誤差是可以針對(duì)某項(xiàng)任務(wù)獲得的最佳理論結(jié)果，不能通過任何自然或人為的功能來改善。
Imagine a data set composed of images of traffic lights where some images have an orientation such, and are so blurry that it is impossible, even for humans to get all the correct light colours from these images.
想象一下一個(gè)由交通信號(hào)燈圖像組成的數(shù)據(jù)集，其中某些圖像具有這樣的方向，并且非常模糊，以至于即使人類也無法從這些圖像中獲得所有正確的燈光顏色。
Unsplash.Unsplash的圖像。For this data set, Bayes optimal performance would be the maximum number of images that we can actually correctly classify, as some of them are impossible both for humans and machines.
對(duì)于此數(shù)據(jù)集，貝葉斯的最佳性能將是我們實(shí)際上可以正確分類的最大圖像數(shù)，因?yàn)槠渲心承﹫D像對(duì)于人和機(jī)器都是不可能的。
For many tasks human level performance is close to Bayes optimal error, so we tend to use human level performance as a proxy or approximation of Bayes optimal error.
對(duì)于許多任務(wù)，人類水平的性能接近貝葉斯最佳誤差，因此我們傾向于使用人類水平的性能作為貝葉斯最佳誤差的近似值或近似值 。
Lets see a more concrete example, with numbers, to get a complete grasp of the relationship between Human level performance, Bayes Optimal error, and the results of our models.
讓我們看一個(gè)帶有數(shù)字的更具體的例子，以全面了解人的水平表現(xiàn)，貝葉斯最佳誤差與模型結(jié)果之間的關(guān)系。
了解人員水平的表現(xiàn)和貝葉斯最佳誤差 (Understanding Human level performance and Bayes Optimal error)
Imagine a medical image diagnosis task, where a typical doctor achieves a 1% error. Because of this, if we take Human level performance as a proxy for Bayes, we can say that Bayes error is lower or equal to 1%.
想象一下醫(yī)學(xué)圖像診斷任務(wù)，典型的醫(yī)生在其中完成了1％的錯(cuò)誤。 因此，如果我們將人的性能作為貝葉斯的代理，那么我們可以說貝葉斯誤差低于或等于1％。
It is important to note that Human level performance has to be defined depending on the context in which the Machine Learning system is going to be deployed.
重要的是要注意，必須根據(jù)將要部署機(jī)器學(xué)習(xí)系統(tǒng)的上下文來定義人員級(jí)別的性能。
Imagine now that we build a Machine learning model and get the following results on this diagnosis task:
現(xiàn)在想象一下，我們建立了機(jī)器學(xué)習(xí)模型，并在此診斷任務(wù)中獲得了以下結(jié)果：
Training set error: 7%
訓(xùn)練集誤差：7％
Test set error: 8%
測(cè)試設(shè)定錯(cuò)誤：8％
Now, if our Human level performance (proxy for Bayes error) is 1%, what do you think we should focus on improving? The error difference between Bayes Optimal error (1%) and our training set error (7%) or the error difference between training and test set error? We will call the first of these two differences Avoidable bias (between human and training set error) and the second one Variance (between train and test errors).
現(xiàn)在，如果我們的人員水平表現(xiàn)(貝葉斯誤差的代理人)為1％， 您認(rèn)為我們應(yīng)該著重改進(jìn)什么？ 貝葉斯最優(yōu)誤差(1％)與訓(xùn)練集誤差(7％)之間的誤差差還是訓(xùn)練與測(cè)試集誤差之間的誤差差？ 我們將這兩個(gè)差異中的第一個(gè)稱為可避免的偏差 (在人為和訓(xùn)練設(shè)置誤差之間)，第二個(gè)差異 (在訓(xùn)練和測(cè)試誤差之間)。
Avoidable bias and Variance. Image by author.可避免的偏見和差異。 圖片由作者提供。Once we know where to optimise, how should we do it? Keep reading to find out!
一旦我們知道在哪里進(jìn)行優(yōu)化，我們應(yīng)該如何做？ 繼續(xù)閱讀以找出答案！
在哪里以及如何改善我們的機(jī)器學(xué)習(xí)模型 (Where and how to improve our Machine Learning models)
Depending on these sizes of these two error differences (avoidable bias and variance) there are different strategies which we ought to apply in order to reduce these errors and get the best possible results out of our models.
根據(jù)這兩個(gè)誤差差異的大小(可避免的偏差和方差)，我們應(yīng)采用不同的策略以減少這些誤差并從模型中獲得最佳結(jié)果。
In the previous example, the difference between human level performance and training set error (6%) is a lot bigger than the difference between training and test set error (1%), so we will focus on reducing the avoidable bias. If training set error was 2% however, then the bias would be 1%, and the variance would be 6% and we would focus on reducing variance.
在前面的示例中，人的水平性能和訓(xùn)練設(shè)置誤差之間的差異(6％)比訓(xùn)練和測(cè)試設(shè)置誤差之間的差異(1％)大得多，因此我們將重點(diǎn)放在減少可避免的偏差上。 但是，如果訓(xùn)練集誤差為2％，則偏差將為1％，方差將為6％，我們將集中精力減少方差。
If bias and variance were very similar, and there was room for improving both, then we would have to see which is least expensive or easier to reduce.
如果偏差和方差非常相似，并且都有改進(jìn)的余地，那么我們將不得不看到哪一種最便宜或更容易減少。
Lastly, if human level performance, training, and test error where all similar and acceptable, we would leave our awesome model just as it is.
最后，如果人員水平的性能，培訓(xùn)和測(cè)試錯(cuò)誤都相似且可以接受，我們將保留我們的出色模型。
How do we reduce each of these gaps? Lets take a look first and how to reduce avoidable bias.
我們?nèi)绾慰s小這些差距？ 讓我們先來看一下如何減少可避免的偏差。
改善模型性能：如何減少可避免的偏差。 (Improving model performance: how to reduce Avoidable Bias.)
In our search for the best possible Machine learning model, we must look to fit the training set really well without producing over-fitting.
在尋找最佳的機(jī)器學(xué)習(xí)模型時(shí)，我們必須看起來非常適合訓(xùn)練集，而不會(huì)產(chǎn)生過度擬合。
We will look at how to quantify and reduce this over-fitting in just a bit, but the first thing we have to try to achieve is an acceptable performance on our training set, making the gap between human level performance or Bayes error, and training set error as small as possible.
我們將稍稍研究一下如何量化和減少這種過度擬合，但是我們要做的第一件事就是在我們的訓(xùn)練集上獲得可接受的性能，從而在人為水平的性能或貝葉斯誤差與培訓(xùn)之間形成差距將誤差設(shè)置得盡可能小。
For this there are various strategies we can adopt:
為此，我們可以采用多種策略：
If we trained a classic Machine learning model, like a Decision Tree, a Linear or Logistic Regressor, we could try to train something more complex like an SVM, or a Boosting model.
如果我們訓(xùn)練了經(jīng)典的機(jī)器學(xué)習(xí)模型 ，例如決策樹，線性或邏輯回歸，則可以嘗試訓(xùn)練更復(fù)雜的東西，例如SVM或Boosting模型 。
If after this we are still getting poor results, maybe our task needs a more complex or specific architecture, like a Recurrent or Convolutional Neural Network.
如果在此之后我們?nèi)匀猾@得較差的結(jié)果，則也許我們的任務(wù)需要更復(fù)雜或更具體的體系結(jié)構(gòu) ，例如遞歸或卷積神經(jīng)網(wǎng)絡(luò)。
- When Artificial Neural Networks still don’t cut it enough, we can train these networks longer, make them deeper or change the optimisation algorithms. 當(dāng)人工神經(jīng)網(wǎng)絡(luò)仍然不能滿足需要時(shí)，我們可以訓(xùn)練這些網(wǎng)絡(luò)更長(zhǎng)的時(shí)間，使其更深，或者改變優(yōu)化算法。
After all this, if there is still a lot of room for improvement, we could try to get more labelled data by humans, to see if there is some sort of issue with our initial data set.
畢竟，如果仍然有很多改進(jìn)的余地，我們可以嘗試由人類獲取更多的標(biāo)簽數(shù)據(jù) ，以查看我們的初始數(shù)據(jù)集是否存在某種問題。
Lastly, we can carry out manual error analysis: seeing specific examples where our algorithm is performing badly. Going back to an image classification example, maybe through this analysis we can see that small dogs are getting classified as cats, and we can fix this by getting more labelled images of small dogs. In our traffic light example we could spot the issue with blurry images and set a pre-processing step for the data set to discard any images that don’t meet a certain quality threshold.
最后，我們可以進(jìn)行手動(dòng)錯(cuò)誤分析 ：查看我們的算法表現(xiàn)不佳的特定示例。 回到圖像分類的例子，也許通過分析，我們可以看到小狗被分類為貓，并且我們可以通過獲取更多帶標(biāo)簽的小狗圖像來解決此問題。 在我們的交通信號(hào)燈示例中，我們可以發(fā)現(xiàn)圖像模糊的問題，并為數(shù)據(jù)集設(shè)置預(yù)處理步驟，以丟棄任何不符合特定質(zhì)量閾值的圖像。
By using these tactics we can make avoidable bias become increasingly low. Now that we know how to do this, lets take a look at how to reduce Variance.
通過使用這些策略，我們可以使可避免的偏差變得越來越小。 現(xiàn)在我們知道如何做到這一點(diǎn)，讓我們看一下如何減少方差。
改善模型性能：如何減少差異。 (Improving model performance: how to reduce Variance.)
When our model has high variance, we say that it is over-fitting: it adapts too well to the training data, but generalises badly to data it has not seen before. To reduce this variance, there are various strategies that we can adopt, which differ mostly from the ones we just saw to reduce bias. These strategies are:
當(dāng)我們的模型具有很高的方差時(shí)，我們說它是過度擬合的：它對(duì)訓(xùn)練數(shù)據(jù)的適應(yīng)性太強(qiáng)，但是對(duì)以前從未見過的數(shù)據(jù)的歸納性很差。 為了減少這種差異，我們可以采用多種策略，這些策略與我們剛剛看到的減少偏差的策略大部分不同。 這些策略是：
Get more labelled data: if our model ins’t generalising well in some cases, maybe it is because it has never seen those kind of data instances in the training, and therefore getting more training data could be of great use for model improvement.
獲取更多帶標(biāo)簽的數(shù)據(jù)：如果我們的模型在某些情況下不能很好地推廣，也許是因?yàn)樗谟?xùn)練中從未見過此類數(shù)據(jù)實(shí)例，因此獲取更多訓(xùn)練數(shù)據(jù)對(duì)于模型改進(jìn)很有用。
Trying data augmentation: if getting more data is not possible, then we could try data augmentation techniques. With images this is a pretty standard procedure, done by rotating, cropping, RGB shifting and other similar strategies.
嘗試數(shù)據(jù)增強(qiáng)：如果無法獲得更多數(shù)據(jù)，則可以嘗試數(shù)據(jù)增強(qiáng)技術(shù)。 對(duì)于圖像，這是一個(gè)非常標(biāo)準(zhǔn)的過程，通過旋轉(zhuǎn)，裁剪，RGB移位和其他類似策略來完成。
Use regularisation: there are techniques that are specifically conceived for reducing over-fitting, like L1 and L2 regularisation, or Dropout in the case of Artificial Neural networks.
使用正則化：存在專門為減少過度擬合而設(shè)計(jì)的技術(shù)，例如L1和L2正則化，或者在人工神經(jīng)網(wǎng)絡(luò)的情況下為Dropout。
After this, we would have also managed to reduce our variance! Awesome, now we have optimised our Machine Learning model to its full potential.
在此之后，我們也將設(shè)法減少差異！ 太棒了，現(xiàn)在我們已經(jīng)優(yōu)化了機(jī)器學(xué)習(xí)模型的全部潛能。
結(jié)論和其他資源 (Conclusion and additional Resources)
That is it! As always, I hope you enjoyed the post, and that I managed to help you understand how to debug and improve the performance of your Machine learning models.
這就對(duì)了！ 與往常一樣，希望您喜歡這篇文章 ，并且我設(shè)法幫助您了解如何調(diào)試和改善機(jī)器學(xué)習(xí)模型的性能。
If you liked this post then feel free to follow me on Twitter at @jaimezorno. Also, you can take a look at my other posts on Data Science and Machine Learning here. Have a good read!
如果您喜歡這篇文章，請(qǐng)隨時(shí) 通過@jaimezorno 在 Twitter上 關(guān)注我 。 另外，您可以在 此處 查看我在數(shù)據(jù)科學(xué)和機(jī)器學(xué)習(xí)方面的其他帖子 。 祝您閱讀愉快！
If you want to learn more about Machine Learning and Artificial Intelligence follow me on Medium, and stay tuned for my next posts! Also, you can check out this repository for more resources on Machine Learning and AI!
如果您想了解有關(guān)機(jī)器學(xué)習(xí)和人工智能的更多信息，請(qǐng) 在Medium上關(guān)注我 ，并繼續(xù)關(guān)注我的下一篇文章！ 另外，您可以在 此存儲(chǔ)庫(kù)中 查看有關(guān)機(jī)器學(xué)習(xí)和AI的更多資源！
翻譯自: https://towardsdatascience.com/the-ultimate-guide-to-debugging-your-machine-learning-models-103dc0f9e421
機(jī)器學(xué)習(xí)模型 非線性模型
總結(jié)
以上是生活随笔為你收集整理的机器学习模型 非线性模型_调试机器学习模型的终极指南的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： ip登录打印机怎么打印_不要打印，登录。
- 下一篇： 您的第一个简单的机器学习项目