當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

机器学习偏差方差_机器学习101 —偏差方差难题

發布時間：2023/12/15 编程问答 34 豆豆

生活随笔收集整理的這篇文章主要介紹了机器学习偏差方差_机器学习101 —偏差方差难题小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

機器學習偏差方差

Determining the performance of our model is one of the most crucial steps in the machine learning process. Understanding the bias-variance trade-off is a significant step towards interpreting the results of our model. Despite its trivial nature, the concepts behind this trade-off are simple to grasp and will allow us to create better and more useful models.

確定模型的性能是機器學習過程中最關鍵的步驟之一。理解偏差方差的權衡是朝解釋模型結果邁出的重要一步。盡管具有微不足道的性質，但這種權衡取舍的概念仍然易于掌握，將使我們能夠創建更好，更有用的模型。

The generalization error of any machine learning model can be defined as the sum of three different errors—

任何機器學習模型的泛化誤差都可以定義為三個不同誤差的總和-

Irreducible Error: As the name suggests, it can’t be reduced regardless of the algorithm we choose. It is introduced into our model because of the way we frame our problem and may be caused by unknown variables that affect the prediction of our target variable.

不可減少的錯誤：顧名思義，無論我們選擇哪種算法，都無法減少錯誤。由于我們將問題框架化的方式而被引入到我們的模型中，并且可能是由影響目標變量預測的未知變量引起的。

Bias Error: It occurs when our model makes the wrong assumptions

偏差錯誤：當我們的模型做出錯誤的假設時會發生

Variance Error: It is caused by sensitivity to small variations in the training set

方差錯誤：這是由于對訓練集中的小變化敏感

When we discuss prediction models, prediction errors can be decomposed into two main subcomponents we care about: error due to “bias” and error due to “variance”. There is a tradeoff between a model’s ability to minimize bias and variance. Understanding these two types of error can help us diagnose model results and avoid the mistake of over- or under-fitting. ~ Scott Fortman-Roe

當我們討論預測模型時，預測誤差可以分解為我們關注的兩個主要子組件：“偏差”引起的誤差和“方差”引起的誤差。在模型最小化偏差和方差的能力之間需要權衡。了解這兩種錯誤類型可以幫助我們診斷模型結果，并避免過擬合或欠擬合的錯誤。 ?斯科特·福特曼·羅

In this blog post, we’re going to focus on the bias error, variance error and the bias-variance trade-off.

在此博客文章中，我們將重點介紹偏差誤差，方差誤差和偏差方差的權衡。

偏差誤差 (Bias Error)

Bias is the amount by which the expected prediction of our model differs from the actual target value, i.e. how far our predictions are from the real values. Essentially, the bias of our model is determined by the assumptions it makes to predict our target value. Simply stated, a high bias means that the underlying patterns are not captured by our learning algorithm. Such models subsequently produce a large error on both the training and test sets.

偏差是模型的預期預測與實際目標值相差的量，即我們的預測與實際值相差多遠。本質上，我們模型的偏差是由其預測目標值的假設所決定的。簡而言之，高偏差意味著我們的學習算法無法捕獲基本模式。這樣的模型隨后會在訓練集和測試集上產生很大的誤差。

Decision Trees, k-Nearest Neighbors and Support Vector Machines are low bias machine learning algorithms
決策樹，k最近鄰和支持向量機是低偏差機器學習算法
Linear Regression and Logistic Regression are high bias machine learning algorithms
線性回歸和邏輯回歸是高偏差機器學習算法

方差誤差 (Variance Error)

It is defined as the amount by which the prediction of our model would changes if we use a different training set. Models with a high variance tend to pay more attention to the data present in the training set and don’t generalize well, i.e. they don’t perform well on the test set. In other words, such machine learning algorithms try to fit themselves to the training data as much as possible. By doing so they make complex assumptions which may only be true for the training data and hence they perform much worse on the test set.

它定義為如果我們使用不同的訓練集，模型預測的變化量。具有高方差的模型傾向于更加關注訓練集中的數據，并且不能很好地概括，即它們在測試集中的表現不佳。換句話說，這樣的機器學習算法試圖使自己盡可能地適合訓練數據。這樣，他們會做出復雜的假設，這可能僅適用于訓練數據，因此它們在測試集上的表現要差得多。

Linear Regression and Logistic Regression are low variance machine learning algorithms
線性回歸和邏輯回歸是低方差機器學習算法
Decision Trees, k-Nearest Neighbors and Support Vector Machines are high variance machine learning algorithms
決策樹，k最近鄰和支持向量機是高方差機器學習算法

偏差-偏差權衡 (Bias-Variance Trade-off)

Now, let’s try and understand the trade-off between bias and variance with the help of a bullseye diagram. One thing we already know is that bias and variance are inversely proportional to one another, i.e. if bias increases then variance decreases and vice versa.

現在，讓我們嘗試通過靶心圖了解偏差和方差之間的權衡。我們已經知道的一件事是，偏差和方差成反比，即，如果偏差增加，則方差減小，反之亦然。

We assume that the center of the diagram is a model that perfectly predicts the target values, and the further we are from the center the worse our predictions get. If we repeat our model building process with a few changes here and there each time we get multiple hits on our target, each of which represents the performance of an individual model.

我們假設圖的中心是一個可以完美預測目標值的模型，并且距離中心越遠，我們的預測就越糟。如果我們重復進行模型構建過程，并且每次在目標上遇到多次打擊時，都會在此處和那里進行一些更改，每個打擊都代表單個模型的性能。

Bulls-eye diagram depicting the Bias-Variance Tradeoff描繪偏差方差折衷的靶心圖

To learn how to interpret our results, let’s go through the different cases we may observe:

要了解如何解釋我們的結果，我們來研究一下我們可能觀察到的不同情況：

Low Bias & Low Variance

低偏差和低方差

Ideal situation for our machine learning model
我們的機器學習模型的理想情況
The error of prediction is as low as possible
預測誤差盡可能低
The predictions don’t change much when we choose a different training set
當我們選擇不同的訓練集時，預測不會有太大變化

2. High Bias & High Variance

2.高偏差和高方差

Worst possible situation for our machine learning model
我們的機器學習模型可能出現的最糟糕情況
The error of prediction is extremely high
預測誤差極高
The predictions fluctuate massively when we use a different training set
當我們使用不同的訓練集時，預測會大幅波動

3. High Bias & Low Variance

3.高偏差低方差

Often referred to as underfitting, which means that our model is unable to capture the underlying patterns present in our data
通常稱為欠擬合，這意味著我們的模型無法捕獲數據中存在的潛在模式
Usually occurs due to the presence of a small amount of data
通常是由于存在少量數據而發生

Underfitting vs Overfitting擬合不足與擬合過度

4. Low Bias & High Variance

4.低偏差和高方差

Also known as overfitting, which means that our model finds underlying patterns present in our data but also interprets the noise as useful information
也稱為過擬合，這意味著我們的模型可以找到數據中存在的潛在模式，但也可以將噪聲解釋為有用的信息
It occurs when we train our model over data which hasn’t been cleaned properly
當我們針對未正確清理的數據訓練模型時會發生這種情況

摘要 (Summary)

At its heart, the bias-variance trade-off aims to avoid both underfitting and overfitting. As the complexity of our model increases the bias reduces and while the variance also increases. In other words, if we keep adding more features to our model our primary concern shifts from reducing the bias to reducing the variance of our model.

從本質上講，偏差方差折衷旨在避免擬合不足和過度擬合。隨著模型復雜度的增加，偏差減小，而方差也增大。換句話說，如果我們繼續向模型添加更多功能，則我們的主要關注點將從減少偏差轉變為減少模型的方差。

Error Complexity Curve誤差復雜度曲線

As mentioned earlier, the generalization error of our model comprises of three different errors and can be depicted mathematically as follows:

如前所述，我們模型的泛化誤差包括三個不同的誤差，可以用以下數學方式表示：

The dotted line in the error complexity curve displayed above denotes the optimum model complexity and is considered the sweet spot for our machine learning model. We can say that the sweet spot has been found when the increase in bias is equal to the reduction in variance of our model. Mathematically we get:

顯示在以上錯誤復雜曲線的虛線表示的最佳模型的復雜性，被認為是甜蜜點我們的機器學習模型。可以說，當偏差的增加等于模型方差的減少時，已經找到了最佳點。數學上我們得到：

If the complexity of our model goes past the sweet spot then we are overfitting our model, and if we do not reach the sweet spot then we are underfitting our model.

如果模型的復雜性超過了最佳點，那么我們就過度擬合了模型，如果我們沒有達到最佳點，那么我們就對模型進行了擬合。

結語… (Wrapping Up…)

In essence, we can define the relationship between bias and variance as follows:

本質上，我們可以定義偏差和方差之間的關系，如下所示：

Increasing the bias will decrease the variance; and
增加偏差將減小方差；和
Increasing the variance will decrease the bias
增加方差將減少偏差

Although there is no definitive method to obtain the so called sweet spot, we can do our best to find it by either using appropriate metrics to analyse the performance of our model or by choosing the correct algorithms (and their proper configuration) for our purposes. Thus, we can conclude that the bias-variance trade-off is an important consideration that we can use as a starting point to determine the predictive performance of our machine learning models.

盡管沒有確定的方法來獲得所謂的最佳位置，但是我們可以通過使用適當的度量來分析模型的性能，或者通過選擇適合我們目的的正確算法(及其正確配置)來盡力找到它。因此，我們可以得出結論，偏差方差折衷是一個重要的考慮因素，我們可以以此為出發點來確定機器學習模型的預測性能。

Gentle Introduction to the Bias-Variance Trade-off in Machine Learning

機器學習中的偏方差權衡的溫和介紹

Understanding the Bias-Variance Tradeoff

了解偏差-方差折衷

Bias-Variance Tradeoff — Bhavesh Bhatt

偏差偏差權衡— Bhavesh Bhatt

Gain Access to Expert View — Subscribe to DDI Intel

獲得訪問專家視圖的權限- 訂閱DDI Intel

翻譯自: https://medium.com/datadriveninvestor/machine-learning-101-the-bias-variance-conundrum-f4143ba9f179