為什么選擇做班級管理系統(tǒng)

Accuracy is a go-to metric because it’s highly interpretable and low-cost to evaluate. For this reason, accuracy — perhaps the most simple of machine learning metrics — is (rightfully) commonplace. However, it’s also true that many people are too comfortable with accuracy.

準(zhǔn)確性是首選指標(biāo)，因為它具有很高的解釋性和低成本。 因此，準(zhǔn)確性(也許是機(jī)器學(xué)習(xí)指標(biāo)中最簡單的一種)(理所應(yīng)當(dāng))是司空見慣的。 然而，這也是事實，很多人都太舒服的準(zhǔn)確性。

Being aware of the limitations of accuracy is essential.

意識到準(zhǔn)確性的局限性是至關(guān)重要的。

Everyone knows that accuracy is misused on unbalanced datasets: for instance, in a medical condition dataset, where the majority of people do not have condition x (let’s say 95%) and the remainder do have condition x.

每個人都知道在不平衡的數(shù)據(jù)集上濫用了準(zhǔn)確性：例如，在醫(yī)療狀況數(shù)據(jù)集中，大多數(shù)人沒有狀況x (比如說95％)，其余人確實有狀況x 。

Since machine learning models are always looking for the easy way out, and especially if an L2 penalization is used (a proportionately less penalty on lower errors), the model can comfortably get away with 95% accuracy only guessing all inputs do not have condition x.

由于機(jī)器學(xué)習(xí)模型一直在尋找簡便的方法，特別是如果使用L2懲罰(對較低的錯誤按比例減少的懲罰)，則僅憑猜測所有輸入都沒有條件x即可輕松獲得95％的精度的模型。

The reply to this common issue is to use some sort of metric that takes into account the unbalanced classes and somehow compensates lack of quantity with a boost of importance, like an F1 score or balanced accuracy.

解決此常見問題的方法是使用某種度量標(biāo)準(zhǔn)，該度量標(biāo)準(zhǔn)考慮了不平衡的類別，并以某種重要的方式補(bǔ)償了數(shù)量不足的情況，例如F1得分或平衡準(zhǔn)確性。

Beyond this common critique, however — which doesn’t address other limitations of accuracy — there are some other problems with using accuracy that go beyond just dealing with balanced classes.

但是，除了這種常見的批評之外(沒有解決準(zhǔn)確性的其他限制)，使用準(zhǔn)確性還有其他一些問題，這些問題不僅僅涉及平衡類。

Everyone agrees that training/testing and deployment of a model should be kept separate. More specifically, the former should be statistical, and the latter should be decision-based. However, there is nothing statistical about turning the outputs of machine learning models — which are (almost) always probabilistic — into decisions, and evaluating its statistical goodness based on this converted output.

每個人都同意，應(yīng)分開進(jìn)行模型的培訓(xùn)/測試和部署。 更具體地說，前者應(yīng)該是統(tǒng)計的，而后者應(yīng)該是基于決策的。 但是，沒有關(guān)于將機(jī)器學(xué)習(xí)模型的輸出(幾乎總是概率)轉(zhuǎn)化為決策，并基于轉(zhuǎn)換后的輸出評估其統(tǒng)計優(yōu)勢的統(tǒng)計信息。

Take a look at the outputs for two machine learning models: should they really be getting the same results? Moreover, even if one tries to remedy accuracy with other decision-based metrics like the commonly prescribed specificity/sensitivity or F1 score, the same problem exists.

看一下兩個機(jī)器學(xué)習(xí)模型的輸出：它們是否真的會得到相同的結(jié)果？ 此外，即使人們試圖用其他基于決策的指標(biāo)(如通常規(guī)定的特異性/敏感性或F1分?jǐn)?shù))來糾正準(zhǔn)確性，也存在相同的問題。

Image created by author圖片由作者創(chuàng)建

Model 2 is far less confident in its results than Model 1 is, but both receive the same accuracy. Accuracy is not a legitimate scoring rule, and hence it is deceiving in an inherently probabilistic environment.

模型2對結(jié)果的信心遠(yuǎn)不如模型1可靠，但兩者的準(zhǔn)確性相同。 準(zhǔn)確性不是一個合理的評分規(guī)則，因此它在固有的概率環(huán)境中具有欺騙性。

While it can be used in the final presenting of a model, it leaves an empty void of information pertaining to the confidence of the model; whether it actually knew the class for most of the training samples or if it was only lucky in crossing on the right side of the 0.5 threshold.

雖然可以在模型的最終展示中使用它，但它留下了與模型的置信度有關(guān)的信息的空白； 無論是實際上對大多數(shù)訓(xùn)練樣本都知道這門課，還是只是幸運(yùn)地越過了0.5個閾值的右側(cè)。

This is also problematic. How can a reliable loss function — the guiding light that shows the model what is right and what is wrong — completely tilt its decision 180 degrees if the output probability shifts 0.01%? If a training sample with label ‘1’ received predictions 0.51 and 0.49 from model 1 and model 2, respectively, is it fair that model 2 is penalized at the full possible value? Thresholds, while necessary for decision-making in a physically deterministic world, are too sensitive and hence inappropriate for training and testing.

這也是有問題的。 如果輸出概率偏移0.01％，可靠的損失函數(shù)(顯示模型正確與錯誤的指示燈)如何將其決策完全傾斜180度？ 如果帶有標(biāo)簽“ 1”的訓(xùn)練樣本分別從模型1和模型2接收到預(yù)測0.51和0.49，那么將模型2懲罰為可能的全部值是否公平？ 閾值雖然在物理確定性世界中進(jìn)行決策是必需的，但過于敏感，因此不適用于培訓(xùn)和測試。

Speaking of thresholds — consider this. You are creating a machine learning model to decide if a patient should receive a very invasive and painful surgery treatment. Where do you decide the threshold to give the recommendation? Instinctively, most likely not at a default 0.5, but at some higher probability: the patient is subjected to this treatment if, and only if, the model is absolutely sure. On the other hand, if the treatment is something less serious like an aspirin, it is less so.

說到閾值，請考慮一下。 您正在創(chuàng)建一個機(jī)器學(xué)習(xí)模型，以決定患者是否應(yīng)該接受侵入性和痛苦性極高的手術(shù)治療。 您在哪里確定提出建議的門檻？ 本能地，最有可能不是默認(rèn)值0.5，而是更高的概率：當(dāng)且僅當(dāng)模型是絕對確定的，患者才接受這種治療。 另一方面，如果像阿斯匹林這樣的不那么嚴(yán)重的治療方法，那么情況就不那么嚴(yán)重了。

The results of the decision dictate the thresholds for forming it. This idea, hard-coding morality and human feeling into a machine learning model, is difficult to think about. One may be inclined to argue that over time and under the right balanced circumstances, the model will automatically shift its output probability distributions to a 0.5 threshold and manually adding a threshold is tampering with the model’s learning.

決策的結(jié)果決定了形成決策的閾值。 將道德和人類感覺硬編碼到機(jī)器學(xué)習(xí)模型中的想法很難考慮。 有人可能會爭辯說，隨著時間的流逝，在正確的平衡情況下，該模型將自動將其輸出概率分布更改為0.5閾值，而手動添加閾值會篡改該模型的學(xué)習(xí)。

The rebuttal would be to not use decision-based scoring functions in the first place, not hard-coding any number, including a 0.5 threshold, at all. This way, the model learns not to cheat and take the easy way out through artificially constructed continuous-to-discrete conversions but to maximize its probability of correct answers.

反對是首先不使用基于決策的評分功能，根本不硬編碼任何數(shù)字，包括0.5閾值。 這樣，該模型就不會通過人工構(gòu)造的連續(xù)到離散轉(zhuǎn)換來欺騙并采取簡單的方法，而是最大限度地提高其正確答案的可能性。

Whenever a threshold is introduced in the naturally probabilistic and fluid nature of machine learning algorithms, it causes more problems than it fixes.

每當(dāng)在機(jī)器學(xué)習(xí)算法的自然概率和流動性中引入閾值時，它都會引起更多的問題，而不是要解決的問題。

Loss functions that treat probability on the continuous scale it is instead of as discrete buckets are the way to go.

損失函數(shù)在連續(xù)尺度上處理概率，而不是像離散的桶那樣走。

What are some better, probability-based, and more informative metrics to use for honestly evaluating a model’s performance?

有什么更好的，基于概率的，更多信息的指標(biāo)可用于誠實地評估模型的性能？

• Brier score
  刺分?jǐn)?shù)
• Log score
  日志分?jǐn)?shù)
• Cross-entropy
  交叉熵

In the end, accuracy is an important and permanent part of the metrics family. But for those who decide to use it: understand that accuracy’s interpretability and simplicity comes at a heavy cost.

最后，準(zhǔn)確性是指標(biāo)系列的重要且永久的組成部分。 但是對于那些決定使用它的人：請理解準(zhǔn)確性的可解釋性和簡單性要付出沉重的代價。

翻譯自: https://medium.com/analytics-vidhya/why-accuracy-is-troublesome-even-with-balanced-classes-590b405f5a06

為什么選擇做班級管理系統(tǒng)

總結(jié)

以上是生活随笔為你收集整理的为什么选择做班级管理系统_为什么即使在平衡的班级下准确性也很麻烦的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。