lime 深度學習

It’s needless to say: machine learning is powerful.

不用說：機器學習功能強大。

At the most basic level, machine learning algorithms can be used to classify things. Given a collection of cute animal pictures, a classifier can separate the pictures into buckets of ‘dog’ and ‘not a dog’. Given data about customer restaurant preferences, a classifier can predict what restaurant a user goes to next.

在最基本的水平上，機器學習算法可用于對事物進行分類 。 給定一組可愛的動物圖片，分類器可以將圖片分成“狗”和“不是狗”的桶。 給定有關客戶餐廳偏好的數據，分類器可以預測用戶將前往哪個餐廳。

However, the role of humans is overlooked in the technology. It does not matter how powerful a machine learning model is if one does not use it. With so little explanation or reasoning as to how these algorithms made their predictions, if users do not trust a model or a prediction, they will not use it.

但是， 人類的作用在技??術中被忽略了。 如果不使用機器學習模型，它的功能有多強大也沒關系。 關于這些算法如何做出預測的解釋或推理很少，如果用戶不信任模型或預測，他們將不會使用它。

“If the users do not trust a model or a prediction, they will not use it.”

“如果用戶不信任模型或預測，他們將不會使用它。”

As machine learning becomes deployed in even more domains, such as medical diagnosis and recidivism, the decisions these models make can have incredible consequences. Thus, it is of utmost importance to understand and explain how their predictions came to be, which then builds trust.

隨著機器學習在醫療診斷和累犯等更多領域中的應用，這些模型做出的決策可能會產生令人難以置信的后果。 因此，最重要的是理解和解釋他們的預測是如何形成的，然后建立信任。

In their paper “‘Why Should I Trust You?’ Explaining the Predictions of Any Classifier”, Ribeiro, Singh, and Guestrin present a new technique to do so: LIME (Local Interpretable Model-agnostic Explanations). This post will summarize their findings and introduce LIME.

在他們的論文“為什么我應該信任你？” Ribeiro，Singh和Guestrin 解釋了“任何分類器的預測” ，提出了一種新的方法：LIME(與局部可解釋模型無關的解釋)。 這篇文章將總結他們的發現并介紹LIME。

一行摘要 (One line summary)

LIME is a new technique that explains predictions of any machine learning classifier and has been shown to increase human trust and understanding.

LIME是一種新技術，可以解釋任何機器學習分類器的預測，并已顯示出它可以增加人們的信任和理解。

解釋預測 (Explaining predictions)

paper紙上圖1

為什么解釋預測有用？ (Why is explaining predictions useful?)

Let’s look at the example use case of medical diagnosis. Given the patient’s symptoms and measurements, a doctor must make their best judgment as to what the patient’s diagnosis is.

讓我們看一下醫學診斷的示例用例。 給定患者的癥狀和測量結果，醫生必須對患者的診斷做出最佳判斷。

Humans (both the doctor and the patient) are more willing to accept (trust) a diagnosis when they have more prior knowledge.

當人類(醫生和患者)擁有更多的先驗知識時，他們更愿意接受(信任)診斷。

A model has the potential to help a doctor even more with greater data and scalability. Adding an explanation into the process, like in the above figure, would then help humans to trust and use machine learning more effectively.

模型有潛力通過更大的數據和可擴展性來幫助醫生。 如上圖所示，在流程中添加說明將幫助人們更有效地信任和使用機器學習。

需要什么解釋？ (What do explanations need?)

1) The explanation needs to be interpretable.

1)說明必須是可解釋的 。

An interpretable model provides qualitative understanding between the inputs and the output.

可解釋的模型提供了輸入和輸出之間的定性理解。

Interpretability must also take into account user limitations and target audience. It is not reasonable to expect a user to understand why a prediction was made if thousands of features contribute to that prediction.

可解釋性還必須考慮用戶限制和目標受眾。 如果成千上萬的特征有助于該預測，則期望用戶理解為何做出預測是不合理的。

2) The explanation needs to be locally faithful.

2)說明必須是本地忠實的 。

Fidelity measures how well the explanation approximates the model’s prediction. High fidelity is good, low fidelity is useless. Local fidelity means the explanation needs to approximate well to the model’s prediction for a subset of the data.

保真度衡量解釋與模型預測的近似程度。 高保真度好，低保真度無用。 局部保真度意味著解釋需要很好地近似于模型對數據子集的預測。

3) The explanation needs to be model agnostic.

3)解釋需要與模型無關 。

We should always treat the original machine learning model as a black box. This helps equalize non-interpretable and interpretable models + adds flexibility for future classifiers.

我們應該始終將原始的機器學習模型視為黑匣子。 這有助于均衡不可解釋和可解釋的模型，并增加了將來分類器的靈活性。

4) The explanation needs to provide a global perspective.

4)說明需要提供全局視角 。

Rather than only explaining one prediction, we should select a few explanations to present to users such that they represent the whole model.

不僅要解釋一個預測，我們還應該選擇一些解釋以呈現給用戶，以便他們代表整個模型。

LIME如何工作？ (How does LIME work?)

“The overall goal of LIME is to identify an interpretable model over the interpretable representation that is locally faithful to the classifier.”

“ LIME的總體目標是在可解釋的表示形式上確定對分類器忠實的可解釋模型。”

LIME boils down to one central idea: we can learn a model’s local behavior by varying the input and seeing how the outputs (predictions) change.

LIME可以歸結為一個中心思想： 我們可以通過更改輸入并查看輸出(預測)如何變化來學習模型的局部行為 。

This is really useful for interpretability, because we can change the input to make sense for humans (words, images, etc.), while the model itself might use more complicated data representations. We call this input changing process perturbation. Some examples of perturbation include adding/removing words and hiding a part of an image.

這對于可解釋性非常有用，因為我們可以更改輸入以使人(單詞，圖像等)有意義，而模型本身可能使用更復雜的數據表示形式。 我們稱這種輸入改變過程為擾動 。 攝動的一些示例包括添加/刪除單詞并隱藏圖像的一部分。

Rather than trying to approximate a model globally, which is a daunting task, it is easier to approximate a model locally (close to the prediction we want to explain). We can do so by approximating a model by an interpretable one learned from perturbations of the original data, and the perturbed data samples are weighted by how similar they are to the original data.

與其嘗試在全局上逼近模型(這是一項艱巨的任務)，不如在本地逼近模型(接近我們要解釋的預測)。 我們可以通過從原始數據的擾動中學到的可解釋模型來近似模型，然后通過與原始數據的相似程度對擾動的數據樣本進行加權。

Examples were shown in the paper with both text classification and image classification. Here is an image classification example:

本文顯示了帶有文本分類和圖像分類的示例。 這是圖像分類示例：

KDNuggets, KDNuggets ， PixabayPixabay

Say we want to explain a classification model that predicts whether an image contains a frog. Given the original image (left), we carve up the photo into different interpretable elements (right).

假設我們要解釋一個預測圖像是否包含青蛙的分類模型。 給定原始圖像(左)，我們將照片分割成不同的可解釋元素(右)。

KDNuggets, KDNuggets ， PixabayPixabay

Then, we generate a data set of perturbed samples by hiding some of the interpretable elements (the parts colored gray). For each sample, as we see in the middle table above, we derive the probability of whether the frog is in the image. We learn a locally-weighted model from this dataset (perturbed samples more similar to the original image are more important).

然后，我們通過隱藏一些可解釋的元素(顏色為灰色的部分)來生成一個擾動樣本的數據集。 對于每個樣本，如我們在上面的中間表中所見，我們得出青蛙是否在圖像中的概率。 我們從該數據集中學習了局部加權模型(與原始圖像更相似的擾動樣本更為重要)。

Finally, we return the parts of the image with the highest weights as the explanation.

最后，我們返回圖像中權重最高的部分作為說明。

與真實人類的用戶研究 (User studies with real humans)

To evaluate the effectiveness of LIME, a few experiments (with both simulated users and human subjects) were conducted with these 3 questions in mind:

為了評估LIME的有效性，針對以下三個問題進行了一些實驗(針對模擬用戶和人類受試者)：

Are the explanations faithful to the model?

解釋是否忠實于模型？

Can the explanations help users increase trust in predictions?

說明可以幫助用戶增加對預測的信任嗎？

Are the explanations useful for evaluating the model as a whole?

這些說明對評估整個模型有用嗎？

解釋是否忠實于模型？ (Are the explanations faithful to the model?)

For each classifier, the researchers kept note of a gold set of features — the most important features. Then, they computed the fraction of the gold features recovered by LIME’s explanations. In the simulated user experiments, LIME consistently provided > 90% recall on all datasets.

對于每個分類器，研究人員都記錄了一組金色的功能-最重要的功能。 然后，他們計算了LIME解釋中回收的黃金特征的比例。 在模擬的用戶實驗中，LIME在所有數據集中始終提供> 90％的召回率。

這些說明可以幫助用戶增加對預測的信任嗎？ (Can the explanations help users increase trust in predictions?)

In the simulated user experiments, the results showed that LIME outperformed other explainability methods. With real human subjects (Amazon Mechanical Turk users), they showed high agreement in choosing the best classifier and improving them.

在模擬的用戶實驗中，結果表明LIME優于其他可解釋性方法。 對于真實的人類受試者(Amazon Mechanical Turk用戶)，他們在選擇最佳分類器并進行改進方面表現出很高的共識。

“Before observing the explanations, more than a third trusted the classifier… After examining the explanations, however, almost all of the subjects identified the correct insight, with much more certainty that it was a determining factor.”

“在觀察解釋之前，超過三分之一的人信任分類器……但是，在研究了解釋之后，幾乎所有受試者都確定了正確的見解，并且更加確定地認為這是決定因素。”

這些說明對評估整個模型有用嗎？ (Are the explanations useful for evaluating the model as a whole?)

From both simulated user and human subject experiments, yes, it does seem so. Explanations are useful for models in the text and image domains especially, in deciding which model is best to use, assessing trust, improving untrustworthy classifiers, and getting more insight about models’ predictions.

從模擬的用戶實驗和人類受試者實驗來看，是的，確實如此。 解釋對于文本和圖像域中的模型很有用，尤其是在確定哪種模型最適合使用，評估信任度，改進不可信分類器以及獲得有關模型預測的更多見解方面。

我的最后想法 (My final thoughts)

LIME presents a new method to explain predictions of machine learning classifiers. It’s certainly a necessary step in achieving greater explainability and trust in AI, but not perfect — recent work has demonstrated flaws in LIME; for example, this paper from 2019 showed that adversarial attacks on LIME and SHAP (another interpretability technique) could successfully fool their systems. I am excited to continue seeing more research and improvements on LIME and other similar interpretability techniques.

LIME提供了一種新方法來解釋機器學習分類器的預測。 當然，這是在AI上獲得更大的可解釋性和信任度的必要步驟，但不是完美的。 例如，2019年的這篇論文表明，對LIME和SHAP(另一種可解釋性技術)的對抗攻擊可能成功使他們的系統蒙蔽。 我很高興繼續看到有關LIME和其他類似可解釋性技術的更多研究和改進。

For more information, check out the original paper on arXiv here and their code repo here.

欲了解更多信息，請查看原文件上的arXiv 這里和他們的代碼回購這里 。

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “‘Why Should I Trust You?’ Explaining the Predictions of Any Classifier.” ACM Conference on Knowledge Discovery and Data Mining (KDD) 2016.

Marco Tulio Ribeiro，Sameer Singh和Carlos Guestrin。 “'我為什么要相信你？' 解釋任何分類器的預測。” 2016年ACM知識發現和數據挖掘(KDD)會議。

Thank you for reading! Subscribe to read more about research, resources, and issues related to fair and ethical AI.

感謝您的閱讀！ 訂閱以了解有關公平，合乎道德的AI的研究，資源和問題的更多信息。

Catherine Yeo is a CS undergraduate at Harvard interested in AI/ML/NLP, fairness and ethics, and everything related. Feel free to suggest ideas or say hi to her on Twitter.

Catherine Yeo是哈佛大學的CS本科生，對AI / ML / NLP，公平與道德以及所有相關方面感興趣。 隨時在Twitter上提出想法或向她打招呼。

翻譯自: https://towardsdatascience.com/explaining-machine-learning-predictions-and-building-trust-with-lime-473bf46de61a

lime 深度學習

總結

以上是生活随笔為你收集整理的lime 深度学习_用LIME解释机器学习预测并建立信任的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。