安裝anaconda，下載assignment作業代碼

作業代碼數據集等2018版基于python3.6 下載提取碼4put

本課程內容參考：

cs231n官方筆記地址

賀完結！CS231n官方筆記授權翻譯總集篇發布

?

CS231n課程筆記翻譯：圖像分類筆記（上）

numpy參考：CS231n課程筆記翻譯：Python Numpy教程

?以下文字部分轉載自：

CS231n——圖像分類（KNN實現）

課程作業基于python3.6.5對應的anaconda 修改了輸入輸出

圖像分類

??目標：已有固定的分類標簽集合，然后對于輸入的圖像，從分類標簽集合中找出一個分類標簽，最后把分類標簽分配給該輸入圖像。

??圖像分類流程

• 輸入：輸入是包含N個圖像的集合，每個圖像的標簽是K種分類標簽中的一種。這個集合稱為訓練集。
• 學習：這一步的任務是使用訓練集來學習每個類到底長什么樣。一般該步驟叫做訓練分類器或者學習一個模型。
• 評價：讓分類器來預測它未曾見過的圖像的分類標簽，把分類器預測的標簽和圖像真正的分類標簽對比，并以此來評價分類器的質量。

??Nearest Neighbor分類器

??數據集：CIFAR-10。這是一個非常流行的圖像分類數據集，包含了60000張32X32的小圖像。每張圖像都有10種分類標簽中的一種。這60000張圖像被分為包含50000張圖像的訓練集和包含10000張圖像的測試集。

??Nearest Neighbor圖像分類思想：拿測試圖片和訓練集中每一張圖片去比較，然后將它認為最相似的那個訓練集圖片的標簽賦給這張測試圖片。
??如何比較來那個張圖片？
??在本例中，就是比較32x32x3的像素塊。最簡單的方法就是逐個像素比較，最后將差異值全部加起來。換句話說，就是將兩張圖片先轉化為兩個向量I_1和I_2，然后計算他們的L1距離：

這里的求和是針對所有的像素。下面是整個比較流程的圖例：

??計算向量間的距離有很多種方法，另一個常用的方法是L2距離，從幾何學的角度，可以理解為它在計算兩個向量間的歐式距離。L2距離的公式如下：

??L1和L2比較：比較這兩個度量方式是挺有意思的。在面對兩個向量之間的差異時，L2比L1更加不能容忍這些差異。也就是說，相對于1個巨大的差異，L2距離更傾向于接受多個中等程度的差異。L1和L2都是在p-norm常用的特殊形式。

??k-Nearest Neighbor分類器(KNN)

??KNN圖像分類思想：與其只找最相近的那1個圖片的標簽，我們找最相似的k個圖片的標簽，然后讓他們針對測試圖片進行投票，最后把票數最高的標簽作為對測試圖片的預測。
??如何選擇k值？
??交叉驗證：假如有1000張圖片，我們將訓練集平均分成5份，其中4份用來訓練，1份用來驗證。然后我們循環著取其中4份來訓練，其中1份來驗證，最后取所有5次驗證結果的平均值作為算法驗證結果。

??這就是5份交叉驗證對k值調優的例子。針對每個k值，得到5個準確率結果，取其平均值，然后對不同k值的平均表現畫線連接。本例中，當k=10的時算法表現最好（對應圖中的準確率峰值）。如果我們將訓練集分成更多份數，直線一般會更加平滑（噪音更少）。

??k-Nearest Neighbor分類器的優劣

??優點：

• 思路清晰，易于理解，實現簡單；
• 算法的訓練不需要花時間，因為其訓練過程只是將訓練集數據存儲起來。

??缺點：測試要花費大量時間計算，因為每個測試圖像需要和所有存儲的訓練圖像進行比較。

??實際應用k-NN

??如果你希望將k-NN分類器用到實處（最好別用到圖像上，若是僅僅作為練手還可以接受），那么可以按照以下流程：

預處理你的數據：對你數據中的特征進行歸一化（normalize），讓其具有零平均值（zero mean）和單位方差（unit variance）。在后面的小節我們會討論這些細節。本小節不討論，是因為圖像中的像素都是同質的，不會表現出較大的差異分布，也就不需要標準化處理了。

如果數據是高維數據，考慮使用降維方法，比如PCA(wiki ref, CS229ref, blog ref)或隨機投影。

將數據隨機分入訓練集和驗證集。按照一般規律，70%-90% 數據作為訓練集。這個比例根據算法中有多少超參數，以及這些超參數對于算法的預期影響來決定。如果需要預測的超參數很多，那么就應該使用更大的驗證集來有效地估計它們。如果擔心驗證集數量不夠，那么就嘗試交叉驗證方法。如果計算資源足夠，使用交叉驗證總是更加安全的（份數越多，效果越好，也更耗費計算資源）。

在驗證集上調優，嘗試足夠多的k值，嘗試L1和L2兩種范數計算方式。

如果分類器跑得太慢，嘗試使用Approximate Nearest Neighbor庫（比如FLANN）來加速這個過程，其代價是降低一些準確率。

對最優的超參數做記錄。記錄最優參數后，是否應該讓使用最優參數的算法在完整的訓練集上運行并再次訓練呢？因為如果把驗證集重新放回到訓練集中（自然訓練集的數據量就又變大了），有可能最優參數又會有所變化。在實踐中，不要這樣做。千萬不要在最終的分類器中使用驗證集數據，這樣做會破壞對于最優參數的估計。直接使用測試集來測試用最優參數設置好的最優模型，得到測試集數據的分類準確率，并以此作為你的kNN分類器在該數據上的性能表現。

??課程作業

課程作業：assignment 1

主函數knn.py 放在根目錄assignment下

#coding:utf-8 '''
#knn.py Created on 2019年4月11日@author: Joel ''' import random import numpy as np from assignment1.data_utils import load_CIFAR10 from assignment1.classifiers.k_nearest_neighbor import KNearestNeighbor import matplotlib.pyplot as plt# This is a bit of magic to make matplotlib figures appear inline in the notebook # rather than in a new window. plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots plt.rcParams['image.interpolation'] = 'nearest' plt.rcParams['image.cmap'] = 'gray'X_train, y_train, X_test, y_test = load_CIFAR10('../datasets')# As a sanity check, we print out the size of the training and test data. print('Training data shape: ', X_train.shape) print('Training labels shape: ', y_train.shape) print('Test data shape: ', X_test.shape) print('Test labels shape: ', y_test.shape)# 從數據集中展示一部分數據 # 每個類別展示若干張對應圖片 classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] num_classes = len(classes) samples_per_class = 7 for y, cls in enumerate(classes):idxs = np.flatnonzero(y_train == y)idxs = np.random.choice(idxs, samples_per_class, replace=False)for i, idx in enumerate(idxs):plt_idx = i * num_classes + y + 1plt.subplot(samples_per_class, num_classes, plt_idx)plt.imshow(X_train[idx].astype('uint8'))plt.axis('off')if i == 0:plt.title(cls) plt.show()# 截取部分樣本數據，以提高本作業的執行效率 num_training = 5000 mask = range(num_training) X_train = X_train[mask] y_train = y_train[mask]num_test = 500 mask = range(num_test) X_test = X_test[mask] y_test = y_test[mask]# reshape訓練和測試數據，轉換為行的形式 X_train = np.reshape(X_train, (X_train.shape[0], -1)) X_test = np.reshape(X_test, (X_test.shape[0], -1))print(X_train.shape) print(X_test.shape)classifier = KNearestNeighbor() classifier.train(X_train, y_train)dists = classifier.compute_distances_two_loops(X_test) print(dists.shape)plt.imshow(dists, interpolation='none') plt.show()# Now implement the function predict_labels and run the code below: # k=1時 y_test_pred = classifier.predict_labels(dists, k=1)# Compute and print the fraction of correctly predicted examples num_correct = np.sum(y_test_pred == y_test) accuracy = float(num_correct) / num_test print('Got %d / %d correct => accuracy: %f' % (num_correct, num_test, accuracy))# k=5時 y_test_pred = classifier.predict_labels(dists, k=5) num_correct = np.sum(y_test_pred == y_test) accuracy = float(num_correct) / num_test print('Got %d / %d correct => accuracy: %f' % (num_correct, num_test, accuracy))####測試三種距離計算法的效率 dists_one = classifier.compute_distances_one_loop(X_test)difference = np.linalg.norm(dists - dists_one, ord='fro') print('Difference was: %f' % (difference, )) if difference < 0.001:print('Good! The distance matrices are the same') else:print('Uh-oh! The distance matrices are different')dists_two = classifier.compute_distances_no_loops(X_test) difference = np.linalg.norm(dists - dists_two, ord='fro') print('Difference was: %f' % (difference, )) if difference < 0.001:print('Good! The distance matrices are the same') else:print('Uh-oh! The distance matrices are different')def time_function(f, *args):"""Call a function f with args and return the time (in seconds) that it took to execute."""import timetic = time.time()f(*args)toc = time.time()return toc - tictwo_loop_time = time_function(classifier.compute_distances_two_loops, X_test) print('Two loop version took %f seconds' % two_loop_time)one_loop_time = time_function(classifier.compute_distances_one_loop, X_test) print('One loop version took %f seconds' % one_loop_time)no_loop_time = time_function(classifier.compute_distances_no_loops, X_test) print('No loop version took %f seconds' % no_loop_time)# 交叉驗證 num_folds = 5 k_choices = [1, 3, 5, 8, 10, 12, 15, 20, 50, 100]X_train_folds = [] y_train_folds = [] ################################################################################ # TODO: # # Split up the training data into folds. After splitting, X_train_folds and # # y_train_folds should each be lists of length num_folds, where # # y_train_folds[i] is the label vector for the points in X_train_folds[i]. # # Hint: Look up the numpy array_split function. # ################################################################################ #數據劃分 X_train_folds = np.array_split(X_train, num_folds); y_train_folds = np.array_split(y_train, num_folds) ################################################################################ # END OF YOUR CODE # ################################################################################# A dictionary holding the accuracies for different values of k that we find # when running cross-validation. After running cross-validation, # k_to_accuracies[k] should be a list of length num_folds giving the different # accuracy values that we found when using that value of k. k_to_accuracies = {}################################################################################ # TODO: # # Perform k-fold cross validation to find the best value of k. For each # # possible value of k, run the k-nearest-neighbor algorithm num_folds times, # # where in each case you use all but one of the folds as training data and the # # last fold as a validation set. Store the accuracies for all fold and all # # values of k in the k_to_accuracies dictionary. # ################################################################################ for k in k_choices:k_to_accuracies[k] = []for k in k_choices:#find the best k-valuefor i in range(num_folds):X_train_cv = np.vstack(X_train_folds[:i]+X_train_folds[i+1:])X_test_cv = X_train_folds[i]y_train_cv = np.hstack(y_train_folds[:i]+y_train_folds[i+1:]) #size:4000y_test_cv = y_train_folds[i]classifier.train(X_train_cv, y_train_cv)dists_cv = classifier.compute_distances_no_loops(X_test_cv)y_test_pred = classifier.predict_labels(dists_cv, k)num_correct = np.sum(y_test_pred == y_test_cv)accuracy = float(num_correct) / y_test_cv.shape[0]k_to_accuracies[k].append(accuracy) ################################################################################ # END OF YOUR CODE # ################################################################################# Print out the computed accuracies for k in sorted(k_to_accuracies):for accuracy in k_to_accuracies[k]:print('k = %d, accuracy = %f' % (k, accuracy))# plot the raw observations for k in k_choices:accuracies = k_to_accuracies[k]plt.scatter([k] * len(accuracies), accuracies)# plot the trend line with error bars that correspond to standard deviation accuracies_mean = np.array([np.mean(v) for k,v in sorted(k_to_accuracies.items())]) accuracies_std = np.array([np.std(v) for k,v in sorted(k_to_accuracies.items())]) plt.errorbar(k_choices, accuracies_mean, yerr=accuracies_std) plt.title('Cross-validation on k') plt.xlabel('k') plt.ylabel('Cross-validation accuracy') plt.show()

# Based on the cross-validation results above, choose the best value for k,
# retrain the classifier using all the training data, and test it on the test
# data. You should be able to get above 28% accuracy on the test data.
best_k = 10

classifier = KNearestNeighbor()
classifier.train(X_train, y_train)
y_test_pred = classifier.predict(X_test, k=best_k)

# Compute and display the accuracy
num_correct = np.sum(y_test_pred == y_test)
accuracy = float(num_correct) / num_test
print ('Got %d / %d correct => accuracy: %f' % (num_correct, num_test, accuracy))

?

?

?

?

assignment.cs231n.classifiers目錄

k_nearest_neighbor.py文件內容：

主要是完成了KNN分類器預測部分，分別用了雙層循環，已經運用numpy廣播的無循環和單循環計算，廣播方法的無循環最快；

預測label的選擇，選擇k個離樣本最近的下標，選擇出現次數最多的下標作為測試樣本的分類；

通過將訓練樣本分成5部分對k的不同取值進行交叉驗證，最終選擇分類效果最好的k，結果為k=12；

np.bincount:統計每個元素出現的次數,np.argmax()將次數出現最多的下標返回；?

numpy.argsort()返回數組值從小到大的索引； [:k]為返回最小的k個距離； ??

#coding:utf-8
#k_nearest_neighbor.py

import numpy as npclass KNearestNeighbor(object):""" a kNN classifier with L2 distance """def __init__(self):passdef train(self, X, y):"""Train the classifier. For k-nearest neighbors this is just memorizing the training data.Inputs:- X: A numpy array of shape (num_train, D) containing the training dataconsisting of num_train samples each of dimension D.- y: A numpy array of shape (N,) containing the training labels, wherey[i] is the label for X[i]."""self.X_train = Xself.y_train = ydef predict(self, X, k=1, num_loops=0):"""Predict labels for test data using this classifier.Inputs:- X: A numpy array of shape (num_test, D) containing test data consistingof num_test samples each of dimension D.- k: The number of nearest neighbors that vote for the predicted labels.- num_loops: Determines which implementation to use to compute distancesbetween training points and testing points.Returns:- y: A numpy array of shape (num_test,) containing predicted labels for thetest data, where y[i] is the predicted label for the test point X[i]. """if num_loops == 0:dists = self.compute_distances_no_loops(X)elif num_loops == 1:dists = self.compute_distances_one_loop(X)elif num_loops == 2:dists = self.compute_distances_two_loops(X)else:raise ValueError('Invalid value %d for num_loops' % num_loops)return self.predict_labels(dists, k=k)def compute_distances_two_loops(self, X):"""Compute the distance between each test point in X and each training pointin self.X_train using a nested loop over both the training data and the test data.Inputs:- X: A numpy array of shape (num_test, D) containing test data.Returns:- dists: A numpy array of shape (num_test, num_train) where dists[i, j]is the Euclidean distance between the ith test point and the jth trainingpoint."""num_test = X.shape[0]num_train = self.X_train.shape[0]dists = np.zeros((num_test, num_train))for i in range(num_test):for j in range(num_train):###################################################################### TODO: ## Compute the l2 distance between the ith test point and the jth ## training point, and store the result in dists[i, j]. You should ## not use a loop over dimension. ######################################################################dists[i][j]=np.sqrt(np.sum(np.square(self.X_train[j,:]-X[i,:])))###################################################################### END OF YOUR CODE ######################################################################return distsdef compute_distances_one_loop(self, X):"""Compute the distance between each test point in X and each training pointin self.X_train using a single loop over the test data.Input / Output: Same as compute_distances_two_loops"""num_test = X.shape[0]num_train = self.X_train.shape[0]dists = np.zeros((num_test, num_train))for i in range(num_test):######################################################################## TODO: ## Compute the l2 distance between the ith test point and all training ## points, and store the result in dists[i, :]. ########################################################################dists[i,:]=np.sqrt(np.sum(np.square(self.X_train-X[i,:]),axis=1))######################################################################## END OF YOUR CODE ########################################################################return distsdef compute_distances_no_loops(self, X):"""Compute the distance between each test point in X and each training pointin self.X_train using no explicit loops.Input / Output: Same as compute_distances_two_loops"""num_test = X.shape[0]num_train = self.X_train.shape[0]dists = np.zeros((num_test, num_train)) ########################################################################## TODO: ## Compute the l2 distance between all test points and all training ## points without using any explicit loops, and store the result in ## dists. ## ## You should implement this function using only basic array operations; ## in particular you should not use functions from scipy. ## ## HINT: Try to formulate the l2 distance using matrix multiplication ## and two broadcast sums. # 矩陣廣播求和的方法 ##########################################################################ab=np.dot(X,self.X_train.T)a_2=np.square(X).sum(axis=1)b_2=np.square(self.X_train).sum(axis=1)print(ab.shape)print(a_2.shape)print(b_2.shape)dists_2=-2*ab+b_2+np.matrix(a_2).Tdists=np.array(np.sqrt(dists_2))########################################################################## END OF YOUR CODE ##########################################################################return distsdef predict_labels(self, dists, k=1):"""Given a matrix of distances between test points and training points,predict a label for each test point.Inputs:- dists: A numpy array of shape (num_test, num_train) where dists[i, j]gives the distance betwen the ith test point and the jth training point.Returns:- y: A numpy array of shape (num_test,) containing predicted labels for thetest data, where y[i] is the predicted label for the test point X[i]. """num_test = dists.shape[0]y_pred = np.zeros(num_test)for i in range(num_test):# A list of length k storing the labels of the k nearest neighbors to# the ith test point.closest_y = []########################################################################## TODO: ## Use the distance matrix to find the k nearest neighbors of the ith ## testing point, and use self.y_train to find the labels of these ## neighbors. Store these labels in closest_y. ## Hint: Look up the function numpy.argsort. # numpy.argsort()返回數組值從小到大的索引； [:k]為返回最小的k個距離； ##########################################################################closest_y=self.y_train[np.argsort(dists[i,:])[:k]]########################################################################## TODO: ## Now that you have found the labels of the k nearest neighbors, you ## need to find the most common label in the list closest_y of labels. ## Store this label in y_pred[i]. Break ties by choosing the smaller ## label. np.bincount:統計每個元素出現的次數,np.argmax()將次數出現最多的下標返回； ##########################################################################y_pred[i]=np.argmax(np.bincount(closest_y))########################################################################## END OF YOUR CODE # #########################################################################return y_pred

?

轉載于:https://www.cnblogs.com/joelwang/p/10697503.html

總結

以上是生活随笔為你收集整理的【cs231n作业笔记】一：KNN分类器的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。