程序示例–多項式回歸

下面，我們有一組溫度（temperature）和實驗產出量（yield）訓練樣本，該數據由博客 Polynomial Regression Examples 所提供：

temperatureyield

50	3.3
50	2.8
50	2.9
70	2.3
70	2.6
70	2.1
80	2.5
80	2.9
80	2.4
90	3.0
90	3.1
90	2.8
100	3.3
100	3.5
100	3.0

我們先通過如下預測函數進行訓練：
$$h(\theta )={\theta }_0+{\theta }_{1}x$$

# coding: utf-8 # linear_regression/test_temperature_normal.py import regression from matplotlib import cm from mpl_toolkits.mplot3d import axes3d import matplotlib.pyplot as plt import matplotlib.ticker as mtick import numpy as npif __name__ == "__main__":X, y = regression.loadDataSet('data/temperature.txt');m,n = X.shapeX = np.concatenate((np.ones((m,1)), X), axis=1)rate = 0.0001maxLoop = 1000epsilon =0.01result, timeConsumed = regression.bgd(rate, maxLoop, epsilon, X, y)theta, errors, thetas = result# 繪制擬合曲線fittingFig = plt.figure()title = 'bgd: rate=%.3f, maxLoop=%d, epsilon=%.3f \n time: %ds'%(rate,maxLoop,epsilon,timeConsumed)ax = fittingFig.add_subplot(111, title=title)trainingSet = ax.scatter(X[:, 1].flatten().A[0], y[:,0].flatten().A[0])xCopy = X.copy()xCopy.sort(0)yHat = xCopy*thetafittingLine, = ax.plot(xCopy[:,1], yHat, color='g')ax.set_xlabel('temperature')ax.set_ylabel('yield')plt.legend([trainingSet, fittingLine], ['Training Set', 'Linear Regression'])plt.show()# 繪制誤差曲線errorsFig = plt.figure()ax = errorsFig.add_subplot(111)ax.yaxis.set_major_formatter(mtick.FormatStrFormatter('%.4f'))ax.plot(range(len(errors)), errors)ax.set_xlabel('Number of iterations')ax.set_ylabel('Cost J')plt.show()

得到的擬合圖像為：

接下來，我們使用了多項式回歸，添加了 2 階項：

$$h(\theta )={\theta }_0+{\theta }_{1}x+{\theta }_2{x}^2$$

因為 $x$ 與 $x2$ 數值差異較大，所以我們會先做一次特征標準化，將各個特征縮放到 [?1,1] 區間

$$z=\frac{x?\mu }{\delta }$$

# coding: utf-8 # linear_regression/test_temperature_polynomial.pyimport regression import matplotlib.pyplot as plt import matplotlib.ticker as mtick import numpy as npif __name__ == "__main__":srcX, y = regression.loadDataSet('data/temperature.txt');m,n = srcX.shapesrcX = np.concatenate((srcX[:, 0], np.power(srcX[:, 0],2)), axis=1)# 特征縮放X = regression.standardize(srcX.copy())X = np.concatenate((np.ones((m,1)), X), axis=1)rate = 0.1maxLoop = 1000epsilon = 0.01result, timeConsumed = regression.bgd(rate, maxLoop, epsilon, X, y)theta, errors, thetas = result# 打印特征點fittingFig = plt.figure()title = 'polynomial with bgd: rate=%.2f, maxLoop=%d, epsilon=%.3f \n time: %ds'%(rate,maxLoop,epsilon,timeConsumed)ax = fittingFig.add_subplot(111, title=title)trainingSet = ax.scatter(srcX[:, 1].flatten().A[0], y[:,0].flatten().A[0])print theta# 打印擬合曲線xx = np.linspace(50,100,50)xx2 = np.power(xx,2)yHat = []for i in range(50):normalizedSize = (xx[i]-xx.mean())/xx.std(0)normalizedSize2 = (xx2[i]-xx2.mean())/xx2.std(0)x = np.matrix([[1,normalizedSize, normalizedSize2]])yHat.append(regression.h(theta, x.T))fittingLine, = ax.plot(xx, yHat, color='g')ax.set_xlabel('Yield')ax.set_ylabel('temperature')plt.legend([trainingSet, fittingLine], ['Training Set', 'Polynomial Regression'])plt.show()# 打印誤差曲線errorsFig = plt.figure()ax = errorsFig.add_subplot(111)ax.yaxis.set_major_formatter(mtick.FormatStrFormatter('%.2e'))ax.plot(range(len(errors)), errors)ax.set_xlabel('Number of iterations')ax.set_ylabel('Cost J')plt.show()

得到的擬合曲線更加準確：

總結

以上是生活随笔為你收集整理的1.7 程序示例--多项式回归-机器学习笔记-斯坦福吴恩达教授的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。