支持向量機matlab工具箱1.0(Support Vector Machine,SVM Matlab Tool %-------------------------------------------------------%
1 聲明

支持向量機Matlab工具箱1.0
使用平臺 - Matlab6.5
版權所有：陸振波，海軍工程大學
電子郵件：luzhenbo@yahoo.com.cn
個人主頁：http://luzhenbo.88uu.com.cn
參數文獻：Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"

Support Vector Machine Matlab Toolbox 1.0
Platform : Matlab6.5 / Matlab7.0
Copyright : LU Zhen-bo, Navy Engineering University, WuHan, HuBei, P.R.China, 430033??
E-mail : luzhenbo@yahoo.com.cn? ?? ???
Homepage : http://luzhenbo.88uu.com.cn? ???
Reference : Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"


Solve the quadratic PageRankogramming PageRankoblem - "quadPageRankog.m"

%-------------------------------------------------------%
2 內容

該工具箱包括了二種分類,二種回歸,以及一種一類支持向量機算法
(1) Main_SVC_C.m? ?? ?? ?? ?? ? --- C_SVC二類分類算法
(2) Main_SVC_Nu.m? ?? ?? ?? ?? ?--- Nu_SVC二類分類算法
(3) Main_SVM_One_Class.m? ?? ???--- One-Class支持向量機
(4) Main_SVR_Epsilon.m? ?? ?? ? --- Epsilon_SVR回歸算法
(5) Main_SVR_Nu.m? ?? ?? ?? ?? ?--- Nu_SVR回歸算法

%-------------------------------------------------------%
3 使用

(1) 目錄下以Main_開頭的文件即是主程序文件,直接按快捷鍵F5運行即可
(2) 工具箱中所有程序均在Matlab6.5環境中調試經過，不能保證在Matlab其它版本正確運行

%-------------------------------------------------------%
4給予共享地址

http://luzhenbo.88uu.com.cn/svm.htm % 支持向量機Matlab工具箱1.0 - Nu-SVC, Nu二類分類算法
% 使用平臺 - Matlab6.5
% 版權所有：陸振波，海軍工程大學
% 電子郵件：luzhenbo@yahoo.com.cn
% 個人主頁：http://luzhenbo.88uu.com.cn
% 參數文獻：Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"
%
% Support Vector Machine Matlab Toolbox 1.0 - Nu Support Vector Classification
% Platform : Matlab6.5 / Matlab7.0
% Copyright : LU Zhen-bo, Navy Engineering University, WuHan, HuBei, P.R.China, 430033??
% E-mail : luzhenbo@yahoo.com.cn? ?? ???
% Homepage : http://luzhenbo.88uu.com.cn? ???
% Reference : Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"
%
% Solve the quadratic PageRankogramming PageRankoblem - "quadPageRankog.m"

clc
clear
close all

% ------------------------------------------------------------%
% 定義核函數及相關參數

nu = 0.2;? ?? ?? ?? ?% nu -> (0,1] 在支持向量數與錯分樣本數之間進行折衷

ker = struct('type','linear');
%ker = struct('type','ploy','degree',3,'offset',1);
%ker = struct('type','gauss','width',1);
%ker = struct('type','tanh','gamma',1,'offset',0);

% ker - 核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).

% ------------------------------------------------------------%
% 構造兩類訓練樣本

n = 50;
randn('state',3);
x1 = randn(n,2);
y1 = ones(n,1);
x2 = 5+randn(n,2);
y2 = -ones(n,1);

figure(2);
plot(x1(:,1),x1(:,2),'bx',x2(:,1),x2(:,2),'k.');
hold on;

X = [x1;x2];? ?? ???% 訓練樣本,n×d的矩陣,n為樣本個數,d為樣本維數
Y = [y1;y2];? ?? ???% 訓練方向,n×1的矩陣,n為樣本個數,值為+1或-1

% ------------------------------------------------------------%
% 訓練支持向量機

tic
svm = Nu_SVC_Train(X,Y,nu,ker);
t_train = toc

% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子

% ------------------------------------------------------------%
% 找尋支持向量

a = svm.a;
epsilon = 1e-8;? ?? ?? ?? ?? ?? ?? ?% 如果小于此值則認為是0
i_sv = find(a>epsilon);? ?? ?? ?? ? % 支持向量下標
plot(X(i_sv,1),X(i_sv,2),'ro');

% ------------------------------------------------------------%
% 測試輸出

[x1,x2] = meshgrid(-2:0.05:7,-2:0.05:7);
[rows,cols] = size(x1);
nt = rows*cols;? ?? ?? ?? ?? ?? ?% 測試樣本數
Xt = [reshape(x1,nt,1),reshape(x2,nt,1)];

tic
Yd = Nu_SVC_Sim(svm,Xt);? ?? ?? ???% 測試輸出
t_sim = toc

Yd = reshape(Yd,rows,cols);
contour(x1,x2,Yd,[0 0],'m');? ?? ? % 分類面
hold off;

function [K] = CalcKernel(ker,x,y)
% Calculate kernel function.? ?
%
% x: 輸入樣本,n1×d的矩陣,n1為樣本個數,d為樣本維數
% y: 輸入樣本,n2×d的矩陣,n2為樣本個數,d為樣本維數
%
% ker??核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).
%
% ker = struct('type','linear');
% ker = struct('type','ploy','degree',d,'offset',c);
% ker = struct('type','gauss','width',s);
% ker = struct('type','tanh','gamma',g,'offset',c);
%
% K: 輸出核參數,n1×n2的矩陣

%-------------------------------------------------------------%
% 轉成列向量
x = x';
y = y';

%-------------------------------------------------------------%

switch ker.type
? ? case 'linear'
? ?? ???K = x'*y;
? ? case 'ploy'
? ?? ???d = ker.degree;
? ?? ???c = ker.offset;
? ?? ???K = (x'*y+c).^d;
? ? case 'gauss'
? ?? ???s = ker.width;
? ?? ???rows = size(x,2);
? ?? ???cols = size(y,2);? ?
? ?? ???tmp = zeros(rows,cols);
? ?? ???for i = 1:rows
? ?? ?? ?? ?for j = 1:cols
? ?? ?? ?? ?? ? tmp(i,j) = norm(x(:,i)-y(:,j));
? ?? ?? ?? ?end
? ?? ???end? ?? ???
? ?? ???K = exp(-0.5*(tmp/s).^2);
? ? case 'tanh'
? ?? ???g = ker.gamma;
? ?? ???c = ker.offset;
? ?? ???K = tanh(g*x'*y+c);
? ? otherwise
? ?? ???K = 0;
end

function svm = Nu_SVC_Train(X,Y,nu,ker)

% 輸入參數:
% X??訓練樣本,n×d的矩陣,n為樣本個數,d為樣本維數
% Y??訓練方向,n×1的矩陣,n為樣本個數,值為+1或-1
% nu? ?控制參數
% ker??核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).

% 輸出參數:
% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子

% ------------------------------------------------------------%
% 解二次優化

n = length(Y);
H = (Y*Y').*Calckernel(ker,X,X);

f = zeros(n,1);
A = -ones(1,n);
b = -nu;
Aeq = Y';
beq = 0;
lb = zeros(n,1);
ub = ones(n,1)/n;
a0 = zeros(n,1);

options = optimset;
options.LargeScale = 'off';
options.Display = 'off';

[a,fval,eXitflag,output,lambda]??= quadPageRankog(H,f,A,b,Aeq,beq,lb,ub,a0,options);
eXitflag

% ------------------------------------------------------------%
% 輸出 svm
svm.ker = ker;
svm.x = X;
svm.y = Y;
svm.a = a;

function Yd = Nu_SVC_Sim(svm,Xt)
% 輸入參數:
% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?? ? type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?? ? poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?? ? gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?? ? tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?? ? degree - Degree d of polynomial kernel (positive scalar).
%? ?? ? offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?? ? width??- Width s of Gauss kernel (positive scalar).
%? ?? ? gamma??- Slope g of the tanh kernel (positive scalar).
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子
%
% Xt??測試樣本,n×d的矩陣,n為樣本個數,d為樣本維數

% 輸出參數:
% Yd??測試輸出,n×1的矩陣,n為樣本個數,值為+1或-1

% ------------------------------------------------------------%

ker = svm.ker;
X = svm.x;
Y = svm.y;
a = svm.a;

% ------------------------------------------------------------%
% 求 b

epsilon = 1e-8;? ?? ?? ?? ?? ?? ?% 如果小于此值則認為是0
i_sv = find(a>epsilon);? ?? ?? ? % 支持向量下標

tmp = (Y.*a)'*Calckernel(ker,X,X(i_sv,);? ?? ?? ? % 行向量
b = 1./Y(i_sv)-tmp';
b = mean(b);

% ------------------------------------------------------------%
% 測試輸出

nt = size(Xt,1);? ?? ?? ?? ?? ?? ?% 測試樣本數
tmp =??(Y.*a)'*Calckernel(ker,X,Xt);
Yd = sign(tmp+b)'; % 支持向量機Matlab工具箱1.0 - Epsilon-SVR, Epsilon回歸算法
% 使用平臺 - Matlab6.5
% 版權所有：陸振波，海軍工程大學
% 電子郵件：luzhenbo@yahoo.com.cn
% 個人主頁：http://luzhenbo.88uu.com.cn
% 參數文獻：Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"
%
% Support Vector Machine Matlab Toolbox 1.0 - Epsilon Support Vector Regression
% Platform : Matlab6.5 / Matlab7.0
% Copyright : LU Zhen-bo, Navy Engineering University, WuHan, HuBei, P.R.China, 430033??
% E-mail : luzhenbo@yahoo.com.cn? ?? ???
% Homepage : http://luzhenbo.88uu.com.cn? ???
% Reference : Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"
%
% Solve the quadratic PageRankogramming PageRankoblem - "quadPageRankog.m"

clc
clear
close all

% ------------------------------------------------------------%
% 定義核函數及相關參數

C = 100;? ?? ?? ?? ?? ? % 拉格朗日乘子上界
e = 0.2;? ?? ?? ?? ?? ? % 不敏感損失函數的參數，Epsilon越大，支持向量越少

%ker = struct('type','linear');
%ker = struct('type','ploy','degree',3,'offset',1);
ker = struct('type','gauss','width',1);
%ker = struct('type','tanh','gamma',1,'offset',0);

% ker - 核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).

% ------------------------------------------------------------%
% 構造兩類訓練樣本

n = 50;
rand('state',42);
X??= linspace(-4,4,n)';? ?? ?? ?? ?? ?? ?? ?? ?? ?% 訓練樣本,n×d的矩陣,n為樣本個數,d為樣本維數,那里d=1
Ys = (1-X+2*X.^2).*exp(-.5*X.^2);
f = 0.2;? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?% 相對誤差
Y??= Ys+f*max(abs(Ys))*(2*rand(size(Ys))-1)/2;? ? % 訓練方向,n×1的矩陣,n為樣本個數,值為期望輸出

figure(4)
plot(X,Ys,'b-',X,Y,'b*');
hold on;

% ------------------------------------------------------------%
% 訓練支持向量機

tic
svm = Epsilon_SVR_Train(X,Y,C,e,ker);
t_train = toc

% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子

% ------------------------------------------------------------%
% 找尋支持向量

a = svm.a;
epsilon = 1e-8;? ?? ?? ?? ?? ?? ?? ?% 如果"絕對值"小于此值則認為是0
i_sv = find(abs(a)>epsilon);? ?? ???% 支持向量下標,那里對abs(a)進行判定
plot(X(i_sv),Y(i_sv),'ro');

% ------------------------------------------------------------%
% 測試輸出

tic
Yd = Epsilon_SVR_Sim(svm,X);? ?? ?? ???% 測試輸出
t_sim = toc

plot(X,Yd,'r--',X,[Yd-e,Yd+e],'g:');
hold off;

function [K] = CalcKernel(ker,x,y)
% Calculate kernel function.? ?
%
% x: 輸入樣本,n1×d的矩陣,n1為樣本個數,d為樣本維數
% y: 輸入樣本,n2×d的矩陣,n2為樣本個數,d為樣本維數
%
% ker??核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).
%
% ker = struct('type','linear');
% ker = struct('type','ploy','degree',d,'offset',c);
% ker = struct('type','gauss','width',s);
% ker = struct('type','tanh','gamma',g,'offset',c);
%
% K: 輸出核參數,n1×n2的矩陣

%-------------------------------------------------------------%
% 轉成列向量
x = x';
y = y';

%-------------------------------------------------------------%

switch ker.type
? ? case 'linear'
? ?? ???K = x'*y;
? ? case 'ploy'
? ?? ???d = ker.degree;
? ?? ???c = ker.offset;
? ?? ???K = (x'*y+c).^d;
? ? case 'gauss'
? ?? ???s = ker.width;
? ?? ???rows = size(x,2);
? ?? ???cols = size(y,2);? ?
? ?? ???tmp = zeros(rows,cols);
? ?? ???for i = 1:rows
? ?? ?? ?? ?for j = 1:cols
? ?? ?? ?? ?? ? tmp(i,j) = norm(x(:,i)-y(:,j));
? ?? ?? ?? ?end
? ?? ???end? ?? ???
? ?? ???K = exp(-0.5*(tmp/s).^2);
? ? case 'tanh'
? ?? ???g = ker.gamma;
? ?? ???c = ker.offset;
? ?? ???K = tanh(g*x'*y+c);
? ? otherwise
? ?? ???K = 0;
end

function svm = Epsilon_SVR_Train(X,Y,C,e,ker)

% 輸入參數:
% X??訓練樣本,n×d的矩陣,n為樣本個數,d為樣本維數
% Y??訓練方向,n×1的矩陣,n為樣本個數,值為期望輸出
% C??拉格朗日乘子上界
% e??不敏感損失函數的參數，Epsilon越大，支持向量越少
% ker??核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).

% 輸出參數:
% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子

% ------------------------------------------------------------%
% 解二次優化

n = length(Y);
Q = Calckernel(ker,X,X);
H = [Q,-Q;-Q,Q];
f = [e*ones(n,1)-Y;e*ones(n,1)+Y];? ?? ?? ? % 符號不相同,決策函數就不相同,實際上是一回事!見文件"Epsilon_SVR_Sim.m"第37,45行
%f = [e*ones(n,1)+Y;e*ones(n,1)-Y];
A = [];
b = [];
Aeq = [ones(1,n),-ones(1,n)];
beq = 0;
lb = zeros(2*n,1);? ?? ?? ?? ?? ?
ub = C*ones(2*n,1);
a0 = zeros(2*n,1);

%第三步：調用優化工具箱quadPageRankog函數乞求二次規劃
options = optimset;
options.LargeScale = 'off';
options.Display = 'off';

[a,fval,eXitflag,output,lambda]??= quadPageRankog(H,f,A,b,Aeq,beq,lb,ub,a0,options);
eXitflag

% ------------------------------------------------------------%
% 輸出 svm
svm.ker = ker;
svm.x = X;
svm.y = Y;
svm.a = a(1:n)-a(n+1:end);

function Yd = Epsilon_SVR_Sim(svm,Xt)
% 輸入參數:
% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?? ? type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?? ? poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?? ? gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?? ? tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?? ? degree - Degree d of polynomial kernel (positive scalar).
%? ?? ? offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?? ? width??- Width s of Gauss kernel (positive scalar).
%? ?? ? gamma??- Slope g of the tanh kernel (positive scalar).
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子
%
% Xt??測試樣本,n×d的矩陣,n為樣本個數,d為樣本維數

% 輸出參數:
% Yd??測試輸出,n×1的矩陣,n為樣本個數,值為+1或-1

% ------------------------------------------------------------%

ker = svm.ker;
X = svm.x;
Y = svm.y;
a = svm.a;? ? % 那里實際值為 a(1:n)-a(n+1:end),見文件"Epsilon_SVR_Train.m"第56行

% ------------------------------------------------------------%
% 求 b

epsilon = 1e-8;? ?? ?? ?? ?? ?? ?? ?% 如果"絕對值"小于此值則認為是0
i_sv = find(abs(a)>epsilon);? ?? ???% 支持向量下標,那里對abs(a)進行判定

tmp = a'*Calckernel(ker,X,X(i_sv,);? ?% 行向量
b = Y(i_sv)-tmp';? ?? ?? ?? ?? ?? ?? ???% 符號不相同,決策函數就不相同,實際上是一回事!見文件"Epsilon_SVR_Train.m"第33行
%b = Y(i_sv)+tmp';
b = mean(b);

% ------------------------------------------------------------%
% 測試輸出

nt = size(Xt,1);? ?? ?? ?? ?? ?? ???% 測試樣本數
tmp =??a'*Calckernel(ker,X,Xt);? ???% 符號不相同,決策函數就不相同,實際上是一回事!見文件"Epsilon_SVR_Train.m"第33行
%tmp =??-a'*Calckernel(ker,X,Xt);
Yd = (tmp+b)'; % 支持向量機Matlab工具箱1.0 - Nu-SVR, Nu回歸算法
% 使用平臺 - Matlab6.5
% 版權所有：陸振波，海軍工程大學
% 電子郵件：luzhenbo@yahoo.com.cn
% 個人主頁：http://luzhenbo.88uu.com.cn
% 參數文獻：Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"
%
% Support Vector Machine Matlab Toolbox 1.0 - Nu Support Vector Regression
% Platform : Matlab6.5 / Matlab7.0
% Copyright : LU Zhen-bo, Navy Engineering University, WuHan, HuBei, P.R.China, 430033??
% E-mail : luzhenbo@yahoo.com.cn? ?? ???
% Homepage : http://luzhenbo.88uu.com.cn? ???
% Reference : Chih-Chung Chang, Chih-Jen Lin. "LIBSVM: a Library for Support Vector Machines"
%
% Solve the quadratic PageRankogramming PageRankoblem - "quadPageRankog.m"

clc
clear
close all

% ------------------------------------------------------------%
% 定義核函數及相關參數

C = 100;? ?? ?? ?? ?? ? % 拉格朗日乘子上界
nu = 0.01;? ?? ?? ?? ???% nu -> (0,1] 在支持向量數與擬合精度之間進行折衷

%ker = struct('type','linear');
%ker = struct('type','ploy','degree',3,'offset',1);
ker = struct('type','gauss','width',1);
%ker = struct('type','tanh','gamma',1,'offset',0);

% ker - 核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).

% ------------------------------------------------------------%
% 構造兩類訓練樣本

n = 50;
rand('state',42);
X??= linspace(-4,4,n)';? ?? ?? ?? ?? ?? ?? ?? ?? ?% 訓練樣本,n×d的矩陣,n為樣本個數,d為樣本維數,那里d=1
Ys = (1-X+2*X.^2).*exp(-.5*X.^2);
f = 0.2;? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?% 相對誤差
Y??= Ys+f*max(abs(Ys))*(2*rand(size(Ys))-1)/2;? ? % 訓練方向,n×1的矩陣,n為樣本個數,值為期望輸出

figure(4)
plot(X,Ys,'b-',X,Y,'b*');
hold on;

% ------------------------------------------------------------%
% 訓練支持向量機

tic
svm = Nu_SVR_Train(X,Y,C,nu,ker);
t_train = toc

% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子

% ------------------------------------------------------------%
% 找尋支持向量

a = svm.a;
epsilon = 1e-8;? ?? ?? ?? ?? ?? ?? ?% 如果"絕對值"小于此值則認為是0
i_sv = find(abs(a)>epsilon);? ?? ???% 支持向量下標,那里對abs(a)進行判定
plot(X(i_sv),Y(i_sv),'ro');

% ------------------------------------------------------------%
% 測試輸出

tic
Yd = Nu_SVR_Sim(svm,X);? ?? ?? ???% 測試輸出
t_sim = toc

plot(X,Yd,'r--');
hold off;

function [K] = CalcKernel(ker,x,y)
% Calculate kernel function.? ?
%
% x: 輸入樣本,n1×d的矩陣,n1為樣本個數,d為樣本維數
% y: 輸入樣本,n2×d的矩陣,n2為樣本個數,d為樣本維數
%
% ker??核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).
%
% ker = struct('type','linear');
% ker = struct('type','ploy','degree',d,'offset',c);
% ker = struct('type','gauss','width',s);
% ker = struct('type','tanh','gamma',g,'offset',c);
%
% K: 輸出核參數,n1×n2的矩陣

%-------------------------------------------------------------%
% 轉成列向量
x = x';
y = y';

%-------------------------------------------------------------%

switch ker.type
? ? case 'linear'
? ?? ???K = x'*y;
? ? case 'ploy'
? ?? ???d = ker.degree;
? ?? ???c = ker.offset;
? ?? ???K = (x'*y+c).^d;
? ? case 'gauss'
? ?? ???s = ker.width;
? ?? ???rows = size(x,2);
? ?? ???cols = size(y,2);? ?
? ?? ???tmp = zeros(rows,cols);
? ?? ???for i = 1:rows
? ?? ?? ?? ?for j = 1:cols
? ?? ?? ?? ?? ? tmp(i,j) = norm(x(:,i)-y(:,j));
? ?? ?? ?? ?end
? ?? ???end? ?? ???
? ?? ???K = exp(-0.5*(tmp/s).^2);
? ? case 'tanh'
? ?? ???g = ker.gamma;
? ?? ???c = ker.offset;
? ?? ???K = tanh(g*x'*y+c);
? ? otherwise
? ?? ???K = 0;
end

function svm = Nu_SVR_Train(X,Y,C,nu,ker)

% 輸入參數:
% X??訓練樣本,n×d的矩陣,n為樣本個數,d為樣本維數
% Y??訓練方向,n×1的矩陣,n為樣本個數,值為期望輸出
% C??拉格朗日乘子上界
% e??不敏感損失函數的參數，Epsilon越大，支持向量越少
% ker??核參數(結構體變量)
% the following fields:
%? ?type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?degree - Degree d of polynomial kernel (positive scalar).
%? ?offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?width??- Width s of Gauss kernel (positive scalar).
%? ?gamma??- Slope g of the tanh kernel (positive scalar).

% 輸出參數:
% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子

% ------------------------------------------------------------%
% 解二次優化

n = length(Y);
Q = Calckernel(ker,X,X);
H = [Q,-Q;-Q,Q];
f = [-Y;+Y];? ?? ?? ? % 符號不相同,決策函數就不相同,實際上是一回事!見文件"Nu_SVR_Sim.m"第37,45行
%f = [+Y;-Y];
A = [];
b = [];
Aeq = [ones(1,n),-ones(1,n);ones(1,2*n)];
beq = [0;C*n*nu];
lb = zeros(2*n,1);? ?? ?? ?? ?? ?
ub = C*ones(2*n,1);
a0 = zeros(2*n,1);

%第三步：調用優化工具箱quadPageRankog函數乞求二次規劃
options = optimset;
options.LargeScale = 'off';
options.Display = 'off';

[a,fval,eXitflag,output,lambda]??= quadPageRankog(H,f,A,b,Aeq,beq,lb,ub,a0,options);
eXitflag

% ------------------------------------------------------------%
% 輸出 svm
svm.ker = ker;
svm.x = X;
svm.y = Y;
svm.a = a(1:n)-a(n+1:end);

function Yd = Nu_SVR_Sim(svm,Xt)
% 輸入參數:
% svm??支持向量機(結構體變量)
% the following fields:
%? ?ker - 核參數
%? ?? ? type? ?- linear :??k(x,y) = x'*y
%? ?? ?? ?? ?? ? poly? ?:??k(x,y) = (x'*y+c)^d
%? ?? ?? ?? ?? ? gauss??:??k(x,y) = exp(-0.5*(norm(x-y)/s)^2)
%? ?? ?? ?? ?? ? tanh? ?:??k(x,y) = tanh(g*x'*y+c)
%? ?? ? degree - Degree d of polynomial kernel (positive scalar).
%? ?? ? offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).
%? ?? ? width??- Width s of Gauss kernel (positive scalar).
%? ?? ? gamma??- Slope g of the tanh kernel (positive scalar).
%? ?x - 訓練樣本
%? ?y - 訓練方向;
%? ?a - 拉格朗日乘子
%
% Xt??測試樣本,n×d的矩陣,n為樣本個數,d為樣本維數

% 輸出參數:
% Yd??測試輸出,n×1的矩陣,n為樣本個數,值為+1或-1

% ------------------------------------------------------------%

ker = svm.ker;
X = svm.x;
Y = svm.y;
a = svm.a;? ? % 那里實際值為 a(1:n)-a(n+1:end),見文件"Nu_SVR_Train.m"第56行

% ------------------------------------------------------------%
% 求 b

epsilon = 1e-8;? ?? ?? ?? ?? ?? ?? ?% 如果"絕對值"小于此值則認為是0
i_sv = find(abs(a)>epsilon);? ?? ???% 支持向量下標,那里對abs(a)進行判定

tmp = a'*Calckernel(ker,X,X(i_sv,);? ?? ? % 行向量
b = Y(i_sv)-tmp';? ?? ?? ?? ?? ?? ?? ?? ?? ?% 符號不相同,決策函數就不相同,實際上是一回事!見文件"Nu_SVR_Train.m"第33行
%b = Y(i_sv)+tmp';
b = mean(b);

% ------------------------------------------------------------%
% 測試輸出

nt = size(Xt,1);? ?? ?? ?? ?? ?? ???% 測試樣本數
tmp =??a'*Calckernel(ker,X,Xt);? ???% 符號不相同,決策函數就不相同,實際上是一回事!見文件"Nu_SVR_Train.m"第33行
%tmp =??-a'*Calckernel(ker,X,Xt);
Yd = (tmp+b)';

總結

以上是生活随笔為你收集整理的陆振波的SVM的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。