RPCA以及LRR
RPCA
關(guān)于RPCA的博客：
原文：http://blog.csdn.net/abcjennifer/article/details/8572994
譯文：http://blog.csdn.net/u010545732/article/details/19066725
數(shù)據(jù)降維的總結(jié)：數(shù)據(jù)降維（RPCA,LRR.LE等）
http://download.csdn.net/detail/tiandijun/8569653
低秩的子空間恢復(fù)：http://download.csdn.net/detail/tiandijun/8569675
LRR
Tutorials
John Wright, Zhouchen Lin, and Yi Ma. Presented at International Conference on Image and Graphics (ICIG), August 2011.?
John Wright, Zhouchen Lin, and Yi Ma. Presented at IEEE International Conference on Image Processing (ICIP), September 2010.
Theory
Emmanuel Candès, Xiaodong Li, Yi Ma, and John Wright. Journal of the ACM, volume 58, no. 3, May 2011.?
John Wright, and Yi Ma. IEEE Transactions on Information Theory, volume 56, no. 7, July 2010.?
John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, and Yi Ma. In Proceedings of Neural Information Processing Systems (NIPS), December 2009.?
Zihan Zhou, Xiaodong Li, John Wright, Emmanuel Candès, and Yi Ma. In Proceedings of IEEE International Symposium on Information Theory (ISIT), June 2010.?
Arvind Ganesh, John Wright, Xiaodong Li, Emmanuel Candès, and Yi Ma. In Proceedings of IEEE International Symposium on Information Theory (ISIT), June 2010.?
Arvind Ganesh, Kerui Min, John Wright, and Yi Ma. submitted to International Symposium on Information Theory, 2012.?
John Wright, Arvind Ganesh, Kerui Min, and Yi Ma. submitted to International Symposium on Information Theory, 2012.
Robust PCA
We provide MATLAB packages to solve the RPCA optimization problem by different methods. All of our code below is Copyright 2009 Perception and Decision Lab, University of Illinois at Urbana-Champaign, and Microsoft Research Asia, Beijing. We also provide links to some publicly available packages to solve the RPCA problem. Please contact?John Wright?or?Arvind Ganesh?if you have any questions or comments. If you are looking for the code to our RASL and TILT algorithms, please refer to the?applications?section.
Usage?- The most basic form of the exact ALM function is?[A, E] = exact_alm_rpca(D, λ), and that of the inexact ALM function is?[A, E] = inexact_alm_rpca(D, λ), where?D?is a real matrix and λ is a positive real number. We solve the RPCA problem using the method of augmented Lagrange multipliers. The method converges Q-linearly to the optimal solution. The exact ALM algorithm is simple to implement, each iteration involves computing a partial SVD of a matrix the size of D, and converges to the true solution in a small number of iterations. The algorithm can be further speeded up by using a fast continuation technique, thereby yielding the inexact ALM algorithm.?
Reference?-?The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, Z. Lin, M. Chen, L. Wu, and Y. Ma (UIUC Technical Report UILU-ENG-09-2215, November 2009).?
Usage?- The most basic form of the full SVD version of the function is?[A, E] = proximal_gradient_rpca(D, λ), where?D?is a real matrix and λ is a positive real number. We consider a slightly different version of the original RPCA problem by relaxing the equality constraint. The algorithm is simple to implement, each iteration involves computing the SVD of a matrix the size of D, and converges to the true solution in a small number of iterations. The algorithm can be further speeded up by computing partial SVDs at each iteration. The most basic form of the partial SVD version of the function is?[A, E] = partial_proximal_gradient_rpca(D, λ), where?D?is a real matrix and λ is a positive real number.?
Reference?-?Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix, Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma (UIUC Technical Report UILU-ENG-09-2214, August 2009).?
Usage?- The most basic form of the function is?[A, E] = dual_rpca(D, λ), where?D?is a real matrix and λ is a positive real number. We solve the convex dual of the RPCA problem, and retrieve the low-rank and sparse error matrices from the dual optimal solution. The algorithm computes only a partial SVD in each iteration and hence, scales well with the size of the matrix?D.
Reference?-?Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix, Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma (UIUC Technical Report UILU-ENG-09-2214, August 2009).?
Usage?- The most basic form of the function is?[A, E] = singular_value_rpca(D, λ), where?D?is a real matrix and λ is a positive real number. Here again, we solve a relaxation of the original RPCA problem, albeit different from the one solved by the Accelerated Proximal Gradient (APG) method. The algorithm is extremely simple to implement, and the computational complexity of each iteration is about the same as that of the APG method. However, the number of iterations to convergence is typically quite large.?
Reference?-?A Singular Value Thresholding Algorithm for Matrix Completion,
J. -F. Cai, E. J. Candès, and Z. Shen (2008).?
Reference?-?Sparse and Low-Rank Matrix Decomposition via Alternating Direction Methods, X. Yuan, and J. Yang (2009).
Matrix Completion
We provide below links to publicly available code and references to solve the matrix completion problem faster than conventional algorithms.
Usage?- The most basic form of the inexact ALM function is?A = inexact_alm_mc(D), where?D?is the incomplete matrix defined in the MATLAB sparse matrix format and the output?A?is a structure with two components -?A.U?and?A.V?(the left and right singular vectors scaled respectively by the square root of the corresponding non-zero singular values). Please refer to the file?test_alm_mc.m?for details on defining?Dappropriately. The algorithm is identical to the inexact ALM method described above to solve the RPCA prblem, and enjoys the same convergence properties.?
Reference?-?The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, Z. Lin, M. Chen, L. Wu, and Y. Ma (UIUC Technical Report UILU-ENG-09-2215, November 2009).?
Reference?-?A Singular Value Thresholding Algorithm for Matrix Completion, J. -F. Cai, E. J. Candès, and Z. Shen (2008).?
Reference?-?Matrix Completion from a Few Entries, R.H. Keshavan, A. Montanari, and S. Oh (2009).?
Reference?-?An Accelerated Proximal Gradient Algorithm for Nuclear Norm Regularized Least Squares Problems, K. -C. Toh, and S. Yun (2009).?
Reference?-?SET: An Algorithm for Consistent Matrix Completion, W. Dai, and O. Milenkovic (2009).?
Reference?-?Online Identification and Tracking of Subspaces from Highly Incomplete Information, L. Balzano, R. Nowak, and B. Recht (2010).
Comparison of Algorithms
We provide a simple comparison of the speed and accuracy of various RPCA algorithms. Each algorithm was tested on a rank-20 matrix of size 400 x 400 with 5% of its entries corrupted by large errors. The low-rank matrix?A?is generated as the product?LRT, where?L?and?R?are 400 x 20 matrices whose entries are i.i.d. according to the standard Gaussian distribution. The error matrix?E?is a sparse matrix whose support is chosen uniformly at random and whose non-zero entries are independent and uniformly distributed in the range [-50,50]. The value of λ was fixed as 0.05. The accuracy of the solution is indicated by the rank of the estimated low-rank matrix?A?and its relative error (in Frobenius norm) with respect to the true solution. All simulations were carried out on a Macbook Pro with a 2.8 GHz processor, two cores, and 4 GB memory.
Please note that the following tables represent typical performance, using default parameters, on random matrices drawn according to the distribution specified earlier. The performance could vary when dealing with matrices drawn from other distributions or with real data.?
Robust PCA Algorithm Comparison
| Algorithm | Rank of estimate | Relative error in estimate of?A | Time (s) |
| Singular Value Thresholding | 20 | 3.4 x 10-4 | 877 |
| Accelerated Proximal Gradient | 20 | 2.0 x 10-5 | 43 |
| Accelerated Proximal Gradient (with partial SVDs) | 20 | 1.8 x 10-5 | 8 |
| Dual Method | 20 | 1.6 x 10-5 | 177 |
| Exact ALM | 20 | 7.6 x 10-8 | 4 |
| Inexact ALM | 20 | 4.3 x 10-8 | 2 |
| Alternating Direction Methods | 20 | 2.2 x 10-5 | 5 |
總結(jié)
- 上一篇： 机器学习和图像资源整理
- 下一篇： Inductive Robust Pri