This R package is a wrapper around the liblinear C/C++ library for machine learning. LIBLINEAR is a linear classifier for data with millions of instances and features. It supports L2-regularized classifiers (such as L2-loss linear SVM, L1-loss linear SVM, and logistic regression) as well as L1-regularized classifiers (such as L2-loss linear SVM and logistic regression). The main features of LiblineaR include multi-class classification (one-vs-the rest, and Crammer & Singer method), cross validation for model selection, probability estimates (logistic regression only) or weights for unbalanced data. The estimation of the models is particularly fast as compared to other libraries. For more information on the C/C++ liblinear library itself, refer to:
R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin.
LIBLINEAR: A Library for Large Linear Classification,
Journal of Machine Learning Research 9(2008), 1871-1874.
Software available at http://www.csie.ntu.edu.tw/~cjlin/liblinear .
Copyright
---------
All of this software is copyrighted by the list of authors in the DESCRIPTION file of
the package and subject to the GNU GENERAL PUBLIC LICENSE, Version 2, see the file
COPYING for details. The LIBLINEAR C/C++ code is copyright Chih-Chung Chang and
Chih-Jen Lin.
Installation
------------
This package should be installed using the `R CMD INSTALL' mechanism, see
the R online help or R manuals for details on how to install packages.
Mathematical Details
--------------------
Formulations:
For L2-regularized logistic regression (-s 0), we solve
min_w w^Tw/2 + C \sum log(1 + exp(-y_i w^Tx_i))
For L2-regularized L2-loss SVC dual (-s 1), we solve
min_alpha 0.5(alpha^T (Q + I/2/C) alpha) - e^T alpha
s.t. 0 <= alpha_i,
For L2-regularized L2-loss SVC (-s 2), we solve
min_w w^Tw/2 + C \sum max(0, 1- y_i w^Tx_i)^2
For L2-regularized L1-loss SVC dual (-s 3), we solve
min_alpha 0.5(alpha^T Q alpha) - e^T alpha
s.t. 0 <= alpha_i <= C,
For L1-regularized L2-loss SVC (-s 5), we solve
min_w \sum |w_j| + C \sum max(0, 1- y_i w^Tx_i)^2
For L1-regularized logistic regression (-s 6), we solve
min_w \sum |w_j| + C \sum log(1 + exp(-y_i w^Tx_i))
where
Q is a matrix with Q_ij = y_i y_j x_i^T x_j.
For L2-regularized logistic regression (-s 7), we solve
min_alpha 0.5(alpha^T Q alpha) + \sum alpha_i*log(alpha_i) + \sum (C-alpha_i)*log(C-alpha_i) - a constant
s.t. 0 <= alpha_i <= C,
If bias >= 0, w becomes [w; w_{n+1}] and x becomes [x; bias].
The primal-dual relationship implies that -s 1 and -s 2 give the same
model, and -s 0 and -s 7 give the same.
We implement 1-vs-the rest multi-class strategy. In training i
vs. non_i, their C parameters are (weight from -wi)*C and C,
respectively. If there are only two classes, we train only one
model. Thus weight1*C vs. weight2*C is used. See examples below.
We also implement multi-class SVM by Crammer and Singer (-s 4):
min_{w_m, \xi_i} 0.5 \sum_m ||w_m||^2 + C \sum_i \xi_i
s.t. w^T_{y_i} x_i - w^T_m x_i >= \e^m_i - \xi_i \forall m,i
where e^m_i = 0 if y_i = m,
e^m_i = 1 if y_i != m,
Here we solve the dual problem:
min_{\alpha} 0.5 \sum_m ||w_m(\alpha)||^2 + \sum_i \sum_m e^m_i alpha^m_i
s.t. \alpha^m_i <= C^m_i \forall m,i , \sum_m \alpha^m_i=0 \forall i
where w_m(\alpha) = \sum_i \alpha^m_i x_i,
and C^m_i = C if m = y_i,
C^m_i = 0 if m != y_i.