Consistency and Robustness Properties of Support Vector Machines for Heavy-Tailed Distributions

Arnout Van Messem

Research output: Chapter in Book/Report/Conference proceedingMeeting abstract (Book)

Abstract

Recently results on consistency and robustness of support vector
machines (SVMs) were derived for non-negative convex losses $L$ of
Nemitski type given some weak moment condition for the joint
distribution $\P$ on $X\times Y$ (Christmann and Steinwart, 2007,
[3]). However, this condition excludes heavy-tailed distributions
such as the Cauchy distribution or several extreme value
distributions.

The condition on $\P$ can be weakened to only a condition on the
marginal distribution $\P_X$ by shifting the loss $L$ downwards, a
trick that will enlarge the applicability of SVMs to heavy-tailed
conditional distributions. More precisely, we define the shifted
loss $\Ls(x,y,f(x)) := L(x,y,f(x)) - L(x,y,0)$. Obviously, this new
``loss'' $\Ls(x,y,f(x))$ can be negative. We define the decision
function of the shifted SVM as $\fPLs$.
%$$
% \fPLs = \arg \inf_{f \in \mathcal{H}} \Ex_\P \Ls(X,Y,f(X)) + \lambda \| f \|_{\mathcal{H}}^2 \, ,
%$$
%with $\mathcal{H}$ the reproducing kernel Hilbert space of a
%measurable kernel $k$ and $\lambda>0$ some regularization parameter.

We will give some properties of $\Ls$ and $\fPLs$ and we will state
a representer theorem and results on both risk-consistency as well
as consistency of the solution. Finally we will show that, given
some conditions, $\fPLs$ is robust in the sense of both Hampel's
influence function as well as the Bouligand influence function
introduced by Christmann and Van Messem (2008) [5].
Original languageEnglish
Title of host publicationVUB PhD Day, May 28, 2010, Brussels (Belgium)
Publication statusPublished - 28 May 2011

Publication series

NameVUB PhD Day, May 28, 2010, Brussels (Belgium)

Keywords

  • SVM
  • consistency
  • robustness
  • heavy tails
  • Bouigand

Fingerprint

Dive into the research topics of 'Consistency and Robustness Properties of Support Vector Machines for Heavy-Tailed Distributions'. Together they form a unique fingerprint.

Cite this