## Abstract

Recently some results on consistency and robustness of support

vector machines for nonparametric classification and nonparametric

regression were derived for non-negative convex loss functions $L$

of Nemitski type, see Christmann and Steinwart (2007) [3] and

Steinwart and Christmann (2008) [15], given that a weak moment

condition for the joint distribution $\P$ on $X\times Y$ is valid.

However, this condition excludes heavy-tailed distributions (e.g.,

Cauchy distribution) or several extreme value distributions which

can occur in financial or actuarial problems.

We will weaken the above condition on $\P$ to only a condition on

the marginal distribution $\P_X$, such that heavy-tailed conditional

distributions are also covered. As was already used by Huber (1967)

in a different setting, the idea is to shift the loss function $L:

X\times Y\times \R\to [0,\infty)$ by some function which is

independent of the estimator. More precisely, we define the shifted

loss function $\Ls(x,y,f(x)) := L(x,y,f(x)) - L(x,y,0)$. It is

obvious that the ``loss'' $\Ls(x,y,f(x))$ can be negative. Let $\fP$

be the decision function of the original support vector machine. We

define

$$

\fPLs = \arg \inf_{f \in \mathcal{H}} \Ex_\P \Ls(X,Y,f(X)) + \lambda \| f \|_{\mathcal{H}}^2 \, ,

$$

where $\mathcal{H}$ is the reproducing kernel Hilbert space of a

measurable kernel $k:X \times X \to \R$ and $\lambda>0$ is some

regularization parameter.

We will first discuss the used ``$\Ls$-trick'', cite some properties

of the new loss $\Ls$, and show that $\fPLs$ exists and is unique.

Furthermore, this function will coincide with $\fP$ if the latter

exists. We then give a representer theorem and results on both

risk-consistency as well as consistency of the solution. We will

also show that $\fPLs$ is robust in the sense of influence functions

if the kernel is bounded and if the loss function $L$ is Lipschitz

continuous. This result holds true for Hampel's influence function

and also for the Bouligand influence function proposed by Christmann

and Van Messem (2008) [5].

vector machines for nonparametric classification and nonparametric

regression were derived for non-negative convex loss functions $L$

of Nemitski type, see Christmann and Steinwart (2007) [3] and

Steinwart and Christmann (2008) [15], given that a weak moment

condition for the joint distribution $\P$ on $X\times Y$ is valid.

However, this condition excludes heavy-tailed distributions (e.g.,

Cauchy distribution) or several extreme value distributions which

can occur in financial or actuarial problems.

We will weaken the above condition on $\P$ to only a condition on

the marginal distribution $\P_X$, such that heavy-tailed conditional

distributions are also covered. As was already used by Huber (1967)

in a different setting, the idea is to shift the loss function $L:

X\times Y\times \R\to [0,\infty)$ by some function which is

independent of the estimator. More precisely, we define the shifted

loss function $\Ls(x,y,f(x)) := L(x,y,f(x)) - L(x,y,0)$. It is

obvious that the ``loss'' $\Ls(x,y,f(x))$ can be negative. Let $\fP$

be the decision function of the original support vector machine. We

define

$$

\fPLs = \arg \inf_{f \in \mathcal{H}} \Ex_\P \Ls(X,Y,f(X)) + \lambda \| f \|_{\mathcal{H}}^2 \, ,

$$

where $\mathcal{H}$ is the reproducing kernel Hilbert space of a

measurable kernel $k:X \times X \to \R$ and $\lambda>0$ is some

regularization parameter.

We will first discuss the used ``$\Ls$-trick'', cite some properties

of the new loss $\Ls$, and show that $\fPLs$ exists and is unique.

Furthermore, this function will coincide with $\fP$ if the latter

exists. We then give a representer theorem and results on both

risk-consistency as well as consistency of the solution. We will

also show that $\fPLs$ is robust in the sense of influence functions

if the kernel is bounded and if the loss function $L$ is Lipschitz

continuous. This result holds true for Hampel's influence function

and also for the Bouligand influence function proposed by Christmann

and Van Messem (2008) [5].

Original language | English |
---|---|

Title of host publication | 17th Annual Meeting of the Belgian Statistical Society, Oct 14-16, 2009, Lommel (Belgium) |

Publication status | Published - 14 Oct 2009 |

### Publication series

Name | 17th Annual Meeting of the Belgian Statistical Society, Oct 14-16, 2009, Lommel (Belgium) |
---|

## Keywords

- SVM
- heavy tails
- robustness
- consistency
- Bouligand