A Review on Consistency and Robustness Properties of Support Vector Machines for Heavy-Tailed Distributions

Arnout Van Messem, Andreas Christmann, H.-h. Bock (Editor), W. Gaul (Editor), A. Okada (Editor), M. Vichi (Editor)

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


Support vector machines (SVMs) belong to the class of modern statistical machine learning techniques and can be described as M-estimatorswith aHilbert norm regularization term for functions. SVMs are consistent and robust for classification and regression purposes if based on a Lipschitz continuous loss and a bounded continuous kernel with a dense reproducing kernel Hilbert space. For regression, one of the conditions used is that the output variable Y has a finite first absolute moment. This assumption, however, excludes heavy-tailed distributions. Recently, the applicability of SVMs was enlarged to these distributions by considering shifted loss functions. In this review paper, we briefly describe the approach of SVMs based on shifted loss functions and list some properties of such SVMs. Then, we prove that SVMs based on a bounded continuous kernel and on a convex and Lipschitz continuous, but not necessarily differentiable, shifted loss function have a bounded Bouligand influence function for all distributions, even for heavy-tailed distributions including extreme value distributions and Cauchy distributions. SVMs are thus robust in this sense. Our result covers the important loss functions epsilon-insensitive for regression and pinball for quantile regression, which were not covered by earlier results on the influence function. We demonstrate the usefulness of SVMs even for heavy-tailed distributions by applying SVMs to a simulated data set with Cauchy errors and to a data set of large fire insurance claims of Copenhagen Re.
Original languageEnglish
Pages (from-to)199-220
Number of pages22
JournalAdvances in Data Analysis and Classification
Issue number4
Publication statusPublished - Sep 2010

Bibliographical note

H.-H. Bock; W. Gaul; A. Okada; M. Vichi


  • Regularized empirical risk minimization
  • Support vector machines
  • Consistency
  • Robustness
  • Bouligand influence function
  • Heavy tails


Dive into the research topics of 'A Review on Consistency and Robustness Properties of Support Vector Machines for Heavy-Tailed Distributions'. Together they form a unique fingerprint.

Cite this