Support Vector Method for Novelty Detection

TLDR

The problem is to estimate a subset S of input space such that the probability that a test point drawn from the underlying distribution lies outside S equals a specified ν, and the algorithm extends the support‑vector method to unlabelled data. The authors propose to estimate a function f that is positive on S and negative on its complement. f is represented as a kernel expansion over a small subset of training data and regularized by controlling the weight‑vector length in feature space. The algorithm’s statistical performance is theoretically analyzed.

Abstract

Suppose you are given some dataset drawn from an underlying probability distribution P and you want to estimate a simple subset S of input space such that the probability that a test point drawn from P lies outside of S equals some a priori specified ν between 0 and 1. We propose a method to approach this problem by trying to estimate a function f which is positive on S and negative on the complement. The functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data; it is regularized by controlling the length of the weight vector in an associated feature space. We provide a theoretical analysis of the statistical performance of our algorithm. The algorithm is a natural extension of the support vector algorithm to the case of unlabelled data.