Message length formulation of Support Vector Machines for binary classification - a preliminary scheme

This paper presents a preliminary attempt at performing extrinsic binary classification by reformulating the Support Vector Machine (SVM) approach in a Bayesian Message Length framework. The reformulation uses the Minimum Message Length (MML) principle as a way of costing each hyperplane via a two-part message. This message defines a separating hyperplane. The length of this message is used as an objective function for a search through the hypothesis space of possible hyperplanes used to dichotomise a set of data points.

Two preliminary MML implementations are presented here, which difier in the (Bayesian) coding schemes used and the search procedures. The generalisation ability of these two reformulations on both artificial and real data sets are compared against current implementations of Support Vector Machines - namely SVM light, the Lagrangian Support Vector Machine and SMOBR. It was found that, in general, all implementations improved as the size of the data sets increased. The MML implementations tended to perform best on the inseparable data sets and the real data set. Our preliminary MML scheme showed itself to be a strong competitor against the classical SVM, despite inefficiencies in the current scheme