ALRn: Accelerated higher-order logistic regression

Nayyar A. Zaidi, Geoff I. Webb, Mark J. Carman, François Petitjean, Jesús Cerquides

    Research output: Contribution to journalArticleResearchpeer-review

    10 Citations (Scopus)

    Abstract

    This paper introduces Accelerated Logistic Regression: a hybrid generative-discriminative approach to training Logistic Regression with high-order features. We present two main results: (1) that our combined generative-discriminative approach significantly improves the efficiency of Logistic Regression and (2) that incorporating higher order features (i.e. features that are the Cartesian products of the original features) reduces the bias of Logistic Regression, which in turn significantly reduces its error on large datasets. We assess the efficacy of Accelerated Logistic Regression by conducting an extensive set of experiments on 75 standard datasets. We demonstrate its competitiveness, particularly on large datasets, by comparing against state-of-the-art classifiers including Random Forest and Averaged n-Dependence Estimators.
    Original languageEnglish
    Pages (from-to)151-194
    Number of pages44
    JournalMachine Learning
    Volume104
    Issue number2-3
    DOIs
    Publication statusPublished - 1 Sep 2016

    Keywords

    • Higher-order logistic regression
    • Low-bias classifiers
    • Generative-discriminative learning

    Cite this

    Zaidi, Nayyar A. ; Webb, Geoff I. ; Carman, Mark J. ; Petitjean, François ; Cerquides, Jesús. / ALRn : Accelerated higher-order logistic regression. In: Machine Learning. 2016 ; Vol. 104, No. 2-3. pp. 151-194.
    @article{250010bcddbc42719a49d8d3a04a2497,
    title = "ALRn: Accelerated higher-order logistic regression",
    abstract = "This paper introduces Accelerated Logistic Regression: a hybrid generative-discriminative approach to training Logistic Regression with high-order features. We present two main results: (1) that our combined generative-discriminative approach significantly improves the efficiency of Logistic Regression and (2) that incorporating higher order features (i.e. features that are the Cartesian products of the original features) reduces the bias of Logistic Regression, which in turn significantly reduces its error on large datasets. We assess the efficacy of Accelerated Logistic Regression by conducting an extensive set of experiments on 75 standard datasets. We demonstrate its competitiveness, particularly on large datasets, by comparing against state-of-the-art classifiers including Random Forest and Averaged n-Dependence Estimators.",
    keywords = "Higher-order logistic regression, Low-bias classifiers, Generative-discriminative learning",
    author = "Zaidi, {Nayyar A.} and Webb, {Geoff I.} and Carman, {Mark J.} and Fran{\cc}ois Petitjean and Jes{\'u}s Cerquides",
    year = "2016",
    month = "9",
    day = "1",
    doi = "10.1007/s10994-016-5574-8",
    language = "English",
    volume = "104",
    pages = "151--194",
    journal = "Machine Learning",
    issn = "0885-6125",
    publisher = "Springer",
    number = "2-3",

    }

    ALRn : Accelerated higher-order logistic regression. / Zaidi, Nayyar A.; Webb, Geoff I.; Carman, Mark J.; Petitjean, François; Cerquides, Jesús.

    In: Machine Learning, Vol. 104, No. 2-3, 01.09.2016, p. 151-194.

    Research output: Contribution to journalArticleResearchpeer-review

    TY - JOUR

    T1 - ALRn

    T2 - Accelerated higher-order logistic regression

    AU - Zaidi, Nayyar A.

    AU - Webb, Geoff I.

    AU - Carman, Mark J.

    AU - Petitjean, François

    AU - Cerquides, Jesús

    PY - 2016/9/1

    Y1 - 2016/9/1

    N2 - This paper introduces Accelerated Logistic Regression: a hybrid generative-discriminative approach to training Logistic Regression with high-order features. We present two main results: (1) that our combined generative-discriminative approach significantly improves the efficiency of Logistic Regression and (2) that incorporating higher order features (i.e. features that are the Cartesian products of the original features) reduces the bias of Logistic Regression, which in turn significantly reduces its error on large datasets. We assess the efficacy of Accelerated Logistic Regression by conducting an extensive set of experiments on 75 standard datasets. We demonstrate its competitiveness, particularly on large datasets, by comparing against state-of-the-art classifiers including Random Forest and Averaged n-Dependence Estimators.

    AB - This paper introduces Accelerated Logistic Regression: a hybrid generative-discriminative approach to training Logistic Regression with high-order features. We present two main results: (1) that our combined generative-discriminative approach significantly improves the efficiency of Logistic Regression and (2) that incorporating higher order features (i.e. features that are the Cartesian products of the original features) reduces the bias of Logistic Regression, which in turn significantly reduces its error on large datasets. We assess the efficacy of Accelerated Logistic Regression by conducting an extensive set of experiments on 75 standard datasets. We demonstrate its competitiveness, particularly on large datasets, by comparing against state-of-the-art classifiers including Random Forest and Averaged n-Dependence Estimators.

    KW - Higher-order logistic regression

    KW - Low-bias classifiers

    KW - Generative-discriminative learning

    UR - http://www.scopus.com/inward/record.url?scp=84979249245&partnerID=8YFLogxK

    U2 - 10.1007/s10994-016-5574-8

    DO - 10.1007/s10994-016-5574-8

    M3 - Article

    AN - SCOPUS:84979249245

    VL - 104

    SP - 151

    EP - 194

    JO - Machine Learning

    JF - Machine Learning

    SN - 0885-6125

    IS - 2-3

    ER -