Solving regression problems using competitive ensemble models

Yakov Frayman, Bernard F. Rolfe, Geoffrey Ian Webb

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

12 Citations (Scopus)


The use of ensemble models in many problem domains has increased significantly in the last fewyears. The ensemble modeling, in particularly boosting, has shown a great promise in improving predictive performance of a model. Combining the ensemble members is normally done in a co-operative fashion where each of the ensemble members performs the same task and their predictions are aggregated to obtain the improved performance. However, it is also possible to combine the ensemble members in a competitive fashion where the best prediction of a relevant ensemble member is selected for a particular input. This option has been previously somewhat overlooked. The aim of this article is to investigate and compare the competitive and co-operative approaches to combining the models in the ensemble. A comparison is made between a competitive ensemble model and that of MARS with bagging, mixture of experts, hierarchical mixture of experts and a neural network ensemble over several public domain regression problems that have a high degree of nonlinearity and noise. The empirical results showa substantial advantage of competitive learning versus the co-operative learning for all the regression problems investigated. The requirements for creating the efficient ensembles and the available guidelines are also discussed.
Original languageEnglish
Title of host publicationAI 2002: Advances in Artificial Intelligence
Subtitle of host publication15th Australian Joint Conference on Artificial Intelligence Canberra, Australia, December 2-6, 2002 Proceedings
EditorsBob McKay, John Slaney
Place of PublicationBerlin Germany
Number of pages12
ISBN (Print)3540001972
Publication statusPublished - 2002
Externally publishedYes
EventAustralasian Joint Conference on Artificial Intelligence 2002 - Canberra, Australia
Duration: 2 Dec 20026 Dec 2002
Conference number: 15th (Proceedings)

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743


ConferenceAustralasian Joint Conference on Artificial Intelligence 2002
Abbreviated titleAI 2002
Internet address

Cite this