Control-based algorithms for high dimensional online learning

Hanwen Ning, Jiaming Zhang, Ting Ting Feng, Eric King wah Chu, Tianhai Tian

Research output: Contribution to journalArticleResearchpeer-review

Abstract

In the era of big data, the high-dimensional online learning problems require huge computing power. This paper proposes a novel approach for high-dimensional online learning. Two new algorithms are developed for online high-dimensional regression and classification problems, respectively. The problems are formulated as feedback control problems for some low dimensional systems. The novel learning algorithms are then developed via the control problems. Via an efficient polar decomposition, we derive the explicit solutions of the control problems, substantially reducing the corresponding computational complexity, especially for high dimensional large-scale data streams. Comparing with conventional methods, the new algorithm can achieve more robust and accurate performance with faster convergence. This paper demonstrates that optimal control can be an effective approach for developing high dimensional learning algorithms. We have also for the first time proposed a control-based robust algorithm for classification problems. Numerical results support our theory and illustrate the efficiency of our algorithm.

Original languageEnglish
Pages (from-to)1909-1942
Number of pages34
JournalJournal of the Franklin Institute
Volume357
Issue number3
DOIs
Publication statusPublished - Feb 2020

Cite this