Adaptive context-aware reinforced agent for handwritten text recognition

Liangke Gui, Xiaodan Liang, Xiaojun Chang, Alexander G. Hauptmann

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

8 Citations (Scopus)


Handwritten text recognition has been a ubiquitous research problem in the field of computer vision. Most existing approaches focus on the recognition of handwritten words without considering the cursive nature and significant differences in the writing of individuals. In this paper, we address these problems by leveraging an adaptive context-aware reinforced agent which learns the actions to determine the scales of context regions during inference. We formulate our approach in a reinforcement learning framework. Specifically, we construct the action set with a number of context lengths. Given an image feature sequence, our model is trained to adaptively choose the optimal context length according to the observed state. An attention mechanism is then used to selectively attend the context region. Our model can generalize well from recognizing isolated words to recognizing individual lines of text while remain low computation overheads. We conduct extensive experiments on three large-scale handwritten text recognition datasets. The experimental results show that our proposed model is superior to the state-of-the-art alternatives.

Original languageEnglish
Title of host publication29th British Machine Vision Conference, BMVC 2018
EditorsHubert P. H. Shum, Timothy Hospedales
Place of PublicationLondon UK
PublisherBritish Machine Vision Association
Number of pages13
Publication statusPublished - 2018
Externally publishedYes
EventBritish Machine Vision Conference 2018 - Newcastle, United Kingdom
Duration: 3 Sept 20186 Sept 2018
Conference number: 29th


ConferenceBritish Machine Vision Conference 2018
Abbreviated titleBMVC 2018
Country/TerritoryUnited Kingdom
Internet address

Cite this