Leveraging Gaussian process approximations for rapid image overlay production

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)


Machine learning models trained using images can be used to generate image overlays by investigating which image areas contribute the most towards model outputs. A common approach used to accomplish this relies on blanking image regions using a sliding window and evaluating the change in model output. Unfortunately, this can be computationally expensive, as it requires numerous model evaluations. This paper shows that a Gaussian process approximation to this blanking approach produces outputs of similar quality, despite requiring signiicantly fewer model evaluations. This process is illustrated using a user-driven saliency generation problem. Here, pairwise image interest comparisons are used to infer underlying image interest and a Gaussian process model trained to predict the interest value of an image using image features extracted by a convolutional neural network. Interest overlays are generated by evaluating model change at blanking image regions selected using the prediction uncertainty of a Gaussian process regressor.

Original languageEnglish
Title of host publicationSAWACMMM'17 - Proceedings of the ACM Multimedia 2017 Workshop on South African Academic Participation
EditorsRiaan Wolhuter, Johan du Preez
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages6
ISBN (Electronic)9781450355056
Publication statusPublished - 2017
Externally publishedYes
EventWorkshop on South African Academic Participation 2017 - Mountain View, United States of America
Duration: 23 Oct 201723 Oct 2017
https://dl.acm.org/doi/proceedings/10.1145/3132711 (Proceedings)
http://www.sigmm.org/opentoc/SAWACMMM2017-TOC (Website)


ConferenceWorkshop on South African Academic Participation 2017
Abbreviated titleSAWACMMM 2017
CountryUnited States of America
CityMountain View
Internet address


  • Gaussian processes
  • Saliency generation

Cite this