Capacity-aware sequential recommendations

Frits De Nijs, Georgios Theocharous, Nikos Vlassis, Mathijs M. De Weerdt, Matthijs T.J. Spaan

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

5 Citations (Scopus)


Personalized recommendations are increasingly important to engage users and guide them through large systems, for example when recommending points of interest to tourists visiting a popular city. To maximize long-term user experience, the system should consider issuing recommendations sequentially, since by observing the user's response to a recommendation, the system can update its estimate of the user's (latent) interests. However, as traditional recommender systems target individuals, their effect on a collective of users can unintentionally overload capacity. Therefore, recommender systems should not only consider the users' interests, but also the effect of recommendations on the available capacity.

The structure in such a constrained, multi-agent, partially observable decision problem can be exploited by a novel belief-space sampling algorithm which bounds the size of the state space by a limit on regret. By exploiting the stationary structure of the problem, our algorithm is significantly more scalable than existing approximate solvers. Moreover, by explicitly considering the information value of actions, this algorithm significantly improves the quality of recommendations over an extension of posterior sampling reinforcement learning to the constrained multi-agent case. We show how to decouple constraint satisfaction from sequential recommendation policies, resulting in algorithms which issue recommendations to thousands of agents while respecting constraints.
Original languageEnglish
Title of host publicationAAMAS’18 - Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems
Subtitle of host publicationJuly 10-15, 2018 Stockholm, Sweden
EditorsMehdi Dastani , Gita Sukthankar
Place of PublicationRichland SC USA
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems
Number of pages9
ISBN (Electronic)9781450356497
ISBN (Print)9781510868083
Publication statusPublished - 2018
Externally publishedYes
EventInternational Conference on Autonomous Agents and Multiagent Systems 2018 - Stockholm, Sweden
Duration: 10 Jul 201815 Jul 2018
Conference number: 17th (Proceedings)


ConferenceInternational Conference on Autonomous Agents and Multiagent Systems 2018
Abbreviated titleAAMAS 2018
Internet address


  • Planning under uncertainty
  • Multi-agent planning
  • Recommender systems

Cite this