Fighting Boredom in Recommender Systems with Linear Reinforcement Learning

Romain Warlop 1 Alessandro Lazaric 2, 1 Jérémie Mary 3, 1
1 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : A common assumption in recommender systems (RS) is the existence of a best fixed recommendation strategy. Such strategy may be simple and work at the item level (e.g., in multi-armed bandit it is assumed one best fixed arm/item exists) or implement more sophisticated RS (e.g., the objective of A/B testing is to find the best fixed RS and execute it thereafter). We argue that this assumption is rarely verified in practice, as the recommendation process itself may impact the user's preferences. For instance, a user may get bored by a strategy, while she may gain interest again, if enough time passed since the last time that strategy was used. In this case, a better approach consists in alternating different solutions at the right frequency to fully exploit their potential. In this paper, we first cast the problem as a Markov decision process, where the rewards are a linear function of the recent history of actions, and we show that a policy considering the long-term influence of the recommendations may outperform both fixed-action and contextual greedy policies. We then introduce an extension of the UCRL algorithm (LINUCRL) to effectively balance exploration and exploitation in an unknown environment, and we derive a regret bound that is independent of the number of states. Finally, we empirically validate the model assumptions and the algorithm in a number of realistic scenarios.
Type de document :
Communication dans un congrès
Neural Information Processing Systems, Dec 2018, Montreal, Canada. 2018
Liste complète des métadonnées
Contributeur : Romain Warlop <>
Soumis le : mercredi 7 novembre 2018 - 16:11:55
Dernière modification le : jeudi 7 février 2019 - 17:09:34
Document(s) archivé(s) le : vendredi 8 février 2019 - 15:34:28


Fichiers produits par l'(les) auteur(s)


  • HAL Id : hal-01915468, version 1


Romain Warlop, Alessandro Lazaric, Jérémie Mary. Fighting Boredom in Recommender Systems with Linear Reinforcement Learning. Neural Information Processing Systems, Dec 2018, Montreal, Canada. 2018. 〈hal-01915468〉



Consultations de la notice


Téléchargements de fichiers