Ray Summit
Using Reinforcement Learning to Optimize IAP Offer Recommendations in Mobile Games
Tuesday, June 22, 8:00PM UTCEmiliano Castro (Principal Data Scientist), Vinicius Alves
(Data Scientist), & Felipe Antunes (Staff Data Scientist), Wildlife Studios
A significant part of mobile games revenue comes from In-App Purchases (IAP), and offers play a relevant role there. Offers are defined as sales opportunities that present a set of virtual items (like gems, for example) with a discount when compared to regular purchases in the game store. Additionally, the player base is very diverse: most of our users never make a purchase in our apps, and there are casual as well as hardcore players. This diversity pushes us to personalize user experience.
The key goal is to define, for any given player at any given time, what is the best offer that we can show to maximize long-term profits. With that in mind, we are encouraged to frame this system as a very particular optimization problem: what is the best policy, the one that will help us make the best sequence of decisions, maximizing revenue in the long run?
In this talk, we'll explain how we used Reinforcement Learning (RL) algorithms and Ray to tackle this problem, from formulating the problem and setting up our clusters, to the RL agents' deployment in production. We'll provide an overview of the main issues we faced and how we managed to overcome them.
Speakers
Emiliano Castro
Principal Data Scientist, Wildlife Studios
Emiliano Castro is a Principal Data Scientist in Wildlife Studios, where he helps the teams experiment with new technical approaches to the most impactful business problems. Before Wildlife, he worked 16 years in the game industry in both development and publishing roles. He later spent five years developing predictive models in a Machine-Learning-as-a-Service startup. After that, Emiliano worked as a Principal Data Scientist at QuantumBlack/McKinsey, where he was also a member of the Global Data Science Leadership Committee and the Global R&D Council. He is specialized in dynamic optimization using machine learning methods and evolution strategies (such as swarm intelligence and neuro-evolution). Emiliano holds a Ph.D. from the University of São Paulo and a Post-Doctorate in Procedural Content Generation using Reinforcement Learning.
Vinicius Alves
Data Scientist, Wildlife Studios
Vinícius Alves is a data scientist in Wildlife Studios currently working with In-App Purchases (IAP) monetization within the company's Sales Squad. More specifically, Vini and the team focus on developing models that provide personalized offer recommendations to all of the major Wildlife's games, helping increase their players' lifetime value. His interests include but are not limited to Reinforcement Learning, Statistics, and plants.
Felipe Antunes
Staff Data Scientist, Wildlife Studios
Felipe currently is part of the LiveOps team at Wildlife Studios and former CDO of Lendico, a Brazilian fintech sold to Lone Star Funds. Before Lendico, he played a major role in creating the Center of Excellence in Data Science and Engineering of Itaú as the third data scientist hired. During his Ph.D., he developed a fraud detection algorithm to point irregular donations in the 2014 Brazilian electoral campaign. In parallel, he created a route optimization algorithm that was acquired by Bora, a technology startup developing a new transportation mode and competing with Uber in Brazil. Simultaneously, he built a trading algorithm during a national competition sponsored by the European Space Agency, which later was incorporated by VCore, a Bitcoin trading platform, sold after one year of operation, right after bitcoin become mainstream. Felipe holds an M.Sc. and a B.Sc. in Computational Physics from Universidade Federal do Rio Grande do Sul.