Ray Summit
Using RLLib in an Enterprise Scale Reinforcement Learning Solution
Wednesday, June 23, 8:35PM UTCJeroen Bédorf (Senior System Architect) & Ishaan Sood (Neural Network and Software Engineer), minds.ai
DeepSim is an optimization platform that can use advanced Reinforcement Learning (RL) methods to develop neural network-based controller software. DeepSim supports various RL libraries, including RLLib. In this talk, we discuss how RLLib, as well as the Tune hyperparameter optimizer, are used to develop controller software. Next, to the default set of features that RLLib offers, DeepSim offers its users a set of custom loggers, actions distributions and network architectures for improved performance of the controllers. The training runs, required to train the neural network, are executed on a Kubernetes based Ray cluster and can be monitored via command line interface tools as well as via TensorBoard. Finally, we show how the trained neural network can be exported, for example via Keras, to be deployed on target hardware.
All the above is demonstrated using two concrete examples, in the first the fuel efficiency of a Hybrid Electric Vehicle is optimized and in the second we develop cruise control software using the Ansys VRXPERIENCE autonomous driving simulator.
Speakers
Jeroen Bédorf
Senior System Architect, minds.ai
Jeroen holds a PhD in Computational Astrophysics from Leiden University and a master’s degree in computer science from the University of Amsterdam. He has experience developing high performance, GPU powered, software and algorithms for a wide variety of HPC systems and architectures including large GPU systems. At minds.ai, Jeroen is responsible for the architecture and heads the development team of the minds.ai DeepSim platform. This platform uses scalable HPC enabled cloud computing resources to create novel reinforcement learning solutions for enterprise customers.
Ishaan Sood
Neural Network and Software Engineer, minds.ai
Ishaan is based out of Bangalore, India and is currently a neural network and software engineer at minds.ai. He holds a Master's degree in Computational and Data Science. He works in the area of reinforcement learning, applying it to industrial control problems. Prior to this, he has also used deep learning for object detection and NLP use cases. When not working, he enjoys playing football, watching documentaries and driving around.