Data Science at Scale: From Notebook to Production with Airflow and Ray

Since the release of Airflow 2.0 in December 2020, we at Astronomer have been working toward the next generation of ML tooling and orchestration. In this talk, we are excited to share multiple integrations with the Ray library that will allow both Airflow and Ray users access to the most powerful features of both of these products. This talk will appeal to Ray users who want to orchestrate and monitor their Ray jobs using Airflow and Airflow users who want to run distributed computation jobs using Ray.