HomeResourcesOptimized AI Workloads on Anyscale with RayTurbo

Optimized AI Workloads on Anyscale with RayTurbo

Ray manages, executes, and optimizes compute needs across AI workloads. It unifies infrastructure by leveraging any compute instance and accelerator on AWS via a single, flexible framework—enabling any AI workload from data processing to model training to model serving and beyond.

Started as a research project in 2016, Ray today is used by leading AI organizations like Uber, Spotify, Netflix, Pinterest, OpenAI, and many more. Across the ecosystem, it is powering 1 million clusters each month for AI and GenAI workloads (and growing).

At Ray Summit 2024, Anyscale released an optimized version of Ray, called RayTurbo, offering improved performance, efficiency, scale, and reliability.

Join this session to learn more details about RayTurbo including how it delivers:

  • Up to 4.5X faster data processing

  • Up to 6X lower costs for LLM Batch Inference compared to Bedrock

  • Up to 60% lower costs on many workloads by supporting spot instances with elastic training

  • Up to 50% fewer nodes for online model serving with Replica Compaction for better efficiency.

  • Up to 60% higher QPS serving with optimized version of Ray Serve

Speakers

  • Richard Liaw, Product Manager Anyscale

  • Edward Oakes, Staff Engineer Anyscale

Ready to try Anyscale?

Access Anyscale today to see how companies using Anyscale and Ray benefit from rapid time-to-market and faster iterations across the entire AI lifecycle.