News

User Story

Engineering

Culture

Eric Liang, Jules S. Damji, Zhe Zhang

We cover 3 methods of batch inference in Ray - from low-level using Ray Actors, to high-level using Ray Data streaming.

Figure 1. Three ways of scaling model batch inference in Ray.


batch_inference_fig_1

Scaling Model Batch Inference in Ray | Actors, ActorPool & Ray Data

Scaling Model Batch Inference in Ray: Using Actors, ActorPool, and Ray Data

This blog covers four methods of [batch inference](https://docs.ray.io/en/latest/ray-overview/use-cases.html#batch-inference) (distributed model evaluation) in Ray: from low-level using Ray Actors, to high-level using Ray AIR libraries. 

We'll see how low-level Ray APIs allow you to control exactly "How" Ray is executing computations, while Ray's libraries enable you to specify just the "What" --- for a more out-of-the-box scalable experience.

Four ways of model batch inference in Ray.

Model Batch Inference in Ray: Actors, ActorPool & Datasets

Model Batch Inference in Ray: Actors, ActorPool, and Datasets

Anyscale + Astronomer

Scalable Machine Learning with Anyscale and Astronomer

Unlocking the Power of Scalable Machine Learning with Anyscale and Astronomer

At Ray Summit 2024, we brought together AI experts and practitioners to dive into the future of distributed AI and scalable machine learning.

image3

Ray Summit 2024: Breaking Through the AI Complexity Wall

Explore the new and improved Anyscale Workspaces. Get advanced observability features so you spend less time on busywork and more time building AI apps.

Anyscale Share Image

Explore Anyscale’s New UX

Anyscale’s New User Experience: A Comprehensive Overview

With Anyscale on AWS Marketplace, you can build, scale, and optimize AI/ML and Generative AI—all within the trusted AWS cloud environment.

Group 491 (1)

Anyscale Available on AWS Marketplace

Anyscale Now Available on AWS Marketplace and Achieves Generative AI Competency

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

Featured Posts and News

Unlocking the Power of Scalable Machine Learning with Anyscale and Astronomer

Ray Summit 2024: Breaking Through the AI Complexity Wall

Anyscale’s New User Experience: A Comprehensive Overview

Scaling Model Batch Inference in Ray: Using Actors, ActorPool, and Ray Data

Model Batch Inference in Ray: Actors, ActorPool, and Datasets