Ray RLlib

Ray Train

Ray Datasets

RayDP

Retail

Gaming

Government

Financial

Ray Serve

Anyscale

Ray Core

Healthcare

Technology

News

User Story

Engineering

Culture

Eric Liang

Eric Liang headshot

Stephanie Wang

Cheng Su

This blog post delves into how Ray Data streaming works and how to use it for your own ML pipelines distributed across both CPU and GPU devices.

main_streaming_image

How to Stream Distributed Execution Across CPUs & GPUs

Streaming distributed execution across CPUs and GPUs

Jules S. Damji

Ben Lorica

Ray_summit_2023_main_image

10 must-attend Ray Summit sessions: Generative AI, Reinforcement learning, scalable ML workloads, and more

10 must-attend Ray Summit sessions: Generative AI, scalable ML workloads, and more

Waleed Kadous

In part 3 of our blog series, we’ll show you how to build an LLM question and answering service in 20 minutes using LangChain & Ray.

qna-answering-questions-with

Building a Self-Hosted Question Answering Service in 20 Minutes

Building a Self Hosted Question Answering Service using LangChain + Ray in 20 minutes

Amog Kamsetty

We conduct a comparison of three different solutions for offline batch inference: AWS SageMaker Batch Transform, Apache Spark, and Ray Data.

Figure 1. Throughput difference among three different solutions for 10GB batch inference


Figure 1

Offline Batch Inference | Ray, Apache Spark & SageMaker

Offline Batch Inference: Comparing Ray, Apache Spark, and SageMaker

Philipp Moritz

In part 2 of this blog series, we show you how to turbocharge embeddings in LLM workflows using LangChain and Ray Data.

Embedding - Stages of our Data Pipeline

Turbocharge LangChain: Guide to 20x Faster Embedding

Turbocharge LangChain: guide to 20x faster embedding

Lance Walter

Lance Walter Headshot

Why I Joined Anyscale by Lance Walter

Why I Joined Anyscale: Powering an Open Source AI Revolution

Richard Liaw

Richard Liaw headshot

Jiajun Yao

main_image

Announcing Ray 2.4.0: Infrastructure for LLM training, tuning, inference, and serving

Sidney Rabsatt

Picture of Sidney Rabsatt, Anyscale Head of Product

Sidney Rabsatt Headshot

why i joined anyscale by sidney rabsatt

Why I Joined Anyscale: Solving Cutting-Edge Problems in a Time of Enormous Change

In part 1 of a new blog series, we show how to build a search engine in 100 lines using LLM embeddings and a vector database.

Build a document index easily with Ray and Langchain

doc-index-starter-langchain

Building an LLM Open-Source Search Engine in 100 Lines

Building an LLM open source search engine in 100 lines using LangChain and Ray

Toby Rahloff

Phi Nguyen

Alex Streed

image1

LiveEO accelerates and optimizes their geospatial workloads by up to 65% using Anyscale and Prefect Cloud. 


liveeo-usecases

LiveEO supercharges their ML infrastructure and accelerates their geospatial workloads practice

Jun Gong

Antoni Baum

In part 4 of our Generative AI series, we share how to build a system for fine-tuning & serving LLMs in 40 minutes or less.

fine-tune-stack

How to Fine-Tune a 6 Billion Parameter LLM for Less Than $7

How to fine tune and serve LLMs simply, quickly and cost effectively using Ray + DeepSpeed + HuggingFace

Kai Fricke

In part 3 of our generative AI blog series, we dive into a concrete example of how you can use Ray to scale the training of generative AI models.

Figure 4. Training times for number of GPUs

Figure-4

Achieve Faster Stable Diffusion Model Fine-Tuning With Ray AIR

Faster stable diffusion fine-tuning with Ray AIR

Jiao Dong

Hao Zhang

Lianmin Zheng

In part 2 of our generative AI blog series, we cover how Ray empowers large language models (LLM) frameworks such as Alpa.

Figure 5: Technical integration layered stack for LLM


Figure-5

High-Performance LLM Training at 1000 GPU Scale With Alpa & Ray

Training 175B Parameter Language Models at 1000 GPU scale with Alpa and Ray

Learn the infrastructure challenges for supporting workloads production deployments around foundation models, and how Ray tackles these challenges.

layered-stack

How Ray Solves Generative AI & LLM Infrastructure Challenges

How Ray solves common production challenges for generative AI infrastructure

Anyscale Ray Team

How companies use Ray to scale many Python jobs at massive scale. Scaling many parallel Python jobs can come in many different forms. You can use Ray to train one million jobs in record time, scale many simulations or train the number of experiments that are part of a hyperparameter job.


summit-stories-scaling-parallel-python-jobs

How companies are scaling parallel python jobs for many models training, simulation or hyperparameter tuning.

Ray Summit Series - Scaling Parallel Python Jobs

Anyscale + Astronomer

Scalable Machine Learning with Anyscale and Astronomer

Unlocking the Power of Scalable Machine Learning with Anyscale and Astronomer

At Ray Summit 2024, we brought together AI experts and practitioners to dive into the future of distributed AI and scalable machine learning.

image3

Ray Summit 2024: Breaking Through the AI Complexity Wall

Explore the new and improved Anyscale Workspaces. Get advanced observability features so you spend less time on busywork and more time building AI apps.

Anyscale Share Image

Explore Anyscale’s New UX

Anyscale’s New User Experience: A Comprehensive Overview

With Anyscale on AWS Marketplace, you can build, scale, and optimize AI/ML and Generative AI—all within the trusted AWS cloud environment.

Group 491 (1)

Anyscale Available on AWS Marketplace

Anyscale Now Available on AWS Marketplace and Achieves Generative AI Competency

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

All Posts