News

User Story

Engineering

Culture

Justin Olsson

Waleed Kadous

Llama 2 70b matches legislative interns for summaries, but GPT-4 does better. Studying the GPT-4 summaries helped us improve the prompt. It worked.

Human rated performance of different bill summarization

LLM-based summarization: A case study of human, Llama 2 70b and GPT-4 summarization quality

Kyle Huang

Wendi Ding

Liguang Xie

Avnish Narayan

Ricky Xu

Anyscale is releasing LLMPerf for benchmarking LLMs on current LLM offerings. See benchmarking results for Anyscale Endpoints vs Fireworks.ai.

End-to-end time

Reproducible Performance Metrics for LLM inference

factuality 373 examples

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30 times cheaper

Using Anyscale Endpoints, we compared Llama 2 7b, 13b and 70b vs. OpenAI's GPT-3.5-turbo and GPT-4 for accuracy and cost.

Llama 2 vs. GPT-4: Nearly As Accurate and 30X Cheaper

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Summarizing is one of the most practical applications of LLM, but you need to know you can trust your summary to be factually accurate. In this experiment, we found Llama-2-70b is almost as strong at factuality as gpt-4, and considerably better than gpt-3.5-turbo.

Kourosh Hakhamaneshi

Fine tuning is one approach to domain-specific model refinement (DSMR), but it’s not a silver bullet for improving domain-specific performance.

fine-tune-img-1

Fine Tuning Is For Form, Not Facts | Anyscale

Fine tuning is for form, not facts

Aviary Explorer

Announcing Aviary: Open Source Multi-LLM Serving

For LLM developers, it's useful to know numbers for back-of-the-envelope calculations. Here are the numbers we use, plus how and why to use them.

numbers-cover-image

Numbers Every LLM Developer Should Know | Anyscale

Numbers every LLM Developer should know

In part 3 of our blog series, we’ll show you how to build an LLM question and answering service in 20 minutes using LangChain & Ray.

qna-answering-questions-with

Building a Self-Hosted Question Answering Service in 20 Minutes

Building a Self Hosted Question Answering Service using LangChain + Ray in 20 minutes

In part 1 of a new blog series, we show how to build a search engine in 100 lines using LLM embeddings and a vector database.

Build a document index easily with Ray and Langchain

doc-index-starter-langchain

Building an LLM Open-Source Search Engine in 100 Lines

Building an LLM open source search engine in 100 lines using LangChain and Ray

Jun Gong

Antoni Baum

Richard Liaw

Richard Liaw headshot

In part 4 of our Generative AI series, we share how to build a system for fine-tuning & serving LLMs in 40 minutes or less.

fine-tune-stack

How to Fine-Tune a 6 Billion Parameter LLM for Less Than $7

How to fine tune and serve LLMs simply, quickly and cost effectively using Ray + DeepSpeed + HuggingFace

Eric Liang

Eric Liang headshot

Chen Shen

Clark Zinzow

3rdGenTasks andActors

Data Ingest in a Third-Generation ML Architecture | Anyscale

Deep Dive: Data Ingest in a Third Generation ML Architecture

mlplatformCropped

Why 3rd Generation ML Platforms are More Performant

Why Third Generation ML Platforms are More Performant

Uber Leveraging Ray

The 3rd Generation of Production ML Architectures | Anyscale

The Third Generation of Production ML Architectures

WaleedKadous345

Why I’m joining Anyscale

Anyscale + Astronomer

Scalable Machine Learning with Anyscale and Astronomer

Unlocking the Power of Scalable Machine Learning with Anyscale and Astronomer

At Ray Summit 2024, we brought together AI experts and practitioners to dive into the future of distributed AI and scalable machine learning.

image3

Ray Summit 2024: Breaking Through the AI Complexity Wall

Explore the new and improved Anyscale Workspaces. Get advanced observability features so you spend less time on busywork and more time building AI apps.

Anyscale Share Image

Explore Anyscale’s New UX

Anyscale’s New User Experience: A Comprehensive Overview

With Anyscale on AWS Marketplace, you can build, scale, and optimize AI/ML and Generative AI—all within the trusted AWS cloud environment.

Group 491 (1)

Anyscale Available on AWS Marketplace

Anyscale Now Available on AWS Marketplace and Achieves Generative AI Competency

This section is used to order the "Types" and "Tags" that show up for filters on the Blog Index

Types

Products / Libraries

Blog

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

Anyscale

Featured Posts and News

Unlocking the Power of Scalable Machine Learning with Anyscale and Astronomer

Ray Summit 2024: Breaking Through the AI Complexity Wall

Anyscale’s New User Experience: A Comprehensive Overview

LLM-based summarization: A case study of human, Llama 2 70b and GPT-4 summarization quality

Reproducible Performance Metrics for LLM inference

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30 times cheaper

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Fine tuning is for form, not facts

Announcing Aviary: Open Source Multi-LLM Serving

Numbers every LLM Developer should know

Building a Self Hosted Question Answering Service using LangChain + Ray in 20 minutes

Building an LLM open source search engine in 100 lines using LangChain and Ray

How to fine tune and serve LLMs simply, quickly and cost effectively using Ray + DeepSpeed + HuggingFace

Deep Dive: Data Ingest in a Third Generation ML Architecture

Why Third Generation ML Platforms are More Performant

The Third Generation of Production ML Architectures

Why I’m joining Anyscale