Skip to content

Introduction

LangBatch is a unified API for accessing Batch APIs from providers like OpenAI, Anthropic, Azure OpenAI, GCP Vertex AI, AWS Bedrock, etc.

  • Utlize Batch APIs for:

    • Requests that don't require immediate responses.
    • Low cost (usually 50% discount)
    • Higher rate limits
    • Example use cases: Data processing pipelines, Evaluations, Classifying huge data, Creating embeddings for large text contents

  • Features:

    • OpenAI format for requests and responses
    • Utilities for handling the complete lifecycle of a batch job: Creating, Starting, Monitoring, Retrying and Processing
    • Pipeline to convert stream of incoming requests into batch jobs

  • 🚀 Get Started

    Learn the basics and become familiar with the LangBatch and how to utilize it.

    Get Started

  • 📚 Core Concepts

    The high-level explanations for building a better understand about the important topics such as how to utlize Batch classes for batch generations.

    Core Concepts

  • 🛠️ How-to Guides

    Practical guides to help you achieve a specific goals. Take a look at these guides to learn how to use LangBatch to solve real-world problems.

    How-to Guides

  • 📖 References

    Technical descriptions of how LangBatch classes and methods work.

    References