LMeterX

<div align="center"> <img src="docs/images/logo.png" alt="LMeterX Logo" width="300"/> <p> <a href="https://github.com/MigoXLab/LMeterX/blob/main/LICENSE"><img src="https://img.shields.io/github/license/MigoXLab/LMeterX" alt="License"></a> <a href="https://github.com/MigoXLab/LMeterX/stargazers"><img src="https://img.shields.io/github/stars/MigoXLab/LMeterX" alt="GitHub stars"></a> <a href="https://github.com/MigoXLab/LMeterX/network/members"><img src="https://img.shields.io/github/forks/MigoXLab/LMeterX" alt="GitHub forks"></a> <a href="https://github.com/MigoXLab/LMeterX/issues"><img src="https://img.shields.io/github/issues/MigoXLab/LMeterX" alt="GitHub issues"></a> <a href="https://deepwiki.com/MigoXLab/LMeterX"><img src="https://deepwiki.com/badge.svg" alt="Ask DeepWiki"></a> </p> <p> <a href="README_CN.md">简体中文</a> | <strong>English</strong> </p> </div>

📋 Project Overview

LMeterX is a professional large language model performance testing platform that can be applied to model inference services based on large model inference frameworks (such as LiteLLM, vLLM, TensorRT-LLM, LMDeploy, and others), and also supports performance testing for cloud services like Azure OpenAI, AWS Bedrock, Google Vertex AI, and other major cloud providers. Through an intuitive Web interface, users can easily create and manage test tasks, monitor testing processes in real-time, and obtain detailed performance analysis reports, providing reliable data support for model deployment and performance optimization.

<div align="center"> <img src="docs/images/images.gif" alt="LMeterX Demo" width="800"/> </div>

✨ Core Features

• Broad Framework Compatibility: Supports mainstream inference frameworks (vLLM, LiteLLM, TRT-LLM) and cloud platforms, ensuring seamless environment migration.
• Full Modality & Scenarios: Supports GPT, Claude, Llama to document parsing models like MinerU and dots.ocr, covering text, multimodal, and streaming.
• Hybrid Protocol Testing: Supports standard Chat APIs and business HTTP interfaces <img src="docs/images/badge-new.svg" alt="NEW" height="16" />, enabling full-stack load testing from base models to upper-level services.
• Multi-mode & High-Concurrency Load: Supports fixed/stepped concurrency <img src="docs/images/badge-new.svg" alt="NEW" height="16" /> strategies, supports simulating ultra-high concurrency, and accurately locates performance inflection points and system capacity limits.
• Built-in Dual-Mode Datasets: Pre-configured with high-quality self-built datasets and ShareGPT standard sets, supporting one-click invocation to lower data preparation barriers.
• Automated Warm-up Mechanism  <img src="docs/images/badge-new.svg" alt="NEW" height="16" />: Supports automatic model service warm-up to eliminate cold-start effects, ensuring the accuracy of test data.
• Multi-dimensional Indicator Visualization: Integrates core indicators such as TTFT, RPS, TPS, and throughput distribution, supporting real-time tracking and visualization of performance data<img src="docs/images/badge-new.svg" alt="NEW" height="16" />.
• Engine Resource Monitoring  <img src="docs/images/badge-new.svg" alt="NEW" height="16" />: Supports real-time monitoring of the load testing machine's CPU, memory, and network bandwidth, accurately identifying local resource bottlenecks.
• AI-Driven Data Insights: AI-powered analysis reports  <img src="docs/images/badge-new.svg" alt="NEW" height="16" /> with multi-model comparison, intuitively identifying optimization directions.
• One-stop Web Console: Manage task scheduling, monitoring, and real-time logs through an intuitive interface, reducing operational complexity.
• Web parsing & Intelligent load testing <img src="docs/images/badge-new.svg" alt="NEW" height="16" />: Enter a web page URL to automatically crawl the page and discover core business APIs, complete connectivity pre-checks, and create load test tasks with zero configuration.
• AI Agent Integration <img src="docs/images/badge-new.svg" alt="NEW" height="16" />: Built-in MCP Server and OpenClaw Skills with native support for AI agents such as Claude Code and Cursor — automatically generate load test configurations and launch tasks via natural language instructions.
• Enterprise-Grade Security & Scaling: Supports distributed elastic deployment and LDAP/AD <img src="docs/images/badge-new.svg" alt="NEW" height="16" />integration for high availability and secure enterprise authentication.

Feature Comparison

| Dimension | LMeterX | EvalScope | llmperf | |----------------------|-------------------------------------------------------------------------|---------------------------------------------------------------------------|----------------------------------------------------------| | Usage | Web UI for full-lifecycle task creation, monitoring & stop (load-test) | CLI for ModelScope ecosystem (eval & load-test) | CLI, Ray-based (load-test) | | Concurrency & Stress | Multi-process / multi-task, fix/stepped load, enterprise-scale load testing | Command-line concurrency (--parallel, --rate) | Command-line concurrency | | Test Report | Multi-model / multi-version comparison, AI analysis, visual dashboard | Basic report + visual charts (requires gradio, plotly, etc.) | Simple report | | Model & Data Support | OpenAI-compatible, custom data & model interfaces | OpenAI-compatible by default; extending APIs needs custom code | OpenAI-compatible | Performance and Resource Monitoring | Supports real-time monitoring of performance metrics and load testing machine resource status. | - | - | | Deployment & Scaling | Docker / K8s ready, easy horizontal scaling | pip install or source code | Source code only |

🏗️ System Architecture

LMeterX adopts a microservices architecture design, consisting of four core components:

1. Backend Service: FastAPI-based REST API service responsible for task management and result storage
2. Load Testing Engine: Locust-based load testing engine that executes actual performance testing tasks
3. Frontend Interface: Modern Web interface based on React + TypeScript + Ant Design
4. MySQL Database: Stores test tasks, result data, and configuration information

<div align="center"> <img src="docs/images/tech-arch.png" alt="LMeterX tech arch" width="700"/> </div>

🚀 Quick Start

Environment Checklist

• Docker 20.10.0+ with the daemon running
• Docker Compose 2.0.0+ (docker compose plugin or standalone docker-compose)
• At least 4GB free memory and 5GB disk space

Need more deployment options? See the Complete Deployment Guide for Kubernetes, air-gapped installs, and advanced tuning.

One-Click Deployment (Recommended)

# Download and run the one-click deployment script
curl -fsSL https://raw.githubusercontent.com/MigoXLab/LMeterX/main/quick-start.sh | bash

After the script finishes:

• Check container health: docker compose ps
• Tail logs if needed: docker compose logs -f
• Scale services (if needed): docker compose up -d --scale backend=2 --scale engine=2
• Open the web UI at http://localhost:8080 (see Usage Guide)

Data & Volume Layout

• ./data → mounted to /app/data in the engine service (large datasets are not baked into the image)
• ./logs → shared log output for backend and engine
• ./upload_files → user-supplied payloads and exported reports

For custom data, please refer to the Dataset Usage Guide.

Usage Guide

LLM API Load Testing

1. Access Web Interface: Open http://localhost:8080
2. Create Test Task: Navigate to Test Tasks → Create Task, configure API request information, test data, and request/response field mappings.
   • 2.1 Basic Information: For OpenAI-like and Claude-like APIs, you only need to configure API path, model, and response mode. You can also supplement the complete payload in request parameters.
   • 2.2 Data & load: Select the dataset type, concurrency, load testing time, etc., as needed.
   • 2.3 Field Mapping: For custom APIs, you need to configure the prompt field path in payload, and response data paths for model output fields, usage fields, etc. This field mapping is crucial for updating request parameters with datasets and correctly parsing streaming/non-streaming responses.

   💡 Tip: For custom multimodal dataset load tests, follow the Dataset Guide for data preparation, mounting, and troubleshooting.

3. API Testing: In Test Tasks → Create Task, click the "Test" button in the Basic Information panel to quickly test API connectivity (use a lightweight prompt for faster feedback).
4. Real-time Monitoring: Navigate to Test Tasks → Logs/Monitoring Center to view full-chain test logs and troubleshoot exceptions
5. Result Analysis: Navigate to Test Tasks → Results to view detailed performance results and export reports
6. Result Comparison: Navigate to Pref Insight to select multiple models or versions for multi-dimensional performance comparison
7. AI Analysis: In Test Tasks → Results or Pref Insight, support intelligent performance evaluation for sin