Mmeval
A unified evaluation library for multiple machine learning libraries
Install / Use
/learn @open-mmlab/MmevalREADME
English | 简体中文
</div>Introduction
MMEval is a machine learning evaluation library that supports efficient and accurate distributed evaluation on a variety of machine learning frameworks.
Major features:
- Comprehensive metrics for various computer vision tasks (NLP will be covered soon!)
- Efficient and accurate distributed evaluation, backed by multiple distributed communication backends
- Support multiple machine learning frameworks via dynamic input dispatching mechanism
| MPI4Py | torch.distributed | Horovod | paddle.distributed | oneflow.comm | | :------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------: | | MPI4PyDist | TorchCPUDist <br> TorchCUDADist | TFHorovodDist | PaddleDist | OneFlowDist |
</details> <details> <summary> Supported metrics and ML frameworks </summary>NOTE: MMEval tested with PyTorch 1.6+, TensorFlow 2.4+, Paddle 2.2+ and OneFlow 0.8+.
| Metric | numpy.ndarray | torch.Tensor | tensorflow.Tensor | paddle.Tensor | oneflow.Tensor | | :--------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------: | :----------: | :---------------: | :-----------: | :------------: | | Accuracy | ✔ | ✔ | ✔ | ✔ | ✔ | | SingleLabelMetric | ✔ | ✔ | | | ✔ | | MultiLabelMetric | ✔ | ✔ | | | ✔ | | AveragePrecision | ✔ | ✔ | | | ✔ | | MeanIoU | ✔ | ✔ | ✔ | ✔ | ✔ | | VOCMeanAP | ✔ | | | | | | OIDMeanAP | ✔ | | | | | | COCODetection | ✔ | | | | | | ProposalRecall | ✔ | | | | | | F1Score | ✔ | ✔ | | | ✔ | | HmeanIoU | ✔ | | | | | | PCKAccuracy | ✔ | | | | | | MpiiPCKAccuracy | ✔ | | | | | | JhmdbPCKAccuracy | ✔ | | | | | | EndPointError | ✔ | ✔ | | | ✔ | | AVAMeanAP | ✔ | | | | | | StructuralSimilarity | ✔ | | | | | | SignalNoiseRatio | ✔ | | | | | | PeakSignalNoiseRatio | ✔ | | | | |
Related Skills
claude-opus-4-5-migration
83.9kMigrate prompts and code from Claude Sonnet 4.0, Sonnet 4.5, or Opus 4.1 to Opus 4.5
model-usage
339.5kUse CodexBar CLI local cost usage to summarize per-model usage for Codex or Claude, including the current (most recent) model or a full model breakdown. Trigger when asked for model-level usage/cost data from codexbar, or when you need a scriptable per-model summary from codexbar cost JSON.
mcp-for-beginners
15.7kThis open-source curriculum introduces the fundamentals of Model Context Protocol (MCP) through real-world, cross-language examples in .NET, Java, TypeScript, JavaScript, Rust and Python. Designed for developers, it focuses on practical techniques for building modular, scalable, and secure AI workflows from session setup to service orchestration.
TrendRadar
49.9k⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载，你的 AI 舆情监控助手与热点筛选工具！聚合多平台热点 + RSS 订阅，支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机，也支持接入 MCP 架构，赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ，数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。
