Pycm
Multi-class confusion matrix library in Python
Install / Use
/learn @sepandhaghighi/PycmREADME
<div align="center">
<img src="https://github.com/sepandhaghighi/pycm/raw/master/Otherfiles/logo.png" width="550">
<h1>PyCM: Python Confusion Matrix</h1>
<br/>
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/built%20with-Python3-green.svg" alt="built with Python3"></a>
<a href="https://github.com/sepandhaghighi/pycm"><img alt="GitHub repo size" src="https://img.shields.io/github/repo-size/sepandhaghighi/pycm"></a>
<a href="/Document"><img src="https://img.shields.io/badge/doc-latest-orange.svg"></a>
<a href="https://codecov.io/gh/sepandhaghighi/pycm"><img src="https://codecov.io/gh/sepandhaghighi/pycm/branch/master/graph/badge.svg"></a>
<a href="https://badge.fury.io/py/pycm"><img src="https://badge.fury.io/py/pycm.svg" alt="PyPI version"></a>
<a href="https://anaconda.org/sepandhaghighi/pycm"><img src="https://anaconda.org/sepandhaghighi/pycm/badges/version.svg"></a>
<a href="https://colab.research.google.com/github/sepandhaghighi/pycm/blob/master"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Document"></a>
<a href="https://discord.com/invite/zqpU2b3J3f"><img src="https://img.shields.io/discord/901883546162065408.svg" alt="Discord Channel"></a>
</div>
Overview
<p align="justify"> PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters. PyCM is the swiss-army knife of confusion matrices, targeted mainly at data scientists that need a broad array of metrics for predictive models and accurate evaluation of a large variety of classifiers. </p> <div align="center"> <img src="https://github.com/sepandhaghighi/pycm/raw/master/Otherfiles/block_diagram.jpg" width="700"> <p>Fig1. ConfusionMatrix Block Diagram</p> </div> <table> <tr> <td align="center">Open Hub</td> <td align="center"><a href="https://www.openhub.net/p/pycm"><img src="https://www.openhub.net/p/pycm/widgets/project_thin_badge.gif"></a></td> </tr> <tr> <td align="center">PyPI Counter</td> <td align="center"><a href="https://pepy.tech/projects/pycm"><img src="https://static.pepy.tech/badge/pycm" alt="PyPI Downloads"></a></td> </tr> <tr> <td align="center">Github Stars</td> <td align="center"><a href="https://github.com/sepandhaghighi/pycm"><img src="https://img.shields.io/github/stars/sepandhaghighi/pycm.svg?style=social&label=Stars"></a></td> </tr> </table> <table> <tr> <td align="center">Branch</td> <td align="center">master</td> <td align="center">dev</td> </tr> <tr> <td align="center">CI</td> <td align="center"><img src="https://github.com/sepandhaghighi/pycm/actions/workflows/test.yml/badge.svg?branch=master"></td> <td align="center"><img src="https://github.com/sepandhaghighi/pycm/actions/workflows/test.yml/badge.svg?branch=dev"></td> </tr> </table> <table> <tr> <td align="center">Code Quality</td> <td align="center"><a class="badge-align" href="https://www.codacy.com/app/sepand-haghighi/pycm?utm_source=github.com&utm_medium=referral&utm_content=sepandhaghighi/pycm&utm_campaign=Badge_Grade"><img src="https://api.codacy.com/project/badge/Grade/5d9463998a0040d09afc2b80c389365c"/></a></td> <td align="center"><a href="https://www.codefactor.io/repository/github/sepandhaghighi/pycm/overview/dev"><img src="https://www.codefactor.io/repository/github/sepandhaghighi/pycm/badge/dev" alt="CodeFactor" /></a></td> </tr> </table>Installation
⚠️ PyCM 4.3 is the last version to support Python 3.6
⚠️ PyCM 3.9 is the last version to support Python 3.5
⚠️ PyCM 2.4 is the last version to support Python 2.7 & Python 3.4
⚠️ Plotting capability requires Matplotlib (>= 3.0.0) or Seaborn (>= 0.9.1)
PyPI
- Check Python Packaging User Guide
- Run
pip install pycm==4.6
Source code
- Download Version 4.6 or Latest Source
- Run
pip install .
Conda
- Check Conda Managing Package
- Update Conda using
conda update conda - Run
conda install -c sepandhaghighi pycm
MATLAB
- Download and install MATLAB (>=8.5, 64/32 bit)
- Download and install Python3.x (>=3.7, 64/32 bit)
- [x] Select
Add to PATHoption - [x] Select
Install pipoption
- [x] Select
- Run
pip install pycm - Configure Python interpreter
>> pyversion PYTHON_EXECUTABLE_FULL_PATH
- Visit MATLAB Examples
Usage
From vector
>>> from pycm import *
>>> y_actu = [2, 0, 2, 2, 0, 1, 1, 2, 2, 0, 1, 2]
>>> y_pred = [0, 0, 2, 1, 0, 2, 1, 0, 2, 0, 2, 2]
>>> cm = ConfusionMatrix(actual_vector=y_actu, predict_vector=y_pred)
>>> cm.classes
[0, 1, 2]
>>> cm.table
{0: {0: 3, 1: 0, 2: 0}, 1: {0: 0, 1: 1, 2: 2}, 2: {0: 2, 1: 1, 2: 3}}
>>> cm.print_matrix()
Predict 0 1 2
Actual
0 3 0 0
1 0 1 2
2 2 1 3
>>> cm.print_normalized_matrix()
Predict 0 1 2
Actual
0 1.0 0.0 0.0
1 0.0 0.33333 0.66667
2 0.33333 0.16667 0.5
>>> cm.stat(summary=True)
Overall Statistics :
ACC Macro 0.72222
F1 Macro 0.56515
FPR Macro 0.22222
Kappa 0.35484
Overall ACC 0.58333
PPV Macro 0.56667
SOA1(Landis & Koch) Fair
TPR Macro 0.61111
Zero-one Loss 5
Class Statistics :
Classes 0 1 2
ACC(Accuracy) 0.83333 0.75 0.58333
AUC(Area under the ROC curve) 0.88889 0.61111 0.58333
AUCI(AUC value interpretation) Very Good Fair Poor
F1(F1 score - harmonic mean of precision and sensitivity) 0.75 0.4 0.54545
FN(False negative/miss/type 2 error) 0 2 3
FP(False positive/type 1 error/false alarm) 2 1 2
FPR(Fall-out or false positive rate) 0.22222 0.11111 0.33333
N(Condition negative) 9 9 6
P(Condition positive or support) 3 3 6
POP(Population) 12 12 12
PPV(Precision or positive predictive value) 0.6 0.5 0.6
TN(True negative/correct rejection) 7 8 4
TON(Test outcome negative) 7 10 7
TOP(Test outcome positive) 5 2 5
TP(True positive/hit) 3 1 3
TPR(Sensitivity, recall, hit rate, or true positive rate) 1.0 0.33333 0.5
Direct CM
>>> from pycm import *
>>> cm2 = ConfusionMatrix(matrix={"Class1": {"Class1": 1, "Class2": 2}, "Class2": {"Class1": 0, "Class2": 5}})
>>> cm2
pycm.ConfusionMatrix(classes: ['Class1', 'Class2'])
>>> cm2.classes
['Class1', 'Class2']
>>> cm2.print_matrix()
Predict Class1 Class2
Actual
Class1 1 2
Class2 0 5
>>> cm2.print_normalized_matrix()
Predict Class1 Class2
Actual
Class1 0.33333 0.66667
Class2 0.0 1.0
>>> cm2.stat(summary=True)
Overall Statistics :
ACC Macro 0.75
F1 Macro 0.66667
FPR Macro 0.33333
Kappa 0.38462
Overall ACC 0.75
PPV Macro 0.85714
SOA1(Landis & Koch) Fair
TPR Macro 0.66667
Zero-one Loss 2
Class Statistics :
Classes Class1 Class2
ACC(Accuracy) 0.75 0.75
AUC(Area under the ROC curve) 0.66667 0.66667
AUCI(AUC value interpretation) Fair Fair
F1(F1 score - harmonic mean of precision and sensitivity) 0.5 0.83333
FN(False negative/miss/type 2 error) 2 0
FP(False p
