Human Library

AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition,
Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis,
Age & Gender & Emotion Prediction, Gaze Tracking, Gesture Recognition, Body Segmentation

<br>

Highlights

• Compatible with most server-side and client-side environments and frameworks
• Combines multiple machine learning models which can be switched on-demand depending on the use-case
• Related models are executed in an attention pipeline to provide details when needed
• Optimized input pre-processing that can enhance image quality of any type of inputs
• Detection of frame changes to trigger only required models for improved performance
• Intelligent temporal interpolation to provide smooth results regardless of processing performance
• Simple unified API
• Built-in Image, Video and WebCam handling

Jump to Quick Start

<br>

Compatibility

Browser:

• Compatible with both desktop and mobile platforms
• Compatible with WebGPU, WebGL, WASM, CPU backends
• Compatible with WebWorker execution
• Compatible with WebView
• Primary platform: Chromium-based browsers
• Secondary platform: Firefox, Safari

NodeJS:

• Compatibile with WASM backend for executions on architectures where tensorflow binaries are not available
• Compatible with tfjs-node using software execution via tensorflow shared libraries
• Compatible with tfjs-node using GPU-accelerated execution via tensorflow shared libraries and nVidia CUDA
• Supported versions are from 14.x to 22.x
• NodeJS version 23.x is not supported due to breaking changes and issues with @tensorflow/tfjs

<br>

Releases

• Release Notes
• NPM Link

Demos

Check out Simple Live Demo fully annotated app as a good start starting point (html)(code)

Check out Main Live Demo app for advanced processing of of webcam, video stream or images static images with all possible tunable options

• To start video detection, simply press Play
• To process images, simply drag & drop in your Browser window
• Note: For optimal performance, select only models you'd like to use
• Note: If you have modern GPU, WebGL (default) backend is preferred, otherwise select WASM backend

<br>

• List of all Demo applications
• Live Examples galery

Browser Demos

All browser demos are self-contained without any external dependencies

• Full [Live] [Details]: Main browser demo app that showcases all Human capabilities
• Simple [Live] [Details]: Simple demo in WebCam processing demo in TypeScript
• Embedded [Live] [Details]: Even simpler demo with tiny code embedded in HTML file
• Face Detect [Live] [Details]: Extract faces from images and processes details
• Face Match [Live] [Details]: Extract faces from images, calculates face descriptors and similarities and matches them to known database
• Face ID [Live] [Details]: Runs multiple checks to validate webcam input before performing face match to faces in IndexDB
• Multi-thread [Live] [Details]: Runs each Human module in a separate web worker for highest possible performance
• NextJS [Live] [Details]: Use Human with TypeScript, NextJS and ReactJS
• ElectronJS [Details]: Use Human with TypeScript and ElectonJS to create standalone cross-platform apps
• 3D Analysis with BabylonJS [Live] [Details]: 3D tracking and visualization of head, face, eye, body and hand
• VRM Virtual Model Tracking with Three.JS [Live] [Details]: VR model with head, face, eye, body and hand tracking
• VRM Virtual Model Tracking with BabylonJS [Live] [Details]: VR model with head, face, eye, body and hand tracking

NodeJS Demos

NodeJS demos may require extra dependencies which are used to decode inputs
See header of each demo to see its dependencies as they are not automatically installed with Human

• Main [Details]: Process images from files, folders or URLs using native methods
• Canvas [Details]: Process image from file or URL and draw results to a new image file using node-canvas
• Video [Details]: Processing of video input using ffmpeg
• WebCam [Details]: Processing of webcam screenshots using fswebcam
• Events [Details]: Showcases usage of Human eventing to get notifications on processing
• Similarity [Details]: Compares two input images for similarity of detected faces
• Face Match [Details]: Parallel processing of face match in multiple child worker threads
• Multiple Workers [Details]: Runs multiple parallel human by dispaching them to pool of pre-created worker processes
• Dynamic Load [Details]: Loads Human dynamically with multiple different desired backends

Project pages

• Code Repository
• NPM Package
• Issues Tracker
• TypeDoc API Specification - Main class
• TypeDoc API Specification - Full
• Change Log
• Current To-do List

Wiki pages

• Home
• Installation
• Usage & Functions
• Configuration Details
• Result Details
• Customizing Draw Methods
• Caching & Smoothing
• Input Processing
• Face Recognition & Face Description
• Gesture Recognition
• Common Issues
• Background and Benchmarks

Additional notes

• Comparing Backends
• Development Server
• Build Process
• Adding Custom Modules
• [Performance Notes](https://github.com/vladmandic/h