MaskCam

MaskCam is a prototype reference design for a Jetson Nano-based smart camera system that measures crowd face mask usage in real-time, with all AI computation performed at the edge. MaskCam detects and tracks people in its field of view and determines whether they are wearing a mask via an object detection, tracking, and voting algorithm. It uploads statistics (not videos) to the cloud, where a web GUI can be used to monitor face mask compliance in the field of view. It saves interesting video snippets to local disk (e.g., a sudden influx of lots of people not wearing masks) and can optionally stream video via RTSP.

MaskCam can be run on a Jetson Nano Developer Kit, or on a Jetson Nano module (SOM) with the ConnectTech Photon carrier board. It was designed to use the Raspberry Pi High Quality Camera but will also work with pretty much any USB webcam that is supported on Linux.

The on-device software stack is mostly written in Python and runs under JetPack 4.4.1 or 4.5. Edge AI processing is handled by NVIDIA’s DeepStream video analytics framework, YOLOv4-tiny, and Tryolabs' Norfair tracker. MaskCam reports statistics to and receives commands from the cloud using MQTT and a web-based GUI. The software is containerized and for evaluation can be easily installed on a Jetson Nano DevKit using docker with just a couple of commands. For production, MaskCam can run under balenaOS, which makes it easy to manage and deploy multiple devices.

We urge you to try it out! It’s easy to install on a Jetson Nano Developer Kit and requires only a web cam. (The cloud-based statistics server and web GUI are optional, but are also dockerized and easy to install on any reasonable Linux system.) See below for installation instructions.

MaskCam was developed by Berkeley Design Technology, Inc. (BDTI) and Tryolabs S.A., with development funded by NVIDIA. MaskCam is offered under the MIT License. For more information about MaskCam, please see the report from BDTI. If you have questions, please email us at maskcam@bdti.com. Thanks!

Table of contents

Start Here!
MQTT Server Setup
Working With the MaskCam Container
- Development Mode: Manually Running MaskCam
- Debugging: Running MaskCam Modules as Standalone Processes
Additional Information
Troubleshooting Common Errors

Start Here!

Running MaskCam from a Container on a Jetson Nano Developer Kit

The easiest and fastest way to get MaskCam running on your Jetson Nano Dev Kit is using our pre-built containers. You will need:

A Jetson Nano Dev Kit running JetPack 4.4.1 or 4.5
An external DC 5 volt, 4 amp power supply connected through the Dev Kit's barrel jack connector (J25). (See these instructions on how to enable barrel jack power.) This software makes full use of the GPU, so it will not run with USB power.
A USB webcam attached to your Nano
Another computer with a program that can display RTSP streams -- we suggest VLC or QuickTime.

First, the MaskCam container needs to be downloaded from Docker Hub. On your Nano, run:

# This will take 10 minutes or more to download
sudo docker pull maskcam/maskcam-beta

Find your local Jetson Nano IP address using ifconfig. This address will be used later to view a live video stream from the camera and to interact with the Nano from a web server.

Make sure a USB camera is connected to the Nano, and then start MaskCam by running the following command. Make sure to substitute <your-jetson-ip> with your Nano's IP address.

# Connect USB camera before running this!
sudo docker run --runtime nvidia --privileged --rm -it --env MASKCAM_DEVICE_ADDRESS=<your-jetson-ip> -p 1883:1883 -p 8080:8080 -p 8554:8554 maskcam/maskcam-beta

The MaskCam container should start running the maskcam_run.py script, using the USB camera as the default input device (/dev/video0). It will produce various status output messages (and error messages, if it encounters problems). If there are errors, the process will automatically end after several seconds. Check the Troubleshooting section for tips on resolving errors.

Otherwise, after 30 seconds or so, it should continually generate status messages (such as Processed 100 frames...). Leave it running (don't press Ctrl+C, but be aware that the device will start heating up) and continue to the next section to visualize the video!

Viewing the Live Video Stream

If you scroll through the logs and don't see any errors, you should find a message like:

Streaming at rtsp://aaa.bbb.ccc.ddd:8554/maskcam

where aaa.bbb.ccc.ddd is the address that you provided in MASKCAM_DEVICE_ADDRESS previously. If you didn't provide an address, you'll see some unknown address label there, but the streaming will still work.

You can copy-paste that URL into your RSTP streaming viewer (see how to do it with VLC) on another computer. If all goes well, you should be rewarded with streaming video of your Nano, with green boxes around faces wearing masks and red boxes around faces not wearing masks. An example video of the live streaming in action is shown below.

This video stream gives a general demonstration of how MaskCam works. However, MaskCam also has other features, such as the ability to send mask detection statistics to the cloud and view them through a web browser. If you'd like to see these features in action, you'll need to set up an MQTT server, which is covered in the MQTT Server Setup section.

If you encounter any errors running the live stream, check the Troubleshooting section for tips on resolving errors.

Setting Device Configuration Parameters

MaskCam uses environment variables to configure parameters without having to rebuild the container or manually change the configuration file each time the program is run. For example, in the previous section we set the MASKCAM_DEVICE_ADDRESS variable to indicate our Nano's IP address. A list of configurable parameters is shown in maskcam_config.txt. The mapping between environment variable names and configuration parameters is defined in maskcam/config.py.

This section shows how to set environment variables to change configuration parameters. For example, if you want to use the /dev/video1 camera device rather than /dev/video0, you can define MASKCAM_INPUT when running the container:

# Run with MASKCAM_INPUT and MASKCAM_DEVICE_ADDRESS
sudo docker run --runtime nvidia --privileged --rm -it --env MASKCAM_INPUT=v4l2:///dev/video1 --env MASKCAM_DEVICE_ADDRESS=<your-jetson-ip> -p 1883:1883 -p 8080:8080 -p 8554:8554 maskcam/maskcam-beta

Another useful input device that you might want to use is a CSI camera (like the Raspberry Pi camera), and in that case you need to set MASKCAM_INPUT=argus://0 instead of the value shown above.

As another example, if you have an already set up our MQTT and web server (as shown in MQTT Server Setup section), you need to define two addtional environment variables, MQTT_BROKER_IP and MQTT_DEVICE_NAME. This allows your device to find the MQTT server and identify itself:

# Run with MQTT_BROKER_IP, MQTT_DEVICE_NAME, and MASKCAM_DEVICE_ADDRESS
sudo docker run --runtime nvidia --privileged --rm -it --env MQTT_BROKER_IP=<server IP> --env MQTT_DEVICE_NAME=<a-unique-string-you-like> --env MASKCAM_DEVICE_ADDRESS=<your-jetson-ip> -p 1883:1883 -p 8080:8080 -p 8554:8554 maskcam/maskcam-beta

If you have too many --env variables to add, it might be easier to create a .env file and point to it using the --env-file flag instead.

MQTT and Web Server Setup

Running the MQTT Broker and Web Server

MaskCam is intended to be set up with a web server that stores mask detection statistics and allows users to remotely interact with the device. We wrote code for instantiating a server that receiv

Maskcam

Install / Use

README