SpeciesNet

An ensemble of AI models for classifying wildlife in camera trap images.

Table of Contents

• Overview
• Running SpeciesNet
  • Setting up your Python environment
  • Running the models
  • Using GPUs
  • Running each component separately
• Downloading SpeciesNet model weights directly
• Contacting us
• Citing SpeciesNet
• Supported models
• Input format
• Output format
• Visualizing SpeciesNet output
• Ensemble decision-making
• Alternative installation variants
• Contributing code
• Animal picture
• Build status

Overview

Effective wildlife monitoring relies heavily on motion-triggered wildlife cameras, or “camera traps”, which generate vast quantities of image data. Manual processing of these images is a significant bottleneck. AI can accelerate that processing, helping conservation practitioners spend more time on conservation, and less time reviewing images.

This repository hosts code for running an ensemble of two AI models: (1) an object detector that finds objects of interest in wildlife camera images, and (2) an image classifier that classifies those objects to the species level. This ensemble is used for species recognition in the Wildlife Insights platform.

The object detector used in this ensemble is MegaDetector, which finds animals, humans, and vehicles in camera trap images, but does not classify animals to species level.

The species classifier (SpeciesNet) was trained at Google using a large dataset of camera trap images and an EfficientNet V2 M architecture. It is designed to classify images into one of more than 2000 labels, covering diverse animal species, higher-level taxa (like "mammalia" or "felidae"), and non-animal classes ("blank", "vehicle"). SpeciesNet has been trained on a geographically diverse dataset of over 65M images, including curated images from the Wildlife Insights user community, as well as images from publicly-available repositories.

The SpeciesNet ensemble combines these two models using a set of heuristics and, optionally, geographic information to assign each image to a single category. See the "ensemble decision-making" section for more information about how the ensemble combines information for each image to make a single prediction.

The full details of the models and the ensemble process are discussed in this research paper:

Gadot T, Istrate Ș, Kim H, Morris D, Beery S, Birch T, Ahumada J. To crop or not to crop: Comparing whole-image and cropped classification on a large dataset of camera trap images. IET Computer Vision. 2024 Dec;18(8):1193-208.

Running SpeciesNet

Setting up your Python environment

The instructions on this page will assume that you have a Python virtual environment set up. If you have not installed Python, or you are not familiar with Python virtual environments, start with our installing Python page. If you see a prompt that looks something like the following, you're all set to proceed to the next step:

Installing the SpeciesNet Python package

You can install the SpeciesNet Python package via:

pip install speciesnet

If you are on a Mac, and you receive an error during this step, add the "--use-pep517" option, like this:

pip install speciesnet --use-pep517

To confirm that the package has been installed, you can run:

python -m speciesnet.scripts.run_model --help

You should see help text related to the main script you'll use to run SpeciesNet.

Running the models

The easiest way to run the ensemble is via the "run_model" script, like this:

python -m speciesnet.scripts.run_model --folders "c:\your\image\folder" --predictions_json "c:\your\output\file.json"

Change c:\your\image\folder to the root folder where your images live, and change c:\your\output\file.json to the location where you want to put the output file containing the SpeciesNet results.

This will automatically download and run the detector and the classifier. This command periodically logs output to the output file, and if this command doesn't finish (e.g. you have to cancel or reboot), you can just run the same command, and it will pick up where it left off.

These commands produce an output file in .json format; for details about this format, and information about converting it to other formats, see the "output format" section below.

You can also run the three steps (detector, classifier, ensemble) separately; see the "running each component separately" section for more information.

In the above example, we didn't tell the ensemble what part of the world your images came from, so it may, for example, predict a kangaroo for an image from England. If you want to let our ensemble filter predictions geographically, add, for example:

--country GBR

You can use any ISO 3166-1 alpha-3 three-letter country code.

If your images are from the USA, you can also specify a state name using the two-letter state abbreviation, by adding, for example:

--admin1_region CA

Using GPUs

If you don't have an NVIDIA GPU, you can ignore this section.

If you have an NVIDIA GPU, SpeciesNet should use it. If SpeciesNet is using your GPU, when you start run_model, in the output, you will see something like this:

<pre>Loaded SpeciesNetClassifier in 0.96 seconds on <b>CUDA</b>. Loaded SpeciesNetDetector in 0.7 seconds on <b>CUDA</b></pre>

"CUDA" is good news, that means "GPU".

If SpeciesNet is <i>not</i> using your GPU, you will see something like this instead:

<pre>Loaded SpeciesNetClassifier in 9.45 seconds on <b>CPU</b> Loaded SpeciesNetDetector in 0.57 seconds on <b>CPU</b></pre>

You can also directly check whether SpeciesNet can see your GPU by running:

python -m speciesnet.scripts.gpu_test

99% of the time, after you install SpeciesNet on Linux, it will correctly see your GPU right away. On Windows, you will likely need to take at least one more step:

1. Install the GPU version of PyTorch, by activating your speciesnet Python environment (e.g. by running "conda activate speciesnet"), then running:

   pip install torch torchvision --upgrade --force-reinstall --index-url https://download.pytorch.org/whl/cu118

2. If the GPU doesn't work immediately after that step, update your GPU driver, then reboot. Really, don't skip the reboot part, most problems related to GPU access can be fixed by upgrading your driver and rebooting.

Running each component separately

Rather than running everything at once, you may want to run the detection, classification, and ensemble steps separately. You can do that like this:

• Run the detector:

  python -m speciesnet.scripts.run_model --detector_only --folders "c:\your\image\folder" --predictions_json "c:\your_detector_output_file.json"

• Run the classifier, passing the file that you just created, which contains detection results:

  python -m speciesnet.scripts.run_model --classifier_only --folders "c:\your\image\folder" --predictions_json "c:\your_clasifier_output_file.json" --detections_json "c:\your_detector_output_file.json"

• Run the ensemble step, passing both the files that you just created, which contain the detection and classification results:

  python -m speciesnet.scripts.run_model --ensemble_only --folders "c:\your\image\folder" --predictions_json "c:\your_ensemble_output_file.json" --detections_json "c:\your_detector_output_file.json" --classifications_json "c:\your_clasifier_output_file.json" --country CAN

Note that in this example, we have specified the country code only for the ensemble step; the geofencing is part of the ensemble component, so the country code is only relevant for this step.

Downloading SpeciesNet model weights directly

The run_model.py script recommended above will download model weights automatically. If you want to use the SpeciesNet model weights outside of our script, or if you plan to be offline when you first run the script, you can download model weights directly from Kaggle. Running our ensemble also requires MegaDetector, so in this list of links, we also include a direct link to the MegaDetector model weights.

• SpeciesNet page on Kaggle
• Direct link to version 4.0.2a weights (the crop classifier)
• Direct link to version 4.0.2b weights (the whole-image classifier)
• Direct link to MegaDetector weights

Contacting us

If you have issues or questions, either file an issue or email us at cameratraps@google.com.

Citing SpeciesNet

If you use this model, please cite:

@article{gadot2024crop,
  title={To crop or not to crop: Comparing whole-image and cro