Rodel.Downloader

<a name="readme-top"></a>

<div align="center"> <img height="160" src="./assets/logo.png"> <h1 align="center">Rodel Downloader</h1>

A dedicated downloader for downloading AI model files, which can stably and reliably download your favorite models from Hugging Face, Model Scope and Civitai. It supports both command line and UI, and is currently only available on Windows.

English · 简体中文

<!-- SHIELD GROUP -->

[![][github-release-shield]][github-release-link] [![][github-releasedate-shield]][github-releasedate-link] [![][github-contributors-shield]][github-contributors-link] [![][github-forks-shield]][github-forks-link] [![][github-stars-shield]][github-stars-link] [![][github-issues-shield]][github-issues-link] [![][github-license-shield]][github-license-link]

</div>

[!WARNING]

The application is developed based on .NET 8, it is not a cross-platform application, and is currently only available on Windows. For users below Windows 10 19041, please use the command line instead of the UI.

<details> <summary><kbd>Table of contents</kbd></summary>

TOC

• ✨ Feature Overview
  • 1 CLI & APP
  • 2 Built-in Aria2
• 🎛️ Environment Support
• 🛠️ CLI Instruction Manual
  • 1 Install
  • 2 Interactive Commands
    • Screenshots of the interactive interface
  • 3 Standard Commands
    • Example
  • 4 Configuration and Saving
    • Overview of Access Token Acquisition
  • 5 Resume from Breakpoint
• 🪄 Application Manual
  • 1 Download and Installation
  • 2 Configuration
  • 3 Download Model
• 🔗 Links

</details>

✨ Feature Overview

1 CLI & APP

The application, with aria2 at its core, offers two modes of use:

• Command line
• UI

This caters to the needs of different users.

Both the command line and UI have simple localization support (supporting en-US and zh-CN), and can switch automatically according to the current system language.

[!TIP]

The UI is based on the Windows App SDK, which requires your system version to be Windows 10 19041 and above. It is highly recommended to download and install from the Microsoft Store.

<p align="left"> <a title="Get it from Microsoft Store" href="https://www.microsoft.com/store/apps/9PJDBLQ239JB?launch=true&mode=full" target="_blank"> <picture> <source srcset="https://get.microsoft.com/images/en-US%20light.svg" media="(prefers-color-scheme: dark)" /> <source srcset="https://get.microsoft.com/images/en-US%20dark.svg" media="(prefers-color-scheme: light), (prefers-color-scheme: no-preference)" /> <img src="https://get.microsoft.com/images/en-US%20dark.svg" width=144 /> </picture> </a> </p>

2 Built-in Aria2

I have to say, the sole motivation for building this tool was that I couldn't find a convenient, reliable download tool to download entire repositories from hugging face. Either there's no progress indication, or the robustness is too poor.

My technical skills aren't great, but aria2 has left a good impression on me in the past, so I chose to build a simple download tool based on it.

The download has the following features:

1. Customizable download directory
2. Support for resuming downloads
3. Complete progress display
4. Ability to operate on individual items, pause/resume/cancel (App only)

Both CLI and APP come with a built-in 1.3.7 aria2c.exe, no additional download required, trying to be as plug and play as possible.

🎛️ Environment Support

||| |-|-| |Framework|.NET 8| |UI Framework|Windows App SDK 1.5| |System Requirements|CLI: Windows 7 and above, APP: Windows 10 19041 and above|

🛠️ CLI Instruction Manual

1 Install

[!WARNING]

CLI relies on .NET 8 framework, please make sure that .NET 8 Desktop Runtime or SDK is installed on your device. You can download .NET SDK or .NET Desktop Runtime at Download .NET 8.0.

1. Open PowerShell
2. Enter the command

   dotnet tool install --global RodelDownloader.CLI
   

3. After the installation is complete, enter the command rodel-downloader to enter the interactive command interface
4. You can also enter the command rodel-downloader --help to view detailed parameter definitions

2 Interactive Commands

The CLI defaults to an interactive mode to guide you through the download process.

After installation, you can simply enter rodel-downloader to access the interactive interface.

The basic download steps are as follows:

1. Choose the model hosting service. Currently, Hugging Face / HF-Mirror, Civitai, and Model Scope are supported.
2. Enter the model ID you want to download. This model ID is usually provided by the hosting service.
3. The CLI will search for the repository and download file list corresponding to the model ID. Once found, it will be listed and you can freely choose which files to download.
4. Start downloading. The CLI will provide progress tips. You can wait for the download to complete or press Ctrl + C at any time to interrupt the download.

Screenshots of the interactive interface

The following image uses the LLM-Research/Phi-3-mini-128k-instruct model from the Model Scope community as an example

3 Standard Commands

The CLI also accepts another mode of operation, which is the regular parameter call.

When you need to use parameter calls, you must first disable the interactive mode, which is -n or --no-interaction.

After that, you need to provide the specified parameters.

  --ignore-config         (Default: false) Ignore the local configuration and use the default process for interaction.

  --clean-up              (Default: false) Kill all aria2c processes before starting the download.

  --edit-config           (Default: false) Edit the configuration file.

  -n, --no-interaction    (Default: false) Disable interaction and use the command-line parameters.

  -m, --model-id          The model ID to download.

  -s, --service           (Default: hf) The service to download the model from. Support hf | civitai | ms

  --use-hf-mirror         (Default: false) Use the hf-mirror (https://hf-mirror.com/) for downloading. Only available
                          when service is hf.

  -t, --token             (Default: ) The token to use for authentication.

  -d, --save-dir          (Default: ) The directory to save the model to.

  -i, --include           (Default: ) The files to include in the download.

  -e, --exclude           (Default: ) The files to exclude in the download.

  --help                  Display this help screen.

  --version               Display version information.

Example

Download the microsoft/Phi-3-mini-4k-instruct model from Hugging Face:

rodel-downloader -n -m "microsoft/Phi-3-mini-4k-instruct" -s hf --token "hf-xxxxxxxx" -d "C:\Models"

4 Configuration and Saving

Repeated input can be quite troublesome. The CLI supports using a configuration file to fix optional parameters, simplifying input each time it is called.

Enter the following command in the command line:

rodel-downloader --edit-config

The application will call the default editor to open the configuration file config.json (if it does not exist, create a new one), the specific parameters are as follows:

{
  "hf_token": "",
  "hf_save_folder": "",
  "hf_backup_folders": {
    "folder1": "path1",
    "folder2": "path2"
  },
  "hf_uri_type": "{official} or {mirror}",

  "civitai_token": "",
  "civitai_save_folder": "",
  "civitai_backup_folders": {
    "folder1": "path1",
    "folder2": "path2"
  },

  "ms_token": "",
  "ms_save_folder": "",
  "ms_backup_folders": {
    "folder1": "path1",
    "folder2": "path2"
  }
}

1. *_token This is the access token for the corresponding service. Here, hf stands for Hugging Face, and ms stands for Model Scope.

2. *_save_folder and *_backup_folders These are a pair of mutually exclusive properties, and the application prefers to use *_save_folder.

   • If the model you download will only be saved in a specific folder, fill in *_save_folder, and the CLI will download the model of the corresponding service to this folder.
   • If you have multiple optional locations, such as downloading the models required for SD-WebUI (check point, lora...), you can fill in the corresponding paths into *_backup_folders, where key is the readable name of the folder path, and value is its absolute path. When running the CLI, you can choose from the defined folder list.

   [!WARNING]

   The "save folder" means the parent folder for storing the model. For Hugging Face and Model Scope, the application will create a subfolder with the same name as the model in this folder as the directory for storing model files.

   For example, if you specify C:\MyFolder as the save folder, then after you download the Llama 3 8B model, the actual model file folder path is C:\MyFolder\Meta-Llama-3-8B.

   But for Civitai, since it is usually a single file download, the application will not create a same-name subfolder, but directly download the model file to the specified storage directory.

Overview of Access Token A