SkillAgentSearch skills...

Ytdlrc

:cloud: Downloads videos and metadata with youtube-dl and moves each file on completion to an rclone remote

GenerateConvertImprove

Install / Use

/learn @bardisty/Ytdlrc
About this skill

Quality Score

0/100

Supported Platforms

Universal

README

ytdlrc

Codacy grade GitHub file size in bytes GitHub

ytdlrc is a simple shell script wrapper for youtube-dl and rclone. It downloads videos and metadata via youtube-dl and moves each file on completion to an rclone remote, e.g. Google Drive.

<details> <summary> <strong>Table of Contents</strong> </summary> <ul> <li><a href="#about">About</a> <li><a href="#example-output">Example Output</a> <ul> <li><a href="#example-initial-execution">Example: Initial execution <li><a href="#example-downloading-all-videos-from-ytuserkurzgesagt">Example: Downloading all videos from ytuser:kurzgesagt <ul> <li><a href="#first-run">First run</a> <li><a href="#second-run">Second run</a> </ul> </ul> <li><a href="#installation">Installation</a> <li><a href="#usage">Usage</a> <li><a href="#requirements">Requirements</a> <li><a href="#license">License</a> </ul> </details>

About

  • Designed to be executed via cron job or manually.

  • Ideal for use on VPS's with small disk space as it moves files on completion rather than copying them. In testing I downloaded / uploaded ~500GB of videos on a VPS with a 15GB SSD.

    • This can easily be changed to keep the files if you have the disk space; see the rclone_command variable to do so.

  • Includes metadata (xattrs, *.info.json, *.description, *.jpg).

  • Reads a file named snatch.list to know which URL's / channels / playlists you want to download and monitor for new videos.

    • Lines beginning with # are ignored.

  • Completed video ID's are saved in a file named archive.list; this prevents youtube-dl from re-downloading videos that have already been processed and moved to the rclone remote.

  • By default, it creates / uses the following structure:

    • ~/ytdlrc/stage - download directory
    • ~/ytdlrc/snatch.list - list of usernames or URL's to monitor / download
    • ~/ytdlrc/archive.list - list of completed video ID's

    These paths can be changed by modifying the ytdl_* variables.

  • In the download directory, subfolders are created for each line in the snatch.list. The name of the subfolders depends on whether the processed line is a playlist or a channel. This results in the following structure:

    • ~/ytdlrc/stage/Some_Channel_Name/{downloaded files}
    • ~/ytdlrc/stage/Another_Channel_Name/{downloaded files}
    • ~/ytdlrc/stage/Some_Playlist_Name/{downloaded files}
    • ~/ytdlrc/stage/Another_Playlist_Name/{downloaded files}

    Upon download completion a file is moved to the rclone remote, creating a structure such as:

    • remote:archive/youtube/Some_Channel_Name/{downloaded files}
    • remote:archive/youtube/Another_Channel_Name/{downloaded files}
    • remote:archive/youtube/Some_Playlist_Name/{downloaded files}
    • remote:archive/youtube/Another_Playlist_Name/{downloaded files}

    The path before the channel / playlist names (remote:archive/youtube) is set via the rclone_destination variable.

  • Downloaded files are saved with the following output template:

    "%(uploader)s.%(upload_date)s.%(title)s.%(resolution)s.%(id)s.%(ext)s"

    This results in filenames such as:

    • Channel_Name.20170307.Video_Title.1920x1080.J---aiyznGQ.mp4
    • Channel_Name.20170307.Video_Title.1920x1080.J---aiyznGQ.info.json
    • Channel_Name.20170307.Video_Title.1920x1080.J---aiyznGQ.description
    • Channel_Name.20170307.Video_Title.1920x1080.J---aiyznGQ.jpg

    See the ytdl_output_template variable if you wish to use a different output template.

Example Output

These examples are what you see with debug=true.

Example: Initial execution

[YTDLRC] Lock file doesn't exist. Attempting to create /tmp/ytdlrc.lock...
[YTDLRC] Creating '/tmp/ytdlrc.lock' succeeded. Continuing...
[YTDLRC] Creating download directory: /home/brian/ytdlrc/stage
[YTDLRC] Creating snatch list: /home/brian/ytdlrc/snatch.list
[YTDLRC] Creating archive list: /home/brian/ytdlrc/archive.list
[YTDLRC] Process complete. Removing lock file.

Example: Downloading all videos from ytuser:kurzgesagt

ytuser:kurzgesagt is the only line in our snatch.list.

First run

[YTDLRC] Lock file doesn't exist. Attempting to create /tmp/ytdlrc.lock...
[YTDLRC] Creating '/tmp/ytdlrc.lock' succeeded. Continuing...
[YTDLRC] Processing ytuser:kurzgesagt...
[YTDLRC] Grabbing 'playlist_title' from 'ytuser:kurzgesagt'...
[YTDLRC] 'playlist_title' is 'Uploads_from_Kurzgesagt_In_a_Nutshell'
[YTDLRC] Trimming off 'Uploads_from_' from 'Uploads_from_Kurzgesagt_In_a_Nutshell'...
[YTDLRC] New 'playlist_title' is 'Kurzgesagt_In_a_Nutshell'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-4', '--continue', '--download-archive', '/home/brian/ytdlrc/archive.list', '--exec', "/usr/bin/rclone move '{}' 'acd:testing/Kurzgesagt_In_a_Nutshell' --config '/home/brian/.rclone.conf' -v --stats 1s", '--ignore-errors', '--no-overwrites', '--restrict-filenames', '--write-description', '--write-info-json', '--write-thumbnail', '--xattrs', '-f', 'bestvideo+bestaudio', '-o', '/home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/%(uploader)s.%(upload_date)s.%(title)s.%(resolution)s.%(id)s.%(ext)s', '--verbose', 'ytuser:kurzgesagt']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.03.07
[debug] Python version 3.6.0 - Linux-4.9.11-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 3.2.4, ffprobe 3.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[youtube:user] kurzgesagt: Downloading channel page
[youtube:playlist] UUsXVk37bltHxD1rDPwtNM8Q: Downloading webpage
[download] Downloading playlist: Uploads from Kurzgesagt – In a Nutshell
[youtube:playlist] playlist Uploads from Kurzgesagt – In a Nutshell: Downloading 58 videos
[download] Downloading video 1 of 58
[youtube] DHyUYg8X31c: Downloading webpage
[youtube] DHyUYg8X31c: Downloading video info webpage
[youtube] DHyUYg8X31c: Extracting video information
[youtube] DHyUYg8X31c: Downloading MPD manifest
[info] Writing video description to: /home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.description
[info] Writing video description metadata as JSON to: /home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.info.json
[youtube] DHyUYg8X31c: Downloading thumbnail ...
[youtube] DHyUYg8X31c: Writing thumbnail to: /home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.jpg
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on 'https://r8---sn-n4v7kne7.googlevideo.com/videoplayback?id=0c7c94620f17df57&itag=299&source=youtube&requiressl=yes&mn=sn-n4v7kne7&mm=31&pl=20&initcwndbps=8067500&mv=m&ms=au&ratebypass=yes&mime=video/mp4&gir=yes&clen=97795996&lmt=1487912067914680&dur=394.266&signature=D4B482B33A060CAD522317EBCF5BA3288C4C95.276EACC1ABBDFFAE143AB016DB9079A47D8329EC&upn=Wj36IpbSSvI&mt=1489800442&key=dg_yt0&ip=104.131.132.15&ipbits=0&expire=1489822135&sparams=ip,ipbits,expire,id,itag,source,requiressl,mn,mm,pl,initcwndbps,mv,ms,ratebypass,mime,gir,clen,lmt,dur'
[download] Destination: /home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.f299.mp4
[download] 100% of 93.27MiB in 00:01
[debug] Invoking downloader on 'https://r8---sn-n4v7kne7.googlevideo.com/videoplayback?pl=20&ei=Vo3MWIusHurD-APq5JnoDg&clen=6926640&itag=251&gir=yes&upn=TwHyVFumtCo&signature=4ADE59C2CE26DABFB4D6418F61D65F10DF13E25B.1B06F7DE826DD862F39066A6D16B90B12D04EE5D&mime=audio%2Fwebm&initcwndbps=8067500&sparams=clen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cupn%2Cexpire&ipbits=0&requiressl=yes&keepalive=yes&mn=sn-n4v7kne7&mm=31&mt=1489800442&dur=394.281&id=o-AM978v3HI34Q_CjNzW-0QaneJHqs_vHmTJJzH32vY0-f&lmt=1487796412297530&key=yt6&ip=104.131.132.15&mv=m&source=youtube&ms=au&expire=1489822134&ratebypass=yes'
[download] Destination: /home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.f251.webm
[download] 100% of 6.61MiB in 00:00
[ffmpeg] Merging formats into "/home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.mkv"
[debug] ffmpeg command line: ffmpeg -y -i file:/home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.f299.mp4 -i file:/home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Conscious.1920x1080.DHyUYg8X31c.f251.webm -c copy -map 0:v:0 -map 1:a:0 file:/home/brian/ytdlrc/stage/Kurzgesagt_In_a_Nutshell/Kurzgesagt_In_a_Nutshell.20170223.Do_Robots_Deserve_Rights_What_if_Machines_Become_Con

bardisty

View profile

View on GitHub
GitHub Stars170
CategoryContent
Updated2mo ago
Forks28
bardisty/ytdlrc

Languages

Shell

Security Score

100/100

Audited on Jan 10, 2026

No findings