iiiLabCrawler

A web crawler for the video parsing API of snapany.com/iiilab.com

简体中文

Introduction

This repository is a Python web crawler for the video parsing API of iiilab.com, a website that supports parsing and downloading video links from multiple platforms.

Update: The video parsing API for some platforms is now redirected to snapany.com. The crawler scripts for the old version can be found in the old-version-202x.

SnapAny currently supports downloading videos and images from TikTok/Douyin, YouTube, Bilibili, X/Twitter, Pinterest, and other platforms. Some platforms still require the old API, and this repository temporarily integrates parsing support for Facebook, Instagram, and NetEase Cloud Music.

If you only need to parse and download a small number of media files, you can also use the original websites directly:

• snapany.com (TikTok/Douyin, YouTube, Bilibili, X/Twitter)
• facebook.iiilab.com (Facebook)
• instagram.iiilab.com (Instagram)
• yinyue.iiilab.com (NetEase Cloud Music)

Preparation

git clone https://github.com/1kuzus/iiiLabCrawler.git
pip install requests

Usage

from iiilab import get_resource

res = get_resource(url="https://www.youtube.com/watch?v=...")

Set url to the resource link to be parsed.

If the request fails, get_resource will return None and prints the error details. Otherwise, it will return the parsed JSON response data.

The script automatically determines whether to call the new or old API based on the domain of the provided URL. If you need to force a specific API, you may pass a second argument to specify the platform.

from iiilab import get_resource, MUSIC163

url = "https://music.163.com/#/song?id=518894020"
res = get_resource(url=url, site=MUSIC163)

Demo

See demo.py for an example:

from iiilab import get_resource
import json

urls = [
    # tiktok/douyin video
    "https://www.tiktok.com/@bbc/video/7142215700784893189",
    "https://www.douyin.com/user/MS4wLjABAAAA8U_l6rBzmy7bcy6xOJel4v0RzoR_wfAubGPeJimN__4?from_tab_name=main&modal_id=7506930749623373071",

    # youtube video
    "https://www.youtube.com/watch?v=RG9TMn1FJzc",

    # bilibili video
    "https://www.bilibili.com/video/BV1az4y1L7dL",

    # X/twitter video
    "https://x.com/TheCatsX/status/1924692026383335840",

    # facebook video
    "https://www.facebook.com/BBCArtsOnline/videos/florence-1501-michelangelo-and-leonardo-go-head-to-head-renaissance-the-blood-an/572612065554780/",

    # instagram video
    "https://www.instagram.com/reel/CzwrRR1SN-0",

    # 163 music
    "https://music.163.com/#/mv?id=5499051",
    "https://music.163.com/#/song?id=518894020",
]

for url in urls:
    res = get_resource(url)
    print(json.dumps(res, indent=4, ensure_ascii=False))

You will get the output:

{
    "text": "Incredible orca hunt🤯 #FrozenPlanet2 #BBCiPlayer #iPlayer ",
    "medias": [
        {
            "media_type": "video",
            "resource_url": "https://v16.tokcdn.com/a4b5559305c7e737b3958e6c2cc48d1a/631d2500/7142215700784893189_original.mp4",
            "preview_url": "https://p16-sign-va.tiktokcdn.com/tos-maliva-p-0068/ed6b06befc12479d8845686ff91d1809_1662926686~tplv-tiktokx-360p.webp?dr=1363&refresh_token=372b800c&x-expires=1768129200&x-signature=M7cuD4kYJv7tzW3juA81JP3e91E%3D&t=bacd0480&ps=d97f9a4f&shp=d05b14bd&shcp=1d1a97fc&idc=maliva&sc=feed_cover&biz_tag=tt_video&s=AWEME_DETAIL"
        },
        {
            "media_type": "audio",
            "resource_url": "https://v16-ies-music.tiktokcdn.com/3a296d023659e028d616967063ed52b4/696b6fdb/video/tos/useast2a/tos-useast2a-v-27dcd7/a8e6e765d48045aa84c13714adb81c50/?a=583965&bti=OUBzOTg7QGo6OjZAL3AjLTAzYCMxNDNg&ch=0&cr=0&dr=0&er=0&lr=default&cd=0%7C0%7C0%7C0&br=250&bt=125&ft=k_ghWID.ZNhWVQl2Gclfu1mNSpM5SbrwuX.7TGbL&mime_type=audio_mpeg&qs=6&rc=ODM6NjUzaDVmZjg0N2hpN0BpM2o3eGg6ZnE2ZjMzNzU8M0AxNWI0NGMyNmAxNWFiMS8xYSNrcmYvcjRfcmNgLS1kMTZzcw%3D%3D&vvpl=1&l=202601101911313F52CFE9CB8F10BB05A6&btag=e00098000&shp=d05b14bd&shcp=-",
            "preview_url": "https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/65c76bec2af5783edfd123d29c90da99~tplv-tiktokx-cropcenter-q:1080:1080:q70.webp?dr=10796&idc=maliva&ps=87d6e48a&refresh_token=e3ef3d3c&s=AWEME_DETAIL&sc=avatar&shcp=1d1a97fc&shp=d05b14bd&t=223449c4&x-expires=1768129200&x-signature=mCy92D%2FIPhRyhWtTpjoKN%2Fo2c%2BI%3D"
        }
    ],
    "overseas": 1,
    "id": "7142215700784893189"
}
{
    "text": "四年了，风吹过稻田，依然想念您。缅怀袁老！",
    "medias": [
        {
            "media_type": "video",
            "resource_url": "https://v11.douyinvod.com/abacb963a53d27fd094b795a9c9dc144/69623e00/video/tos/cn/tos-cn-ve-15/osAue2Egtt4faWdG0ZCM2AADMYC8ARx57ifjQf/?a=1128&ch=0&cr=0&dr=0&er=0&cd=0%7C0%7C0%7C0&cv=1&br=1250&bt=1250&cs=0&ds=4&ft=ahhJkQQqUnXfmoZmo0OW_QYaUqiXjy4EkVJEHNuMIbPD-Ipz&mime_type=video_mp4&qs=0&rc=aGk8aGY8ZWhpOmg0MzQ6aEBpamV3NnA5cmx5MzMzNGkzM0BfNTJeYzMuNjYxYS80YWIwYSMuazBnMmQ0amphLS1kLTBzcw%3D%3D&btag=c0000e000a5000&cc=33&cquery=100y&dy_q=1768042428&feature_id=46a7bb47b4fd1280f3d3825bf2b29388&l=202601101853483493F9DDA6B3FA973FC1&req_cdn_type=",
            "preview_url": "https://p3-pc-sign.douyinpic.com/image-cut-tos-priv/4e48fbbf9b2c40ced0a64c3d186d836e~tplv-dy-resize-origshort-autoq-75:330.jpeg?lk3s=138a59ce&x-expires=2083399200&x-signature=MWg1XP1E8m3ykf%2B%2BwHgaJL2oN10%3D&from=327834062&s=PackSourceEnum_AWEME_DETAIL&se=false&sc=cover&biz_tag=pcweb_cover&l=20260110185348A824162A2DBC4B6C6069",
            "headers": {
                "User-Agent": ""
            }
        }
    ],
    "overseas": 0
}
{
    "text": "The incredible ibex defies gravity and climbs a dam | Forces of Nature with Brian Cox - BBC",
    "medias": [
        {
            "media_type": "video",
            "resource_url": "https://redirector.googlevideo.com/videoplayback?expire=1768064029&ei=vS9iaaLbKKTKi9oPyIK48Q0&ip=176.6.151.223&id=o-AH-1ra1NuwXrE8j74ndWvtROvuZQpSVioKVvtrmy9G6L&itag=18&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&cps=2&met=1768042429%2C&mh=S-&mm=31%2C29&mn=sn-uxax4vopj5qx-cxgk%2Csn-4g5ednd7&ms=au%2Crdu&mv=m&mvi=11&pl=20&rms=au%2Cau&initcwndbps=1988750&bui=AYUSA3AkQPcnfxxeLIohyTRTJ1Sl63noNCWTJojvd9OUK5QeYH237Qb2ZrW1y6_US2e1zvV0RZKxjOYo&spc=wH4Qq0a8kQsZ-izNzoN7&vprv=1&svpuc=1&mime=video%2Fmp4&rqh=1&gir=yes&clen=16332781&ratebypass=yes&dur=232.408&lmt=1725424927317407&mt=1768041895&fvip=3&fexp=51552689%2C51565115%2C51565682%2C51580968&c=ANDROID&txp=4538434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Crqh%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=AJfQdSswRQIhAJh_IG0dKiTxLmH7ShT5ZKjU04KbT_nqQnSNNjKZytmCAiAGrFkceITNwulL8qE9DVgPsrQ92EfRR8OTEpAG1Y1PIA%3D%3D&lsparams=cps%2Cmet%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=APaTxxMwRgIhALOWqdh77NGEIWEEm5tI1jvk6JEUUG3KHNY6Et7jN77CAiEAukv_E-4KSO5aHwTYrmjd8Gsk0KCdoJ_5D_obdat5R6Q%3D",
            "preview_url": "https://i.ytimg.com/vi_webp/RG9TMn1FJzc/sddefault.webp",
            "formats": [
                {
                    "quality": 1080,
                    "quality_note": "1080p",
                    "video_url": "https://redirector.googlevideo.com/videoplayback?expire=1768064029&ei=vS9iaaLbKKTKi9oPyIK48Q0&ip=176.6.151.223&id=o-AH-1ra1NuwXrE8j74ndWvtROvuZQpSVioKVvtrmy9G6L&itag=137&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&cps=2&met=1768042429%2C&mh=S-&mm=31%2C29&mn=sn-uxax4vopj5qx-cxgk%2Csn-4g5ednd7&ms=au%2Crdu&mv=m&mvi=11&pl=20&rms=au%2Cau&initcwndbps=1988750&bui=AYUSA3DbztcfTVHlDa4Jc6m5vUBrykJZQ1kx4NSYttXWx1a8l6ns92wrkcNCPty1_46dwa9oq0ZlX9cB&spc=wH4Qqz69OQsR&vprv=1&svpuc=1&mime=video%2Fmp4&rqh=1&gir=yes&clen=56556197&dur=232.360&lmt=1725425356139113&mt=1768041895&fvip=3&keepalive=yes&fexp=51552689%2C51565115%2C51565682%2C51580968&c=ANDROID&txp=4532434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRAIgdEcEDIe8vamqIvCP4qlMgP85tWAvSzjK4Lu4PCHRM3ACIGKlbJg-SkOAO0j7vlHnXg3tuLRN5gOSwonqPvxCGr2J&lsparams=cps%2Cmet%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=APaTxxMwRgIhALOWqdh77NGEIWEEm5tI1jvk6JEUUG3KHNY6Et7jN77CAiEAukv_E-4KSO5aHwTYrmjd8Gsk0KCdoJ_5D_obdat5R6Q%3D",
                    "video_ext": "mp4",
                    "video_size": 56556197,
                    "audio_url": "https://redirector.googlevideo.com/videoplayback?expire=1768064029&ei=vS9iaaLbKKTKi9oPyIK48Q0&ip=176.6.151.223&id=o-AH-1ra1NuwXrE8j74ndWvtROvuZQpSVioKVvtrmy9G6L&itag=140&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&cps=2&met=1768042429%2C&mh=S-&mm=31%2C29&mn=sn-uxax4vopj5qx-cxgk%2Csn-4g5ednd7&ms=au%2Crdu&mv=m&mvi=11&pl=20&rms=au%2Cau&initcwndbps=1988750&bui=AYUSA3DbztcfTVHlDa4Jc6m5vUBrykJZQ1kx4NSYttXWx1a8l6ns92wrkcNCPty1_46dwa9oq0ZlX9cB&spc=wH4Qqz69OQsR&vprv=1&svpuc=1&mime=audio%2Fmp4&rqh=1&gir=yes&clen=3762111&dur=232.408&lmt=1725424428667227&mt=1768041895&fvip=3&keepalive=yes&fexp=51552689%2C51565115%2C51565682%2C51580968&c=ANDROID&txp=4532434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRgIhANNfHPVOxVJPLAcAwY_t4kUWRCaiJvEkZaYHWfd3M5O1AiEArNacL7nJuIL5Qg_-3gx2p567WYmrhHImijW83W3c4v4%3D&lsparams=cps%2Cmet%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=APaTxxMwRgIhALOWqdh77NGEIWEEm5tI1jvk6JEUUG3KHNY6Et7jN77CAiEAukv_E-4KSO5aHwTYrmjd8Gsk0KCdoJ_5D_obdat5R6Q%3D",
                    "audio_ext": "m4a",
                    "audio_size": 3762111,
                    "separate": 1,
                    "video_proxy_url": "https://stream.aipark.top/download?payload=Flt