scrapling-skill
Install, troubleshoot, and use Scrapling CLI to extract HTML, Markdown, or text from webpages. Use this skill whenever the user mentions Scrapling, `uv tool install scrapling`, `scrapling extract`, WeChat/mp.weixin articles, browser-backed page fetching, or needs help deciding between static and dyn…
Install / Use
npx skills add daymade/claude-code-skills --skill scrapling-skillInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
Content & MediaSupported Platforms
Tags
Our assessment of scrapling-skill
scrapling-skill scores 89/100 on our quality scale, 515th of 1,153 Content & Media skills we index (top 45%).
Its SKILL.md is 5.5 KB long, well organised into 24 sections with 14 code examples: a solid amount of guidance for an agent.
With 1,426 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so scrapling-skill is actively maintained.
- Our last check on 2026-10-31 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
WarningOur scan of the whole file found 1 high-risk pattern. Read the lines below before installing scrapling-skill, and do not run it with automatic approvals. An AI review of the same text found nothing harmful.
- highTells the agent to hide actions from the userline 174
- Do not tell the user to reinstall blindly. Verify first.
AI review by kimi-k2.7-code on 2026-10-03. Automated pattern scan on 2026-10-03. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
scrapling-skill compared with similar skills
All 4 of these similar skills score higher than scrapling-skill; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| scrapling-skill (this skill)by daymade | 89 | 1.4k | 53d ago | SKILL.md |
| siyuanby siyuan-note | 100 | 47.3k | today | MCP Server |
| x402by browser-use | 100 | 121.0k | 3d ago | SKILL.md |
| algorithmic-artby anthropics | 100 | 177.9k | 55d ago | SKILL.md |
| pptxby anthropics | 100 | 177.9k | 55d ago | SKILL.md |
Frequently asked questions
- How do I install scrapling-skill?
- Run
npx skills add daymade/claude-code-skills --skill scrapling-skill. The install tabs above show the steps for each supported agent. - Which AI agents does scrapling-skill work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is scrapling-skill safe to use?
- Our scan of the whole file found 1 high-risk pattern. Read the lines below before installing scrapling-skill, and do not run it with automatic approvals. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is scrapling-skill still maintained?
- The repository was last updated about 2 months ago, so scrapling-skill is actively maintained.
Skill content
View source on GitHubname: scrapling-skill
description: Install, troubleshoot, and use Scrapling CLI to extract HTML, Markdown, or text from webpages. Use this skill whenever the user mentions Scrapling, uv tool install scrapling, scrapling extract, WeChat/mp.weixin articles, browser-backed page fetching, or needs help deciding between static and dynamic extraction.
Scrapling Skill
Overview
Use Scrapling through its CLI as the default path. Start with the smallest working command, validate the saved output, and only escalate to browser-backed fetching when the static fetch does not contain the real page content.
Do not assume the user's Scrapling install is healthy. Verify it first.
Default Workflow
Copy this checklist and keep it updated while working:
Scrapling Progress:
- [ ] Step 1: Diagnose the local Scrapling install
- [ ] Step 2: Fix CLI extras or browser runtime if needed
- [ ] Step 3: Choose static or dynamic fetch
- [ ] Step 4: Save output to a file
- [ ] Step 5: Validate file size and extracted content
- [ ] Step 6: Escalate only if the previous path failed
Step 1: Diagnose the Install
Run the bundled diagnostic script first:
python3 scripts/diagnose_scrapling.py
Use the result as the source of truth for the next step.
Step 2: Fix the Install
If the CLI was installed without extras
If scrapling --help fails with missing click or a message about installing Scrapling with extras, reinstall it with the CLI extra:
uv tool uninstall scrapling
uv tool install 'scrapling[shell]'
Do not default to scrapling[all] unless the user explicitly needs the broader feature set.
If browser-backed fetchers are needed
Install the Playwright runtime:
scrapling install
If the install looks slow or opaque, read references/troubleshooting.md before guessing. Do not claim success until either:
scrapling installreports that dependencies are already installed, or- the diagnostic script confirms both Chromium and Chrome Headless Shell are present.
Step 3: Choose the Fetcher
Use this decision rule:
- Start with
extract getfor normal pages, article pages, and most WeChat public articles. - Use
extract fetchwhen the static HTML does not contain the real content or the page depends on JavaScript rendering. - Use
extract stealthy-fetchonly afterfetchstill fails because of anti-bot or challenge behavior. Do not make it the default.
Step 4: Run the Smallest Useful Command
Always quote URLs in shell commands. This is mandatory in zsh when the URL contains ?, &, or other special characters.
Full page to HTML
scrapling extract get 'https://example.com' page.html
Main content to Markdown
scrapling extract get 'https://example.com' article.md -s 'main'
JS-rendered page with browser automation
scrapling extract fetch 'https://example.com' page.html --timeout 20000
WeChat public article body
Use #js_content first. This is the default selector for article body extraction on mp.weixin.qq.com pages.
scrapling extract get 'https://mp.weixin.qq.com/s/ARTICLE_ID?scene=1' article.md -s '#js_content'
Step 5: Validate the Output
After every extraction, verify the file instead of assuming success:
wc -c article.md
sed -n '1,40p' article.md
For HTML output, check that the expected title, container, or selector target is actually present:
rg -n '<title>|js_content|rich_media_title|main' page.html
If the file is tiny, empty, or missing the expected container, the extraction did not succeed. Go back to Step 3 and switch fetchers or selectors.
Step 6: Handle Known Failure Modes
Local TLS trust store problem
If extract get fails with curl: (60) SSL certificate problem, treat it as a local trust-store problem first, not a Scrapling content failure.
Retry the same command with:
--no-verify
Only do this after confirming the failure matches the local certificate verification error pattern. Do not silently disable verification by default.
WeChat article pages
For mp.weixin.qq.com:
- Try
extract getbeforeextract fetch - Use
-s '#js_content'for the article body - Validate the saved Markdown or HTML immediately
Browser-backed fetch failures
If extract fetch fails:
- Re-check the install with
python3 scripts/diagnose_scrapling.py - Confirm Chromium and Chrome Headless Shell are present
- Retry with a slightly longer timeout
- Escalate to
stealthy-fetchonly if the site behavior justifies it
Command Patterns
Diagnose and smoke test a URL
python3 scripts/diagnose_scrapling.py --url 'https://example.com'
Diagnose and smoke test a WeChat article body
python3 scripts/diagnose_scrapling.py \
--url 'https://mp.weixin.qq.com/s/ARTICLE_ID?scene=1' \
--selector '#js_content' \
--no-verify
Diagnose and smoke test a browser-backed fetch
python3 scripts/diagnose_scrapling.py \
--url 'https://example.com' \
--dynamic
Guardrails
- Do not tell the user to reinstall blindly. Verify first.
- Do not default to the Python library API when the user is clearly asking about the CLI.
- Do not jump to browser-backed fetching unless the static result is missing the real content.
- Do not claim success from exit code alone. Inspect the saved file.
- Do not hardcode user-specific absolute paths into outputs or docs.
Resources
- Installation and smoke test helper:
scripts/diagnose_scrapling.py - Verified failure modes and recovery paths:
references/troubleshooting.md
Related Skills
siyuan
47.3kAn open-source, privacy-first, self-hosted knowledge workspace where humans and AI agents work together 开源、隐私优先、自托管的知识工作空间,让人与智能体在此协作
x402
121.0kSet up Browser Use Cloud payments with x402 — pay per request from a crypto wallet (USDC on Base mainnet), no signup or API key.
algorithmic-art
177.9kCreating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems.
pptx
177.9kUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an em…
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
