web-fetch
Fetches web content as clean markdown by preferring markdown-native responses and falling back to selector-based HTML extraction. Use for documentation, articles, and reference pages at http/https URLs.
Install / Use
npx skills add aiskillstore/marketplace --skill web-fetchInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
Content & MediaSupported Platforms
Our assessment of web-fetch
web-fetch scores 87/100 on our quality scale, 668th of 1,148 Content & Media skills we index.
Its SKILL.md is 3.3 KB long, well organised into 11 sections with 10 code examples: a solid amount of guidance for an agent.
It has 428 GitHub stars, a meaningful sign that others use it.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so web-fetch is actively maintained.
- Our last check on 2026-11-02 found the source still online.
- No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
- Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
web-fetch compared with similar skills
All 4 of these similar skills score higher than web-fetch; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| web-fetch (this skill)by aiskillstore | 87 | 428 | 55d ago | SKILL.md |
| siyuanby siyuan-note | 100 | 47.3k | today | MCP Server |
| algorithmic-artby anthropics | 100 | 177.9k | 57d ago | SKILL.md |
| pptxby anthropics | 100 | 177.9k | 57d ago | SKILL.md |
| prospectingby coreyhaines31 | 100 | 60.1k | 7d ago | SKILL.md |
Frequently asked questions
- How do I install web-fetch?
- Run
npx skills add aiskillstore/marketplace --skill web-fetch. The install tabs above show the steps for each supported agent. - Which AI agents does web-fetch work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is web-fetch safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is web-fetch still maintained?
- The repository was last updated about 2 months ago, so web-fetch is actively maintained.
Skill content
View source on GitHubname: web-fetch description: Fetches web content as clean markdown by preferring markdown-native responses and falling back to selector-based HTML extraction. Use for documentation, articles, and reference pages at http/https URLs.
Web Content Fetching
Fetch web content in this order:
- Prefer markdown-native endpoints (
content-type: text/markdown) - Use selector-based HTML extraction for known sites
- Use the bundled Bun fallback script when selectors fail
Prerequisites
Verify required tools before extracting:
command -v curl >/dev/null || echo "curl is required"
command -v html2markdown >/dev/null || echo "html2markdown is required for HTML extraction"
command -v bun >/dev/null || echo "bun is required for fetch.ts fallback"
Install Bun dependencies for the bundled script:
cd ~/.claude/skills/web-fetch && bun install
Default Workflow
Use this as the default flow for any URL:
URL="<url>"
CONTENT_TYPE="$(curl -sIL "$URL" | awk -F': ' 'tolower($1)=="content-type"{print tolower($2)}' | tr -d '\r' | tail -1)"
if echo "$CONTENT_TYPE" | grep -q "markdown"; then
curl -sL "$URL"
else
curl -sL "$URL" \
| html2markdown \
--include-selector "article,main,[role=main]" \
--exclude-selector "nav,header,footer,script,style"
fi
Known Site Selectors
| Site | Include Selector | Exclude Selector |
|------|------------------|------------------|
| platform.claude.com | #content-container | - |
| docs.anthropic.com | #content-container | - |
| developer.mozilla.org | article | - |
| github.com (docs) | article | nav,.sidebar |
| Generic | article,main,[role=main] | nav,header,footer,script,style |
Example:
curl -sL "<url>" \
| html2markdown \
--include-selector "#content-container" \
--exclude-selector "nav,header,footer"
Finding the Right Selector
When a site isn't in the patterns list:
# Check what content containers exist
curl -s "<url>" | grep -o '<article[^>]*>\|<main[^>]*>\|id="[^"]*content[^"]*"' | head -10
# Test a selector
curl -sL "<url>" | html2markdown --include-selector "<selector>" | head -30
# Check line count
curl -sL "<url>" | html2markdown --include-selector "<selector>" | wc -l
Universal Fallback Script
When selectors produce poor output, run the bundled parser:
bun ~/.claude/skills/web-fetch/fetch.ts "<url>"
If already in the skill directory:
bun fetch.ts "<url>"
Options Reference
--include-selector "CSS" # Keep only matching elements
--exclude-selector "CSS" # Remove matching elements
--domain "https://..." # Convert relative links to absolute
Troubleshooting
Empty output with selectors: The page might be markdown-native. Check headers first:
curl -sIL "<url>" | grep -i '^content-type:'
Wrong content selected: The site may have multiple article/main regions:
curl -s "<url>" | grep -o '<article[^>]*>'
html2markdown not found: Install it, then retry selector-based extraction.
bun or script deps missing: Run cd ~/.claude/skills/web-fetch && bun install.
Missing code blocks: Check if the site uses non-standard code formatting.
Client-rendered content: If HTML only has "Loading..." placeholders, the content is JS-rendered. Neither curl nor the Bun script can extract it; use browser-based tools.
Related Skills
siyuan
47.3kAn open-source, privacy-first, self-hosted knowledge workspace where humans and AI agents work together 开源、隐私优先、自托管的知识工作空间,让人与智能体在此协作
algorithmic-art
177.9kCreating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems.
pptx
177.9kUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an em…
prospecting
60.1kWhen the user wants to find and qualify prospects for outbound, or asks who to target first, which signal to prioritize, or where to spot buyers with the problem: buying signals, enrichment, verification, account research.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
