SkillAgentSearch skills...

web-fetch

Fetches web content as clean markdown by preferring markdown-native responses and falling back to selector-based HTML extraction. Use for documentation, articles, and reference pages at http/https URLs.

Install / Use

npx skills add aiskillstore/marketplace --skill web-fetch

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

87/100

Supported Platforms

Universal

Our assessment of web-fetch

web-fetch scores 87/100 on our quality scale, 668th of 1,148 Content & Media skills we index.

Its SKILL.md is 3.3 KB long, well organised into 11 sections with 10 code examples: a solid amount of guidance for an agent.

It has 428 GitHub stars, a meaningful sign that others use it.

Substance
26/30
Structure
20/20
Description
15/15
Adoption
11/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so web-fetch is actively maintained.
  • Our last check on 2026-11-02 found the source still online.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

web-fetch compared with similar skills

All 4 of these similar skills score higher than web-fetch; compare them before choosing.

SkillScoreStarsUpdatedFormat
web-fetch (this skill)by aiskillstore8742855d agoSKILL.md
siyuanby siyuan-note10047.3ktodayMCP Server
algorithmic-artby anthropics100177.9k57d agoSKILL.md
pptxby anthropics100177.9k57d agoSKILL.md
prospectingby coreyhaines3110060.1k7d agoSKILL.md

Frequently asked questions

How do I install web-fetch?
Run npx skills add aiskillstore/marketplace --skill web-fetch. The install tabs above show the steps for each supported agent.
Which AI agents does web-fetch work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is web-fetch safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is web-fetch still maintained?
The repository was last updated about 2 months ago, so web-fetch is actively maintained.

name: web-fetch description: Fetches web content as clean markdown by preferring markdown-native responses and falling back to selector-based HTML extraction. Use for documentation, articles, and reference pages at http/https URLs.

Web Content Fetching

Fetch web content in this order:

  1. Prefer markdown-native endpoints (content-type: text/markdown)
  2. Use selector-based HTML extraction for known sites
  3. Use the bundled Bun fallback script when selectors fail

Prerequisites

Verify required tools before extracting:

command -v curl >/dev/null || echo "curl is required"
command -v html2markdown >/dev/null || echo "html2markdown is required for HTML extraction"
command -v bun >/dev/null || echo "bun is required for fetch.ts fallback"

Install Bun dependencies for the bundled script:

cd ~/.claude/skills/web-fetch && bun install

Default Workflow

Use this as the default flow for any URL:

URL="<url>"
CONTENT_TYPE="$(curl -sIL "$URL" | awk -F': ' 'tolower($1)=="content-type"{print tolower($2)}' | tr -d '\r' | tail -1)"

if echo "$CONTENT_TYPE" | grep -q "markdown"; then
  curl -sL "$URL"
else
  curl -sL "$URL" \
    | html2markdown \
        --include-selector "article,main,[role=main]" \
        --exclude-selector "nav,header,footer,script,style"
fi

Known Site Selectors

| Site | Include Selector | Exclude Selector | |------|------------------|------------------| | platform.claude.com | #content-container | - | | docs.anthropic.com | #content-container | - | | developer.mozilla.org | article | - | | github.com (docs) | article | nav,.sidebar | | Generic | article,main,[role=main] | nav,header,footer,script,style |

Example:

curl -sL "<url>" \
  | html2markdown \
      --include-selector "#content-container" \
      --exclude-selector "nav,header,footer"

Finding the Right Selector

When a site isn't in the patterns list:

# Check what content containers exist
curl -s "<url>" | grep -o '<article[^>]*>\|<main[^>]*>\|id="[^"]*content[^"]*"' | head -10

# Test a selector
curl -sL "<url>" | html2markdown --include-selector "<selector>" | head -30

# Check line count
curl -sL "<url>" | html2markdown --include-selector "<selector>" | wc -l

Universal Fallback Script

When selectors produce poor output, run the bundled parser:

bun ~/.claude/skills/web-fetch/fetch.ts "<url>"

If already in the skill directory:

bun fetch.ts "<url>"

Options Reference

--include-selector "CSS"  # Keep only matching elements
--exclude-selector "CSS"  # Remove matching elements
--domain "https://..."    # Convert relative links to absolute

Troubleshooting

Empty output with selectors: The page might be markdown-native. Check headers first:

curl -sIL "<url>" | grep -i '^content-type:'

Wrong content selected: The site may have multiple article/main regions:

curl -s "<url>" | grep -o '<article[^>]*>'

html2markdown not found: Install it, then retry selector-based extraction.

bun or script deps missing: Run cd ~/.claude/skills/web-fetch && bun install.

Missing code blocks: Check if the site uses non-standard code formatting.

Client-rendered content: If HTML only has "Loading..." placeholders, the content is JS-rendered. Neither curl nor the Bun script can extract it; use browser-based tools.

Related Skills

View on GitHub
GitHub Stars428
CategoryContent
Updated1mo ago
Forks45

Languages

Python

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium