webcrawler-deep-crawl
Deep-crawl any website from start URLs, return per-page LLM-ready text/markdown/HTML plus metadata (title, description, author, language, canonical URL, OG) and in-scope outbound links
Install / Use
npx skills add browser-act/skills --skill webcrawler-deep-crawlInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
AI & Machine LearningSupported Platforms
Our assessment of webcrawler-deep-crawl
webcrawler-deep-crawl scores 87/100 on our quality scale, 318th of 729 AI & Machine Learning skills we index (top 44%).
We have not analysed the SKILL.md file itself yet, so the content part of this score is an estimate until our crawler reaches it.
With 6,001 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated 33 days ago, so webcrawler-deep-crawl is actively maintained.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
webcrawler-deep-crawl compared with similar skills
All 4 of these similar skills score higher than webcrawler-deep-crawl; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| webcrawler-deep-crawl (this skill)by browser-act | 87 | 6.0k | 33d ago | SKILL.md |
| claude-memby thedotmack | 100 | 94.7k | today | CLAUDE.md |
| Understand-Anythingby Egonex-AI | 100 | 84.3k | 14d ago | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 73.9k | today | CLAUDE.md |
| CowAgentby zhayujie | 100 | 47.1k | today | CLAUDE.md |
Frequently asked questions
- How do I install webcrawler-deep-crawl?
- Run
npx skills add browser-act/skills --skill webcrawler-deep-crawl. The install tabs above show the steps for each supported agent. - Which AI agents does webcrawler-deep-crawl work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is webcrawler-deep-crawl safe to use?
- It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is webcrawler-deep-crawl still maintained?
- The repository was last updated 33 days ago, so webcrawler-deep-crawl is actively maintained.
Related Skills
claude-mem
94.7kPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More
Understand-Anything
84.3kGraphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.
headroom
73.9kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
CowAgent
47.1kOpen-source super AI assistant & Agent Harness. Plans tasks, runs tools and skills, self-evolves with memory and knowledge. Multi-agent, multi-model, multi-channel. Lightweight, extensible, one-line install.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.