x-tweet-search
Scrapes tweets from X (Twitter) by search query, user handle, or direct URL — returns full tweet data including text, author info, engagement metrics, media, and hashtags
Install / Use
npx skills add browser-act/skills --skill x-tweet-searchInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
MarketingSupported Platforms
Our assessment of x-tweet-search
x-tweet-search scores 93/100 on our quality scale, 129th of 642 Marketing skills we index (top 21%).
Its SKILL.md is 11 KB long, well organised into 17 sections with 3 code examples: a thorough specification that gives an agent plenty to work with.
With 6,001 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 3 months ago, so x-tweet-search is actively maintained.
- Our last check on 2026-10-26 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-06. Automated pattern scan on 2026-10-06. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
x-tweet-search compared with similar skills
All 4 of these similar skills score higher than x-tweet-search; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| x-tweet-search (this skill)by browser-act | 93 | 6.0k | 3mo ago | SKILL.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 79.8k | today | CLAUDE.md |
| Scraplingby D4Vinci | 100 | 103.6k | 2d ago | MCP Server |
| crawl4aiby unclecode | 100 | 87.4k | 3d ago | MCP Server |
Frequently asked questions
- How do I install x-tweet-search?
- Run
npx skills add browser-act/skills --skill x-tweet-search. The install tabs above show the steps for each supported agent. - Which AI agents does x-tweet-search work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is x-tweet-search safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is x-tweet-search still maintained?
- The repository was last updated about 3 months ago, so x-tweet-search is actively maintained.
Skill content
View source on GitHubname: x-tweet-search description: "Scrapes tweets from X (Twitter) by search query, user handle, or direct URL — returns full tweet data including text, author info, engagement metrics, media, and hashtags. Use when user mentions X, Twitter, tweet scraping, scrape tweets, get tweets, fetch tweets, twitter data, x.com data, tweet collector, tweet search, twitter search, social media scraping, twitter handle scraping, scrape user tweets, from:username, twitter keyword search, hashtag search, twitter timeline, twitter list scraping, collect twitter posts, export tweets, tweet analysis, twitter monitoring, search twitter, get twitter data, twitter api alternative, tweet dump."
X (Twitter) — Tweet Search & Scraper
Search query / handle / URL → structured tweet list (text, author, metrics, media, entities)
Language
All process output to user (progress updates, process notifications) follows the user's language.
Objective
Collect tweets matching a search query, from specific user profiles, or from direct URLs, extracting complete structured data for each tweet.
Prerequisites
- Target page is already open in the browser:
https://x.com/search?q=...orhttps://x.com/{handle}or a direct tweet/list URL - User must be logged in to X (user avatar or username visible in left sidebar)
Pre-execution Checks
1. Tool Readiness
If browser-act has been confirmed available in the current session → skip this step.
Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.
2. Login Verification
If login status for X has been confirmed in the current session → skip this step.
Otherwise: open https://x.com and observe the left sidebar:
- User avatar or "@username" visible at the bottom → logged in, continue
- "Sign in" or "Log in" button visible → not logged in, inform user that X login is required and assist with the login flow
User refuses or cannot log in → terminate execution.
Capability Components
This Skill's operational boundary = what the user can manually do in their browser. It reads tweet data already rendered in the X DOM, never bypassing authentication. JS is encapsulated in
scripts/files, invoked viaeval "$(python scripts/xxx.py)". Use the bash tool for execution.
DOM: Tweet list extraction (React Fiber)
Extracts all currently visible tweets from the page via React internal state (React Fiber). Works on search pages, profile pages, list pages, and any X page rendering tweet articles.
Wait for tweets to appear before extracting:
wait --selector "article[data-testid='tweet']" --state attached --timeout 15000
Extract: eval "$(python scripts/extract-tweets.py)"
Returns a JSON array. Each element:
[
{
"id": "2059255862548738182", // tweet ID
"url": "https://x.com/NASA/status/2059255862548738182", // direct link
"text": "Full tweet text including hashtags and URLs", // full_text field
"created_at": "2026-05-26T12:50:31.000Z", // ISO 8601
"lang": "en", // ISO 639-1 language code, null if unknown
"author_id": "11348282", // author user ID
"author_name": "NASA", // display name
"author_screen_name": "NASA", // @handle (without @)
"author_profile_image": "https://pbs.twimg.com/profile_images/.../photo.jpg",
"author_followers": 92080161, // follower count
"author_following": 305, // following count
"author_verified": false, // legacy blue checkmark
"author_blue_verified": true, // X Blue / Gold / Gray checkmark
"author_location": "Washington, D.C.", // profile location, null if not set
"author_description": "Explore the universe...", // bio, null if empty
"like_count": 82579,
"retweet_count": 11952,
"reply_count": 4230,
"quote_count": 1850,
"bookmark_count": 12400,
"view_count": 25923006, // null if not available
"is_retweet": false,
"is_quote": false,
"is_reply": false,
"in_reply_to_tweet_id": null, // parent tweet ID if is_reply=true
"in_reply_to_user": null, // @handle of replied-to user
"conversation_id": "2059255862548738182",
"hashtags": ["AI", "Space"], // without #
"urls": ["https://example.com/article"], // expanded URLs from entities
"mentions": ["SpaceX", "ESA"], // @handles without @
"media": [
{
"type": "video", // "photo", "video", "animated_gif"
"url": "https://pbs.twimg.com/amplify_video_thumb/.../img/thumb.jpg",
"alt_text": null,
"video_variants": [
{"bitrate": 2176000, "url": "https://video.twimg.com/.../1280x720/video.mp4"},
{"bitrate": 832000, "url": "https://video.twimg.com/.../640x360/video.mp4"}
]
}
],
"source_name": "Twitter for iPhone", // client used to post
"source_url": "http://twitter.com/download/iphone"
}
]
URL Construction Guide
Input type → URL mapping
searchTerms (keyword / advanced query):
- Sort Latest:
https://x.com/search?q={url_encoded_query}&src=typed_query&f=live - Sort Top:
https://x.com/search?q={url_encoded_query}&src=typed_query - Sort Latest+Top: run both URLs in sequence, deduplicate by tweet ID
twitterHandles (scrape a user's profile tweets):
- Option A (profile page):
https://x.com/{handle}— shows all tweets/retweets - Option B (search): use
from:{handle}as the search query — more filter-compatible
startUrls (direct URLs): navigate to the URL as-is. Supported types:
- Tweet URL:
https://x.com/{user}/status/{id}— single tweet conversation - Profile URL:
https://x.com/{handle}— profile timeline - Search URL:
https://x.com/search?q=...— use directly - List URL:
https://x.com/i/lists/{list_id}— list timeline
Filter parameters → query operators
Append these operators to the base query string (space-separated):
| Parameter | Query operator | Example |
|-----------|---------------|---------|
| tweetLanguage | lang:{code} | lang:en |
| onlyVerifiedUsers | filter:verified | |
| onlyTwitterBlue | filter:blue_verified | |
| onlyImage | filter:images | |
| onlyVideo | filter:videos | |
| onlyQuote | filter:quote | |
| author | from:{handle} | from:NASA |
| inReplyTo | to:{handle} | to:NASA |
| mentioning | @{handle} | @NASA |
| minimumRetweets | min_retweets:{n} | min_retweets:100 |
| minimumFavorites | min_faves:{n} | min_faves:500 |
| minimumReplies | min_replies:{n} | min_replies:10 |
| start | since:{YYYY-MM-DD} | since:2024-01-01 |
| end | until:{YYYY-MM-DD} | until:2024-06-01 |
| geotaggedNear + withinRadius | near:"{location}" within:{radius} | near:"New York" within:15mi |
| geocode | geocode:{lat},{lon},{radius} | geocode:40.7,-74.0,10km |
| -filter:retweets | exclude retweets | |
Example: Scrape English tweets from NASA since 2024 with ≥100 likes, excluding retweets:
query = "from:NASA lang:en since:2024-01-01 min_faves:100 -filter:retweets"
url = "https://x.com/search?q=" + encodeURIComponent(query) + "&src=typed_query&f=live"
Pagination
DOM Pagination (scroll to load more):
X dynamically appends new tweet articles to the DOM as the user scrolls. Tweets already rendered remain in the DOM (no virtualization for typical result sets < ~500 tweets).
Loop pattern:
- Record current tweet count: extract → note
len(results) scroll down --amount 2000wait stable- Extract again → compare IDs, add new ones to collection
- Termination conditions:
- New extraction returns 0 new tweet IDs (end of results reached)
collected >= max_items(if limit specified)- Same tweet IDs returned 3 consecutive times (no more data loading)
Deduplication: track seen IDs in a Python set, filter before appending to output.
Success Criteria
result count >= 1 and id field non-null rate = 100% and text field non-null rate = 100%
Known Limitations
- React Fiber key: The
__reactFiberprefix includes a session-specific hash (e.g.,__reactFiber$ozawbbp0gp). The script usesstartsWith('__reactFiber')which is stable across deployments; only fails if React is replaced with a different framework. - DOM virtualization: For very large result sets (500+ tweets), X may virtualize older DOM nodes to reclaim memory. If extraction suddenly returns fewer tweets than expected after extended scrolling, the remaining tweets may have been removed from DOM. Mitigation: save results incrementally after each scroll batch.
- View count:
view_countis null for older tweets, low-engagement tweets, or when X doesn't expose it. This is a data availability issue, not an extraction failure. - List URL: Requires a valid, accessible public list ID. Private lists return no tweets.
filter:blue_verified: Whether X's search engine respects this operator may vary; verify results if blue-verified filtering is critical.customMapFunction: Not applicable — this is a source-platform-specific JS transform parameter, not a data filter relevant to browser extraction.- Rate limiting: X may temporarily stop loading new tweets after aggressive scrolling. Add 1–2s delays between scroll batches for large collections.
Execution Efficiency
- Batch orchestration: For multiple search queries or handles, process them serially in one browser session with short delays (1–2s) between navigations. Do not open parallel searches in the same browser — use separate sessions for parallel work.
- Test before batch execution: Test with 1 query first, verify 10+ tweets extract correctly, then run the full batch.
- Reduce redundant pre-operations: Login check only needed once per session; skip on subsequent queries.
- Error resumption: Save results to a file after each URL/query. On failure, resume from the last saved query rather than restarting from scratch.
- Incremental save: After each scroll batch, append new tweets to the output file rather than holding everything in memory.
Experience Notes
Path: {working-directory}/browser-act-skill-forge-memories/x-tweet-scraper-x-tweet-search.memory.md
Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.
After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line:
{YYYY-MM-DD}: {what happened} → {conclusion}
Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.
Related Skills
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
headroom
79.8kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Scrapling
103.6k🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
crawl4ai
87.4kOfficial Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
